This reminds me of the rather puzzling fact that intelligence and rationality are only moderately correlated. We seem to be particularly prone to thinking errors when the matter under consideration is directly related to us as humans and individuals and to our role in society and the world.
“It follows from the way in which the brain works that there are inevitably gaps in the model and thus gaps in our cognitive abilities.”
In addition to gaps that evolution saw no need to fill (metaphorically speaking), it may have deliberately built cognitive blinkers in us to prevent us from seeing certain things, things that would have been detrimental to our survival and reproduction. I wrote a brief post on the constraints of human intelligence here, and developed the idea further here.
“Let’s hope someone’s smart enough to fund serious research into the topic of human reasoning, before it’s too late.”
Daniel Kahneman has concluded that even a full understanding of our cognitive biases is not enough to correct them. Perhaps we might still learn to do so by finding the right cognitive mechanisms. Or we might find that to be impossible, and conclude that the only way to circumvent our reasoning limitations is to delegate our thinking to a capable artificial intelligence—or enhance our brains somehow. Either possibility seems quite out of reach for now, though. We’ll have to continue to manage with our current intrinsic capabilities, at least for the time being.