The people who rise fastest in most organisations are rarely the most competent, they are simply the most confident, and decades of research suggests we are almost incapable of telling the two apart.

You have almost certainly worked with the person this piece is about. They speak up first in meetings. They volunteer opinions on subjects they have never studied. They are described, in their performance reviews, as having presence, gravitas, or executive potential. They are promoted more quickly than the quiet colleague sitting three desks over, who has read everything on the subject and could answer the same questions correctly if anyone thought to ask them. And when the promoted person eventually reaches a level of responsibility their actual abilities cannot support, the organisation is genuinely surprised.

The specific pattern has been documented in one form or another since at least the late 1990s, across dozens of laboratory experiments and workplace field studies. What the accumulated research has established is not that competent people are being deliberately overlooked. It is that the human brain, on the current experimental evidence, does not have a reliable mechanism for distinguishing between people who are good at something and people who merely appear to be good at something.

And the two things tend to look, from the outside, almost identical.

The paradox at the bottom

According to a foundational 1999 paper by Dr Justin Kruger and Dr David Dunning at Cornell University, published in the Journal of Personality and Social Psychology and hosted in full text by the University of Michigan’s Social Sciences and Society Initiative, titled “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments”, the relationship between how competent a person is at a task and how competent they believe themselves to be is not what most people assume. Across four studies, testing participants on skills as varied as humour, grammar, and logical reasoning, Kruger and Dunning found that the participants who performed worst on the objective tests consistently estimated themselves to be performing well above average.

The specific numbers are worth pausing on. Participants who scored in the bottom quartile, at the twelfth percentile of the actual test population, estimated their own performance to be at the sixty-second percentile. They believed themselves to be more capable than roughly two thirds of the people around them, when they were in fact less capable than roughly seven eighths of them.

The reason, on Kruger and Dunning’s analysis, is genuinely difficult to unwind. The metacognitive skill required to accurately evaluate one’s own performance at a task is the same skill required to perform the task well in the first place. A person who lacks basic competence at grammar also, on the same measurement, lacks the ability to recognise the errors they are making. Their confidence is not despite their incompetence. It is a direct symptom of it. They cannot see what they cannot see, and their inability to see it registers, subjectively, as certainty rather than as ignorance.

The finding has been replicated many times, refined in various directions, and generalised beyond the specific undergraduate populations Kruger and Dunning originally studied. What it has never really lost is its central observation. The people who are worst at something, on the accumulated evidence of the past twenty-five years, are the ones most likely to feel confident that they are excellent at it.

Related to this – we came across a video that dives into the psychology of why society is producing so many toxic people. From work to personal lives, there’s been a shift in how we value people, and it’s not been for the better…

Why the rest of us cannot tell

The other side of the problem, and the one that actually matters for how organisations make decisions about who to promote, was documented in careful detail by a 2012 paper by Cameron Anderson, Sebastien Brion, Don Moore and Jessica Kennedy, working across the Haas School of Business at Berkeley, the IESE Business School at the University of Navarra, and what was then the Wharton School, published in the Journal of Personality and Social Psychology under the title “A Status-Enhancement Account of Overconfidence”. Anderson and his colleagues ran six separate studies, using both naturalistic group settings and controlled experiments, testing whether the confidence a person displays about their own abilities influences how competent other people believe them to be.

What they found was that it does, powerfully, and in a specific direction. Participants who were more overconfident, meaning that they believed themselves to be more skilled than they actually were, were consistently rated as more competent by their peers. They were awarded higher social status within their groups. They were listened to more often. Their contributions to group tasks were weighted more heavily in the group’s final decisions.

The mechanism was not that overconfident people were persuasive through explicit argument. It was that overconfidence produces, on Anderson’s analysis, what the researchers called a behavioural signature. Overconfident people speak first. They speak more often. They use more assertive body language. They express fewer hedges and qualifications. They interrupt more. They appear to know things. And observers, watching this pattern, register it as evidence of underlying competence rather than as evidence of self-belief that has become unmoored from the underlying ability.

The uncomfortable finding, when the Anderson results are combined with the Kruger and Dunning findings, is that the two effects reinforce each other in exactly the direction that produces bad organisational outcomes. The least competent people are the most confident. The most confident people are read as the most competent. And the organisation, which cannot tell them apart, promotes them.

What this actually does inside a workplace

The specific consequence for how organisations function was laid out in an August 2013 Harvard Business Review article by Tomas Chamorro-Premuzic, professor of business psychology at University College London and Columbia University, under the title “Why Do So Many Incompetent Men Become Leaders?”. Chamorro-Premuzic’s argument, drawing on his own research and a wide reading of the surrounding literature, was that the everyday inability of hiring managers and promotion committees to distinguish between confidence and competence has produced a systematic drift, across most large organisations, toward a leadership population that is on average less capable than the people they lead.

The pattern he described has since been elaborated in a full-length book of the same name, in which Chamorro-Premuzic argues that the personality traits which help someone rise into a leadership role, meaning overconfidence, self-promotion, willingness to interrupt, comfort with dominance, and a certain kind of thick-skinned charm, are almost the exact opposite of the traits that predict actually performing well in the role once obtained. Which produces, at the level of the organisation, the pattern that everyone recognises but few organisations know how to correct. The confident rise. The competent, on average, do not. And the resulting leadership population underperforms measurably against a hypothetical population selected on ability alone.

What the whole body of research points at, at the level of a proposition most working adults will recognise from experience, is that the person in the corner office is not there because someone examined the evidence and concluded that they were the best person for the job. They are there because at some earlier stage in their career they projected the specific behavioural signature that the human brain, on the accumulated evidence of thousands of hiring decisions, mistakes for competence. Some of them turn out to be genuinely capable anyway. Some do not. The organisations that promoted them do not, on average, have a reliable way of telling the difference.