On Confidence, Competence and the Dunning-Kruger Effect

I started obsessing over health and wellness when I was 18. After my athletic aspirations imploded, I channelled my focus toward mastering all things exercise, nutrition, and health. I completed a personal training certification. I delved deeply into nutritional biochemistry. I got a job sweeping floors and coaching classes at a local gym. I was passionate and motivated, a classic case of a nascent health nut. This all happened over the course of only a few months.

Here’s the odd thing: despite being such a beginner, I truly felt like an expert.

When people came to me asking for advice, whether they wanted to lose weight, rehab their shoulder, or fix their irritable bowel syndrome, I didn’t hesitate to offer advice. I felt like I knew what I was talking about.

“It’s simple. You need to do ‘x.’ You need to stop doing ‘y.’ You’re welcome.”

It’s painful reflecting on those interactions. That advice came from an infant of the industry with next to no real-world coaching experience.

Here’s the issue: I didn’t know what I didn’t know. I was unskilled and correspondingly unaware of my lack of skill. Incompetent, but blissfully confident.

Fast forward a decade. I now have two science degrees, I’ve spent thousands of hours studying and I've have treated thousands of clients.

But if you came to me today asking how to rehab your painful shoulder, my answer would drown you in nuance:

It’s complicated. It depends on your goals, your anatomy, your injury history, your training experience. I’d need to thoroughly assess you to even know where to start. Go see your local rehab professional.

Strangely, years of study and practice have actually made me less bold in my convictions. As the saying goes, “the further you get from shore, the deeper the water.”

My story is a textbook example of a psychological phenomenon called the Dunning-Kruger Effect.

The Dunning-Kruger Effect emerged from studies in which individuals rated their own skill in areas like grammar, humour, and logic, and were then tested objectively in these same domains. Researchers then compared the self-perception data with the objective testing data. The punchline is that the least competent individuals are the most likely to overestimate their own performance [1]. For example, individuals in the 12th percentile predict their scores to be at the 62nd, on average. This effect holds up to about the 80th percentile.

It turns out that the skills which reflect competence in a domain are the very same skills needed to assess competence in that domain. While we all overestimate our abilities to some degree, beginners do so most outrageously.

As Darwin so wisely pointed out in 1871:

“Ignorance more frequently begets confidence than does knowledge.”

This figure, which plots confidence as a function of wisdom, sums it up well:

Source: Wikipedia Commons

At 18, I was shouting from the rooftops of Mt. Stupid, too naïve to zoom out and notice where my feet were planted. Many years later, I am humbly residing at Basecamp, re-examining the basics and acutely aware of what I don't know.

So, here's the question: how can one expedite the slow climb from the Valley of Despair up the Slope of Enlightenment?

Many people will tell you the key is to 'fake it until you make it.' While this advice is well-intentioned, it is shortsighted.

We have a word to describe someone with lots of self-confidence in the absence of competence: delusional. No one is an expert at anything until they practice, fail, and learn from their mistakes for a very long time. 

The unfortunate truth is that there are no "hacks" to building confidence. You simply need the willingness to put in the repetitions required to gain genuine expertise. 

Rather than fake it until you make it, just engage in consistent action towards your goals. As the author James Clear describes in his excellent book Atomic Habits, each action we take is like a vote for the type of person we want to be. Over time, when you put in the work to generate proof of expertise, your identity will change.

This goes for anything, whether it's becoming an expert in a medical field or becoming an exerciser. 

Competence precedes confidence, not the other way around. 

Here's a great video discussing the Dunning-Kruger Effect through the lens of the importance of intellectual humility: 

References

[1] Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of personality and social psychology, 77(6), 1121.