Better, not wiser: confidence and expertise after AI

In 1999 Justin Kruger and David Dunning tested Cornell students on logic, grammar and humour, then asked them how well they thought they had done.

Almost everyone put themselves above average. The students who scored lowest, around the 12th percentile, thought they had beaten about 60% of their classmates¹; the best underestimated themselves. Confidence still rose with skill, just not nearly as steeply, and a short lesson made the weakest much better judges of their own answers.

Statisticians have argued ever since about how much of the famous pattern is psychology and how much is the way the numbers are grouped; much of it appears even in random data². The fair summary: most people overestimate, the gap is widest at the bottom, and why is still debated. What matters here is the part nobody disputes. Doing something and judging how well you did it draw on the same skill.

In 2025, researchers at Aalto University asked whether AI changes that. Their answer: it raises the work, and leaves the judgement where it was.

They gave people hard reasoning questions from the law-school admissions test, and randomly gave half of them ChatGPT³. With AI, people scored about three and a half points higher out of twenty. But both groups guessed their scores too high, by almost exactly the same amount, about four points, even when they were paid to guess accurately. Their confidence barely told their right answers from their wrong ones.

Nearly half asked the AI just once per question and took what it said. That is not laziness; it is what a fluent, confident answer invites. In a separate survey of 319 knowledge workers, Microsoft and Carnegie Mellon⁴ found that the more people trusted the AI, the less critical thinking they reported, and the more they trusted themselves, the more they reported. It is self-reported and shows a link, not a cause, but the direction is worth noticing: confidence in yourself, not in the tool, kept judgement switched on.

AI helps the least experienced most. That is good news, with a catch.

In a call centre, AI help made novice agents 34% more productive⁵ and barely changed the experts. In a study of professional writing, ChatGPT helped the weakest writers most⁶. Among 758 consultants at the Boston Consulting Group, those with AI did more work, faster and better, on tasks the AI was good at. On a task just outside what it could do, they were about 19 points less likely⁷ to get the right answer than those without it. The researchers called the edge a jagged frontier, because nothing on the screen tells you which side of it you are on.

Put together: the people gaining most from AI are often the ones with the least experience to spot a confident mistake. That is not a reason to keep them away from the tools. It is a reason to build the checking in.

For a designer, the danger is polish. AI makes work look finished long before anyone has checked that it works for the people who will use it.

Developers using an AI assistant in one study wrote less secure code⁸ and were more likely to believe it was secure; it was a small study, but the pattern is the one to watch for. Novice designers shown AI images while brainstorming fixated on them⁹ and came up with fewer, less original ideas. And the Nielsen Norman Group warns that polished AI prototypes can sabotage your stakeholder communication¹⁰, because a finished-looking screen reads as a decision already made.

Designers seem to sense this. In Figma’s own 2025 survey of its users, a vendor’s figures, 78% said AI made them more efficient¹¹, but only 32% said they could depend on what it produced.

None of this is about other people being foolish. It is about all of us, experts first.

In a Stanford study in 1990, people tapped out the rhythm of well-known songs and predicted listeners would name half of them. Listeners named 3 out of 120¹⁵. Once you know something, you cannot easily imagine not knowing it. A design team has always been the tapper, sure the page is clear because they can hear the tune. AI adds a second voice that sounds just as sure.

The way out is the one the research keeps pointing to: answer first, check cheaply, explain it back, and watch real people use what you made.

This one runs here, on this page: it measures you, not a design. Use what it shows you when you review a prototype or a live page with the other tools.