The Illusion of Understanding

When an explanation feels convincing, how do we know whether we have actually understood it?

There is a particular satisfaction in encountering an explanation that suddenly makes something difficult seem simple. A complex physiological process becomes intelligible. An unfamiliar diagnosis begins to make sense. What was uncertain a moment earlier now appears obvious.

Artificial intelligence has become remarkably good at producing these moments.

Ask it to explain almost any medical concept and it can construct an answer that is coherent, accessible and often persuasive. It can adapt the explanation to our level of knowledge, simplify difficult relationships and respond patiently to questions we might hesitate to ask a colleague.

The experience can feel remarkably like learning.

But feeling that we understand something is not necessarily evidence that we do.

Medicine has always required us to distinguish confidence from competence. We recognise the dangers of premature diagnostic closure, confirmation bias and the tendency to favour explanations that fit our existing assumptions. Clinical training attempts to cultivate an awareness of these weaknesses, although experience offers no immunity from them.

AI introduces another dimension to this problem. It can make an explanation convincing before we have established whether it is correct.

In a 2026 randomised study published in npj Digital Medicine, 111 junior medical students were exposed to correct AI explanations, misleading explanations or no explanations while answering clinical questions. Students receiving misleading explanations performed significantly worse than those given no assistance. Correct explanations, surprisingly, did not produce a statistically significant improvement in diagnostic accuracy.

But the more revealing finding concerned confidence.

Students exposed to AI explanations reported greater confidence regardless of whether those explanations were accurate. Those receiving misleading information showed poor alignment between their confidence and the correctness of their answers.

The study was conducted under controlled educational conditions, with inexperienced learners and a limited set of questions. It cannot establish that the same effects occur among experienced clinicians or during real patient care.

Nevertheless, it exposes an uncomfortable possibility.

AI may influence not only what we believe, but how certain we feel about believing it.

Consider a clinician reviewing an unfamiliar clinical presentation. An AI system produces a plausible diagnosis, supported by a detailed explanation and apparently logical reasoning. Nothing seems obviously wrong. The explanation is sufficiently sophisticated to be reassuring, yet sufficiently accessible to feel understood.

What happens if one of its assumptions is incorrect?

Recognising that error requires more than reading the explanation carefully. It requires independent knowledge, an appreciation of competing possibilities and enough intellectual distance to question a conclusion that appears coherent.

The difficulty is that fluency can be mistaken for reliability. An answer that is clearly expressed may seem more credible than one that is hesitant or incomplete, even when that hesitation reflects genuine uncertainty.

And medicine is rarely as orderly as the explanations we construct afterwards.

This does not mean that AI should be distrusted simply because it communicates well. Clear explanations are valuable, particularly when they make complex subjects accessible. Nor are clinicians naturally better at judging their own understanding without technological assistance. Human reasoning has always been vulnerable to misplaced confidence.

The question is whether AI helps us recognise those limitations or merely makes them less visible.

That distinction should shape how we use it.

An AI system that agrees with our diagnosis and provides an elegant justification may reinforce an existing belief. One that identifies missing information, challenges our assumptions and presents credible alternatives may improve the quality of our judgement.

The difference is not necessarily in the sophistication of the technology. It is in the intellectual relationship we establish with it.

Medical education and clinical practice increasingly emphasise AI literacy. But this cannot be reduced to knowing how to construct prompts or recognising that models occasionally hallucinate. It must include the ability to recognise when we lack the knowledge required to evaluate an answer, and the judgement to seek independent verification.

Perhaps the greatest danger is not that AI will give us incorrect answers. We already understand that possibility.

It is that an incorrect answer, delivered with clarity and confidence, may leave us feeling more certain than we were before.

In medicine, knowing that we might be wrong is often what keeps us safe. We should be cautious of any technology that makes that uncertainty too easy to forget.