When AI Gets It Wrong, It Can Be Very Convincing
Artificial intelligence is rapidly becoming part of everyday life. We use it to research, write, summarise documents, generate images and help make sense of complex information. Most of the time, it is remarkably useful.
But every now and then, it gets something completely wrong.
Not obviously wrong.
Convincingly wrong.
That is what makes AI hallucinations so fascinating and, in some situations, so dangerous.
A World Cup Mistake
I had two reminders of this recently.
The first came during a family sweepstake for the World Cup. I happened to draw Argentina while my son drew Spain, so we’re getting ready for the final decider on Sunday.
The wider family is around 12 people, and I’m the banker each year for any big football competition. This year, instead of the usual spreadsheet list of names and teams, I did it all with AI. Ahead of the knockout stages, I asked AI to analyse the remaining teams and produce an update to the leaderboard graphic.
When I got the stunning looking graphic, with all the teams, the points that individual had accumulated with those teams, who was in and who was out, it did a remarkable job. For the final 8, it confidently included Canada.
The only problem was that Canada had already been eliminated.
This wasn’t a complex legal opinion or an obscure technical question. It was simply a football tournament. Thankfully, my son spotted the mistake immediately.
The Yacht That Was Never Requested
The second example was even more interesting.
I asked AI to create an image featuring the yachts from ten brochures I had uploaded. The instructions were straightforward: use only the yachts featured in those brochures.
Instead, the AI introduced an entirely different yacht. Which I hadn’t noticed as I was so impressed with the results I just proceeded to edit a video and social media post.
It even misspelled the name so it looked similar to another yacht before presenting it as though it belonged in the collection.
Out of curiosity, I checked after someone commented that they liked that particular yacht.
The yacht actually existed. It was genuinely for sale, with realistic specifications and pricing.
The AI hadn’t invented a fantasy yacht. It had blended real-world information with the wrong context and then presented the result with complete confidence.
Why Hallucinations Matter
This is why AI hallucinations deserve attention.
They are rarely dramatic.
More often, they’re subtle.
A believable statistic. An incorrect date. A misquoted source. The wrong person. An extra yacht.
The answers are usually well written, logically structured and presented with complete confidence. That’s precisely what makes them easy to accept without question.
As AI becomes embedded in business, education, marketing, software development and research, this matters more than ever.
Professional presentation should never be mistaken for factual accuracy.
Fluency is not evidence.
AI Still Needs Human Judgement
None of this is an argument against AI.
Quite the opposite.
These tools are transforming how we work, learn and create. Used well, they save enormous amounts of time and often uncover ideas we might never have considered.
The key is understanding what AI is.
It is an exceptionally capable assistant.
It is not an unquestionable authority.
Whenever names, dates, figures, prices, legal information or anything destined for publication are involved, verification still matters.
Human judgement hasn’t become less valuable.
It’s become more valuable.
The New Skill: Healthy Skepticism
Both of my examples ended well because someone checked.
One mistake was spotted by my son.
The other was caught because I took the time to verify the result.
Perhaps that’s one of the most important skills of the AI era.
Not simply learning how to write better prompts, but learning when to pause, question the answer and check the facts.
The better AI becomes, the more convincing its mistakes will be.
And that makes critical thinking one of the most valuable skills we can develop.
Have you experienced an AI hallucination that looked completely believable until you checked it?