AI can produce an answer that sounds plausible and is still wrong. That risk is often called hallucination or confabulation. It does not mean the model is deliberately lying; it means generated output is not automatically verified information.
AI doesn't lie. It fills gaps.
There's a word that comes up a lot when people talk about AI getting things wrong.
Hallucination.
It sounds dramatic. Like the AI is seeing things. But what it actually means is much more ordinary, and much more dangerous for that.
What hallucination actually means
NIST, the US National Institute of Standards and Technology, defines it as confabulation: AI produces content that is confident, fluent, and wrong. Not randomly wrong. Plausibly wrong. Wrong in a way that fits the pattern of how a correct answer would read.
OpenAI has written about why this happens. Models can be trained in ways that reward producing an answer over admitting uncertainty. So when the model doesn't have reliable information, it doesn't always say so. It produces something that sounds like an answer.
The result can be fluent and convincing even when a detail is missing or incorrectly filled in. Modern systems may show uncertainty, cite sources or use web search, but verification can still be necessary.
What it looks like in practice
A lawyer in New York submitted a legal brief to a court in 2023. The brief cited several court cases as precedents.
The cases didn't exist.
ChatGPT had generated them. Real-sounding names, real-sounding case numbers, real-sounding summaries. The lawyer hadn't checked. The judge noticed. The lawyer was fined.
The case was covered by the New York Times and became one of the most widely reported examples of AI hallucination causing real damage.
That's an extreme case. But the same thing happens at a much smaller scale every day.
A business owner asks AI for the average price of web design in Spain. Without current sources, a precise-looking number may appear without solid evidence. With web search, the answer can be better grounded, but you should still inspect which sources were used and how current they are.
Someone asks about employment rules in Spain. The answer may be well structured and still use the wrong exception, region or date. Legal and employment information should therefore be checked against a current official source or qualified professional.
Someone asks for the opening hours of a Costa Blanca business. An AI system with web search may retrieve a current listing, but the listing itself may still be outdated. Important local details should be checked with the business or its official listing.
In each case, the answer reads the same whether it's accurate or invented.
Fake sources are a specific problem
AI can misstate facts, citations or references. In systems without real source retrieval, fully fabricated references can also occur.
If you ask for a source, check that the link actually exists and that the source supports the claim being made. A convincing-looking citation is not proof by itself.
This is particularly dangerous because a source feels like proof. You see a citation and your guard drops. You stop reading critically. That's the moment the invented information gets through.
The rule is simple: if a source matters, open it yourself. Don't trust the title. Don't trust the summary. Open the actual page, check the date, check the author, check whether it says what the AI claimed it says.
Why it happens more with local information
AI models are trained on large amounts of international data, and modern systems can also retrieve current sources. Coverage still varies significantly by topic, language and region.
For local regulation, tax, property and permit questions on the Costa Blanca, general European or national information may be too broad. The issue is not simply that local information is 'missing from training data'; it is that you must verify whether a source actually applies to Spain, the Valencian Community and, where relevant, the municipality.
This is the pattern I see most often: someone uses AI to research something specific to Spain or to the Costa Blanca, and the answer they get is technically about Europe, or worse, about a completely different country.
It reads correctly. The context is wrong.
The thing that makes this hard to spot
If AI said "I'm not sure about this" before every answer it was uncertain about, hallucination would be easy to manage.
Sometimes it does, sometimes it does not. Modern systems may show uncertainty, warnings or citations, but this is not reliable for every error.
Confident wording tells you little about factual certainty. A genuine and an incorrect source can look equally convincing at first glance, which is why source checking still matters.
You can't tell from the text itself. You have to check.
That's not a criticism of AI as a tool. It's just the reality of how it works right now. And knowing it changes how you should use it.
What to do about it
Check anything that matters. That means:
Pricing: look at actual current sources for Spain specifically. Legal and tax rules: check with a professional or a verified Spanish government source. Sources and citations: open every link before you trust it. Local business information: check directly with the business. Statistics: find the original study, not the AI summary.
AI is still useful for structure, for drafts, for generating ideas, for rewriting rough text. For those things, the exact accuracy of every claim matters less.
But for anything that goes on your website, into a contract, into advice you give to a client, or into a decision that costs money: check it independently.
The next article is about what happens when AI-generated misinformation doesn't stay in one place. How it spreads, how it gets amplified, and why "ChatGPT said so" is a phrase that should make you more cautious, not less.
Read part 3: "ChatGPT said so" is not a source.
If you want to know whether the content on your website can be trusted, send me your URL on WhatsApp and I'll take a look.
Read more:
