Tornado Alley (PROG)

[fuagf]

Right, LLMs like Gemini, Perplexity et al do hallucinate (confabulate), like humans. They
do as humans do. It doesn't follow that all humans that do are unreliable for all things.

? "the fact LLM like Gemini hallucinaate (confabulate) as humans do doesn't mean they are unreliable" ?

The fact that LLMs like Gemini confabulate does not by itself make them unreliable; it means they can produce fluent but ungrounded outputs, so reliability depends on the task, the model’s constraints, and whether outputs are checked or grounded in sources.

Why the distinction matters

Human confabulation is not the same as deliberate deception, and research on LLMs uses the term similarly to describe arbitrary or incorrect generations rather than intent. A model can be highly useful when it is used for drafting, brainstorming, summarizing, or retrieval-assisted answering, even if it sometimes produces false statements.

When it is reliable

LLMs are more reliable when the answer can be verified, the prompt is specific, and the system is supported by retrieval, tools, or uncertainty detection. One Nature paper notes that detecting confabulations can help systems avoid answering questions likely to trigger ungrounded outputs, which improves practical reliability. Another study argues that confabulations can mirror human sense-making patterns, which explains why they can sound convincing without being accurate.

When caution is needed

They become less reliable when asked for exact facts, rare details, - [see red below] - or high-stakes decisions without verification, because fluent text can mask errors. So the right conclusion is not “LLMs are unreliable,” but “LLMs are conditionally reliable, and their outputs need context-specific checks”.

Plain-language version

A better way to say it is: hallucination is a failure mode, not a total indictment. Like a calculator that is great at arithmetic but bad at prose, an LLM can be very capable in some settings and risky in others.

Follow-ups

When Gemini hallucinates, does that really mean it’s unreliable? The evidence cuts both ways Computer
?
A practical risk matrix for using Gemini safely in high-stakes work Computer
?
What are the lowest hallucination rates for major LLMs in 2026 benchmarks

How does semantic entropy detect LLM confabulations

Why do LLMs produce fluent but incorrect outputs

https://www.perplexity.ai/search/the-fact-llm-like-gemini-hallu-O7i4R9gYRriT1v3w9EJokQ

Each of those followups gives another summary, whatever to call it, as the above.

**

Good, i was having trouble with their use of hallucination vs confabulation, and this 2nd of yours helps:

Are AI Models Hallucinating—Or Just Confabulating?

June 10, 2025
Blog Post ML NLP Python R Data Science

Alternative text

These days, it feels like the term hallucinations is everywhere—from media headlines to blog posts, technical reports, and peer-reviewed papers. At its core, the term hallucination is often used to describe what happens when a large language model (LLM) generates an answer that’s incorrect, misleading, or just plain nonsensical.

But thinking about the term itself, I started to wonder whether hallucination is really the right word to describe this well-known issue in natural language generation with LLMs. After some research and discussion, I don’t think it is. A more fitting term, I believe, is confabulation.

These perspectives are also reflected in the scientific literature: some sources acknowledge that the terms are often used interchangeably,1 others argue that while confabulation may be more accurate, hallucination remains more widely recognized,2 and still others explicitly advocate for adopting confabulation as the preferred term.1 3 4

Where These Terms Come From—and What They Actually Mean

But what do hallucinations and confabulation actually mean—and where do these terms come from?

Both terms have a clinical background.2 It makes sense that the field borrows from clinical language because when we trace the origins of artificial intelligence, we go back to 1956, when a summer workshop at Dartmouth College laid the foundation for what we now understand as AI. While different approaches emerged—like mathematical logic and deductive reasoning—some participants advocated drawing from biology and psychology to mimic human brain-like behavior.5

According to the ICD-11—the eleventh edition of the International Classification of Diseases and the global standard for documenting health information—hallucinations are described as “sensory perceptions of any modality occurring in the absence of the appropriate (external) stimulus. The person may or may not have insight into the unreal nature of the perception.”6

Confabulation, by contrast, is defined as “the filling of memory gaps with fabricated, distorted, or misinterpreted memories about oneself or the world, without the conscious intention to deceive.”7

Contrasting both definitions makes clear: When someone experiences hallucinations, someone might see, hear, feel, or even smell something that isn’t actually there.8 Confabulation, on the other hand, is about memory and cognition. Individuals experiencing confabulation aren’t trying to mislead anyone; their brain is simply generating plausible (but false) memories to maintain a coherent sense of reality. Confabulation is also called “honest lying” where the individual believes that the statement is true.9 This is also supported by research in psychiatry where researchers state that “confabulations are false memories within the context of retrieval, in which patients are not generally aware that they are confabulating”.10
Understanding The Phenomenon in the World of LLMs

While hallucination and confabulation have clinical definitions, in the context of AI, these terms describe how LLMs generate text that “seemingly fills gaps in the information contained in the model by plausibly sounding words”11, yet is often “nonsensical or unfaithful to the provided source content”12. Some research further distinguishes between intrinsic hallucinations, where generated text contradicts the source, and extrinsic hallucinations, where it cannot be verified against the source13.

From a technical perspective, this phenomenon is inherent to how these models operate. The generation process relies on probability: at each step, the model predicts the most likely next word based on the preceding context.14 Essentially, with each new word generated, the model assigns a probability to it, favoring words with higher probabilities. If the probability at any point drops too low, the model stops generating text. While this probabilistic approach is fundamental, it can lead to factually incorrect outputs—a consequence generally undesirable, unless the goal is creative or fictional writing.

Here’s an example of how this might look in practice: I asked ChatGPT which special event happened on May 5, 1960, related to Trader Joe’s. I know Trader Joe’s wasn’t founded until after 1967, yet ChatGPT claimed their first store opened on May 5, 1960. This kind of error is what the literature often refers to as both hallucination and confabulation.

[Insert: So a specific date was asked for. Not a good idea.]

Alternative text

Researchers in the realm of LLMs also observe that the terms confabulation and hallucination are still often used interchangeably.1 They highlight, however, that using the term confabulation instead of hallucination has certain advantages.

While hallucination remains the dominant term in both media and technical literature, the use of confabulation is gaining traction—particularly in research areas focused on explainability, cognitive alignment, and ethical framing. Several studies argue that confabulation more accurately describes how LLMs generate plausible-sounding yet incorrect responses — without implying sensory experience or intent.1 3 4 For instance, the model isn’t perceiving something nonexistent; rather, it is fabricating information.3 8 Moreover, confabulation is a more neutral term that respects sensitivities around mental health. Neurological studies support this by showing that everyday memory reconstruction involves confabulation in both healthy and impaired individuals, which helps reduce the stigma attached to the term.

However, others note that despite its conceptual accuracy, confabulation remains less recognized than hallucination in both public and professional discourse.2 Consequently, some researchers adopt a hybrid approach, using confabulation to denote a subset of hallucinations—specifically, those that are fluent, arbitrary, and confidently incorrect.12

Interestingly, research shows that “presenting patients with contradictory information may further perpetuate confabulation in an attempt to explain their account.”9 This behavior can also be observed in LLMs when given contradictory inputs. Although these models are improving in handling such contradictions, this confabulatory behavior may still occur:

Your 2nd - https://cosimameyer.com/post/are-ai-models-hallucinating-or-just-confabulating/

So earlier an AI told me they were not LLMs (apparently wrong), but now am told they are LLMs just a LLM of a different kind than others. That is a live, lol, example of them giving incorrect information. Like a person not understanding something fully, but believing the info they know is the answer.

"There's much, much ore on the subject."

There sure is, one earlier we saw here:

Ethical Considerations in AI Chatbot Design
[...]
Why Do LLMs Hallucinate?

Large language models (LLMs) are incredibly powerful, but they aren’t perfect. One of their biggest challenges is hallucination—when an AI confidently generates false or misleading information. Unlike human errors, LLM hallucinations don’t always come from a lack of knowledge. Instead, they result from how these models are built and trained.

At their core, LLMs predict the most likely sequence of words based on patterns in their training data. They don’t have an inherent sense of truth. This means they can produce responses that sound convincing but lack factual accuracy. Several key factors contribute to hallucinations:

https://investorshub.advfn.com/boards/read_msg.aspx?message_id=176228163

So as in chat with humans be aware mistakes are made, and most of all be careful what you ask for.

LOL every day in every way...