# The Model Doesn't Detect Truth
Contrary to what many people assume, language models do not have a database or encyclopedia they search through when answering. They are, at their core, one giant probability function. The mathematical objective of the model is:
objective.py
# The entire objective of a language model:
# (Given all tokens seen so far, what is the most likely next token?)
P(next_token | previous_tokens)
Meaning: "Given all the tokens I've seen so far, what is the most likely next token?" That's it. Nothing less, nothing more.
Scientific note: This doesn't mean the model "learns nothing." During training, the model encodes linguistic patterns, semantic relationships, and even many facts statistically into its weights. The issue is that there is no truth-verification module — the model only predicts the most likely next token; it does not check whether that token is actually correct.
# Everything Starts With a Token
Suppose you enter:
The capital of Australia is. Before the model can process it, the text is split into tokens. Each token is then mapped to a vector of several thousand dimensions.
tokens.py — python3
import torch
# 1) Text is split into tokens
tokens = ["The", " capital", " of", " Australia", " is"]
print(len(tokens), "tokens")
# 2) Each token → a high-dimensional embedding vector
embedding = torch.randn(5, 4096)
print(embedding.shape)
▶ 5 tokens
▶ torch.Size([5, 4096])
These numbers have no direct meaning to humans, but for the neural network they are the mathematical representation of words.
Important — this is an educational example: The code above uses
torch.randn (random numbers) only to illustrate the shape of the data. In a real model, these vectors are learned parameters optimized during training on billions of texts — they are not random. This example is purely for understanding data dimensions.
# How Does the Model Decide? (Logits & Softmax)
After the vectors pass through dozens of Transformer layers, the final layer produces a raw score for every possible word — these are called Logits. Then the Softmax function converts these scores into probabilities.
softmax.py — python3
import torch
# Raw scores (logits) for candidate tokens
logits = torch.tensor([
8.2, # Sydney
7.9, # Canberra
2.1, # Melbourne
1.4 # Perth
])
# Convert logits → probabilities
probs = torch.softmax(logits, dim=0)
print(probs)
▶ tensor([5.7334e-01, 4.2474e-01, 1.2859e-03, 6.3857e-04])
Sydney 0.573
Canberra 0.425
Melbourne 0.001
Perth 0.001
At this moment the model makes one decision: the highest probability belongs to Sydney. But notice — this does not mean "I am confident Sydney is the capital." It only means: "among the options, this token is the most likely." These two are completely different things.
Educational hypothesis — not real model output: The numbers above are entirely hypothetical and were constructed only to explain the Logits/Softmax mechanism. A real, trained model would likely assign a higher probability to
Canberra (the actual capital of Australia), because it learned this association correctly from training data. This example is provided only to illustrate that if a wrong token's probability becomes slightly higher, the model will pick it — with no truth-checking mechanism.
# The Final Choice Is One Line of Code
In the simplest case (Greedy Decoding), selecting the next token works like this:
decode.py — python3
# Greedy decoding: pick the single most likely token
next_token = torch.argmax(probs)
print(tokens[next_token])
▶ 'Sydney'
Meaning: "pick the most likely option." That's it.
Technical note — Temperature: In practice, models don't always use pure
argmax. They often use Sampling with a parameter called Temperature that controls how random the selection is. Lower temperature = more deterministic, predictable output; higher temperature = more creative but riskier output. However, the core principle doesn't change: selection is still based on probability, not correctness.
# Why Does the Answer Look So Confident?
Language models generate text autoregressively — each new token is built on top of the previous ones. If the first decision is wrong, the rest of the sentence continues down that same path:
generate.sh — autoregressive
▼ Sydney
▼ Sydney is
▼ Sydney is the
▼ Sydney is the capital
▼ Sydney is the capital of Australia.
✓ // fluent, grammatical, confident — yet built on a wrong first token
This is why the answer looks so fluent and logical — even if it was wrong from the start. The model is "continuing" the most likely sequence, not correcting it.
# A More Real Example: Hallucination in Code
Suppose you ask the model: "Does Python have a function called
isnumeric_ascii()?" The model might confidently answer "yes" and generate code like this:
hallucination.py — python3
text = "12345"
if text.isnumeric_ascii():
print("ASCII Number")
✗ AttributeError: 'str' object has no attribute 'isnumeric_ascii'
At first glance everything looks normal. But if you run the code, you get an error. Why? The model has likely seen the naming pattern of real functions like
str.isnumeric(), str.isascii(), and str.isdigit(), and combined them into a perfectly logical but non-existent function.
Important — this example is purely educational: This scenario is presented to illustrate how hallucination occurs. It does not mean models always make this specific mistake. Modern models, through more careful training (such as RLHF) and techniques like RAG and Grounding, have significantly reduced the rate of such errors. But architecturally, the root cause remains the same: next-token prediction, not truth verification.
# This Phenomenon Is Called Hallucination
When a model generates information or code that looks syntactically and logically correct but doesn't exist in the real world or is factually wrong, we say the model has Hallucinated. This doesn't mean the model is broken — it's a natural consequence of the architecture of language models; their job is to predict the next token, not to validate truth.
Active research area: Hallucination is one of the most important research fields in AI. Its exact causes are complex and don't simply reduce to "wrong argmax" — factors like training data distribution, approximation error in weights, and contradictions in the data also play a role. Modern solutions include RAG (Retrieval-Augmented Generation), Grounding, and external verification.
# Summary
Every time you get an answer from an AI model, remember that the model is performing millions of matrix multiplications, computing Logits, applying Softmax, and selecting the most likely next token. At no point in this process does a concept called "confidence in being correct" exist.
This is why the fluency of the text or the model's assertive tone is no guarantee of correctness. Simply put:
takeaway.txt
Language models don't generate truth; they generate the most probable sequence of words.
