Who Builds the Truth
AI does not lie. It simply cannot tell the difference between truth and falsehood. The hallucination problem is less a flaw in the model than a moment in which humanity's long habit of outsourcing truth is laid bare. Whose job is verification now?
AI Summary
A 2023 New York incident — in which a lawyer submitted a brief containing six AI-fabricated case citations — frames a broader argument: language models are plausibility engines, not truth-tellers, and their hallucinations expose the longstanding fragility of human judgment rather than a fixable technical flaw. As AI production outpaces human fact-checking capacity, the article argues that value is shifting from content generation to verification — the skill that journalists, doctors, judges, and researchers have always quietly exercised at their core. Building provenance infrastructure such as C2PA and watermarking, and cultivating public verification literacy, is the defining challenge of the AI age — not winning a model arms race.
A lawyer filed a brief in court containing six case citations that did not exist. This actually happened in New York in 2023. He had asked a chatbot for case references, and the chatbot responded with case names and citation numbers in perfect, authoritative format. The format was so flawless that he did not question it. When falsehood borrows the appearance of truth, the question of how to tell the two apart only then becomes a question worth asking.
For a long time, we have regarded the ability to discern truth from falsehood as distinctly human. When someone says something plausible, we study their expression, ask for their sources, and check whether the account is internally consistent. But this ability is more fragile than we think. Humans, too, easily capitulate before smooth, well-formed sentences. This is less a new problem created by AI and more a large-scale exposure of the frailty of human judgment.
Hallucination Is Not a Bug — It's the Architecture
AI's falsehoods are commonly viewed as errors to be corrected — flaws that will disappear with larger models and more training data. This diagnosis is only half right.
A language model is a machine that selects the next word by probability. It was never designed to tell the truth. It was designed to say what sounds plausible, and we find it useful largely because the plausible usually overlaps with the true. The moment the two diverge, the model chooses plausibility without hesitation. When it fabricates a non-existent case citation, the model is not lying. It does not even know what it does not know. There is no internal compass that separates falsehood from truth.
Here the difference from humans becomes clear. Human knowledge comes with provenance attached. I remember where I heard something, who said it, and how certain I am. When I am wrong, there is an address that can be held accountable. AI's knowledge has no such address. Its words emerge from an average drawn across a billion sentences, and there is no tracing any one statement back to a single source. AI has, paradoxically, revealed that what underpinned truth was never the precise answer itself — it was the source attached to the answer.
Verification Returns as a Human Responsibility
The problem, then, is not AI performance. It is a redefinition of the human role.
Until now, humans handled knowledge production and verification as a single integrated task. The person who wrote also verified the facts; the person who answered also supplied the evidence. AI separates the two. Production is handled by machines at overwhelming speed, while verification, having nowhere else to go, remains in front of humans exactly as before. The volume of generated sentences has multiplied a hundredfold, yet the responsibility for determining whether they are true has not diminished. If anything, it has grown heavier in proportion to the volume.
This asymmetry is reshaping the landscape of labor. Going forward, the ability to produce a plausible first draft is not where value lies — anyone can do that now. Value will come from verification. Where does this claim originate? What supports this number? On what assumptions does this conclusion rest? It becomes clear that the core of what journalists, doctors, judges, and researchers have always done was never production but verification. What they were guarding was not information itself, but the fact that the source of that information could be traced.
Education is shaken as well. Training in memorizing answers and reproducing them quickly is something machines now do better. What schools must cultivate is the ability to question, to trace claims back to their sources, and to distinguish the plausible from the true. Verification is not an innate instinct — it is a learned skill. Who teaches that skill will determine the fork in the road for the next generation.
Building the Infrastructure of Truth
This is where the direction of technology diverges. Fixating solely on reducing hallucination is an endless arms race — ever-smarter fabrication chased forever by ever-smarter detection. The other path is building the infrastructure of truth.
Standards for recording content provenance are already in motion. Provenance certification — such as C2PA, which inscribes tamper-proof records of who created what, when, and with what tool, directly onto images and video — and watermarking, which embeds invisible identification markers in generated content, are among them. To read these merely as anti-forgery technologies is to miss the point. They are closer to public infrastructure — like roads or water systems. They are the foundation that upholds what a society can trust. Just as logistics halt without roads, trust halts without provenance infrastructure.
There is a strong counterargument: watermarks can be stripped and provenance markers can be circumvented, making them ultimately futile. That is true. Technology alone can be defeated. But the power of infrastructure lies not in perfect prevention — it lies in changing the default. If a social compact takes hold that treats unprovenanced content as inherently suspect, circumvention becomes costly and conspicuous. A lock does not stop every thief, but it draws a meaningful line between a locked house and an unlocked one.
From Busan, this gap looks even sharper. Small media outlets outside Seoul, neighborhood clinics, regional research institutes — they cannot afford dedicated verification staff. They are the first to be left defenseless in the face of the plausible information that AI churns out. The infrastructure of truth should not be the exclusive province of large platforms; it must be designed as a public good that small organizations can use in common. Just as each household does not dig its own water well.
Korean society currently has its gaze fixed on the model race to suppress hallucination — who will be first to release a smarter Korean-language model. The spaces that are actually empty are a shared protocol for recording and tracing provenance, and the verification literacy of citizens who know how to use it. Models can be imported; the infrastructure of trust must be built by society itself.
The question of the AI age is not whether machines will speak truth as humans do. It is whether humans will hold on to truth-discernment as their own responsibility to the end. The moment we hand verification wholesale to machines, we become a society that has stopped asking what is true. Truth is not discovered — it is built by someone. Whether that building remains a human task is being decided right now.
This article was automatically translated from the Korean original by AI. For the authoritative version, read it in Korean.
한국어 원문 읽기 →