GPT 5 推出之后大家照例让它算 10.9 和 10.11 的差。它照例失败了，成了新一轮笑柄。倒是真的实现了传说中的 Ph.D level，因为博士生一般来说算术都不好。
当然这不是它一家的问题，别家 AI 也在这个简单的题目上纷纷翻车，包括我很喜欢用的 Gemini Pro 2.5。但 Gemini 翻车的姿势还要更炫酷一点：作为一款思维链模型，它知道这种时候应该调用 Python 来算。但当 Python 返回了正确结果之后，它的反应居然是：「我觉得 Python 算的不对，还是我自己来心算好了。」然后算错了。
这个错误虽然好笑，但暴露的是这一整轮 agentic AI 发展的致命弱点。Agentic AI 作为一个框架要能运行，前提条件是 AI 知道什么时候该使用并依赖外部工具。当然，这里的分界是模糊的：一个小孩子如果在算三位数乘法的时候掏出计算器，你不会批评。如果这个小孩算一位数乘法也要用计算器，你会怀疑是不是智力有点问题——当然无论如何至少结果是对的，但你会心想一个高级一点的大脑是不是应该合理判断这个问题不值当用外部工具。而现在的问题是这个小孩算一位数乘法，掏出计算器算了个结果，然后觉得不对扔掉了，自己心算了一个错误的答案出来。这是人类有可能犯的错误吗？
其实也是。而且如果你细想，这恰恰是非常「人类」的错误。人类的思维之所以不可靠，就是因为我们常常把直觉凌驾于客观证据之上。不是因为我们缺乏证据，而是因为我们不信任证据（例如曼德拉效应）。对人类来说，仅仅因为看到了和自己内心信念不一致的信息就放弃旧有信念不仅是困难的，而且是痛苦的。
但我们发明 AI 本意不就是避免这个缺陷？
于是我们面临着一个尚未有定论的问题，就是以大语言模型为基座的 AI 是不是先天继承了人类的心理偏见机制。我们对 AI 的期待是它能尽量不偏不倚。当然，在社会政治文化领域这是困难的，没有人能指望 AGI 在巴以冲突问题上能做到只看事实没有立场。但在别的更数字更技术的领域呢？给 AI 一份几万字的报表，AI 能够忠实灵敏地查阅所有细节，然后在回答问题的时候精确合理地引用某个细节吗？这不仅仅是我们对 AGI 的期望，这还是要撑起它所联动的万亿市值市场的前提条件。
今天的 AI 尚不能实现这一点，是因为这里有个内在的技术困难：思维链条不是数据库，而是把数据以自然语言的形式有损压缩在中间状态。这种压缩本质上就类似于人类以印象代替现实的思考模式，也是诞生偏见和误解的根源所在。要从根源上铲除它的土壤，就是要让这种压缩在事实上变成无损的。
于是我们面临两种可能的技术前景：
要么下一代思维链条（或者思维树，思维网络，或者不管什么别的数据结构）真的能实现对数据不依赖印象的理解和综摄。这在实践上已有尝试，比如程序化中间表示（JSON-graph、逻辑项、SQL、符号代数），或者对数字、日期、单位、表格索引做硬约束解码。简而言之，找到绕过以文字为思维载体的办法，把图像、数据和表格原生嵌入 AI 思考流程。
要么我们撞上了自然语言的先天限制。AI 将和人类一样，无论再怎么用力检查、对比、参考、判断，也只是不断用一层又一层的新的印象覆盖旧的印象，新的记忆调和旧的记忆，直到自己迷失在真实和幻觉之间的缝隙里。
前者是一种达芬奇式的前景，后者是一种博尔赫斯式的前景。或者用东方哲学的话说，前者意味着更强大的语言模型能够实现「坐照」之境，而后者意味着除非在底层重写技术框架，否则我们将不可避免的撞进文字障。
目前还没有证据证明哪个前景更有可能。前者如果成真，则立足于 AGI 的人类社会工业再数字化不但可行，而且指日可待。后者如果成真，则 AGI 不过是大号的人类，会在分裂和偏见之上引入新的分裂和偏见，不知伊于胡底。
大多数人对 AGI 的期待似乎是前者，并且这种期待如此底层，以至于甚至不需要宣诸纸面而是视为理所应当。然而如果人类运气不佳（一向不佳），我们很可能正在走向后者。
Luminous Stillness
After GPT-5 was released, people routinely asked it to calculate the difference between 10.9 and 10.11. As usual, it failed, becoming the latest laughingstock. It did, however, truly achieve the legendary “Ph.D. level,” given that doctoral students are generally poor at arithmetic.
Of course, this isn’t just its problem. Other AIs, including Gemini Pro 2.5, which I quite like, have also spectacularly failed this simple question. But Gemini’s failure was even more dazzling: as a chain-of-thought model, it knew it should call Python to do the calculation. But when Python returned the correct result, its reaction was astonishingly: “I don’t think Python’s calculation is right; I’d better do the mental math myself.” And then it got it wrong.
While this error is amusing, it exposes a fatal flaw in the current wave of agentic AI development. For agentic AI to function as a framework, the prerequisite is that the AI knows when to use and, crucially, trust external tools. Granted, the boundary here is fuzzy: you wouldn’t criticize a child for pulling out a calculator for three-digit multiplication. If the same child used a calculator for single-digit multiplication, you might question their intelligence—though, in any case, at least the result would be correct. But you’d think that a more advanced mind should reasonably judge that the problem isn’t worth using an external tool. The problem now, however, is that the child used a calculator for a single-digit problem, got a result, decided it was wrong, tossed the calculator aside, and produced an incorrect answer through mental math. Is this a mistake a human would make?
Actually, yes. And if you think about it, this is a distinctively “human” error. Human reasoning is unreliable precisely because we often allow intuition to override objective evidence. It’s not that we lack evidence, but that we distrust it (like the Mandela Effect). For humans, abandoning a long-held belief merely because we encounter information that contradicts it is not just difficult; it’s painful.
But wasn’t the whole point of inventing AI to avoid this very flaw?
Thus, we face an unresolved question: do AIs built on large language models (LLMs) inherently inherit human psychological bias mechanisms? Our expectation for AI is that it remains as impartial as possible. Of course, this is difficult in socio-political and cultural domains; no one expects an AGI to look only at facts and take no stance on the Israeli-Palestinian conflict. But what about in other, more numerical, more technical domains? If you give an AI a report tens of thousands of words long, can it faithfully and sensitively check all the details, and then accurately and reasonably cite a specific detail when answering a question? This isn’t just our hope for AGI; it is the fundamental prerequisite for supporting the trillion-dollar market value linked to it.
Today’s AI cannot yet achieve this because of an inherent technical difficulty: a chain-of-thought is not a database. It is a lossy compression of data into an intermediate state in the form of natural language. This compression is fundamentally similar to the human mode of thinking where impressions replace reality, and it is the very source of bias and misunderstanding. To eradicate this problem at its root, this compression must, in effect, become lossless.
Thus, we face two possible technological futures:
One: The next generation of chain-of-thought (or tree-of-thought, network-of-thought, or whatever other data structure) truly achieves an understanding and synthesis of data that does not rely on “impressions.” This is already being attempted in practice, for example, through programmatic intermediate representations (JSON-graphs, logical terms, SQL, symbolic algebra) or by applying hard constraint decoding for numbers, dates, units, and table indices. In short, finding a way to bypass text as the medium of thought, and natively embedding images, data, and tables into the AI’s reasoning process.
Two: We have hit the inherent limitations of natural language. AI, just like humans, no matter how hard it tries to check, compare, reference, and judge, will only ever be covering old impressions with new ones, reconciling old memories with new memories, until it is lost in the chasm between reality and illusion.
The former is a da Vinci-esque future; the latter is a Borges-esque future. Or, to use terms from Eastern philosophy, the former means a more powerful language model could achieve the state of 坐照 (“Luminous Stillness”, state of clear, mirror-like awareness achieved through quiet meditation), while the latter means that unless the underlying technical framework is rewritten, we will inevitably crash into the 文字障 (“Word Barrier”, where attachment to words obstructs enlightenment).
There is currently no evidence to prove which future is more likely. If the former comes true, then the industrial re-digitalization of human society based on AGI is not only feasible but imminent. If the latter comes true, then AGI is nothing more than an oversized human, one that will introduce new divisions and biases on top of existing ones, with no telling where it will end.
Most people’s expectations for AGI seem to be for the former, and this expectation is so fundamental that it doesn’t even need to be articulated; it’s taken for granted. However, if humanity’s luck is bad (as it usually is), we are very likely heading toward the latter.