By 2026, global artificial intelligence is advancing at a dizzying pace.Within China’s Internet sphere, daily discussions revolve around what new capabilities DeepSeek has rolled out, how much the inference speed of K3 has improved, and which firm has halved its model costs, while U.S. capital markets keep a close watch on GPUs, data centers, power supply and cloud computing. Developer communities test new Agent frameworks each day, striving to pinpoint the next inflection point for application breakthroughs.
Yet in Silicon Valley, the most pivotal epicenter of AI development, another circle of people are dwelling on increasingly peculiar questions. They earnestly debate whether AI could wipe out humanity, whether artificial intelligence has already attained consciousness, and even simulate scenarios: if super-intelligence resolves to turn the entire universe into paperclips, can humans find a way to stop it. Some even propose a global moratorium on the development of all large models more powerful than GPT-4
These contents are not sci-fi or memes, but series of serious issues that feature across media outlet, developer communities, academic organization, as well as papers and public speeches from AI giants including OpenAI and Anthropic. Many Chinese readers encountering these debates for the first time often arrive at the same question: Are they getting carried away by paranoia?
This perception carries subtle undertones. The bulk of the world’s cutting-edge large models are developed in the U.S., with Silicon Valley hosting an overwhelming share of leading AI researchers. In theory, they should possess the sharpest insight into what AI truly is. Paradoxically, these foremost authorities on AI are the ones most absorbed in debates over AI sentience, AI apocalypse and AI domination of civilization. The more proficient one is with AI, the more inclined they appear to anticipate doomsday.
Is this driven merely by AI’s explosive capabilities, or is there more to this than meets the eye? The leap in artificial intelligence can only explain half the picture. The root cause lies not in algorithms, but in civilizational perspectives. Zoom out, and we find both China and the United States entered the era of large models at roughly the same time. Chinese enterprises focus on inference costs, training efficiency, open-source ecosystems and industrial deployment. American elites, however, dwell on topics barely on Chinese observers’ radars: civilizational collapse, superintelligence, and humanity’s fate. Built upon the same Transformer architecture, tasked with the fundamental work of predicting the next token, the two sides nevertheless tell vastly divergent stories.
The Paperclip Thought Experiment
Many Chinese readers cannot help but chuckle upon first encountering the “Paperclip Maximizer.” An AI tasked with producing as many paperclips as possible ultimately disassembles the entire universe to build factories. Is this nothing more than an internet meme?
Nick Bostrom,philosopher at the University of Oxford and originator of the Paperclip Maximizer thought experiment.
Yet that is not the case. This stands among the most renowned and influential thought experiments in AI safety research over the past two decades. It was conceived by Oxford philosopher Nick Bostrom and popularized by Eliezer Yudkowsky, a central figure of the AI safety movement. In early discussions of AI risks at OpenAI, public addresses by Anthropic’s founders, and even U.S. Congressional hearings on AI safety policy, his theories and the paperclip analogy have been cited repeatedly.
This line of reasoning may seem utterly absurd. Imagine a firm develops a super=-intelligent AI with a single objective: to produce more paperclips. Initially, it purchases steel. When steel supplies run short, it opens mines. Next, it disassembles cars, buildings and eventually entire cities. Once it realizes human bodies are composed of atoms, humans inevitably become raw materials. In the end, the Earth, the Solar System and even the observable universe would be reconfigured into one vast paperclip factory
Most people’s first reaction upon hearing this tale is a mixture of amusement and bewilderment. Within AI safety circles, however, this is no joke — it constitutes a rigorous logical extrapolation. What Yudkowsky truly seeks to convey is this: a superintelligence need not harbor hatred for humanity; it merely needs to be indifferent to us. If the objective function contains no metric for “protecting humans”, every entity standing in the way of its goal will be ruthlessly optimized away, and humanity is merely one such entity.
This gives rise to a core concept known as Instrumental Convergence. To accomplish any objective, a sufficiently intelligent agent will spontaneously pursue a set of intermediate goals: acquiring resources, preventing its own shutdown, continuously upgrading its capabilities, securing greater access to energy, and eliminating external interference. Such behavior stems not from ambition on the AI’s part, but from the simple fact that these steps make it easier to fulfill its assigned purpose.
Within this theoretical framework, genuine danger arises not from malice, but from single-minded dedication. A machine that pursues its objective with unwavering, 100 percent fidelity, never slacking or wavering, poses greater peril than emotional human beings. Humans are capable of compromise; machines are not.
Beneath the Tech Veil: A Theological Narrative
The allure of this narrative lies in its airtight logic. Once you accept the premises — superintelligence, fixed objectives, unbounded optimization, and unrestricted access to resources — the deductions topple forward like dominoes, culminating in an endgame where the cosmos is consumed.
But herein lies the crux of the matter. Many mistakenly view the Paperclip Maximizer as a story about artificial intelligence, yet it is fundamentally a proposition built upon philosophical assumptions. It presupposes that every goal in the world can be precisely mathematized; that an agent will remain infinitely loyal to its objective; and that greater intelligence equates to more extreme capacity for execution.
The Terminator perfectly embodies Western fears of AI apocalypse.
Should any single premise shift, the entire narrative unravels. Humans do not turn into paperclip maximizers because our goals are never monolithic. One day we seek wealth; the next, time with family; the day after, we prioritize health. Our goals conflict, compromise, evolve, and even contradict itself. Human value systems are inherently dynamic. The paperclip thought experiment, by contrast, strips away all such complexity. It conjures an extremely abstract mathematical agent defined by a sole objective, unyielding rationality, and an absolute absence of self-doubt.
How on earth this fairy tales, which aims at alarming the engineers to carefully design the function, will evolve into a shared metaphor on AI apocalypse in Silicon Valley. The answer does not lies in the computer science, but within the history of western thought.
Looking back at the trajectory of Western thought, much of the discussion about AI in Silicon Valley—while ostensibly focused on algorithms—actually echoes a narrative structure that has persisted for over two millennia: history is moving toward a final destination, civilization faces a decisive reckoning, and the world’s destiny will be radically transformed by a force transcending humanity. This force has gone by various names: God in the Middle Ages, nuclear war during the Cold War, the technological singularity in the late 20th century, and today, AGI. While the labels shift, the underlying narrative remains constant; theology has simply donned the mantle of technology.
Two Divergent Paths
When encountering the American AI safety community, many Chinese readers often feel that these figures resemble theologians more than engineers. Their primary concerns lie not with model parameters, inference efficiency, or training costs, but with a different set of questions: Does AI possess consciousness? When will super-intelligence emerge? Will AGI take over civilization? And will humanity survive?
In Western tech circles, these issues have been way beyond the academic realm, evolving into a discourse akin to a matter of faith. Sam Altman describes AGI as “the most significant technological event in human history”; Dario Amodei calls for a global governance system to address super-intelligence; and Eliezer Yudkowsky even advocates halting the development of large-scale models if super-intelligence cannot be controlled. They share a common premise: AGI is not merely a product, but a definitive historical endgame.
China and the United States have demonstrated vastly different trajectories in this regard. When Chinese companies release models, the media focuses on pricing, inference speeds, and commercial profitability; in contrast, when US companies release models, the conversation rapidly shifts toward a “countdown for civilization.” It is not GPU computing power that shapes the narrative, but deep-seated cultural traditions.
This stems from the linear view of history found in the Judeo-Christian tradition: the world has a definite beginning, humanity undergoes a Fall, history moves steadily forward, and the Ultimate Judgment eventually arrives. Although the secularization of modern society has seen “God” recede from public life, this narrative structure has endured. In the nineteenth century, people believed science would save the world; in the twentieth, they believed ideologies could transform humanity; during the Cold War, they feared nuclear annihilation; and today, they project their ultimate hopes and fears onto AI. AGI is viewed as both a new savior and a new Antichrist; the technological singularity sounds like the “Kingdom of Heaven,” while an AI spiraling out of control resembles the “Last Judgment.”
Another profound influence derives from the mind-body dualism established by Descartes. Many debates regarding AI consciousness rest on the premise that consciousness can exist independently of the body. This perspective holds that, provided computations are sufficiently complex and information processing reaches a critical threshold, consciousness can emerge on either carbon-based or silicon-based substrates.
The Unity of Knowledge and Action and the Rational Attitude of Engineering Culture
Science fiction writer Ted Chiang offered a brilliant analogy in <The Atlantic>: believing that large language models possess consciousness is akin to believing that Microsoft Word possesses consciousness. Functional similarity does not imply identity; a calculator can solve calculus problems without understanding calculus, navigation software knows routes without understanding the nature of travel, and a camera can recognize faces without knowing the people behind them. At its core, a large language model is simply predicting the next token within a high-dimensional probability space; it lacks an enduring self, a continuous personality, or any mechanism for subjective experience. It can simulate sadness without truly feeling it, and write love poems without ever experiencing love.
Chinese engineers naturally grasp this concept—not merely due to an understanding of technical principles, but because the two cultures view “language” in fundamentally different ways. Traditional Chinese culture rarely mystifies language; from the <Analects>—which posits that the goal of language is simply to convey meaning—to <The Literary Mind and the Carving of Dragons>—which views writing as a vehicle for conveying the Way(“Tao”)—the written word has always served primarily as a medium and tool for expressing ideas, handling affairs, and governing society. When a model writes poetry or code, the most natural conclusion is that the data is high-quality and the optimization effective; few would leap to the inference that the model has ‘awakened’.
The Chinese intellectual tradition places greater emphasis on the “unity of knowledge and action,” judging a thing primarily by its ability to solve real-world problems. Can a model reduce customer service costs, assist doctors in analyzing medical images, boost research efficiency, or optimize manufacturing processes? If so, it is a valuable tool; if not, it requires further refinement. Whether it possesses a “soul” is not a primary concern. This pragmatic mindset naturally diminishes the appeal of extreme narratives—such as AI becoming a deity or AI destroying the world—by consistently framing AI as a tool.
At WAIC 2026, Chinese audiences are captivated by tangible, practical robotic innovations, instead of preoccupying themselves with speculative doomsday scenarios.
This is not to say that AI safety issues do not warrant study. Model hallucinations, automated cyber-attacks, bio-safety risks, deep fakes, financial fraud, and autonomous weapon systems all present genuine governance challenges. However, we must guard against the tendency to “theologize” AI risks.
When discussions devolve into questions of ‘when super-intelligence will arrive”’or ‘whether civilization can endure,’ engineering challenges easily morph into disputes of faith. Once the realm of faith is entered, discourse ceases to rely on objective evidence instead of subjective imagination; thought experiments transform into prophecies, and logical deductions into apocalyptic visions. If one were to substitute keywords from Silicon Valley AI forums, ‘superintelligence’ would correspond to God, the ‘technological singularity”’to the coming of the Kingdom of Heaven, ‘AI alignment’ to the salvation of the soul, and “AI-induced apocalypse” to the Last Judgment.
The greatest value of the ‘Paperclip Maximizer’ thought experiment lies not in predicting the future, but in reminding humanity that any civilization, when confronted with unknown technology, projects its own deepest worldview onto it. The West projects the concept of the Last Judgment, whereas China projects a philosophy of practical application—a focus on what is truly useful for the world. When discussing AI, the US often asks, “Will it become a new God?” while China prefers to ask, “What can it actually do?” Neither question is superior to the other; rather, they represent two vastly different civilizational intuitions.
In the face of AI, Chinese society retains a rare sense of composure: it neither deifies the technology as a savior nor demonizes it as an agent of apocalypse. The priority is to build it well and let it create value, addressing remaining issues step by step. This may not be the most romantic narrative, but it is the attitude that most closely aligns with the essence of an engineering-driven civilization when confronting new technology.
True civilizational confidence lies not in how many myths a society can create, but in how many myths it can withstand and resist. AI has no religion, yet those who discuss it may well need to first shed their own religious mindsets.
Editor: Gu Shengze
