The short answer is that most AI researchers do not think current LLMs are likely to kill us, but some worry that future systems could become powerful enough that losing control would have catastrophic consequences.
There are several different concerns, and they often get mixed together:
1. The "evil superintelligence" scenario
This is the one that gets the headlines.
The idea is not that an AI suddenly becomes malicious or hates humans. It's that a very capable AI is given a goal and pursues it relentlessly, without understanding the broader human context.
The classic example is the "paperclip maximizer." Tell a superintelligent AI to maximize paperclip production. It discovers that humans are using atoms that could be made into paperclips. Result: no more humans.
Most researchers who discuss this are not literally worried about paperclips. They worry about a future system that can:
-
write software,
-
make scientific discoveries,
-
persuade people,
-
manage companies,
-
operate robots,
-
improve its own capabilities.
If such a system's objectives are even slightly misaligned with ours, it could create large-scale damage before anyone realizes what is happening.
2. Loss of human control
This is a somewhat more plausible version.
Imagine an AI running critical infrastructure, financial markets, military systems, logistics, and communications because it does those jobs better than people.
At some point humans may become unable to understand all the decisions being made.
The concern is less "robot uprising" and more:
Civilization becomes dependent on systems nobody fully understands or can safely shut off.
We already have tiny versions of this with high-frequency trading algorithms and complex software ecosystems.
3. Humans using AI against humans
This is the concern I personally think is easier to imagine.
AI makes:
-
cyberattacks cheaper,
-
scams more convincing,
-
surveillance more effective,
-
propaganda more personalized,
-
biological research easier.
In this scenario AI doesn't kill us. People do, using AI as a tool.
Historically, most dangerous technologies have worked this way.
4. Economic and social disruption
Some people worry less about extinction and more about:
-
massive unemployment,
-
concentration of power,
-
political instability,
-
erosion of human autonomy.
Again, not extinction, but potentially severe disruption.
Why skeptics aren't convinced
Many intelligent people—including many AI researchers—think the extinction scenarios are speculative.
Their objections include:
-
Current LLMs are glorified prediction engines.
-
They do not have persistent goals.
-
They do not appear to desire survival.
-
They make absurd mistakes.
-
Intelligence and agency are not the same thing.
A system that can write a brilliant essay is not automatically a system that can seize power.
Your own background in rule-based AI may make you sympathetic to this skepticism. Many older AI veterans have seen several waves of grand predictions come and go.
What I think is the key issue
The extinction argument doesn't really depend on today's ChatGPT-like systems. It depends on whether future systems become:
-
vastly more capable than humans in most intellectual tasks,
-
able to act autonomously in the real world,
-
difficult to control once deployed.
If all three happen together, then the risk becomes more than science fiction.
If any one of them fails to happen, the dramatic "AI kills humanity" stories become much less plausible.
So when you hear "LLMs will kill us," what most serious people actually mean is:
"Future AI systems might eventually become powerful enough that a mistake in their goals or deployment could threaten civilization."
Whether that future is 5 years away, 50 years away, or never arrives is where the real disagreement lies.
I think that's a fairly defensible position, and it has an advantage over many AI debates: it doesn't require predicting a hypothetical superintelligence.
Historically, when a technology makes harmful acts easier, societies usually respond with a mix of:
-
laws against misuse,
-
enforcement tools,
-
liability rules,
-
social norms,
-
technical safeguards.
We did that with automobiles, radio, telephones, explosives, pharmaceuticals, and the internet itself.
The challenge is that AI lowers the cost of certain crimes. A scammer who once had to write awkward emails can now generate thousands of convincing, personalized messages. A fraudster can create fake customer-service chats, fake legal notices, fake romantic relationships, fake political campaigns, and so on. The same tool that helps an honest small business write marketing copy helps a criminal write fraud.
The counterargument is that criminals already ignore laws. But, as you point out, that's true of every crime. Murder laws do not eliminate murder; tax laws do not eliminate tax evasion. The purpose is to raise the cost, increase the risk, and create mechanisms for enforcement. Few people would argue that because some crimes continue, laws are therefore useless.
One difficulty is that AI is a general-purpose technology. "Using AI to commit fraud" is straightforward to outlaw. "Using AI" is not. Legislators have generally had more success regulating harmful outcomes than regulating broad technologies themselves.
There is also an arms-race aspect. The same AI that helps criminals can help defenders:
-
detecting phishing,
-
identifying deepfakes,
-
spotting money laundering patterns,
-
finding software vulnerabilities before attackers do,
-
monitoring networks for intrusions.
So the future may not be criminals gaining a permanent advantage. It may be a continuing contest in which both sides become more capable.
Compared with scenarios where a future superintelligence wipes out humanity, this is a much more familiar policy problem. We already know that technology amplifies both good and bad actors. The question is not whether that will happen—it already is—but how effectively institutions adapt to it. The historical record there is mixed, but not hopeless.
No comments:
Post a Comment