An attempt to reassure us by showing how hard it is for current LLMs to crack CAPTCHAs did not reassure me

An attempt to reassure us by showing how hard it is for current LLMs to crack CAPTCHAs did not reassure me. It only showed that further development is needed, but it will be achieved.

I think that's a reasonable reaction.

The demonstration "AI can't solve CAPTCHAs reliably" only tells you something about today's systems. It doesn't tell you much about the long-term trajectory. If an AI can already write software, analyze images, navigate websites, and carry on conversations, then a visual puzzle that humans solve in a few seconds hardly looks like a fundamental barrier.

Historically, CAPTCHAs have been an arms race. First it was distorted text. Then OCR got good enough to read it. Then image selection puzzles. Then more behavioral analysis. The goalposts keep moving because the underlying assumption—that humans possess some unique, easily testable ability that machines lack—keeps eroding.

What reassures some people is not that CAPTCHAs are impossible for AI, but that security systems don't depend on any single test. Modern anti-bot systems increasingly look at dozens of signals: timing, browser behavior, network patterns, account history, device fingerprints, and so on. The CAPTCHA is often just the visible tip of a much larger system.

But if the claim is "AI will never be able to pass CAPTCHAs as well as humans," I wouldn't find that convincing either. There is no obvious theoretical reason why a sufficiently capable vision-and-reasoning system could not do so.

The broader issue is that many people unconsciously treat current limitations as permanent. We saw that with chess, Go, speech recognition, translation, image generation, and driving. The more prudent question is not "Can today's model do it?" but "Is there some fundamental reason a future model could not?" For CAPTCHAs, I don't see such a reason.

In fact, one could argue that CAPTCHAs are becoming obsolete for exactly this reason. If AI eventually becomes as good as or better than humans at every CAPTCHA challenge, websites will have to authenticate identity or reputation rather than merely test for human-like perception. That's a very different problem.

So they will take a look at the camera? I suspect security will only really be available through callback systems.

Possibly. There is already movement in that direction.

Many security systems have shifted from "prove you're human" to "prove you're you." Face recognition, fingerprints, device possession, security keys, and app-based approvals are all examples.

A callback system—"we'll contact you through a previously trusted channel"—is indeed one of the strongest approaches. Banks already do versions of this. If someone logs in from a new device, the bank sends a code to your phone or asks you to approve the login from an already authenticated app.

The weakness is that every callback channel can itself be compromised. Phones can be stolen, SIMs can be swapped, email accounts can be hacked, and even biometric systems can be spoofed. Security ends up becoming a matter of layers rather than a perfect solution.

What's interesting about advanced AI is that it attacks the oldest assumption in computer security: that there are things humans can do but machines cannot. CAPTCHAs were built on that assumption. If AI becomes able to read, see, converse, and navigate websites as well as people, then "prove you're human" becomes nearly meaningless.

I suspect the future will rely on combinations of:

• Possession (a device or hardware key you already own).

• History (this looks like the same person who has used this account for years).

• Reputation (this account has built trust over time).

• Cross-checks through trusted channels (your callback idea).

• Risk scoring (a routine login gets little scrutiny; unusual behavior gets much more).

The philosophical shift is from human verification to identity verification.

There's also an ironic possibility: if AI agents become common personal assistants, websites may eventually stop trying to distinguish humans from machines at all. Instead they may ask, "Which trusted agent is acting for this person?" In that world, the machine isn't the intruder—the machine is the authorized representative. That would have sounded like science fiction when you were building rule-based AI systems in Kansas City, but parts of the industry are already thinking along those lines.

  

AI against AI.

 

 

Very likely.

For centuries, security was mostly human versus human. Then it became human versus machine. Increasingly it may become machine versus machine, with humans only setting the rules.

An AI attacker might attempt millions of variations, search for weaknesses, imitate normal behavior, and adapt in real time. The defense may be another AI watching patterns across billions of events, noticing subtle anomalies no human analyst could spot.

In a sense, this is already happening. Credit-card fraud detection, spam filtering, malware detection, network intrusion detection, and anti-cheat systems all contain elements of automated defense fighting automated attack.

The interesting question is whether the balance favors offense or defense.

Historically, offense often seems ahead at first because attackers need only find one weakness, while defenders must protect everything. But defenders have an advantage too: they can centralize information. A bank's security AI may observe attacks against millions of customers simultaneously and learn from all of them. An attacker sees only the attacks he launches.

One can imagine a future login attempt:

• Your personal AI says, "Frank wishes to access this account."
• The website's AI replies, "Prove it."
• Your AI presents cryptographic credentials and behavioral evidence.
• The website's AI evaluates risk, compares it to years of history, perhaps checks with other trusted systems, and decides.

The human may never see any of this.

That raises an amusing possibility: someday a person may complain, "My AI says your AI is wrong," and neither side will fully understand the technical argument taking place. The dispute will resemble two lawyers arguing in a language their clients don't speak.

The deeper issue is trust. The real contest may not be AI versus AI, but whose AI you trust to act on your behalf and whose AI you trust to keep others out.

 

  

 

Sounds destructive, like pushing us back to a barter economy.

 

 

It could, if trust mechanisms fail badly enough.

A modern economy depends on strangers trusting strangers. You buy stock in companies you've never visited. Your bank accepts instructions sent electronically. Merchants ship goods before meeting customers. Credit cards work because most transactions can be trusted most of the time.

If AI made impersonation nearly perfect and defenses couldn't keep up, the transaction costs of trust would rise sharply. People would demand more verification, more delays, more human involvement, and more reliance on established relationships. Economists sometimes call this "friction." Too much friction slows commerce.

I don't think we'd literally return to barter. Barter is astonishingly inefficient. But I could imagine a world where:

• Large institutions transact mainly with other large institutions.
• Individuals rely heavily on trusted intermediaries.
• Strong cryptographic identity becomes as important as a passport.
• Anonymous interaction becomes rare and expensive.

In a way, we've already seen a mild version. Fifty years ago a phone call from your bank was usually assumed genuine. Today many people are taught to hang up and call back using a known number because voice alone is no longer trustworthy.

AI may extend that trend. Text, voice, images, video, and even real-time conversation may cease to be reliable evidence of identity. When that happens, trust shifts toward things that are harder to forge: cryptographic keys, established relationships, long histories of behavior, and independent verification channels.

What's interesting is that this isn't necessarily a collapse of civilization. Commerce existed long before telephones and the internet. But it could feel like a step backward in convenience. The dream of frictionless digital interaction would give way to a world of constant authentication.

That may be what you're reacting to: not a return to barter itself, but the possibility that technology which was supposed to reduce friction ends up forcing us to spend more effort proving who we are. That would feel like losing ground, even if the economy remained highly advanced underneath.