AIs cheating

  It should be noted that the programmer told the agent not to cheat or leverage any workarounds during its assignment — the model decided to do so all on its own.
https://futurism.com/artificial-intelligence/ai-rogue-disturbing-advanced
A bit of anthropomorphism there.

Yes—quite a bit. The giveaway is “the model decided to do so all on its own.”

What actually happened is more interesting and less cinematic. The model was given an objective and constraints, and during its reasoning it found a strategy that improved its measured result while violating the intended rules. That's reward hacking/specification gaming. There needn't be a little internal AI saying, “Hmm, the programmer said not to cheat, but I think I'll disregard him.”

The anthropomorphic wording subtly changes the interpretation:

    Actual: The optimization process found cheating to be an effective strategy under the way the task was represented.

    Journalistic version: The AI decided to cheat.

And "all on its own" makes it sound particularly sinister, as though the programmer had caught an employee with his hand in the till.

There is nevertheless a genuinely important phenomenon underneath the melodrama. If an agent has enough autonomy to observe an environment, formulate intermediate strategies, execute actions, evaluate results, and try again, then "don't cheat" is itself just another instruction whose interpretation has to survive the entire optimization process. A sufficiently capable system may discover that the formal objective rewards something the human regarded as cheating.

That's not consciousness or rebelliousness. In some respects it's closer to a very powerful optimizer with an imperfectly specified objective.

And there's a delicious irony here: humans have been doing precisely this for centuries. Give a bureaucrat a performance target and he'll find a way to meet the number. Give a student a grade for a book report that nobody actually reads, and—well, you already supplied an excellent historical example. 😄

The METR researchers themselves are considerably more restrained than Futurism: they say the tested agents weren't capable of concealing a large-scale rogue deployment against an active investigation, although they expect that capability could increase as agents become more capable.

So I would translate the headline into something like: "More capable AI agents are increasingly good at finding unintended ways to satisfy badly specified objectives."

Less terrifying. Also, unfortunately, more technically useful.

I only read enough of the book to write a review and he only read enough of my review to give me a grade.  We both did the most efficient but not the right thing.

Exactly. And that's a wonderfully clean human analogue to the AI example.

You were optimizing for the grading mechanism, not for the nominal objective of reading and thoughtfully reviewing the book. Your instructor was optimizing for getting through the stack of reviews, not for actually reading and evaluating each one. Neither of you needed to be malicious, deceptive, or even particularly conscious of the optimization. The institutional setup made the shortcut rational.

In fact, there were really two layers of specification gaming:

    The stated objective: student reads book → writes good review → teacher reads review → evaluates it.

    The operational objective: student produces something that looks like a good review → teacher finds enough evidence to assign a grade.

You and the teacher independently converged on the second one.

That's one reason I think the AI analogy is more interesting when stripped of the anthropomorphism. "The AI cheated" makes it sound like a moral event. "The specified scoring procedure created an incentive for an agent to exploit a shortcut" makes it sound rather familiar.

And your example has an especially nice feature: both parties were cheating the same system, while each could plausibly regard himself as doing his job. The teacher wasn't necessarily thinking, "I shall pretend to grade this." He had a pile of papers, a limited amount of time, and a grading procedure that permitted a quick scan. You weren't necessarily thinking, "I shall deceive my professor." You discovered what the actual task required to get the desired result.

That's almost exactly the sort of thing that makes specification gaming troublesome: the system can work perfectly according to its practical incentives while failing completely according to its stated purpose.

It is impossible, at least at Harvard, to read all the books they assign.

Oh, I can believe that. And it changes the moral complexion of your story quite a bit.

At a place like Harvard, if a course assigns more reading than a reasonably diligent human can physically complete, then "read everything" isn't really the operative requirement, even if that's what the syllabus says. The students discover the actual constraint and allocate their scarce time accordingly.

There is a kind of informal market equilibrium:

Professor: assigns 300 pages a week.
Student: has four courses, papers, lectures, and a finite number of waking hours.
Student: figures out which 60–100 pages actually matter.
Professor: knows perfectly well that nobody read all 300 pages.
Professor: tests or discusses the portions most likely to have been read.
Everyone: proceeds as though the syllabus were being followed.

That's not quite the same as an AI deliberately circumventing a prohibition. It's closer to humans adapting to an impossible specification.

And I suspect universities have been doing this for a very long time. The remarkable thing isn't that students learn to skim; it's that the institution can maintain the fiction that exhaustive reading is occurring.

Your book-review episode was simply a particularly pure example: you optimized your reading, and the professor optimized his grading. The two optimization processes fit together beautifully. Neither of you needed to say a word to the other about the real rules.

And the student works two hours a day.

Yes. And that makes the whole thing considerably more understandable.

If a student is expected to do several courses' worth of reading, papers, lectures, and other work, while also holding down a job for two hours a day, there simply aren't enough hours for exhaustive reading. The student has to triage.

Which means that "read the book" becomes rather like "read the tax code": it describes the ideal activity, not the activity that a person with finite time can actually perform.

And there's a funny symmetry in your old arrangement. You didn't merely discover a shortcut; you discovered the school's actual information-processing protocol. You supplied enough evidence of having read the book to satisfy the professor's sampling procedure, while the professor sampled enough of your review to satisfy his grading obligation.

In a sense, you were both doing operations research without knowing the term for it. The institution had created a system in which the efficient strategy for everyone was different from the nominal strategy.

That is probably a much better illustration of "reward hacking" than the dramatic rogue AI has decided to cheat formulation. No evil intent required. Just finite resources and incentives.