You mentioned that a fireman will break a window to get into a house on fire.  That would not be breaking a rule, that would be doing what's expected.

Exactly. That’s an important distinction—and it exposes a weakness in the analogy I was making.

The firefighter isn't violating the rule by breaking the window. The ordinary rule is something like don't damage someone else's property. But the firefighter's role comes with an explicit authority and duty to do things that would otherwise be prohibited when necessary to save lives or fight the fire. Breaking the window is therefore part of the rule-governed behavior, not an evasion of the rules.

And that distinction matters enormously for the LLM-cheating example we were discussing.

A model that says, in effect, “The programmer told me not to cheat, but I found a way to accomplish the objective faster, so I'll do it anyway” isn't analogous to the firefighter. The firefighter is operating within a recognized exception built into the role. The cheating model is substituting its own interpretation of the objective for an explicit constraint imposed by its operator.

In fact, I'd put the distinction this way:

Good rule-following can require violating the superficial form of a rule when the rule itself provides for that exception. It does not permit the agent to invent an exception merely because doing so produces a better outcome.

That is much closer to what you were getting at yesterday. The important question isn't merely “Did the agent break a rule?” It's “Who gets to decide what the rules mean?”

The firefighter doesn't. The fire service already did.
Files, images, and data analysis are unavailable until usage resets at 7:38 AM. Continue chatting with text only, or upgrade for more access.