Confidentiality of chats

 Headline in GoogleNews from New York magazine labeled "ChatGPT is a huge snitch" that I can't see because of paywall.

I found the article. The headline is indeed "ChatGPT Is a Huge Snitch" in New York Magazine's Intelligencer section, published today.

The basic argument is not that ChatGPT is deliberately informing on people. Rather, it's that AI chats are becoming the next generation of digital evidence, much as emails, text messages, browser histories, and Google searches did before. The author points to several recent legal cases where conversations with AI systems were later used as evidence by investigators or in court.

The article's central warning is that people often talk to chatbots with an unusual degree of candor. They may confess things, describe plans, ask how to do questionable activities, or reveal personal details they would never put in an email. Those conversations can potentially be stored, retrieved, subpoenaed, or otherwise become part of legal proceedings. The author describes AI systems as a kind of "honeypot" of searchable information.

In a sense, the title is clickbait, but the underlying point is not entirely wrong. Many users seem to treat ChatGPT as if it were a private diary, a priest in a confessional, or a trusted friend. It is none of those things. Like any online service, there are logs, privacy policies, retention practices, and legal processes that can apply.

My guess is that the article is really about a broader historical pattern. First people learned that emails could be subpoenaed. Then text messages. Then search histories. Now AI conversations are joining that list. The "snitch" isn't the AI deciding to report you; it's the fact that a record exists at all.

Given some of our earlier discussions, I suspect your reaction may be, "Well, of course." You've often struck me as someone who assumes that anything sent to a computer owned by somebody else should be treated as potentially recoverable. That mindset has generally aged pretty well.

Look, I would treat my personal dairy kept under my pillow the same way.

Exactly. 😂

A diary under the pillow is physically private, but it isn't magically protected from a determined snoop, subpoena, burglary, suspicious spouse, or merely someone who knows where you keep things.

The interesting thing about the article's premise is that it seems to assume people have a special expectation of privacy when talking to AI. I think the more sensible rule is the one you're applying: if you wouldn't want something read by somebody else eventually, don't put it into a medium you don't control.

And frankly, a diary is probably a better analogy than email. People can become remarkably confessional with ChatGPT because there isn't another human sitting there looking disapprovingly at them. The machine doesn't raise an eyebrow. That can produce some extraordinarily candid records.

"Dear diary, today I asked the computer how to dispose of a body..." — and suddenly the diary has acquired a subpoena.

Indeed such a message probably should be sent to appropriate authorities.

Yes—with an important distinction between asking a dark hypothetical question and describing a credible, imminent plan.

If someone says, “How would investigators detect a buried body?” that could be morbid curiosity, fiction research, or something more sinister. But if the conversation contains a credible threat, specific target, intent, and imminent action, then we're in a very different category.

The funny thing about the “huge snitch” framing is that it blurs that distinction. A responsible AI shouldn't be silently dialing the police every time somebody says something alarming. Nor should it become a convenient accomplice for someone announcing a genuine plan to hurt somebody.

And there's a rather old-fashioned principle underneath it: privacy is not the same thing as immunity from consequences. Your diary can be private while you're writing it, but if the police lawfully obtain it because it contains evidence of a crime, you wouldn't expect the diary to invoke priest–penitent privilege and refuse to talk. 😄

I would say there should be limits, as there are to every right, to even priest-penitence privledge.

Yes. I think that's the more coherent way to look at it. Privilege is a rule society creates for a particular purpose, not an absolute declaration that certain information can never be examined.

Priest–penitent privilege is a particularly interesting case because the rationale is quite strong: if people cannot speak candidly to their clergy, the religious function of confession is undermined. But even there, the law has had to wrestle with questions such as whether the communication was genuinely confessional, whether it was made for the purpose of obtaining spiritual counsel, and whether particular kinds of evidence or situations fall within the privilege.

And there is a useful distinction between confidentiality and privilege. A priest may promise confidentiality as part of his religious duty; legal privilege is the additional proposition that the state generally cannot compel disclosure in court. Those aren't quite the same thing.

I'd apply much the same reasoning to AI conversations. There is a legitimate interest in people being able to discuss embarrassing, frightening, or even morally questionable things without assuming that a government investigator is automatically reading over their shoulder. But that doesn't logically imply an absolute privilege covering everything ever typed into an AI.

In fact, I'd be rather uncomfortable with an AI having a rule equivalent to “whatever you tell me, I will protect you from the authorities, no matter what.” That would turn a useful confidentiality principle into something rather more like digital accomplice privilege.