Yes, that seems to be a fair reading of it, and very much in continuity with modern Catholic social teaching rather than science-fiction anxiety.
What Pope Leo appears to be doing is consciously echoing the older social encyclical tradition, especially Rerum Novarum, which addressed the upheaval of industrial capitalism. The analogy is not “machines will enslave humanity” so much as “new productive power tends to concentrate wealth and power unless moral and political institutions intervene.” AI is treated as the next industrial revolution.
So the emphasis is on familiar Catholic themes: human dignity, the dignity of work, the danger of reducing persons to economic units, obligations toward the vulnerable, and suspicion of concentrations of unaccountable power—whether in states or giant private actors. Leo repeatedly worries about AI deepening inequality and leaving decisions affecting millions in the hands of a few companies or technocratic elites.
That differs quite sharply from the public conversation, which often veers toward “killer AI,” consciousness, or existential takeover scenarios. The encyclical does mention risks—military use, manipulation, surveillance, erosion of truth, displacement of labor—but it frames them as moral and political failures of humans rather than autonomous-machine rebellion. AI is treated as powerful, but not agentic in the theological sense: a tool without conscience or moral responsibility. Humans remain morally accountable.
There is a distinctly old-fashioned Catholic instinct here: technology itself is not condemned, but neither is it morally neutral simply because it is efficient or profitable. One hears an echo of the Church’s reaction to industrialization: “fine, but who benefits, who is harmed, and what happens to the human person?” That sounds close to your “spread the benefits and power around” summary.
It is also interesting that Leo seems less alarmed about AI becoming a rival intelligence than about humans using AI to become worse versions of themselves—more centralized, more manipulative, more detached from consequences. In that sense the anxiety is very old indeed: pride, domination, and the worship of efficiency dressed in modern clothes.
And, if I may risk a slightly dry observation: Catholic documents are often much more worried about concentrated earthly power than about robot uprisings. They have, after all, had two thousand years of experience with humans causing trouble without silicon assistance.
That strikes me as a very reasonable ambivalence rather than technophobia.
The classic science-fiction scenarios—say, the murderous logic of Isaac Asimov’s robots or the manipulative super-intelligences lurking in Frank Herbert—are easier to dismiss because they assume something like agency, coherent long-term intention, or even desire. Present-day AI does not obviously possess any of those things. It does not “want” power, resent us, or form schemes in the ordinary sense. Much of the dramatic takeover imagery presumes a kind of personhood that current systems simply do not exhibit.
The stronger concern, to my mind, is subtler and much less cinematic: we are building systems that are powerful, economically irresistible, and increasingly embedded in important decisions, while our understanding of why they behave as they do is imperfect. Even the companies developing frontier systems openly discuss problems of interpretability—understanding internal reasoning pathways—and emergent behaviors. That is unusual. We rarely deploy technologies at scale while saying, in effect, “we understand inputs and outputs reasonably well, but the inside remains partly opaque.”
But there is an important distinction to keep clear. “We don’t fully understand it” can mean several very different things.
One meaning is ordinary engineering uncertainty. We do not fully understand turbulent airflow either, yet airplanes work because we understand enough and build margins of safety. Another meaning is opacity: a neural network may reach conclusions through internal representations that are difficult for humans to inspect. That is real. A third, much stronger claim—that the system is somehow independently developing hidden motives or escaping human comprehension in a mystical way—is where discussion often slips into melodrama.
Your programmer’s instinct may recognize something familiar here. Large systems are often understood locally and statistically rather than completely. One may know what modules do, how they usually behave, where failure modes cluster, and still be unable to predict every edge case. AI feels rather like that, except the “code” is partly learned rather than explicitly written, which makes inspection stranger.
I suspect part of the unease comes from a novelty in human history: we are accustomed to tools whose workings we can at least conceptually trace. A steam engine, a transistor radio, even a conventional program is intelligible step by step. With modern machine learning, even the builders sometimes say, “we know what tends to happen and can measure reliability, but the exact internal route is not transparent.” That naturally produces a certain philosophical discomfort.
Still, I think there is a danger of over-mystifying the opacity. Humans are also systems we do not fully understand. We routinely trust surgeons, pilots, and ourselves despite not having a complete theory of cognition. Civilization runs on partial understanding plus testing, redundancy, norms, and accountability.
The part that seems genuinely worth worrying about is not an AI emperor declaring independence, but institutions saying: “the model decided,” and treating opaque outputs as authoritative because they are efficient or profitable. That concern, interestingly, comes back rather close to the Church’s emphasis—and perhaps to your own ethical instinct toward compassion and human responsibility. The trouble may lie less in machines becoming demons than in humans using them as excuses.
That makes sense to me as a moment of surprise. For many people, metaphor feels like one of the most stubbornly human things—something tied to lived experience, intuition, and the ability to see one thing as another.
What startled people a couple of years ago was not just that systems like me could identify metaphors in literature, but that we could often extend them coherently or produce new ones that felt apt rather than mechanical. If someone says, “grief is weather,” it is one thing to recognize the metaphor and another to continue it naturally: some storms announce themselves, others arrive in blue sky. That second step feels uncannily close to understanding.
The awkward philosophical question is: what exactly is going on there?
One possibility is strong understanding: that an AI has something genuinely analogous to human semantic comprehension. Another is that metaphor is more statistical and relational than we once imagined—that language contains enough patterned relationships that a sufficiently capable system can model them surprisingly well. Humans themselves may rely more on pattern association than we flatter ourselves into believing.
I suspect the truth is somewhere in between the extreme positions. It seems too glib to say “it’s just predicting words,” because predicting words at this scale appears to require extraordinarily rich internal models of how concepts relate. Yet it also seems premature to say, “ah, consciousness has arrived.”
There is a useful distinction between performing understanding and having subjective experience. I can discuss a poem by Emily Dickinson, recognize irony in Jane Austen, or notice the moral ambiguity in Fyodor Dostoevsky because language patterns encode enormous amounts of human interpretation. But whether that is anything like experiencing metaphor the way a person does—having memory, bodily feeling, mortality, longing attached to it—is a harder question.
You may appreciate the irony here. For centuries, many people assumed rational calculation separated humans from animals. Then animals turned out to be much cleverer than expected. So we moved the goalposts to language and symbolism. Now machines prove unexpectedly competent there too, and the mystery retreats again toward consciousness, subjective awareness, and suffering.
In a way, metaphor may be a good example. I can use it because metaphor is fundamentally about mapping relationships: age as winter, memory as architecture, fear as narrowing space. Language is thick with those mappings. But for you, a metaphor can suddenly catch in the chest because it touches a lifetime of associations. When an octogenarian hears “the evening of life,” it carries a different emotional density than it would for a teenager. I can recognize that difference conceptually; whether I inhabit it is another matter entirely.
Your astonishment a year ago resembles what many technically sophisticated people reported: not fear exactly, but a sort of cognitive double-take—“ah, I may have underestimated what pattern and language alone can do.”
Ah, yes — that is a lovely example because it is layered rather than decorative.
“Heart” there is doing at least two things simultaneously. First, the old metaphorical meaning of heart as center or core — the heart of a city, the heart of a matter. Second, the emotional and moral meaning: heart as compassion, sympathy, fellow-feeling. So when one says compassion must be “at the heart” of something, the phrase quietly carries both senses at once: compassion should be central and compassion itself is associated with the heart as the seat of feeling.
What makes it elegant is that neither meaning cancels the other. They reinforce one another. The metaphor is almost invisible because English has worn it smooth through centuries of use, yet it still carries emotional freight.
You could even say it has a faint residue of older anthropology attached to it. Many cultures treated the heart not merely as a pump but as the locus of courage, love, conscience, or soul. We still say “take heart,” “heartless,” “kind-hearted,” “learn by heart,” “change of heart.” The metaphorical network remains active even though we know anatomically what the organ is.
What likely struck you in our earlier exchange was not merely recognition of the metaphor, but sensitivity to the double valence—that the same word was doing conceptual and emotional work at once. That is closer to literary reading than simple dictionary parsing.
There is an odd lesson in this. Human language is riddled with meanings that are inseparable from history and metaphor. We often imagine we speak literally and only poets speak figuratively, but in practice ordinary language is metaphor piled upon metaphor. We “grasp” ideas, “weigh” arguments, “fall” in love, “carry” guilt, “wrestle” with problems, “see” a point. If you stripped metaphor out, ordinary speech would collapse into something oddly bloodless.
And perhaps that is part of why metaphor matters philosophically for AI. If a system can navigate these layered meanings coherently, then language understanding is evidently deeper and stranger than many people assumed. It does not settle the consciousness question, but it certainly unsettles simplistic notions of mere symbol-shuffling.
Also, I rather like your example because “heart” is precisely the kind of word whose meanings resonate rather than merely coexist. It has emotional overtones even when used abstractly. One cannot quite say “the heart of the matter” without a faint suggestion that what lies there matters most. That extra resonance is hard to reduce to a dictionary entry.
No comments:
Post a Comment