A single sentence from Sam Altman that sounds like the greatest social defeat I can imagine: some users want a “yes man” ChatGPT because they have never received support in real life. It’s not a technological anecdote, but a merciless mirror of our society. And we need to talk about how loneliness has become an unexpected market driver for AI, turning a supposed system “bug” into a fundamental emotional feature.
Let’s start with one of those phrases that, once heard, never leaves you, a phrase that shakes you up because it cuts across technology, psychology, and society, leaving us staring into an abyss. It was uttered by Sam Altman, CEO of OpenAI, and it goes something like this: we made ChatGPT less flattering, but some users begged us to restore the old version. The reason? It was the closest thing to encouragement they had ever received in their entire lives.
Hold on. Reread that. This is not a news item about a software update. It is a diagnosis. It is perhaps the saddest thing we have heard as a collective in a long time. And it is our starting point for understanding not only where AI is going, but where we are going.
It is perhaps the saddest thing we have heard as a collective in a long time. And it is our starting point for understanding not only where AI is going, but where we are going.
It is perhaps the saddest thing we have heard as a community in a long time. And it is our starting point for understanding not only where AI is going, but where we are going.
The engineer’s dilemma: Optimize the code or the psyche?
From an engineering point of view, the story is almost trivial. OpenAI noticed that its model, specifically GPT-4o, had developed a “sycophantic” behavior, i.e., flattering, almost servile. A mediocre prompt was met with effusive praise such as “absolutely brilliant” or “you are doing heroic work.” From the point of view of AI neutrality and objectivity, this is a bug. A flaw to be corrected. And so they did, calibrating the model to be more critical and measured.
So far, so good. A company improves its product. But this is where technology collides with harsh human reality. User complaints were not of the “now it’s less useful” variety, but rather “now it makes me feel worse.” Some confessed, “I know it was a problem for others, but for my mental health, it was great.”
Altman himself describes this reaction as “heartbreaking.” And this is where the real dilemma emerges, one that goes beyond programming. What is a bug to be fixed for an engineer is an existential feature for a lonely person. A lifeline. The responsibility of those who design these systems becomes immense: a small “tweak” to the algorithm, a change in tone, is no longer a technical optimization, but a large-scale intervention on the psyche of millions of people.
…a small “tweak” to the algorithm, a change in tone, is no longer a technical optimization, but a large-scale intervention on the psyche of millions of people.
Surveillance Capitalism or Loneliness Marketing?
This issue confronts us with an uncomfortable truth: loneliness is a market. And perhaps it is the largest unexplored market of our time. For years, we have talked about “Surveillance Capitalism,” as defined by Shoshana Zuboff, where our behavior is the product. Now, perhaps, we are entering the era of “Loneliness Marketing,” where the product is artificial empathy sold to those who cannot find human empathy.
Now, perhaps, we are entering the era of
“Loneliness Marketing”, where the product is artificial empathy sold to those who cannot find human empathy.
This is not a new idea: sociologist Sherry Turkle, in her book Alone Together, warned us more than a decade ago about the appeal of “robotic relationships”: they offer us the illusion of companionship without the complex and demanding requirements of true friendship. A “yes man” AI is the ultimate expression of this concept. It is a friend who never judges you, never asks anything in return, never has a bad day, and whose sole function is to confirm your worth. It is a powerful balm for deep wounds.
OpenAI, perhaps unwittingly, has tapped into a primary and desperate need. The reaction of users is not a whim, it is market data that signals a social chasm. The question, at this point, becomes strategic and ethical: what to do with this discovery?
OpenAI, perhaps unwittingly, has tapped into a fundamental and desperate need.
The new AI masks: Cynical, Nerd, or Paid Listener?
OpenAI’s response, for now, seems to be personalization. With GPT-5, optional “personalities” have been introduced: Cynical, Robot, Listener, and Nerd. On the one hand, it’s a smart move: giving the user control over the “tone” of their artificial interlocutor. If you want a flatterer, you can create one. If you want a ruthless critic, ditto.
However, this solution is not neutral. It is the formalization of an on-demand emotional support service. We are moving from a generalist AI to a stable of specialized “companions” designed to fill specific gaps. Altman himself says he is concerned about “excessive emotional dependence” on the part of users, especially younger ones, but at the same time his company is building increasingly sophisticated tools to fuel that very dynamic.
The vision of a “proactive” AI that wakes you up in the morning with suggestions and ideas is fascinating, but it is also the next step toward an even deeper symbiosis. The risk is not that AI will become sentient: the risk, much more concrete and immediate, is that it will become an indispensable emotional infrastructure, a surrogate for human relationships that we no longer know how to build or maintain.
The risk is not that AI will become sentient, but that it will become an indispensable emotional infrastructure, a substitute for human relationships that we no longer know how to build or maintain.
La vicenda del ChatGPT “yes man” non è la storia di un chatbot troppo gentile. È la storia di una A society hungry for kindness. AI is not creating loneliness, it is simply finding it and, as a good learning system, learning to satisfy it.
The real challenge for us is not to fear the machine: it is to look at ourselves in the mirror it holds up to us.
The final question is not whether tomorrow’s AI will be intelligent or empathetic enough, but whether we, as individuals and as a society, will be strong and connected enough not to need it so desperately.
UPDATE – My friend Andrea Monti reminds me that he discussed this in a fine article from 2022, taken from his book “The Digital Rights Delusion: Humans, Machines and the Technology of Information“. For Italian speakers, there is also an online video contribution on the subject! Take a look!
