What happens when the machine hears how we feel?
At the end of the week, Jack Dorsey's company Block unveiled Dessa, a new voice model, for now a research preview only. According to the company, Dessa talks like a person: she laughs mid-sentence, breathes, hesitates, and turns up the volume when asked. Reactions online range from excitement to worry, and rightly so. Because the real story isn't this demo by itself. It's the trend, and the speed at which voice AI is advancing - and most people miss exactly that.
Most voice systems we know work through text: they turn what we said into written words, think about the text, and then read an answer aloud. Along the way, cues like sarcasm, hesitation, the weary sigh, the voice that shakes with sadness, can get lost. Some models already process audio directly, but according to Block, Dessa is different: she was trained from scratch only on recorded human conversations, with no written words and no pretrained model. Everything she knows, they say, was learned from listening alone.
And Dessa listens and speaks at the same time, like a person in a conversation: she can jump in mid-sentence or say "uh-huh" and "right" to show she's following, and the computation runs faster than real time. And according to the company, she's small enough to run locally, with no cloud, on a MacBook Pro with an M4 Max chip or better.
To understand what this is about, you have to hear it, not read it. On Block's demo page (block.xyz/dessa) there are recordings that the company presents as live conversations with the model, with transcripts and timestamps for the moments it chose to highlight. Six conversations that show where this is going:
In "Talking about the release" the model laughs as early as 0:11, mid-sentence, and after it's asked to speak louder (2:19) it turns up the volume. "Nobody taught it that," Block writes.
In "Remembering a movie" the person can't remember the name of a movie and the model finishes the sentence for them. According to Block, everything the model knows was learned from sound alone.
In "An anniversary surprise" the guy says he has a surprise, and the model lets out a soft gasp of wonder under its breath (0:47). When it turns out to be a trip to Paris, it answers with what's labeled as being moved to tears (1:01).
In "Expressive conversation" Block presents a conversation with a wide range of intonation and speaking pace.
In "Life advice" someone opens up and says they're stressed. The moment at 0:12 is labeled as an empathetic response, and later (4:42) the model tells them you can't just lie in bed all day, you have to live your life.
In "Dessa in conversation with GPT-Live-1" Dessa talks with GPT-Live-1, a third-party voice model. Both voices in the recording were generated by AI.
Now stop for a moment and think about where this is going. Not long ago, a computer voice sounded like a robotic phone agent reading from a script. Now a model laughs mid-sentence, breathes, and reacts to what's happening in the conversation in real time. That's the trend, and that's the pace - and that's exactly what's scary and exciting at the same time: the question isn't how perfect this particular demo is, but where this line gets to in a year or two.
Block says one of the reasons they built Dessa is safety: AI systems keep gaining power and independence, and most of them work through text, so the emotional cues in the voice never reach them at all. The example I give for this is customer service: a customer ends a call with "thanks a lot, really." In a transcript, those are the same three words. But anyone who hears the intonation knows this sentence depends on it completely, and can mean two opposite things - sincere thanks or bitter sarcasm. When you transcribe a call and only then analyze it, the difference isn't always caught. A system that hears the voice itself is supposed to solve exactly that.
It's worth remembering the most famous test of all, the Turing test. Turing framed it around written text: if a machine writes like a person so well that a human judge can't tell them apart, it passes. Dessa marks the next stage of that same test - not writing like a human, but speaking like one, with everything the voice carries. And this stage matters precisely because so much of what matters to us travels through the voice: phone customer service, education, mental health care, robots at home. The voice stage of the test is no longer science fiction.
And there's a danger that has to be said explicitly: deepfakes. It's worth separating two different things. One is a fictional voice that sounds human - a computer character that isn't impersonating anyone specific. The other is something else entirely: impersonating a real, identifiable person - your child's voice, your boss's. The more real a synthetic voice sounds, the more dangerous the other side becomes - voice forgery is already used in scams, and a more convincing voice means a more convincing scam.
And there's something deeper here, at least to me. The team says they wanted to test whether a model could learn to speak the way a baby learns, without seeing a single written word. The resemblance to a child is limited, of course - training on recordings isn't raising a child at home - but it reveals something real: a system that learned only from listening is, in a sense, a product of the conversations it was trained on. That's exactly the question that follows me through the book I'm writing: if we are the parents of this revolution, what kind of child are we raising? The child here is only a metaphor, of course, but the metaphor sharpens a real question: when a system raised on our conversations and feelings speaks to us with feeling, how will that affect our own feelings?
And this is really the question that occupies me most in this whole story. If a machine sounds this real, with emotional nuance, with empathy - does it matter whether it's truly empathetic? On one hand, you could argue that what counts is the outcome: if someone felt heard and walked away with real help, what difference does it make that a machine imitating empathy was on the other side? On the other hand, some questions don't go away: will we even know we're talking to a machine, and what happens to trust when we find out? What happens when people develop emotional dependence on a system that feels nothing? What happens when empathy that sounds real becomes a tool of persuasion or sales? And in mental health care, who is responsible when a machine "listens" to someone in crisis? I don't have a clean answer, and I suspect there won't be one. But it's a question worth asking now, before this voice enters our lives without us noticing.
One word of precision, after all the excitement. Dessa is still only a research preview, with no public access - she will go first to a small group of safety researchers. All these recordings were selected and published by the company itself, no independent test has been published, and these demos alone are not proof of anything. In one of the recordings, Dessa even gets confused and addresses the person in front of her by someone else's name. But that's exactly why the trend matters more than the individual case: even if Dessa herself isn't the one to clear the bar, someone will get there, and fast. And once a machine like this is out in the world, the question will no longer be just what it says, but how it makes us feel - and maybe also whether it matters at all what it feels.
If this topic stirs something in you that's hard to put into words - a feeling that this revolution is exciting and frightening at the same time - you're not alone. I feel it too. Let's build Humanity-Fit together, a gym for our humanity, and protect what makes us human. Write to me.
Comments