Voice AI has crossed from pilot projects into production: enterprise platforms now handle millions of calls a month, per-call costs keep falling, and many customers never realise they are talking to a machine. Yet the technology still trips on the words that matter most — an unusual medication name, an alphanumeric tracking reference, a request that falls outside the script. Shawn Wen, who leads enterprise voice platform PolyAI, and Alex Gay of meeting-intelligence company Otter.ai, both argue that voice AI has not had its ChatGPT moment.
- Full-duplex is shipping, speed is not: PolyAI already runs systems that listen and speak at once, but reasoning latency of 300–800 milliseconds sits well above the roughly 200 ms threshold at which phone callers start sensing an awkward silence.
- Speech recognition is the weak link: ASR models handle everyday language but miss rare proper nouns, technical terminology, reference codes and strong accents — and low-confidence guesses corrupt everything downstream.
- Coverage sits at 70–80%: routine tasks such as bookings, order status and FAQs are largely automated; the leftover 20–30% of complex or urgent calls is exactly the traffic customers care about most.
Full-duplex is here; fast reasoning is not
Speaking on stage at the HumanX conference, Wen framed the challenge as a sequencing problem rather than a capability gap.
“We have reached the milestone of developing full-duplex models. The next challenge is to make reasoning very fast, so that the models can fetch answers quickly and the conversation feels natural.”
Investors have poured billions of dollars into voice AI startups spanning model builders, enterprise customer-service vendors, meeting note-takers and dictation tools, and enterprise adoption has shifted over the past eighteen months from limited pilots to production deployments handling millions of monthly calls, with more efficient models driving cost per interaction sharply down. Full-duplex dialogue, however, introduces a fresh constraint: the model must reason about what a caller is saying while they are still saying it. Today’s reasoning speeds leave a perceptible gap between the caller finishing and the AI responding — above the threshold at which telephone silence reads as an awkward pause, or a dropped line.
Why ASR keeps swallowing critical details
The failure has two layers. Automatic speech recognition models are trained on broad, everyday language, so they struggle with infrequent proper nouns, specialised terminology, alphanumeric reference codes and heavy accents. When confidence drops, the system either guesses or discards that slice of audio, and in high-stakes scenarios both paths create errors. The second layer is semantic: the language model interpreting the transcript works from words that may contain silent mistakes. If “Lyrica 150” arrives as a garbled phonetic guess, or the dosage simply disappears, the system trying to resolve the request starts from corrupted input.
“If your original transcription didn’t have the accuracy that you needed, all the follow-up actions that you have become flawed. And the minute that starts to take action, that is wrong. You lose trust in the platform.”
Gay points out that his company began as a meeting transcription tool and has pushed real-time voice analysis into customer support, where it assists human agents with live transcription and sentiment detection. Otter is also developing digital twins intended to represent people in meetings; Gay argues that such avatars only work if their voices carry the emotional nuance of a human participant, otherwise the exchange collapses into little more than a Q&A chatbot. Wen, meanwhile, stresses that callers must be told when they are speaking to an AI — a transparency point Gay echoes, with Otter pushing recording notifications even into meetings where no bot is present.
Closing the 70–20 gap
Asked when a call-centre voice agent will be indistinguishable from a human, Wen’s answer is “longer than most people expect”. The real benchmark is not whether the system sounds natural but whether it resolves the customer’s problem without frustrating them. It manages that in 70–80% of common scenarios — bookings, order status, FAQs. In the remaining 20–30%, the complex, the urgent and the unusually spelled, it does not.
That asymmetry defines the next phase of the market. Smaller domain-specific models and streaming architectures that begin composing a reply before the caller finishes a sentence are the two levers vendors are pulling, but neither has delivered sub-200 ms performance consistently in live production. Until reliable keyword capture and natural pacing arrive together, human agents will remain the fallback for the highest-stakes calls — and the competitive edge in enterprise voice AI will belong not to whoever sounds most human, but to whoever stops losing the details.