Multilingual Voice Agents: What Actually Breaks

Code-switching is normal, and most agents handle it badly

Bilingual speakers routinely switch languages inside a single sentence — starting a question in one language and finishing it in another, or using one language for everyday words and another for numbers, technical terms, or brand names. This is not an edge case; in many markets it is the default way people speak. Systems that pick a language at the start of the call and lock to it will mis-transcribe every switch, and systems that re-detect too eagerly will flip mid-sentence and produce nonsense. Neither behaviour is acceptable to a caller who was speaking perfectly naturally. If you serve a bilingual market, this specific behaviour is worth testing before anything else, because it is both the most common failure and the one least likely to appear in a vendor demo.

The variation inside a language is bigger than between languages

Naming a language implies a uniformity that does not exist. English spoken in Glasgow, Lagos, Chennai, Alabama, and Singapore differs enough in vowels, rhythm, and vocabulary that recognition quality can vary sharply across them, and models generally perform best on whichever varieties dominated their training data. The practical consequence is uncomfortable but worth stating plainly: an agent can work well for some of your callers and noticeably worse for others, and if you only test with your own team you will never see it. This is a fairness issue as much as a quality one, because the callers who get the worst experience are frequently the ones with the least patience for being misunderstood by a machine. Sample your test calls deliberately across the accents you actually serve.

Formality is not decoration

Several languages encode social distance grammatically — the formal and informal second person in French, German, Spanish, and Hindi among many others, and honorific systems in Japanese and Korean that go considerably further. Choosing wrong is not a stylistic wobble; addressing an older customer with the familiar form can read as disrespectful in a way that has no equivalent in English. A prompt translated literally from English will usually default to whichever form the translation model prefers, with no awareness of who is being addressed. Decide the register deliberately per language and market, and have a native speaker review the actual output rather than the source prompt, because the two can diverge substantially once the model starts generating.

Compliance follows the caller, not the company

Adding a language usually means adding a market, and disclosure and recording rules differ by jurisdiction rather than by language. If a call is being recorded, the notice generally has to be given in a language the caller understands, and any requirement to disclose that the caller is speaking to an automated system applies in that language too. It is straightforward to translate the sales copy and forget the disclosure, which is the one line where getting it wrong carries legal consequences rather than commercial ones. Treat each new market as a compliance review, not just a translation job, and confirm which rules apply where your callers are rather than where your servers are.

Key takeaways

• A language list on a vendor page tells you almost nothing about whether an agent works for your callers.
• Real bilingual callers code-switch mid-sentence. Agents locked to one language per call handle this badly.
• Accent coverage varies enormously within a single named language, and that variation is where most failures happen.
• Formality and register are not cosmetic — getting them wrong reads as rude in several languages, not just unusual.