3 min readMadrugo team

Voice commands without misunderstanding

The problem is not that the speech recogniser fails. The problem is that nobody tells you what language it is listening in.

Almost every productivity app with voice input shares the same flaw: it does not show what language it is listening in. You open the app, tap the microphone, dictate "remind me to call the dentist tomorrow", and the transcription comes out as if you had spoken in Spanish. The recogniser was fine. The language was wrong.

Why it happens

Apple's and Google's speech recognisers need to know the language before they start listening. They do not guess on the fly: they are trained on a specific language model and transcribe whatever they hear according to that model. If the model is Spanish and you speak English, the result is high-confidence garbage.

Most apps pick the device language and never touch it again. If your iPhone is set to English, they transcribe in English. If you set it to Spanish because some apps look better that way, they transcribe in Spanish even though you speak English. And they never tell you which one they chose.

What should happen

A well-made microphone button needs three things:

1. A visible badge showing "EN", "ES" or "CA" before you speak. If you do not know what language it is listening in, you cannot correct it before wasting thirty seconds dictating.

2. A quick menu on long press, with your three main languages. Without having to open settings, find the option and choose from 63 locales.

3. The actual language in the server request. There is no point in the recogniser getting it right if the backend then receives "locale: es" and the AI interprets your command in the wrong language.

The third point is subtle and the most damaging. If you dictate in English, the transcription comes out perfectly, but the server reads locale: es because that is what the profile says, and the AI understands something else. The user sees a correct transcription and does not understand why the command fails.

Live transcription is not optional

Another common mistake: recording thirty seconds of audio and showing the transcription only at the end. If the recogniser was in the wrong language, the user discovers it after they have finished speaking. The two seconds of silence the app needs to decide you are done become the two most frustrating seconds of the day.

Partial transcription — the kind that appears word by word while you speak — is not a nice touch. It is control information. If you see that what appears on screen makes no sense, you stop talking immediately. If you cannot see it, you speak the whole thing, wait, and discover that everything went wrong.

In Madrugo, the microphone button carries a badge with the active language, a quick menu to switch it on the fly, and the transcription appears in real time with the language displayed. Because dictating a command and having it work first time should not require blind faith.

Keep reading

All news