Code-switching is not an edge case

Listen to a few minutes of real conversation in an office in Dar es Salaam and you will hear something like this:

Nimeshatuma ile invoice jana, ila sijapata confirmation bado.

That is one sentence, one speaker, one thought. It is Swahili grammar carrying English nouns, and nobody listening would find it unusual. Yet many speech systems break on exactly this kind of sentence. They expect a call to be in one language, choose a model, and then fail on every word that doesn't fit.

At Waridi we start from the opposite assumption. Mixing languages is how people talk. A speech system for East Africa has to treat it as the normal case.

What code-switching actually looks like

Code-switching is not random. It follows patterns, and those patterns are what a model has to learn.

• English nouns inside Swahili sentences. Meeting imesogezwa hadi kesho. Words like meeting, invoice, password, account and network often stay in English.
• Swahili grammar wrapped around English roots. Naomba unitumie link ya ku-reset password. Here ku- is the Swahili infinitive prefix attached directly to an English verb.
• Whole clauses switching. A speaker might start in Swahili and finish a technical explanation in English, then switch back for the polite closing.
• Street and youth language. In Tanzania, everyday slang such as mshkaji (friend) or Bongo (Tanzania, or Dar es Salaam) appears alongside standard Swahili. In Kenya, Sheng mixes Swahili, English and other local languages with its own fast-changing vocabulary.

Each of these has a different structure, and a system that only knows "Swahili" or "English" misses all of them.

Why per-utterance language ID fails

The common shortcut is to detect the language of a whole utterance and route it to the matching model. For mixed speech this goes wrong in two ways.

First, the detector has to pick one label for a sentence that genuinely contains two languages. Whichever it picks, part of the sentence is now being decoded by the wrong model.

Second, the errors land on the most important words. In Nimeshatuma ile invoice jana, the word a support agent needs is invoice. In a banking call it might be PIN, statement or balance. These are often the English words, so a Swahili-only decoder mangles exactly what matters.

Our approach in Waridi Sikia is to use a single model that handles both languages together, so there is no routing step to get wrong. Where we need language labels, for example to choose pronunciation in text-to-speech, we predict them per word rather than per utterance.

Lexicon and tokenization

A mixed-language model needs a vocabulary that represents both languages fairly. If the tokenizer is trained mostly on English text, Swahili words get chopped into many small pieces, which makes them harder to learn and slower to decode. If it is trained mostly on Swahili, English loanwords suffer.

We build our subword vocabulary from a balance of Tanzanian Swahili and the English that actually appears in Tanzanian speech, not general web English. We also pay attention to hybrid forms like ku-reset or ame-download, so the Swahili prefix and the English root are both recognisable units.

Spelling is another decision. Do we transcribe password as English or as pasiwedi? We follow how people write in practice, which today is mostly the English spelling for technical terms, and we keep that rule consistent across our data so the model isn't punished for a choice we made inconsistently.

Text-to-speech has the same problem in reverse

For Waridi Sauti, the challenge is pronunciation. A Tanzanian speaker does not say invoice the way a London newsreader does. The vowels shift toward Swahili ones, and the rhythm follows the surrounding sentence.

If a TTS system switches to an English accent for one word and back again, it sounds like two people sharing a sentence. Our goal is the opposite: one consistent Tanzanian voice that says English words the way a Tanzanian would.

Collecting the right data

You cannot learn code-switching from clean, single-language recordings. We collect conversational speech from consented, paid speakers, using prompts that invite natural mixing: describing a work day, explaining a problem with a phone, giving directions. We also review transcripts with native speakers so that slang and hybrid words are written down consistently.

Sheng and street language change quickly, so this is not a one-off task. We plan to refresh our lexicon and test sets regularly.

How we evaluate it

A single word error rate across a whole test set can hide poor performance on mixed speech. In our ASR benchmark work we report results separately for:

• monolingual Swahili utterances,
• utterances with English insertions,
• the English words themselves inside mixed utterances,
• slang and street-language terms.

The third number is the one we watch most closely, because it tracks the words callers most need the system to get right.

The point

People in East Africa should not have to change how they speak to be understood by a machine. If someone says Nimeshatuma ile invoice jana, the system should simply understand. That is the standard we are building Waridi Sikia toward.

You can read more about how mixed speech is handled in the speech-to-text docs.