Multilingual RAG: Cross-Lingual Retrieval for Indian and European Languages

What you'll build, in plain English

By the end of this guide you will know how to build a retrieval-augmented generation system that answers a question in the language it was asked, no matter which language your source documents happen to be written in. A user in Coimbatore can ask in Tamil and get an answer grounded in an English policy PDF. A support agent in Lyon can ask in French and pull the right paragraph from a German technical manual. That capability — a query in language A retrieving documents in language B — is called cross-lingual retrieval, and it is the single most under-built part of most production RAG stacks serving India and Europe.

The plan looks like this. Detect the language and script of the incoming query. Normalise the text so Unicode quirks and code-mixing do not sabotage matching. Embed the query with a multilingual model that has learned a shared cross-lingual space. Retrieve with a hybrid of dense, sparse and multi-vector signals. Rerank the shortlist with a multilingual cross-encoder. Then evaluate the whole thing per language, because an aggregate score hides the languages that are quietly broken. We will build each stage with concrete, well-supported choices, and we will keep one eye on data residency the entire way — because for Indian and European users, where the vectors are computed is not an afterthought, it is a legal constraint.

Why English-first RAG breaks on Indian and European languages

Most RAG tutorials assume the query and the corpus are both in English, so they reach for an English-only embedding model, chunk everything on whitespace, and match on cosine similarity. The moment real users arrive, three things break at once.

The embedding space does not span the languages. An English-centric model has never learned that the Hindi ऋण, the Tamil கடன் and the English loan should live near each other. So a Hindi query lands in an empty region of the space and retrieves noise. This is not a tuning problem; the model simply does not represent the language.

Tokenisation and chunking assumptions collapse. Whitespace splitting is meaningless for scripts and languages that do not delimit the way English does, and naive character-count chunks slice Devanagari or Bengali graphemes mid-cluster. German compounds such as Datenschutz­grundverordnung behave nothing like the short tokens an English chunker expects. If you have not already sorted your segmentation strategy, our companion guide on chunking and embedding strategies for production is worth reading alongside this one.

Code-mixing is the norm, not the exception. Indian users routinely type romanised Hindi (Hinglish), mix English technical terms into a Tamil sentence, or switch scripts mid-query. European users paste an English error message into a French support question. An English-first pipeline treats all of this as gibberish. The uncomfortable truth, and a well-documented weakness rather than a theoretical one, is that embeddings align low-resource Indian languages far more weakly than they align English — so you must not assume English-level quality anywhere. You have to measure it.

Choosing a multilingual embedding model

The first real decision is which embedding model anchors your shared space. In 2026 the practical shortlist is small, and the differences that matter are less about the headline MTEB average and more about language coverage, licence, and whether the model gives you sparse signals for free. The table below aggregates multilingual MTEB positioning; treat the scores as directional, since MTEB is a moving target and your own corpus will reorder them.

Model	Multilingual MTEB (approx.)	Languages	Licence / hosting	Notable trait
Cohere embed-v4	~65.2	100+	Hosted API (in-region options)	Leads the multilingual leaderboard; no self-host
OpenAI text-embedding-3-large	~64.6	Broad	Hosted API	Strong all-rounder; residency depends on provider region
BGE-M3 (BAAI)	~63.0	100+	Open weights (self-host)	Dense + sparse + ColBERT multi-vector in one model, 8K context
multilingual-e5 (mE5)	Competitive	~100	Open weights (self-host)	The mE5 report notes it surpasses Cohere multilingual-v3 by ~0.4 points

For most teams serving India and Europe, BGE-M3 is the pragmatic default, and the rest of this guide builds on it. Not because it tops the leaderboard — it does not — but because it is open-weights (so you can run it inside AWS Mumbai, London or Frankfurt for residency), it covers 100-plus languages including a wide sweep of Indian ones, it handles 8K-token context, and, crucially, it produces dense, sparse (lexical) and ColBERT-style multi-vector representations from a single model. That last property means one model gives you the entire hybrid retrieval toolkit without stitching together a separate BM25 index and a separate late-interaction model.

If your traffic is dominated by one hosted region and you are willing to pay per token, Cohere embed-v4 will squeeze out the best raw quality, and OpenAI's text-embedding-3-large is a comfortable middle. But the moment residency, cost at scale, or the need for sparse signals enters the picture, the open-weights options pull ahead. If you want to run this comparison rigorously on your own data rather than trusting a leaderboard, our walkthrough on choosing an embedding model with a retrieval benchmark shows how to build the harness.

Cross-lingual retrieval, explained

Here is the mechanism that makes all of this work. A good multilingual model is trained so that a sentence and its translation land close together in vector space. Over many languages, the model learns a shared cross-lingual semantic space: meaning, not surface form, decides position. BGE-M3 is explicitly built this way, which is why it supports both in-language retrieval (Hindi query, Hindi document) and cross-lingual retrieval (English query, Hindi document) out of the same index.

The practical payoff is enormous. You do not need to translate your corpus into every user language, and you do not need a separate index per language. You embed each document once, in whatever language it was written in, and you embed each query once, in whatever language it arrives in. Because both sit in the same space, an English query for "data protection obligations for processors" will surface a French passage about obligations du sous-traitant and a Hindi passage on the same theme, ranked by meaning. For a dual-market product, that is the difference between maintaining one clean index and maintaining a fragile translation pipeline that drifts every time a document changes.

Two caveats keep this honest. First, cross-lingual scores are noisier than in-language scores, which is exactly why the reranking stage later on earns its keep. Second, the quality of the shared space is deeply uneven: high-resource pairs (English–French, English–German) are excellent, while low-resource Indian languages align more weakly. Cross-lingual retrieval is a capability you switch on and then verify per language — never one you assume works uniformly.

A hybrid pipeline: detect, embed, retrieve, rerank

Hybrid retrieval — combining dense semantic search, sparse lexical search and a reranking pass — is the 2026 production default, and it matters even more multilingually because no single signal is reliable across every language. Dense vectors capture meaning but blur exact tokens; sparse vectors catch names, codes and rare terms that dense models smear; the reranker cleans up the shortlist. BGE-M3 ships all three retrieval modes in one model, so the code below stays compact. Our fuller treatment of the pattern lives in the production hybrid retrieval guide, and the reranking mechanics are unpacked in reranking with cross-encoders, ColBERT and hosted rerankers.

The pipeline has four stages: detect the language and script, normalise and (if code-mixed) transliterate, hybrid-retrieve with BGE-M3, then rerank with a multilingual cross-encoder. Here is a compact, production-shaped implementation.

import unicodedata
from fast_langdetect import detect            # language + script detection
from indic_transliteration import sanscript   # romanised -> native script
from FlagEmbedding import BGEM3FlagModel, FlagReranker

# One model gives dense + sparse (lexical) + ColBERT multi-vector.
embedder = BGEM3FlagModel("BAAI/bge-m3", use_fp16=True)
reranker = FlagReranker("BAAI/bge-reranker-v2-m3", use_fp16=True)  # multilingual

def normalise(text: str) -> str:
    # NFC keeps Devanagari/Bengali grapheme clusters intact and
    # collapses compatibility variants across scripts.
    return unicodedata.normalize("NFC", text).strip()

def prepare_query(raw: str) -> dict:
    text = normalise(raw)
    lang = detect(text)["lang"]              # e.g. "hi", "ta", "fr", "de", "en"
    variants = [text]
    # Code-mixed / romanised Hindi (Hinglish): add a native-script variant
    # so lexical matching fires on the Devanagari tokens too.
    if lang == "en" and _looks_romanised_indic(text):
        variants.append(sanscript.transliterate(
            text, sanscript.ITRANS, sanscript.DEVANAGARI))
    return {"lang": lang, "variants": variants}

def hybrid_retrieve(query: dict, k: int = 50) -> list[dict]:
    # Encode every query variant; BGE-M3 returns dense + sparse weights.
    enc = embedder.encode(
        query["variants"],
        return_dense=True, return_sparse=True, return_colbert_vecs=True,
    )
    # Fuse dense and sparse scores against the index (weights tuned per corpus).
    hits = index.hybrid_search(
        dense=enc["dense_vecs"],
        sparse=enc["lexical_weights"],
        weight_dense=0.6, weight_sparse=0.4,
        top_k=k,
    )
    return hits

def retrieve_and_rerank(raw_query: str, k_final: int = 8) -> list[dict]:
    query = prepare_query(raw_query)
    candidates = hybrid_retrieve(query, k=50)
    # Cross-encoder rerank: score each candidate against the ORIGINAL query.
    pairs = [[raw_query, c["text"]] for c in candidates]
    scores = reranker.compute_score(pairs, normalize=True)
    for c, s in zip(candidates, scores):
        c["rerank_score"] = s
    ranked = sorted(candidates, key=lambda c: c["rerank_score"], reverse=True)
    return ranked[:k_final]

A few design notes. Language detection drives everything downstream, so treat a low-confidence detection as a signal to keep both the original and a transliterated variant rather than committing to one. The dense/sparse weighting (0.6/0.4 here) is a starting point, not a law — tune it per corpus, and expect sparse to matter more when your documents are full of product codes, statute numbers or transliterated names. Reranking always scores against the original query text, because the cross-encoder is where cross-lingual noise gets corrected: a multilingual reranker such as BGE-reranker-v2 or Cohere Rerank-3.5 typically lifts recall by roughly 10 to 25 percent, and that lift is largest exactly where first-stage cross-lingual scores are weakest.

Handling code-mixed and low-resource languages

This is where multilingual RAG earns or loses its reputation. Two failure modes dominate, and both need explicit handling rather than hope.

Code-mixed queries. Hinglish — romanised Hindi peppered with English — is how a huge share of Indian users actually type. "Mera loan ka EMI kitna hai" is neither clean Hindi nor clean English, and a dense-only model will smear it. The fix is layered: detect the script, transliterate the romanised portion back to Devanagari so it can match native-script documents, and keep the original so English tokens like "EMI" and "loan" still hit via the sparse channel. BGE-M3's lexical component is what rescues these exact tokens; a purely dense pipeline would lose them. The same discipline applies to European code-mixing, where users drop English product names and error strings into French or German questions.

Script and Unicode normalisation. The same Tamil or Bengali string can arrive in several Unicode encodings, with combining marks ordered differently, or with visually identical characters from different code blocks. Without NFC normalisation your index and your query silently disagree and recall craters for reasons no dashboard will explain. Normalise on the way into the index and on the way in from the query, identically.

Realistic coverage targets help you scope this. Sarvam AI spans 22 Indian languages and the EU recognises 24 official languages — those two numbers frame the ceiling for each market. You will not evaluate all 46 equally on day one, but you should know which subset carries your traffic and treat the low-resource members of that subset as first-class citizens in testing, not stragglers you check later.

Evaluating per language

An aggregate retrieval score is the enemy of a working multilingual system. A pipeline that reports 0.82 recall overall can be hiding 0.94 on French and German and 0.55 on Tamil and Bengali — and the average looks fine while a fifth of your users get nonsense. The discipline is simple to state and easy to skip: build a per-language golden set and report metrics per language, never pooled.

For each language you serve, assemble twenty to fifty real queries in that language, each mapped to the document chunks that genuinely answer it (in whatever language those chunks are written — this is where cross-lingual pairs go). Then track recall@k and MRR per language, and set a floor below which a language is considered failing and gets targeted work. The decision table below summarises which technique to reach for when a specific language underperforms.

Symptom	Likely cause	Technique to apply
One low-resource language far below the rest	Thin shared-space alignment for that language	Per-language golden set; consider a specialist or per-language model for that language only
Romanised / Hinglish queries miss native-script docs	No transliteration bridge	Transliterate to native script; index a transliterated field; lean on sparse matching
Recall craters for no visible reason	Inconsistent Unicode across index and query	NFC normalise identically on ingest and at query time
Right documents retrieved but ranked low	Noisy first-stage cross-lingual scores	Add a multilingual cross-encoder reranker (BGE-reranker-v2 / Rerank-3.5)
Exact terms (codes, names) not matching	Dense-only retrieval blurs rare tokens	Enable BGE-M3 sparse channel; raise sparse weight for that corpus
Multilingual vs per-language model unclear	No head-to-head on your data	Benchmark both on the per-language golden set; keep multilingual unless a specialist wins clearly

The recurring choice in that table — multilingual model versus per-language model — deserves a clear default. Start multilingual: one model, one index, cross-lingual retrieval for free. Only split to a per-language model when your evaluation shows a specific high-traffic language is materially underserved and a specialist model beats the multilingual baseline on your golden set, not on a leaderboard. Splitting multiplies your operational surface, so make the model earn it.

Data residency for India and the EU

For a dual-market build this is not optional polish; it shapes your architecture. The EU's GDPR and the UK's own UK GDPR make any cross-border transfer of personal data something you have to justify, while India's Digital Personal Data Protection (DPDP) Act 2023 governs transfers through a government blacklist model — permitted except to notified countries — rather than a blanket localisation rule. A user's question is personal data, and its processing includes embedding and inference. Keeping that processing in-region is the cleanest way to stay on the right side of all three regimes and to avoid a cross-border transfer you would otherwise have to defend.

The clean pattern is region-pinned processing with request routing. Run your embedding and generation stack in AWS Mumbai for Indian users, AWS London for UK users and AWS Frankfurt for EU users, and route each request to the nearest compliant region rather than defaulting to a single global endpoint. This is one of the strongest arguments for open-weights models such as BGE-M3: you can host the model inside the region you need, instead of depending on where a third-party API happens to run. If you are designing the routing layer itself, our guide to data residency for AI apps under DPDP and GDPR covers the request-routing and storage patterns in depth.

The knock-on for retrieval is that your index, too, may need to be regional. If Indian and European user data cannot mingle, you may run parallel regional indexes — which is entirely compatible with the single-model, cross-lingual approach here, since the same BGE-M3 weights run in every region. Residency changes where the vectors live, not how you compute or match them.

Bringing it together

Multilingual RAG done properly is not a bigger model bolted onto an English pipeline. It is a sequence of deliberate choices: a multilingual embedding model with a genuinely shared cross-lingual space; a hybrid retrieve-and-rerank pipeline that fuses dense, sparse and cross-encoder signals; explicit handling for code-mixing and Unicode; per-language evaluation that refuses to hide behind an average; and region-pinned processing that respects DPDP and GDPR from the first request. Get those right and a single, clean index will serve a Tamil query and a German query with equal seriousness — which is exactly what building for India and Europe demands.