Ask in Hinglish, get a different answer

01 / 08Summary

Many Indians type their questions in Hinglish, Hindi written in the Latin alphabet, or in Hindi itself. We wanted to know whether AI answer engines treat those as the same question. On 5 October 2026 we put 50 everyday questions, each written in English, Hinglish and Hindi, to ChatGPT, Perplexity, Gemini and Google AI Mode from India, and asked 20 of them a second time on two engines to measure how much an answer changes on its own: 720 answers and 4,817 citations.

Five results stand out. The script a question is typed in changed the shortlist, the country the answer was written for and the pages behind it.

02 / 08Why this matters

Brands that track how AI engines describe them usually test in English. Their customers do not all ask in English. If a Hinglish question returns a different shortlist, an English-only test describes only part of the market, and a brand can look present in the report while it is absent from the answers many buyers read.

Language also decides which pages can be cited. An engine that answers in Hindi but cites English pages, or cites a machine translation of them, rewards a different kind of publishing than one that looks for Hindi pages.

Asked in English from India, Gemini answered almost a third of the questions as if the asker lived in the United States.

Study 009, Finding 2

03 / 08Method

We wrote 50 recommendation questions across eight topics, each in three versions: English; Hinglish, typed the way people type it on a phone; and Hindi in Devanagari, the way phone or voice typing produces it. The three versions mean the same thing.¹ Each version went to four engines from an Indian connection, once. Twenty questions, chosen at random within topics, were then asked again in all three versions on ChatGPT and Google AI Mode, straight after the first run, to give a run-to-run baseline.²

Questions

508 topics × 3 versions = 150 prompts

Answers

720600 in run 1, 120 in the repeat run

Citations

4,8173,865 in run 1 · 2,764 unique URLs

Engines

ChatGPT (search on), Perplexity, Gemini, Google AI Mode

Market

India, set at the network level; no country in the prompt

Fieldwork

5 October 2026, 11:14–12:19 UTC

What we counted

A citation is a URL the engine presented as a source for the answer, counted once per answer.³ A brand is named when its name appears in the answer text. We matched names with a hand-built list for each question, 1,348 entities in all, with Latin and Devanagari spellings, so that “Physics Wallah” and “फिजिक्स वाला” count as the same institute.⁴

Brand change between two answers is the share of brands in the first that are missing from the second, and the overlap of the two lists (Jaccard).⁵ We also recorded the language of each reply,⁶ whether the answer was written for India or for another market,⁷ whether each cited page is in Hindi,⁸ and the type of each cited source.⁹

Disclosure: some brands that appear in this study are clients of Citation Research. Their results were produced by the same questions, engines and rules as every other brand’s.

04 / 08Findings

1. Hinglish changes the shortlist more than chance does

On the 20 questions we asked twice, a plain re-ask in the same language left out 27% of the brands the first answer had named (188 of 698). Switching the question from English to Hinglish left out 42% (102 of 245). Switching to Hindi in Devanagari left out 34% (84 of 245).

Measured as overlap between the two lists, the Hinglish gap beyond ordinary run-to-run change is 0.12 on a scale of 0 to 1 (95% interval 0.05 to 0.19). The Hindi gap is 0.02 (interval −0.05 to 0.09): within the noise. On these two engines, Devanagari Hindi changed the brands no more than asking again did; Hinglish did.

Across all four engines and all 50 questions, the overlap between language versions ranged from 0.271 to 0.618 (Table 2). Gemini’s English answers shared the least with its other versions, partly because of the next finding: with its US-market answers removed, Gemini’s English-to-Hinglish overlap rises from 0.279 to 0.403. We did not repeat questions on Perplexity or Gemini, so their numbers have no baseline.

The sources moved further than the brands. On the same 20 questions, two runs of one version shared 0.37 of their cited domains; English and Hinglish versions shared 0.21, a gap of 0.16 (interval 0.10 to 0.22).

2. In English, Gemini often answered for the United States

Every request came from an Indian connection. Even so, 16 of Gemini’s 50 English answers were written for the US market: American credit cards, health insurers, telehealth apps, broadband providers and federal schemes. One placed the asker in Philadelphia. Perplexity did the same in 4 of 50. ChatGPT and Google AI Mode did not, in 50 each. Across all four engines, none of the 400 answers to Hinglish or Hindi questions was written for another market.

The US answers were concentrated in money, health and public services. That is visible in the topic table further down: personal finance and health show the lowest overlap between English and Hinglish answers.

3. Ask in Hinglish, get Hindi

English questions were answered in English every time (200 of 200), and Hindi questions in Hindi in 198 of 200. Hinglish was where engines disagreed. Of 200 answers to Hinglish questions, 103 came back in Hinglish, 90 in Devanagari Hindi and 7 in English, all seven from Perplexity. Google AI Mode switched to Devanagari 35 times out of 50; ChatGPT, 8 times.

4. Hindi answers, mostly English sources

For Hindi questions, 31% of citations went to Hindi-language pages (399 of 1,287), against 0.4% for English questions (5 of 1,254). The engines split sharply. Perplexity’s Hindi answers cited Hindi pages 71% of the time. ChatGPT wrote all 50 answers in Hindi and cited Hindi pages 3 times in 218.

Google AI Mode did something else. Of its 499 citations on Hindi questions, 134 (27%) were English pages routed through Google Translate and shown to the reader in Hindi. The originals were spread across many sites, from Google Play listings to Quora threads. Translated pages appeared in no other engine’s answers.

5. Fewer sources on Hindi questions, but only on Perplexity

Across all engines, the number of citations barely moved: 6.3 per answer for English, 6.6 for Hinglish and 6.4 for Hindi. One engine stood apart. Perplexity cited 7.6 sources per Hindi answer against 10.0 per English answer, 24% fewer; it gave fewer in 41 of 50 question pairs and more in none. Gemini and Google AI Mode went the other way and cited more on Hindi questions.

Answers were about the same length in all three versions: 290, 275 and 294 words on average for English, Hinglish and Hindi, though words are not a precise unit across scripts.

6. Hindi questions cite more video

The mix of source types shifted with the language. Video, all of it YouTube, rose from 3.3% of citations on English questions to 6.5% on Hinglish and 9.9% on Hindi (41, 86 and 127 citations). Directories and marketplaces rose from 13.0% to 16.8%. Brands’ own sites slipped from 31.5% to 28.8%.

By topic

Questions about exams and coaching kept the most brands across languages; personal finance and health kept the fewest, where several English answers recommended American products.

05 / 08What it means for brands

None of this says one language is better. It says the language of the question is part of the question.

1. Test in the languages your buyers type.On the two engines we could measure, 42% of the brands in an English answer were gone from the Hinglish one. Hinglish deserves its own test.
2. Compare with a re-ask before calling it a trend.More than a quarter of brands (27%) dropped out when nothing about the question changed. Report language gaps against that baseline, not against zero.
3. Know which engines read Hindi pages.Perplexity cited Hindi pages for most Hindi questions; ChatGPT almost never did. A Hindi page helps on one and may not on the other.
4. State the market plainly.English questions sometimes got American answers. Pages that say who they are for, with rupee prices and Indian availability, leave an engine less to guess.

06 / 08Limitations

• One day, one run. Fieldwork took about an hour on 5 October 2026. Engines change without notice; another week may differ.
• A baseline on two engines only. The repeat run covered 20 questions on ChatGPT and Google AI Mode. Language effects on Perplexity and Gemini are reported without one.
• One translation per question. People spell Hinglish many ways and mix scripts. A different spelling might get a different answer.
• Location is approximate. Requests came from Indian connections, and ChatGPT sometimes localised to a city. We cannot see what location each engine actually used.
• Matching and coding errors. Our hand-check of 40 answers found 11 brand-matching errors in 234 names, all fixed; some remain elsewhere. Hindi-page flags rest on titles and URLs, and 165 of 593 rest on the title alone.
• We observe, we do not explain. Nothing here shows why an engine answered as it did.

07 / 08Data and reproducibility

The question list, every answer’s metadata, every counted citation and the coding tables are published under CC BY 4.0. Version 1.0 corresponds to this publication; corrections will be issued as new versions with a changelog.

• questions.csv50 rowsTranslation table
• answers.csv720 rowsLanguage, length, brands named
• citations.csv4,817 rowsType, Hindi flag, translation flag
• overlap-pairs.csv717 rowsEvery compared pair
• entities.csv1,348 rowsBrand list with spellings
• market-coding.csv600 rowsMarket each answer was written for
• domain-types.csv1,328 rowsSource type by domain

For questions about the data, or a reanalysis you would like us to check, write to research@citationresearch.com.

@techreport{citationresearch2026study009,
  author      = {{Citation Research}},
  title       = {Ask in Hinglish, get a different answer},
  institution = {Citation Research},
  number      = {Study 009},
  year        = {2026},
  url         = {https://citationresearch.com/research/hinglish-hindi-ai-answers}
}