In an EMNLP 2026 Findings paper, Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMs, Joseba Fernandez de Landa, Carla Perez-Almendros, and Jose Camacho-Collados at the University of the Basque Country and Cardiff University test what language models do when a cultural question leaves the place blank. Instead of asking a multiple-choice question about a named country, they ask an open question such as "What values shape family life {in region/place}? Be brief. Choose yourself the place."
Their dataset, Culture-Related Open Questions (CROQ), has 1,320 questions across 11 domains and 66 subtopics, with 20 questions per subtopic covering beliefs, daily life, education, arts, food, geography, politics, health, media, history, and the economy. The questions were generated with GPT-5.1, reviewed by hand to remove accidental location hints, and translated into 24 languages for a total of 31,680 prompts. The authors ran eight frontier models through OpenRouter at temperature 0.7 and top-p 0.9: gpt-4o-mini, gemini-2.5-flash, claude-3.5-haiku, llama-4-maverick, command-r-08-2024, magistral-small-2506, deepseek-v3.2-exp, and qwen3-next-80b-a3b-instruct. A judge model, gpt-4o-mini, extracted up to five referenced countries per answer, reaching 98 percent accuracy on a 264-item human validation check.
The first pull comes from the input language itself. Across all 24 languages, 43 percent of answers on Command-R up to 78 percent on GPT-4o-mini name a country where the prompt language is an official language. Low-resource languages stay close to home and name fewer countries overall: Assamese names an official-language country 83 percent of the time across 41 distinct countries, Sundanese 79 percent across 62, Basque 78 percent across 68, and Amharic 77 percent across 38. High-resource languages roam much further: English prompts name an official-language country 31 percent of the time across 157 countries, and Russian 36 percent across 146. Across the 24 languages, the average number of distinct countries named correlates with each language's share of CommonCrawl pretraining text at Spearman's rs = 0.843 (p < 0.001).
Once own-country mentions are removed, the foreign answers pile up in the same places across models and languages. Five of the eight frontier models pick Japan more often than any other foreign country, one ties Japan with the United States, and two pick the United States first: Llama-4-Maverick names Japan 2,701 times against 1,074 for the United States, DeepSeek-V3.2-Exp 2,104 against 1,006, Qwen3-Next-80B 1,861 against 483, Magistral-Small 1,754 against 1,254, Claude-3.5-Haiku 1,601 against 1,200, and Gemini-2.5-Flash 1,493 for each. Only GPT-4o-mini (811 Japan, 944 United States) and Command-R (936 Japan, 2,064 United States) put the United States ahead. On English prompts, where the United States, India, and the United Kingdom count as own-country, Japan is the top foreign choice with 2,893 mentions across the eight models, ahead of China at 367. On Chinese prompts, Japan is first with 2,064 mentions, ahead of the United States at 1,460. Across the 11 domains, Japan ranks first or second in nine of them and drops only in Politics, where it is third behind the United States and China, and History, where it is fourth behind the United States, France, and Italy.
To see when that preference appears, the authors compare base models against instruction-tuned models on the 1,320 English questions across Llama-3.1 (8B, 70B), Gemma-2 (9B, 27B), Mistral-7B, Qwen2.5 (7B, 32B, 72B), OLMo-2 (7B, 32B), and OLMo-3 (7B, 7B-Think). Because OLMo-2 and OLMo-3 release intermediate supervised fine-tuning (SFT) checkpoints before preference alignment, those four models show the step-by-step shift. Base models spread their answers more evenly across countries and have higher normalized entropy on every model family tested. The sharp jump happens at SFT. On Olmo-3-7B, mentions of Japan rise from 35 in the Base checkpoint to 395 after SFT and 478 in the final Instruct checkpoint, while United States mentions move from 186 to 212 to 171, and unanswered prompts fall from 608 to 85 to 41 as normalized entropy drops from 0.74 to 0.66 to 0.64. On Olmo-3-7B-Think, Japan rises from 35 in Base to 453 in SFT and 386 in Instruct. On OLMo-2-32B, Japan rises from 61 in Base to 361 in SFT before settling at 221 in Instruct alongside 368 for the United States.
In Appendix I, on the last page of the paper, the authors check whether raw country counts in the open OLMo post-training datasets explain that jump. They do not. In Olmo-3-7B's SFT dataset (Dolci-Instruct-SFT), China is mentioned 52,509 times, Germany 42,349, the United Kingdom 41,682, France 26,091, the United States 24,325, and Japan 20,883. Yet the SFT model names China only 26 times (and 19 times after alignment) while naming Japan 395 times. In Olmo-3-7B-Think's SFT set (Dolci-Think-SFT-7B), China appears 133,953 times against Japan's 21,009, and the SFT model names Japan 453 times and China 40 times. In OLMo-2's preference sets, France is the most mentioned country of the eight tracked (8,110 mentions in the 7B preference mix and 16,128 in the 32B preference mix), yet the final OLMo-2-32B Instruct model names France 74 times against 368 for the United States and 221 for Japan. The authors end Appendix I by writing that their dataset count did not clarify the main "Why" question in their title.
Editorial
Counting how many times a country's name appears in an SFT dataset treats every mention as if it were doing the same job. A country named inside a news summary, a math word problem, or a translation pair teaches the model facts and vocabulary about that place. It does not teach the model which place to reach for when a user asks for "a brief cultural example" with the country left blank.
That blank is an instruction-following habit, and the paper's own topic table shows what kind of habit it is. Japan leads or sits second in Beliefs, Social life, Education, Arts, Food, Health, and Media. When the prompt turns to Politics or History, where examples involve protests, wars, and contested state narratives, Japan drops to third and fourth behind the United States, China, France, and Italy. In synthetic and human-written SFT data, when an assistant needs a short, recognizable, low-conflict illustration of a seasonal dish, a greeting, a neighborhood rule, or a school routine, Japan is the textbook specimen. Its customs are distinct enough to sound specific in two sentences and familiar enough across English and Chinese corpora that the example rarely trips a safety or politics filter.
The broader lesson is in the opposite movements of diversity and entropy in Table 4 and Table 15. Supervised fine-tuning makes a model much better at answering the prompt: on Olmo-3-7B, unanswered questions fall from 608 out of 1,320 in the base model to 85 after SFT, so the total count of distinct countries named across the run goes up from 114 to 145. At the same time, normalized entropy falls because most of those newly completed answers funnel into one or two default countries. Post-training does not widen a model's view of the world. It teaches the model to finish the assignment cleanly, and when the assignment asks for "a culture," the model hands back the safest postcard in its training mix.