How accurate is it?
Each test is a hypothetical game with comparable games. Truth is Google Trends interest by country for a real shipped game like it, which is never used as a comp. Every method ranks the same candidate countries. ✓ marks a correct #1.
Held-out set: 23 games with no home-country setting
Written and committed before it was ever scored, with the scorer frozen. Crowdspawn lost to an LLM with no Qloo data here. Its #1 city was Stockholm for 9 and Istanbul for 6 of the 23 games, whatever the genre. Removing that city bias afterwards did not close the gap (section 4 of docs/scenario-results.md).
| Method | Trends #1 is our #1 | Trends #1 in our top 3 | Mean ρ |
|---|---|---|---|
| Crowdspawn (Qloo spawn score) | 1 of 23 | 1 of 23 | 0.11 |
| LLM only (gemma4:31b) | 4 of 23 | 7 of 23 | 0.54 |
| Popularity (all gamers, no game signal) | 0 of 23 | 1 of 23 | -0.33 |
| Hypothetical game | Real game used as truth | Trends top 3 | Crowdspawn top 3 (#1 city) | LLM-only top 3 |
|---|---|---|---|---|
| Jackpot Run: a roguelike deckbuilder about chasing huge scoring combos | Balatro | CA, KR, AU | SE, PL, DE (Stockholm) | PL, SE, CA |
| Dead Drop: a hardcore extraction shooter where you keep only what you carry out | Escape from Tarkov | KR, SE, CA | TR, IN, ID (Delhi) | PL, CZ, SE |
| Conveyor: a factory automation game about building ever larger production lines | Factorio | CZ, SE, DE | TR, PL, SE (Istanbul) | DE, PL, SE |
| Gridlock: a modern city builder about traffic, zoning and services | Cities: Skylines | KR, SE, PL | GB, AU, SE (London) | DE, PL, CZ |
| Deep Shelf: an underwater survival and crafting game | Subnautica | CZ, AU, CA | ID, TR, IN (Jakarta) | SE, CA, AU |
| Pulse Cutter: a VR rhythm game where you slash blocks to the beat | Beat Saber | CZ, CA, US | SE, CA, PH (Stockholm) | JP, KR, US |
| Apex Line: a realistic circuit racing sim for car lovers | Gran Turismo 7 | JP, DE, AU | SE, CA, GB (Stockholm) | DE, GB, SE |
| Knockdown: a 3D one-on-one fighting game | Tekken 8 | KR, PH, AU | GB, SE, DE (London) | JP, KR, US |
| Last Light: asymmetric multiplayer horror, four survivors against one killer | Dead by Daylight | JP, KR, US | PL, AR, CZ (Warsaw) | BR, MX, PH |
| Turbo Goal: arcade ball games played with rocket-powered cars | Rocket League | FR, GB, AU | TR, ID, IN (Istanbul) | BR, TR, PL |
| Drop Zone: a realistic 100-player battle royale shooter | PUBG: Battlegrounds | KR, TR, IN | TR, IN, ID (Delhi) | BR, PH, TR |
| Starfall Chronicles: an open-world action RPG where you collect characters through gacha banners | Genshin Impact | JP, KR, PH | ✓ JP, CA, US (Tokyo) | ✓ JP, KR, SG |
| Tap Empire: an idle incremental game about making numbers grow | Cookie Clicker | SE, AU, US | TR, IN, ID (Istanbul) | SG, KR, JP |
| Lantern Deep: a hand-drawn metroidvania about exploring a ruined underground kingdom | Hollow Knight | KR, MX, CA | SE, DE, PL (Stockholm) | PL, SE, CA |
| Ashen Crown: an open-world dark fantasy soulslike | Elden Ring | CA, KR, US | SE, PL, JP (Stockholm) | PL, KR, JP |
| Squad Zero: turn-based squad tactics against an alien invasion | XCOM 2 | KR, SG, CZ | SE, PL, DE (Stockholm) | PL, CZ, DE |
| Wrench Time: a garage sim about repairing and restoring cars | Car Mechanic Simulator 2021 | PL, CZ, SE | SE, JP, CA (Stockholm) | ✓ PL, CZ, DE |
| Dig Deep: a 2D sandbox adventure of digging, building and boss fights | Terraria | KR, PH, AU | TR, IN, ID (Istanbul) | SE, PL, CZ |
| Wobble Rush: a chaotic party game where dozens of players race through obstacle courses | Fall Guys | KR, JP, MX | CA, IN, ID (Toronto) | US, KR, JP |
| Far Drift: space exploration and survival across endless planets | No Man's Sky | CA, US, AU | SE, CZ, PL (Stockholm) | SE, CZ, PL |
| Breach Point: a 5v5 tactical shooter with hero abilities | Valorant | KR, PH, SG | TR, IN, PL (Istanbul) | ✓ KR, PL, TR |
| Lane Clash: real-time card battles in three-minute mobile matches | Clash Royale | MX, AR, ES | TR, IN, ID (Istanbul) | KR, JP, PH |
| Harvest Machine: a realistic farming sim with heavy machinery | Farming Simulator 22 | PL, CZ, DE | SE, JP, GB (Stockholm) | ✓ PL, CZ, DE |
Tuning set: 10 games
The scorer was designed on these games, so they are a sanity check, not a test. Half have a real-world setting, where culture anchors carry the signal. One truth game, Mumbai Gullies, was cancelled and never shipped; without it the results are: Crowdspawn (Qloo spawn score) 5 of 9, LLM only (gemma4:31b) 5 of 9, Popularity (all gamers, no game signal) 0 of 9.
| Method | Trends #1 is our #1 | Trends #1 in our top 3 | Mean ρ |
|---|---|---|---|
| Crowdspawn (Qloo spawn score) | 6 of 10 | 8 of 10 | 0.45 |
| LLM only (gemma4:31b) | 6 of 10 | 7 of 10 | 0.61 |
| Popularity (all gamers, no game signal) | 0 of 10 | 0 of 10 | -0.43 |
| Hypothetical game | Real game used as truth | Trends top 3 | Crowdspawn top 3 (#1 city) | LLM-only top 3 |
|---|---|---|---|---|
| Delhi Gullies: open-world crime game set in the streets of Delhi | Mumbai Gullies (cancelled, never shipped) | IN, US, CA | ✓ IN, AE, SG (Mumbai) | ✓ IN, AE, SG |
| Neon Kabukicho: a story-driven street brawler set in Tokyo's nightlife district | Yakuza (series) | JP, KR, SG | ✓ JP, KR, SG (Tokyo) | ✓ JP, KR, US |
| Leshy: a dark fantasy RPG built on Slavic folklore | The Witcher 3: Wild Hunt | PL, CZ, SE | ✓ PL, JP, CA (Warsaw) | ✓ PL, CZ, DE |
| Hanok Online: a fantasy MMORPG from a Seoul studio | Lost Ark | KR, CA, SG | PH, SG, KR (Manila) | ✓ KR, PH, ID |
| Sultan's March: a medieval sandbox of battles and politics in Anatolia | Mount & Blade II: Bannerlord | TR, KR, SE | ✓ TR, AE, IN (Istanbul) | ✓ TR, AE, ID |
| Moss Hollow: a cozy farming and village life sim | Stardew Valley | CA, KR, PH | JP, GB, CA (Tokyo) | JP, KR, US |
| Swarmline: a cheap, fast bullet-heaven roguelite | Vampire Survivors | JP, KR, CA | ✓ JP, SE, GB (Tokyo) | PL, KR, JP |
| Lane Legends: a 5v5 mobile MOBA with 10-minute matches | Mobile Legends: Bang Bang | PH, ID, SG | TR, PL, CZ (Istanbul) | ✓ PH, ID, KR |
| Touchline: a deep football club management sim | Football Manager (series) | KR, TR, GB | TR, IN, ID (Delhi) | GB, DE, ES |
| Long Haul: a relaxing long-distance trucking sim | Euro Truck Simulator 2 | TR, PL, CZ | ✓ TR, PL, IN (Istanbul) | PL, CZ, DE |
Audience check: Qloo's gender skew vs a player survey
For 10 genres, Qloo's female index for a panel of well-known games against the female share of each genre's players in Quantic Foundry's 2017 survey (about 270,000 gamers). Design fixed before any call. Spearman ρ 0.997; an LLM's guess (gemma4:31b) 0.991, which may be recall of this published survey. Qloo measures the skew for any comp set, including ones no survey covers.
| Genre | Survey: % female | Qloo female index |
|---|---|---|
| Match 3 | 69 | +0.37 |
| Family/Farm Sim | 69 | +0.50 |
| Casual Puzzle | 42 | +0.10 |
| Atmospheric Exploration | 41 | +0.10 |
| High Fantasy MMO | 36 | -0.08 |
| Sci-Fi MMO | 16 | -0.28 |
| Fighting | 13.4 | -0.29 |
| Racing | 6 | -0.39 |
| Tactical Shooter | 4 | -0.43 |
| Sports | 2 | -0.48 |
Partner check: Qloo's lists vs an LLM's
For 20 games, an LLM (gemma4:31b, no Qloo data) and Qloo each name 6 artists, podcasts, films and brands that fans of the game's comps like more than gamers in general. "Generic" is the share of names a method gives for 3 or more of the games, whatever the game: it doesn't use Qloo's scores. Lift for the LLM's names is measured with Qloo, which picked its own list by lift, so that column favours Qloo by design. Design fixed before any call.
| Type | Generic names: Qloo | Generic names: LLM | LLM names found in Qloo | Median lift: Qloo | Median lift: LLM's names |
|---|---|---|---|---|---|
| artist | 5% | 32% | 115 of 120 | ×1.096 | ×1.006 |
| podcast | 20% | 32% | 51 of 103 | ×1.126 | ×0.988 |
| movie | 3% | 15% | 107 of 113 | ×1.108 | ×0.998 |
| brand | 67% | 51% | 88 of 119 | ×1.023 | ×0.986 |
Caveats
- Google Trends measures search interest share per country, not downloads, wishlists or sales.
- Truth is country level; Crowdspawn ranks cities, averaged per country here.
- Small samples: one game moves a hit rate by several points.
- Each LLM ran once, at temperature 0, on one prompt.
- Method, code and every number:
docs/scenario-results.md,docs/backtest-results.md,docs/audience-check.mdanddocs/partner-check.mdin the repository.