Kimi K3 landed this week — Moonshot’s new flagship, a million-token context window, and the usual chorus: Chinese models are the cheap way to run AI now. I happen to keep a benchmark rig for exactly this claim. Not leaderboards — real work, with a bill at the end. So I gave the new arrival the same job I’ve given seven other frontier models this month. The “cheap Chinese AI” story survived exactly half the test.
The rig: same job, hard pass mark, only the model changes
The job is real: build a small leads CRM database on Supabase — three tables, security policies, triggers, seed data. Every model gets the identical spec, and nothing counts as done until supabase db reset && supabase test db comes back green. No vibes, no demo videos. Passing tests or it didn’t happen.
The trick is that an AI coding loop has two different jobs in it. The planner thinks once: it reads the spec and breaks it into tasks. The doer types: it grinds through every task, writing the actual code. You can point each job at a different model — and that’s where the money story gets interesting. I measure two numbers per run: what it cost all-in, and how many passing tests came out the other end.
The receipts: eight planners, one job
Same spec, same cheap doer typing the code. Only the planning brain changes. Prices are list price per million tokens (input/output); “all-in” is what the whole run actually cost me.
| Planner | List price /1M | All-in run | Tests passed | Cost per test |
|---|---|---|---|---|
| GPT-5.6 sol-pro | $5 / $30 | $1.73 | 113 | 1.5¢ |
| Gemini 3.1 Pro | — | $1.08 | 103 | 1.0¢ |
| GPT-5.6 luna | $1 / $6 | $0.92 | 101 | 0.9¢ |
| Kimi K3 (China, new this week) | $3 / $15 | $0.82 | 81 | 1.0¢ |
| Grok-4.5 | $2 / $6 | $0.74 | 92 | 0.8¢ |
| DeepSeek V4-Pro (China) | $0.43 / $0.87 | $0.65 | 83 | 0.8¢ |
| GLM-5.2 (China) | $0.92 / $2.89 | $0.58 | 43 | 1.4¢ |
| Claude Opus (workhorse tier) | — | $0.27 | 36 | 0.8¢ |
Read that middle column again. Kimi K3 is not the cheap option. At $3 in and $15 out it lists above Grok and luna — Western flagships — and its single planning call cost me 46 cents, thirteen times what luna’s cost, because “thinking” tokens bill at the full $15 output rate. It wrote a genuinely beautiful plan, the most detailed task notes of the whole field. It still landed mid-table on cost per passing test.
GLM-5.2 makes the opposite mistake: cheapest run of the reasoning tier, but its plan was thin — the most tasks and the fewest tests of any frontier model. A cheap plan that produces 43 tests where a rival produces 101 isn’t cheap. And Claude Opus at the bottom shows the other trap: lowest bill, decent cost per test, but a third of the coverage — the plan just asked for less. The honest Chinese value pick as a planner is DeepSeek V4-Pro: 83 passing tests at $0.65. Respectable. But it’s beaten on value by Grok-4.5, an American model.
The half that’s true: Chinese hands are unbeatable
Now the doer — the model doing the typing. Here the story flips completely. Earlier in this experiment I ran the identical plan through two different doers:
| Doer | List price /1M | Run cost | Result |
|---|---|---|---|
| DeepSeek V4-flash (China) | $0.10 / $0.20 | $0.27 | 36 tests, all green |
| Claude Sonnet | $3 / $15 | $5.20 | 36 tests, all green |
Nineteen times the price for the same green tests. DeepSeek’s little flash model has passed every gate I’ve put in front of it this month, at prices that round to nothing. For the grinding, typing, test-writing part of AI coding, the Chinese models really are the cheapest thing running, and it isn’t close.
Three traps the price sheet won’t tell you about
1. Reasoning models bill you for thoughts you never see. Kimi K3’s 46-cent plan is mostly hidden reasoning tokens, billed at $15 a million. Even my three smoke-test calls — each supposed to reply with one word — cost 51 cents total. With reasoning models, list price and real price are different currencies. This is the same quiet-bill mechanic that once cost me £700 in an afternoon, wearing a new hat.
2. Some benchmark stars return literally nothing through real tooling. Kimi K3, GLM-5.2, DeepSeek V4-Pro and the GPT-5.6 family all top coding leaderboards — and all of them come back empty when driven through the Claude Code CLI, because their answers arrive wrapped in reasoning blocks the tooling throws away. My rig auto-falls back to the raw API for planning, so they still work as planners. But a model that returns nothing in your harness doesn’t cost less. It costs everything. One-line check before you believe any leaderboard: claude -p "Reply with exactly: WORKING" --model <id>. Empty reply, disqualified.
3. The routing rots under you. On cheap aggregator routes, the provider behind a model id can change mid-day. A model that answered cleanly this morning went dark by afternoon — same id, different backend. Cheap needs re-verifying, per run, not per month.
What I actually run now
The winning recipe from eight rounds of this: brilliant plan, cheap hands. A frontier reasoning model plans once — Grok-4.5 or GPT-5.6 luna, both under a dollar all-in — and DeepSeek’s flash model types everything, with the test gate keeping it honest. That combination gets me roughly 100 passing tests for under a pound. The all-premium version of the same run cost me $7.41 and produced fewer tests.
So: is Chinese AI the cheapest? For the hands, yes — emphatically, 19-to-1 yes. For the brain, not this month: the newest Chinese flagships price like Western ones and lose on value, and the one that undercuts everyone writes the thinnest plans. Buy the roles separately, measure cost per verified output instead of cost per token, and smoke-test through your own tooling before trusting anyone’s leaderboard — including mine.
