Methodology

Does Thinking Twice as Long Make a Go AI Twice as Good?

Our Go engine uses Monte Carlo Tree Search: to pick a move it plays out thousands of fast random games and keeps the move that wins most. The obvious knob is how many playouts it runs per move — so the obvious question is what you actually buy by turning it up. We measured it, on a 9×9 board, with a headless self-play harness that plays the shipped engine against weaker copies of itself, thousands of games, no human at the keyboard.

 ·  7 min read  ·  all numbers from a re-runnable, seeded benchmark

The Numbers

+233
mean Elo per doubling of playouts
171→357
Elo gain, first vs last doubling
600
headless self-play games
41%
Black win rate under 7.5 komi

Strength climbs — and keeps climbing

0 300 600 900 1200 50 100 200 400 800 1600 +1167 Elo cumulative Elo playouts per move (log scale) → lkforge.com

What each doubling is worth

0 90 180 270 360 +171 50→100 73% wins +162 100→200 72% wins +180 200→400 74% wins +297 400→800 85% wins +357 800→1600 89% wins Elo gained lkforge.com

Each bar is a direct head-to-head: the stronger budget against the next one down, 100 games, with the two engines swapping colours halfway so Black's first-move edge and the 7.5 komi cancel out. Win rate above 50% converts to an Elo gap. The bars grow left-to-right — the opposite of the usual diminishing-returns curve. A plausible read: at 50–200 playouts the search is still too noisy to convert its reading into the right move, so the extra rollouts are half-wasted; from 400 playouts up it reliably reads out the captures and life-and-death that actually decide a 9×9 game, so each doubling unlocks more, not less. Saturation must arrive eventually — but not by 1,600 playouts a move.

Why We Measure Playouts, Not Seconds

The live game gives its AI a time budget — roughly a second per move on the 9×9 board — because that's what keeps the page responsive. But a time budget is the wrong unit for a benchmark: "0.9 seconds of thinking" buys a fast laptop three times as many playouts as a phone, so a strength number tied to milliseconds would say more about the reader's hardware than about the engine. Playouts per move is the hardware-independent knob. Fix the playout count and the same seed produces the same games on any machine, which is the whole point of publishing a number someone else can check.

One honesty note up front: this is an honest-amateur engine, strongest on 9×9. The Elo figures here are internal — they measure the engine against itself at different budgets, not against human ranks or a professional program like KataGo. A 9×9 result also won't transfer unchanged to 19×19, where the search space is far larger and every playout costs more.

The Harness

The benchmark imports the exact mcts() function the browser ships — same rules, same eye-aware playouts, same scoring — and drives it with no DOM and no human. For each doubling step it plays 100 games between two playout budgets, alternating which side is Black so that first-move advantage and komi don't leak into the result. Games run to two consecutive passes and are scored by area (Chinese) rules with 7.5 komi, so there are no ties to adjudicate. The random number generator is seeded, so every game — and therefore every Elo figure on this page — reproduces exactly.

Reproduce this benchmark Run it yourself →
  • Engine: the same MCTS engine that runs at lkforge.com/games/go/ (commit a4ab763), 9×9, komi 7.5, area scoring.
  • Elo ladder: budgets 50, 100, 200, 400, 800, 1600 playouts/move; each adjacent pair played 100 colour-balanced games. Expected: stronger side wins 73% at the first doubling, rising to 89% at the last.
  • Balance check: 100 games at 800 playouts a side, fixed colours. Expected: Black wins ≈ 41% under 7.5 komi.

Seeded (mulberry32, master seed 1234567); counts are deterministic per seed and stable across re-runs. Elo per pairing = 400·log₁₀(p/(1−p)) from the colour-balanced win rate p.

Is 9×9 Go Fair? A Side Question the Harness Answers

Black moves first in Go, which is an advantage, so White is compensated with komi — 7.5 points added to White's score. Whether that number balances the game is exactly the kind of thing self-play can measure. Holding both sides at 800 playouts and fixing the colours, Black won 41% of 100 games — so at this strength on 9×9, 7.5 komi slightly over-compensates, leaving a small edge to White rather than Black. Close to fair, not a landslide. It's a modest sample, so read it as a pointer, not a verdict.

Related reading: why our game engines are search algorithms and not neural nets, in Your Game Opponent Isn't an LLM; the three classic search algorithms behind the rest of the board games, in Six Games, Three Classic Algorithms; and the same measure-before-you-claim discipline applied to tic-tac-toe and 2048, in How We Benchmark a Game AI.

Play the Engine

The same MCTS engine these numbers came from, live in your browser — 9×9, 13×13 and 19×19, three strength levels, no sign-up.

Play Go →