The three comparisons at the heart of the DFS study, worked through: fill order (row-major wins, a bad order is catastrophic), heuristics (MRV rescues border-first but costs throughput; more propagation did not help), and breaks (they smash the depth wall; the break schedule is the lever, not the per-cell cap).
Three comparisons carry the DFS study. Each isolates one decision by holding everything else fixed. All scores are the mean matched-edge count over the ten corner-pinned variants, single core, sixty seconds. Throughput is search-nodes per second and is never compared across families.
The two baselines run the same strict row-major algorithm: NAIVE-CLEAN as a
readable general engine, NAIVE-CODEGEN as a hand-specialised, row-major-only
16×16 hot loop. The specialisation delivers what it should on the axis it targets.
NAIVE-CODEGEN is meaningfully faster per node, by up to a third on some
instances. On score the two are a wash, a point or two apart at sixty seconds
and well inside the run-to-run spread, which is the fair reading rather than a
claim that specialisation hurts. At a fixed wall-clock a faster engine reaches a
different point of the same tree, and a backtracker's best partial does not move
monotonically with how fast it got there. The baselines are therefore a clean
speed comparison and are deliberately not presented as a score comparison. The
score effects worth studying are all on the axes below, where the search itself
changes.
Fix the engine (strict, no heuristics) and change only the order cells are filled. The six orders differ only in where the search sends its frontier, shown below.
Same engine, same sixty seconds, a swing of more than 300 points from the fill order alone. This is the measured form of a piece of community wisdom: border-first is only good with a heuristic to choose cells within the ring. On its own it is one of the worst orders available.
Test the lever yourself: pick any of the engine's nine scan orders below and watch the same real backtracker reach a different depth on the same puzzle.
Now fix the path near the frame and add one thing at a time. The largest lever is MRV, filling the most-constrained empty cell next, chosen dynamically. It turns the stalled border-first (mean 67) into a search that averages 324 (best 341) at depth 190 or so. It also recomputes the most-constrained cell over the whole frontier at every step, so node throughput falls by three orders of magnitude, from tens of millions of nodes per second to a few thousand. Each node is worth far more, and far fewer of them are visited. Node rate and score are different axes.
Adding heavier look-ahead on top did not buy more score at this budget.
The lesson is not that propagation is useless. It is that at a small fixed budget, on this instance, the cheapest useful prune (forward-checking) already captures whatever benefit is available, and more expensive reasoning does not recover its extra per-node cost within sixty seconds.
Strict backtracking, whatever its order or heuristic, hits a wall well short of a
full board: row-major tops out around depth 208 of 256, and even the fastest
strict variant (NAIVE-CODEGEN) only reaches 216. The record engines get past it
by breaking: allowing a bounded number of interior edge mismatches,
released on a depth schedule, with a rule that no cell may carry too many broken
edges. The score of a full board is then 480 − #breaks.
These break numbers sit alongside the sibling benchmark's from-scratch Blackwood-style and Verhaard-style reimplementations, which reach the high 430s on the same five clues. The agreement from an independent engine cross-validates the break machinery here.
Blackwood and McGavin are the high end of this same family, and both build and run on this machine, so this study ran them on both a pinned and an unpinned grid, with every score canonically rescored from the engine's own board. The result is a finding in its own right, and it has two halves.
On the pinned grid, they collapse. McGavin's C, built with its author's own ARM flags (native tuning plus link-time optimisation), reaches depth 211 on the plain centre-clue puzzle at about 85 million tiles per second, well ahead of our fastest strict engine, yet pinning three corners collapses it to depth 21, a canonical score of 13. Its generated scan path never visits the corners early, so a pinned corner constrains its neighbourhood at once and dead-ends the fixed path almost immediately. Blackwood's C# hardcodes its piece set and scan, so it cannot even express an arbitrary corner pin; on the matching five-clue constraint its break heuristic thrashes to depth 47, a canonical score of 75, because it is tuned for the near-unconstrained one-clue instance where its 470 record was set.
That collapse is the point. A record engine built around one clue configuration does not transfer to another, and a fixed scan path cannot absorb an arbitrary pin. Our from-scratch engines treat a pin as a pre-placed cell the scan simply skips, which is why they, rather than the community binaries, are the stand-ins on the pinned grid.
On a fair unpinned grid, they run as designed. Give every engine the official pieces with only the one mandatory centre clue, and nothing dead-ends on a corner. In sixty seconds on one core, McGavin reaches 392, our strongest break engine 344, the Verhaard reimplementation 286, and Blackwood 214. Read those as what a single core buys in a minute from a cold start, not as any engine's ceiling: Blackwood's 470 and McGavin's deep runs came from days on hundreds of cores, which this budget cannot show. What the grid does show is that all four run properly once the pins that break a fixed scan path are gone, which is exactly what the pinned grid denied the two foreign engines. Both grids are on the study's leaderboard above.
One theme runs through all three comparisons: node count is not score. The fastest variants per node (the codegen baseline and the strict row-major engine) do not reach the best boards; the slowest per node (MRV with propagation) reaches far better ones; and the breaks that win do so by changing which branch is legal, not by visiting branches faster. Raw speed is a constant factor, while where and whether the search is allowed to go is the exponential one.