# What the study found

> The results, worked through: on these boards the five clue-shaped hints never help a backtracker, they range from a mild cost to a catastrophe, and the fill order decides how much damage they do; the scores are bimodal, not a smooth gradient; and the hint-count question is confounded by a free pinned-seam floor.

- Canonical page (with interactive figures/demos): https://eternity2.dev/research/lab/experiments/raphael-anjou/hint-study/findings/
- Updated: 2026-07-21
- Topics: backtracking, structure, search-space
- Source: Committed per-run results (results.jsonl) and the analysis script — https://github.com/raphael-anjou/eternity2/tree/main/research/experiments/hint-study

---
The study fixes five hints in the shape of Eternity II's own five clues, runs eight
fill orders on the identical boards, and asks a simple question: what are those five
hints worth? Every number below is over fifteen distinct generated instances,
single core, eight seconds a run, re-scored out of 480 by one canonical
scorer. The comparison is paired, every fill order sees the same boards, and the
per-instance spread is shown rather than averaged away, because it turns out to
matter more than any median.

## The hints don't help, and the fill order decides how much they hurt

Start with the distribution. Each dot is one board; the tick is the median.
> **[Interactive: HintStudyCharts]** Rendered on the canonical page (link above); not shown in this markdown export.
Two things are visible at once. First, the scores are **bimodal**: on most orders a
board either climbs into the 360s or stalls in the double digits, with little in
between. A
median drawn through that is a summary of a gap, not a centre, which is exactly why
the dots are shown. Second, the orders split hard: the three compact sweeps
(row-major, its bottom-up mirror, Verhaard's comb) sit high on most boards; the
fragmenting orders sit low throughout.

But the compact-beats-fragmenting split is not the real finding, because it invites
the wrong question, "which order best *uses* the hints?" The question that matters
is whether the hints help *at all*. The second chart answers it, and the answer is no.
It shows the **paired change** from adding the five hints: each order's score on a
board minus its score on the *same board with no hints*.

Every bar is at or below zero. The five clue-shaped hints do not help a single fill
order. On the compact sweeps they cost only ten to twenty points. On spiral-out they
cost about ninety. And on the two hint-seeking orders they are ruinous: the
`trace-hints` order, which draws a skeleton between the clues before filling, loses
about three hundred and twenty-five, and the `connect-hints-first` flood loses
roughly **three hundred and forty-five**, taking an order that scores among the
*best* of all eight with no hints to the *worst* with them. The more deliberately an
order chases the hints, the more they cost it. Handing the solver five correct
pieces, in the puzzle's own clue geometry, made every version of it worse.

## Why a correct hint hurts

A pinned piece is not free information to a chronological backtracker; it is a **hard
constraint the fixed fill order must satisfy on arrival**. When a compact sweep rolls
down to a pinned interior cell, the piece is already there, and the row it just built
has to match that piece's edges. Most of the time it can, at a small cost: the
sweep steers around the constraint and loses a few tens of points. But on some boards
the pinned piece contradicts what the frontier has committed to, and there is no
local repair: the search hits a wall it cannot pass and thrashes below it. That is
the stalled mode, and it is the hints that create it. On row-major, the boards that
collapse into the double digits are precisely the ones where a clue-shape pin lands
where the sweep cannot honour it; the same board with no pins climbs into the 360s
and 370s.

This reframes the fill-order result. The order still matters (a compact sweep
survives the hint constraints with a scar of ten to twenty points while
`connect-hints-first` is destroyed by them), but what the order is buying is not
"using the hints well." It
is **surviving them**. The frontier is why: an order that keeps a single tight
frontier has room to route around a bad pin; an order that has already fragmented
into five open blobs has committed everywhere at once and cannot.

That frontier relationship, across the eight orders, is strikingly clean, and it is
worth showing precisely because the frontier can be computed from an order's geometry
with no solver at all, then checked against the measured scores:
> **[Interactive: FrontierPhaseChart]** Rendered on the canonical page (link above); not shown in this markdown export.
The average open frontier an order holds predicts its median score closely across
these eight orders. It is a strong descriptive relationship, not a law proved on
eight points, and it stops settling the ranking among the fragmenting orders on the
right (spiral-in holds a larger frontier than spiral-out yet scores higher). But the
direction is exactly what the mechanism predicts: branching cost is multiplicative in
the frontier, so an order that keeps the frontier small keeps the room to absorb a
hostile pin.

For contrast, the beam solver, which is not a chronological backtracker and does not
pay the frontier cost the same way, reaches a median in the 450s on these same
hinted boards. The hints and boards are nowhere near unsolvable. It is specifically
the *chronological, fixed-order* backtracker that cannot turn five correct pieces
into progress.

## Count: a threshold, not a gradient, and a floor that hides it

The same question one level out: does adding *more* hints help? The answer is not a
smooth "more is better", and the raw number hides which part is real.

The lower chart above splits each layout's score into two parts. The **floor** is
the seams the pins complete for free, because both their endpoints are pinned to the
true solution; the **earned** part is what the search actually found. A solid
clustered block banks a tall floor (five 4×4 blocks pin a quarter of the whole
board's seams before the search takes a single step), while a spread lattice, whose
hints never touch, banks nothing. So a raw-score comparison hands clustered layouts
a hundred-point head start that says nothing about whether the search made headway.

Read the earned column across the spread lattices, whose floor is zero so earned
*is* the score, and a threshold appears. A sparse spread, four to sixteen hints,
earns almost nothing (in the twenties): the pins are just scattered constraints the
sweep keeps tripping over, exactly the five-clue effect. But keep adding them and
the picture flips. Twenty-five spread hints earn 127, and thirty-six, a six-per-line
lattice, **solve the board outright on most instances**. Below the threshold the
spread hints only get in the way; above it there are finally enough of them to
carve the board into pieces small enough for the sweep to finish. It is not a
gradient of help, but a wall the count has to clear.

The clustered blocks show the mirror image. Their raw score is mostly *floor*: five
4×4 blocks (eighty hints, a quarter of the board) do solve every instance, but they
have pinned so much of the board that they have half-solved it by hand. Strip the
floor away and the clustered layouts short of that extreme earn only double digits,
coasting on the free seams. So the real count story is not "more hints help" or
"more hints hurt", but: it takes a *lot* of correct hints, spread or clustered, to
move a chronological backtracker at all, and until you reach that amount the extra
pins are as likely to trip the search as to speed it.

The [method page](/research/lab/experiments/raphael-anjou/hint-study/method) works
the floor arithmetic through in full.

## Scattered versus contiguous: we get the opposite

The community's
[hint-geometry write-up](/research/why/hint-geometry) makes a sharper version of the
placement claim: eighteen hints *scattered* on a lattice solve a 16×16 E2-like
puzzle in minutes, while eighteen piled into *contiguous* top rows barely help, and
you need eighty or more contiguous hints to match the scattered eighteen. That
result was measured on one specific puzzle with one specific engine. We put its two
exact layouts, the scattered rows-{1,3,5} lattice and the eighteen-cell top block,
on our own generated boards and solvers to see whether the direction holds.
> **[Interactive: HintGeoComparison]** Rendered on the canonical page (link above); not shown in this markdown export.
It does not. On our boards the **contiguous** eighteen score *higher* than the
scattered eighteen for seven of the eight fill orders, and by a wide margin: a
row-major sweep reaches a median of 377 with the contiguous block and only 138 with
the scattered lattice. This looks like a flat contradiction, and it is worth being
precise about why it is not quite one.

The two studies measure different things. The community result is about *time to a
full solution*: scattered hints reach into the deep endgame where a backtracker
spends almost all of its time, so they prune the expensive part, while a contiguous
top block prunes only the cheap opening. Our number is *matched-edge score at a
short budget*, and at eight seconds none of these strict backtrackers reaches the
endgame at all. What a top-anchored contiguous block does buy, immediately, is a
large correct region for the row-major sweep to build against, so the score climbs
fast even though the hard part of the board is untouched. Scattered hints, by
contrast, fragment the early fill exactly the way the five-clue shape did. So the
two results are consistent once you separate "solves the whole board eventually"
from "scores well in the first eight seconds": scattered placement helps the former
and hurts the latter. The next section makes that split visible.

## Solving it: where scattered finally wins

At 16×16 nothing solves in eight seconds, so the score is always a snapshot of a
search still in its opening. To see the *endgame* effect the community reported, we
need a board that fully solves. An 8×8 built to the same colour recipe does, in well
under a second, so on it we can measure the quantity that actually matters: the
number of search nodes a row-major backtracker needs to reach a complete solution. Fewer nodes means
the hints did real pruning work. Below, a spread lattice and a matched contiguous
block, at rising hint counts.
> **[Interactive: SolveSpeedChart]** Rendered on the canonical page (link above); not shown in this markdown export.
This is the whole story in one chart, and it finally lines up with the community
claim. At four hints the scattered lattice is *worse* than useless, solving only two
of thirty boards, the same sparse-hints-trip-the-search effect from every other
axis, while the matched contiguous block solves almost all of them. But cross a
threshold around sixteen hints and the lines swap hard: a sixteen-hint scattered
lattice solves in about four thousand nodes, while the matched contiguous block
still grinds through nearly two million, a five-hundred-fold difference at the same
hint count. Push further and both layouts become easy (by thirty-six hints a quarter
of the board is pinned and either one solves in a few hundred nodes), so the
scattered advantage is a window, widest where the count is enough to reach the deep
search but not so large that the board is half-solved by hand. In that window,
spread hints reach into the part of the search a backtracker actually struggles with
and cut it off, exactly as Joe and Peter McGavin described; a contiguous block only
ever prunes the easy opening. The 16×16 score comparison looked like it contradicted
them only because at a short budget the search never lives long enough to reach the
region where scattered placement pays off.

## What it does and does not say

This is a result about **strict, chronological depth-first backtrackers** on
generated 16×16 boards built to Eternity II's colour recipe, at a short (eight
second) budget. Each of those scoping words earns its place. *Chronological*: the
result is specifically about solvers that fill cells in a fixed order and must
satisfy a pinned piece when they reach it, a beam solver, which does not, reaches
the 450s on the same boards. *Generated*: the boards share E2's colour recipe and
its five-clue *shape*, but they are not the official puzzle, and the study transfers
no number to it. *Short budget*: none of these strict backtrackers solves within
eight seconds, so this measures how far each reaches, not a race to 480; whether the
"hints hurt" effect survives at much longer budgets is untested here.

Within that scope the lesson is sturdy and, we think, counterintuitive: for a
chronological backtracker, five correct pieces placed in the puzzle's own clue
geometry are not a gift but a constraint, and can cost far more than they give. What
decides the damage is not the hints but the fill order that has to live with them
and the order pays for a hint the way it pays for everything else, in the size of the
frontier it keeps open. It is one more reading of
[why the puzzle resists](/research/why/hint-geometry): even correct information helps
only a solver built to receive it.

## Related

- [The hint study](https://eternity2.dev/research/lab/experiments/raphael-anjou/hint-study) — Give a backtracker five correct pieces for free, in the puzzle's own clue geometry. It turns out not to help, and depending on the fill order it can hurt badly, because a pinned piece is a hard constraint a fixed fill order must satisfy on arrival. A family of fill paths, run on the same hinted boards, single core, measured against no hints at all.
- [How the study is built](https://eternity2.dev/research/lab/experiments/raphael-anjou/hint-study/method) — The apparatus behind the hint study: a parametric board generator faithful to Eternity II's colour recipe at every size, the family of fill-path backtrackers, the one canonical scorer, and the piece of arithmetic that keeps the count axis meaningful, the pinned-seam floor.
- [Where you place the hints beats how many](https://eternity2.dev/research/why/hint-geometry) — On a 16×16 puzzle built like Eternity II, eighteen hints scattered across the board solve it in minutes, while the same puzzle needs eighty or more hints piled into contiguous rows to be as easy. Position, not count, is the lever, and it points straight at the endgame.
- [The DFS study](https://eternity2.dev/research/lab/experiments/raphael-anjou/dfs-study) — One question, asked carefully: among depth-first backtrackers for Eternity II, what does each fill order, each heuristic, and the break mechanism actually buy? A family of from-scratch backtrackers, each one change apart, run on the same ten corner-pinned variants, single core, sixty seconds.