This page is the editorial standard behind everything in the lab. It exists so
that a finding reaches a reader as an authored artifact, not a raw note, and so
the same rules apply no matter who writes it.
The ground rule everything rests on:
- Unpublished work does not appear here. Where a researcher keeps their work
in progress is their own business. It is simply not in this repository. There
is no half-finished draft sitting in the published notebook.
- Published work lives in the lab. Public, distilled, written for a reader.
It got here through review.
Publishing happens by pull request. A researcher opens a PR that adds or promotes
a page, and the review on that PR is where the decision to publish, and at what
tier, is made. Nothing enters the public record without a PR, and the PR is where
a second reader checks it against the standard below. This keeps review
collegial rather than a heavyweight gate, and it scales to many authors.
Three axes describe every page#
Keeping these separate is the whole method. It is easy to blur "polished,"
"reproducible," and "reviewed" into one vague notion of "done." They are not the
same thing.
- Contribution: what kind of result is this?
- Tier: how far through review has it come?
- Rigor and reproducibility: how firmly is the claim backed?
Plus attribution: who did what.
Not every result is a solver, and the notebook stopped pretending otherwise. A
page declares its contribution:
- solver: produces a competitive board by searching. Its score is a real
search result. Only these earn a row on the leaderboard.
- analysis: proves or computes a property of an existing board or of the
instance. It does not produce a new board.
- reconstruction: decodes or rebuilds the community's known work to extract
an insight. The number is theirs, decoded.
- theory: a mathematical property, law, or impossibility proof.
- method: a technique described for reuse, not a scored run.
- measurement: a benchmark or empirical observation about solvers or
instances.
- negative: a rigorously-run dead end. First-class, not a footnote.
- tool and exposition: a software artifact, or an explainer.
The load-bearing distinction is solver versus everything else. A meet-in-the-
middle band solver that proves an endgame is optimal is an analysis, not a
solver, and it does not belong on a score chart even though it emits a number.
Getting this axis right is what lets the leaderboard mean one thing.
This axis is also a browse surface: the
by-contribution index gathers the
pages by the label they declare, so the analyses, the theory, the measurements,
and the dead ends kept as first-class
negatives each have a shelf
of their own.
Once a page is published, it carries a tier that says how firmly it has been
reviewed:
- Technical report: public, but the review confirmed only that it is sound
and fairly stated, not that it has been independently checked. The page
carries a "technical report" badge, the way a preprint is stamped "not yet
reviewed." A legitimate resting state: not everything needs to go further.
- Reviewed finding: a second reader, or the same author after a
cooling-off, confirmed it holds. Cited, treated as close to immutable;
corrections happen in place, annotated.
Two principles carry over from how research publishing works elsewhere. The tier
is stamped on the page, so a reader always knows what they are looking at. And
promotion turns on rigor, not on outcome: a sound method that found nothing gets
published as a negative result, while a striking result on an unsound method does
not.
Every page states how firmly its central claim is established (proven, measured,
or conjectured) and how re-runnable it is. For anything quantitative the gate is
simple: a number publishes only when its exact configuration and seed are
archived and it can be re-run from the documentation. A benchmark score with no
re-runnable config does not get published. Cheaper contributions carry a lighter
bar: a negative result or an explainer needs sound reasoning and clearly stated
limits, not a reproduction script.
Each researcher's pages gather automatically on their contributor page, derived
from the byline, never hand-listed. Adding an author is adding an entry to the
registry and writing pages under their name. When a page has more than one hand,
a light contributor role (who ran it, who analysed it, who validated it, who
wrote it) records the split, so credit stays accurate as the lab grows.
When a piece of work is done:
- Name the contribution. Solver, analysis, reconstruction, theory, method,
measurement, negative, tool, or exposition.
- Decide if it publishes. Publish on rigor, not on excitement. A sound dead
end publishes. A number with no re-runnable config waits.
- Open a PR at a tier. Technical report if it is written up but not yet
independently checked; reviewed finding once a second reader signs off.
- Place it. Solvers and their kin under the experiments; structural results
under why it is hard; techniques under building a solver; cross-solver
measurements with the benchmarks.
- Chart it only if it is a solver.
That is the whole standard. It is deliberately light, because the point is to
raise the floor on every page without slowing the notebook down.