Method: Deep research, from question to sealed book

This is how one civic issue — homelessness, transit, housing, whatever the subject — goes from a list of open questions to a finished, citizen-ready book with a straight answer under every claim. It is the deepest research process this project runs, and everything else we publish (backgrounders, briefs, recommendation cards, discussion guides) is a lighter version of the same loop's later steps.

What it produces

A sealed book: one issue, treated end to end, with a plain-language explainer, a graded evidence base behind it, and receipts a reader can follow back to the original source for every number that matters. "Sealed" is our word for a specific state — every load-bearing claim in the piece has been independently re-checked against its original source, not just copied forward from an earlier draft.

How it works, step by step

We start from a written list of the questions a book on this issue needs to answer — agreed in advance, so the research has a target instead of drifting. We prioritize by what matters most for a voter deciding in the next election, not by what happens to be easiest to find.

A research pass then works through that list, building the evidence base for each question: first checking what we already have on file for this issue, so we never re-research ground we've already covered; then going out and finding what's missing; then closing the specific gaps that are still open. Every dead end — a document we couldn't get, an expert who didn't answer — gets written down with what we tried, not silently dropped.

Every piece of evidence that comes out of that pass ends with a plain record: the figure, where it came from, and what we think its status is. That status is a claim, not yet a fact — nothing is marked verified by the same people who drafted it.

Then comes the step that actually earns the word "checked." A separate pass — run by someone other than whoever drafted the material — goes back to the original source for every claim that matters and re-confirms it independently. This is not a read-over of the draft; it is re-fetching the original document or page and checking the number, the quote, or the date against it directly. A copy of what was actually read is kept on file, so the check itself can be checked later. Where that re-check disagrees with the draft, the draft is fixed — or, if it can't be resolved cleanly, flagged honestly rather than smoothed over.

Only after that independent verification does a piece get "sealed." Sealing is a yes/no decision, never a partial pass in disguise — a piece is either sealed, or it goes back on a fix list with exactly what still needs work. A sample of sealed material is also spot-checked by a separate reviewer on every batch; if the error rate in that sample is too high, the whole batch goes back for another verification round rather than letting a few bad checks slip through with the good ones.

Once the evidence base for an issue is sealed, it gets synthesized into the book itself: the core argument, the graded evidence behind each claim, and a plain-language explainer written so someone without a policy background can follow it. From that same sealed substance, we then build audience-specific versions — a summary for a busy reader, a longer version for someone who wants the detail — but every version traces back to the same sealed core, and any place a public version softens a claim for readability is logged, not silently smoothed away.

How it's checked

The verification step is deliberately adversarial and deliberately separate from drafting. Concretely, an outsider could challenge us on any of the following, and we should be able to show our work:

• Independent re-fetch. Every claim that carries real weight in the final piece is checked by going back to its original source directly — not by trusting a tool's summary of that source, and not by trusting the drafter's own read of it. We learned this the hard way: an automated fetching tool we used early on fabricated a plausible-sounding quote for a page it could not actually read, and it was only caught because a separate check went back to the raw document itself. Every quote extraction now gets that raw cross-check as a matter of course.
• Someone else does the checking. The session that verifies a claim is never the session that drafted it. That separation is structural, not a courtesy.
• A copy is kept. Whatever was actually read to support a verdict is archived at the time it's read, attached to the specific entry it supports — so a challenge later can be answered by pointing at what was actually seen, not by re-describing it from memory.
• Sealing is binary. A piece is sealed or it is on a fix list. There is no quiet middle state where something ships half-checked.
• Sample audits. A blinded sample of already-sealed material is re-checked on every batch; too high an error rate in that sample sends the whole batch back, not just the sampled pieces.
• Corrections sweep everywhere they touch. If a claim is later found to be wrong or disputed, every other place in the corpus that cited it gets checked and updated mechanically — by a systematic, dated sweep across everything downstream, not by trying to remember what might have referenced it. If that sweep means a piece we previously called verified no longer is, we say so and show the date it changed, rather than quietly leaving the old status standing. That sweep, and its visible history, is exactly what How we verify describes for the site as a whole.
• Sources that can't be reached get flagged, not hidden. If a source is paywalled, offline, or otherwise unreachable, that's stated plainly next to the claim it supports — we do not seal a page on a dead link and hope nobody checks.

What it honestly costs

The verification stage is the expensive part of this process, and it's expensive on purpose — that expense is what the word "sealed" is actually paying for. Based on the one full cycle we've run start to finish: a first verification round on a book-length piece involved two full adversarial review passes plus a same-day round of fixes across several files, all in one sitting. A separate cleanup pass, closing a gap in how earlier work had been logged, worked through several dozen records in a single sustained session. On a backlog of roughly a thousand claims awaiting this kind of check, our own working pace works out to a real multi-session effort — not something that happens in an afternoon.

It also costs a person's attention, not just computer time. At minimum, someone has to make a judgment call on any blocker the verification pass can't resolve on its own, and any new kind of public version needs a human review pass before it goes out.

Known limits

We're naming these plainly rather than papering over them.

• We don't yet have a written playbook for what happens right after a sample audit fails. We know the sampling check works — we've tested it against material we deliberately seeded with errors — but we don't yet have a standing, step-by-step runbook for what a session does in the moments right after a real rollback fires on real material.
• The paper trail from a verification pass back to its final verdict isn't standardized yet. The detail of how a given verification pass reached its conclusions currently lives in that pass's own notes rather than in one consistent, reusable format.
• We don't yet have a standard way to show an unreachable source on the page itself. We know the rule — flag it honestly, never seal on a dead link — but we don't yet have one settled template for how that flag should look to a reader.

What stays internal, and why

The working files behind a sealed book — the raw evidence extracts, the internal seal records, the in-progress drafts of the synthesis — stay internal. They're written for people doing the verification work, not for a general reader, and they reference internal tracking that wouldn't mean anything outside this project. Publishing anything from this pipeline is always a separate, deliberate decision after sealing is complete — sealing an artifact internally is not the same thing as deciding to publish it, and both decisions are made on purpose, not by default.

Every load-bearing claim on this site carries a receipt regardless of who drafted it, including where AI assistance was used to draft the underlying research. Run this in your municipality if you want to build the same kind of process for your own community, or read How we verify for how this checking discipline runs across the whole site.

— The Unknown Soldier · uniteTOlove · Toronto