Method: Question banks — what the research owes its readers

A question bank is the bridge between everything we've researched and what a voter actually gets to ask a candidate. This is how we build that bridge: every question tied back to a real, sourced claim, the whole set checked against every issue in our corpus so nothing important gets skipped, and every revision built so a candidate's earlier answer never gets silently orphaned.

What it produces

Two connected things. First, inside each research project, a running set of open questions the research itself still owes an answer to, tracked honestly as answered, partly answered, still open, or out of scope — never quietly dropped. Second, and the one citizens actually see: the Citizen Question Bank — the set of questions we and the public put directly to candidates, published on the site, built to span the full range of municipal issues the corpus covers.

How it works, step by step

Every research project seeds its own question list the moment work on it starts, with each question marked honestly as open unless something already in hand answers it — never marked answered in advance to look tidier than it is.

For the citizen-facing bank, every question is checked against the full map of municipal issues the corpus tracks, so the bank as a whole covers the range of things a municipal government actually touches, not just the topics that were easiest to write questions about. Our most recent full check ran against 52 separate issue areas: 51 of them had at least one question covering them, and the one that didn't was named openly as outside municipal jurisdiction rather than papered over with a forced, artificial question.

The bank's size and shape follow what we're told is actually needed, not a habit or a house default. When an earlier draft of 73 questions wasn't enough — the ask was for genuinely open-ended questions candidates could answer at real length, spanning every issue area, with room for roughly 300 total because these are meant as real ammunition for a real conversation — the next draft answered that directly: 300 questions, with a meaningful number of deliberately open-ended, longer-form questions spread across every category, re-checked against the 52-issue map.

Underneath the citizen-facing bank sits a set of category-level banks that hold the sourcing and underlying claims each question is built from — the receipts, in other words. The citizen-facing bank is not a rough draft of those; it's a separate, purpose-built instrument that draws on them. When the bank is revised, every question already asked keeps its identity: a revision expands or improves the set, but it never renumbers or drops a question a candidate may already be partway through answering. The frozen prior version stays available so anyone can compare exactly what changed between drafts.

Published versions of the bank go live only once we've been told the instrument is ready to publish — that's a deliberate, separate step from drafting it.

How it's checked

The bank is graded on coverage, not word count: does it actually reach every issue area in the corpus, checked directly against that map, rather than just producing a large number of questions. Every research project's open-question tracker has to reflect real, current status before that project's work is considered closed for the day — no research session ends with stale or invented status. When a revision expands the bank, we verify that every prior question survived the change unchanged in substance, by keeping the previous frozen version available to compare directly against the new one, question for question. And the underlying claims that sit behind the questions were independently re-fetched from their original sources in a dedicated recheck pass, which found zero discrepancies against what had originally been cited — a check on the sourcing layer itself, not just on whether the documents are internally consistent.

What it honestly costs

Drafting the bank is work an assistant can do at volume, but every sizing decision, every redline, and every publish decision runs through a real review before it counts as final. As one data point: expanding a 73-question draft to 300 — including the full 52-issue span check and writing dozens of new open-ended questions — was completed as a single same-day pass. That's one precedent from one round of work, not a promise about how long the next expansion will take; a larger rebuild, or a first-time bank for a brand-new area, would take longer.

Attention from the person ultimately accountable for the bank is required at two points specifically: whenever the bank's size or shape is being decided, and before any published version goes live. Day-to-day tracking of which questions are answered doesn't require that same level of attention.

Known limits

The relationship between the category-level sourcing banks and the citizen-facing instrument is written down as a process, but it isn't yet built as a repeatable, automatic step — right now it lives in documentation, not in a script anyone could just re-run. There's no standing method yet for catching duplicate questions across the sourcing banks and the many project-level question trackers behind them. The 52-issue coverage check itself was done by hand as a one-time pass — there's no automated check that would catch drift the next time the bank is revised without someone re-running that same manual review. And publishing a new version of the bank is currently governed by direct instruction at the time, not by a written checklist anyone else could follow to know when a version is ready.

What stays internal, and why

The detailed sourcing behind each category bank, and the full research trail behind each project's internal question tracker, are kept internal — useful for auditing our own work and for building the next version of the bank, but not published themselves. What's public is the citizen-facing bank and its published categories. If you're building this for your own municipality, what you'd copy is the shape of the method — sourcing layer feeding a public instrument, published only once reviewed — not our internal sourcing files.

Want to run this in your own municipality? Start here: Run this in your municipality. See how we check other kinds of claims across the whole site: How we verify.

— The Unknown Soldier · uniteTOlove · Toronto