Methods page preface — /assembly/methods.html opening section

Committed page source (old-estate fires dispatch Leg 1, 2026-08-12): the opening section of the methods/limits page, rendered by tools/assembly_site_gen.py ABOVE the imported METHODS and VALIDITY memos (archived-imports/phase0-pilot/). Exists as a file so page substance lives in markdown sources, never inside a generator (five-verbs law). The H1–H3 statements below are condensed from the engine's own research/assembly/phase0_pilot_import/CODE_REVIEW.md — update them only against that source. Everything above the first --- is stripped at render.

How the simulation works — and where it honestly falls short

This page publishes the engine's own methods documentation and its threats-to-validity memo, unedited, exactly as the project's internal review wrote them. Read the limits before the methods: no number this system produces is a measurement of what real Torontonians think.

The three open validity issues (H1–H3)

Our own code review found three high-severity issues that remain open. They are the core of why run-1 results are unpublished, and any future results will state how each was addressed:

• H1 — agents never update opinions across rounds (live mode). In live-LLM runs there is no opinion-updating mechanism: each round is independent voting plus proposal authoring. Agents receive expert briefings into memory, but nothing makes them reconsider earlier positions in light of the emerging cluster structure or others' votes. Until fixed, live runs are "iterated independent polling with idea generation," not deliberation with opinion change — which is the central gap between this simulation and a real assembly.
• H2 — the raw consensus score is not comparable across runs. Group-informed consensus is a product of per-group agreement probabilities, so it shrinks mechanically as the number of opinion groups grows: a 4-group run reports lower "consensus" than a 2-group run for identical agreement. Comparisons must use the per-group floor or a k-normalized geometric mean; the raw product is k-dependent wherever it appears.
• H3 — seeded live runs are not reproducible. The random seed fixes the roster, routing, and mock votes, but live model calls are stochastic (temperature 0.85 plus model nondeterminism), so two live runs with the same seed will differ. Live reproducibility is statistical, not exact — a limit for any calibration story that assumes a run is a deterministic function of its inputs.

What follows

The two documents below are published verbatim from the project archive: Methods (what the system does, precisely enough to reproduce) and Threats to validity (what its outputs can and cannot support, with the published failure modes of simulating people with language models).

Methods

System: a simulated deliberative mini-public ("Citizens' Assembly") in which a statistically representative population of LLM-backed agents deliberates a policy question, votes on statements, authors proposals, and (optionally) requests expert briefings. Outputs are an opinion map, ranked cross-group-agreed proposals, and a full transparency log.

Status / claim. This is a rehearsal and hypothesis-generation instrument, not a measurement of public opinion. It is designed to (a) pressure-test the design of a real assembly (agenda, framing, materials, process) cheaply and repeatably, and (b) generate testable hypotheses about where a population might find common ground. Every output is labelled SIMULATED. Threats to validity are documented separately in VALIDITY.md; this file documents what the system does, precisely enough to reproduce.

This document describes the Phase-0 reference implementation (single machine, ≤ a few hundred agents). Section 12 notes what changes at population scale.

1. Pipeline overview

A run is a deterministic function of (seed, config, agenda, model) over seven stages:

1. Sampling — draw a roster of N agent profiles matched to target population marginals, with realistic joint structure (§3–4).
2. Agent construction — turn each profile into a persona with values, per-issue leanings, psychographic traits, a latent opinion vector, and a one-page narrative (§5).
3. Seeding — load the agenda: a question + balanced two-sided seed statements (§6).
4. Deliberation — R rounds of routed voting, proposal authoring, and (optional) expert-briefing requests, with inter-round memory and reflection (§7).
5. Opinion analysis — Pol.is-style clustering and group-informed consensus over the accumulated vote matrix; semantic de-duplication of proposals (§8).
6. Expert panel / knowledge base — balanced briefings, cached and reusable (§9).
7. Reporting — report, opinion map, dashboard, and machine-readable artifacts (§10).

2. Precedents (what each component reproduces)

Full citations in VALIDITY.md. The implementation is original; these are the conceptual sources.

3. Population data and provenance

Population data lives in swappable region datasheets (regions/<id>.json; default toronto.json). The reference population is the City of Toronto (CSD 3520005, 2021 Census). Each datasheet declares one-way marginals, pairwise affinities, and supplementary populations (below). Full figures, sources, and census-vs-illustrative flags are in DATA_PROVENANCE.md (not separately published on this site). region_tool.py scaffolds/validates new places; --region <id> selects one. Ontario and Canada datasheets ship as scaling examples.

• Census-grounded marginals (Statistics Canada 2021 Census Profile; City of Toronto census backgrounders): area (4 community-council areas), gender, age band, visible- minority group, immigrant status, tenure, household income band, education, Indigenous identity. Values are rounded to the figures cited in the project blueprint.
• Illustrative marginals (plausible, not census-exact; flagged in code): household type, employment status, commute mode, disability, years-in-city, caregiving.
• Attitudinal strata: trust-in-government, openness-to-change, political lean, religiosity — included so the room is not implicitly stacked with enthusiasts (cf. Climate Assembly UK, which stratified on attitude to climate).

Provenance discipline. Each marginal is a named distribution in one file; changing the modelled population means changing only MARGINALS (+ AFFINITIES, §4). No figure is hard-coded elsewhere. Illustrative vs census-grounded status is documented in census.py and must be re-stated in any published result.

4. Sampling: marginal-exact quota + targeted-swap raking

Goal. Produce N profiles that (i) match every target marginal exactly (up to integer rounding, eliminating small-sample drift) and (ii) exhibit realistic joint structure (e.g., seniors tend to be retired and owners), all without a full joint cross-tabulation (which Statistics Canada does not publish freely).

Step 1 — quota allocation (exact marginals). For each variable with target distribution p, integer category counts are assigned by the largest-remainder method: base_k = floor(p_k · N), then the N − Σ base_k residual seats go to the categories with the largest fractional remainders. Small mandatory cells (Indigenous, non-binary/trans) are floor-guaranteed ≥ 1 for N ≳ 50. Each variable's category list is shuffled and assigned to agents independently, so every one-way marginal is hit to the nearest integer (verified: tests.py §2, all N).

Step 2 — correlation pass (joint structure). Target conditional rates P(B = b | A = a) are declared in AFFINITIES (e.g., P(retired | 65+) = 0.70). A targeted swap exchanges the B-value between one agent with A=a, B≠b and one with A≠a, B=b: this raises the joint count of (a,b) by one while leaving the B-marginal unchanged (it is a permutation of B-values). Swaps are applied per affinity toward its target count round(target · |A=a|); several passes resolve conflicts among affinities that share a B-dimension. The procedure is deterministic given the seed and converges to within ≤ 0.03 of every target across N ∈ {50,100,400} and 5 seeds, with marginals still exact (verified: tests.py §2; cross-seed sweep in VALIDATION.md).

Modelled affinities (illustrative, editable): age↔employment (retired/student/FT), education↔income, income↔tenure, age↔tenure, household↔income, politics↔tenure, area↔commute, immigrant↔years-in-city, age↔disability.

Honest scope. This is a constrained maximum-plausibility construction, not inference from a real joint distribution. It is faithful to the marginals and to the declared pairwise targets; higher-order interactions are not modelled. The production endpoint remains true iterative proportional fitting (IPF) against a commissioned StatCan custom cross-tabulation. method="independent" reproduces naïve per-variable sampling for ablation.

5. Agent construction

For each profile, agents.py::build_agents computes, in order:

1. Values — three ranked priorities sampled (without replacement) from 12 value tokens with profile-conditioned weights (e.g., low income ⇒ +affordability, +fairness).
2. Per-issue leanings — issue_bias[theme] ∈ ℝ, a signed scalar per deliberation theme derived from profile (e.g., car commuter = −0.5 on mobility; renter = +0.3 on housing). Positive = leans toward the change/regenerative pole of that theme.
3. Latent opinion vector vec = (x, y): - x (incremental/centralized ↔ bold/decentralized) = OPEN[openness] + {left:+.15, centre:0, right:−.15} + youth/low-trust adjustments + 𝒩(0, .30); - y (market ↔ collective) = LEAN_Y[politics] + 0.4·TRUST[trust] + renter/low-income/ racialized/high-income adjustments + 𝒩(0, .30); both clamped to [−1.3, 1.3].
4. Traits ∈ [0,1]: engagement, numeracy, persuadability, risk-tolerance, optimism, verbosity — profile-conditioned with bounded noise.
5. Narrative — a one-page persona that renders all of the above in plain language, including an explicit "at a glance" footer of trait levels. This is the system prompt in LLM mode, so behaviour is conditioned on the full, legible profile.

Fidelity note. Narratives are synthesized from demographics, not real interviews — the largest single fidelity gap (see VALIDITY.md). The architecture deliberately matches Park et al.'s narrative-grounding so that real interview transcripts can be substituted in Phase 2 with no interface change.

6. Agenda and seed statements

The agenda is a question plus balanced, two-sided seed statements grouped by theme (agenda.py). The default is a regenerative-Toronto agenda (22 statements / 11 themes); arbitrary agendas load from a human-readable Markdown file (--agenda, agenda_tool.py), with check warning on any one-sided theme. Two-sidedness is a deliberate guard against a leading instrument. In mock mode, statements missing latent coordinates receive a keyword-estimated position (estimate_xy); in LLM mode only the text is used.

7. Deliberation protocol

Each of R rounds (deliberation.py::run_round):

1. Routing. Each agent is shown a subset of statements (default 12), preferring unseen ones — a simplified analogue of Pol.is comment routing (information-gain / consensus-potential weighting is not yet implemented; see VALIDITY.md).
2. Decision (concurrent, bounded). Each agent returns votes (agree=+1 / disagree=−1 / pass=0) on the shown subset, optionally one authored proposal, optionally one evidence question. Concurrency is semaphore-bounded; per-agent failures are isolated (one malformed reply cannot abort the run).
3. Proposal integration with de-duplication. Authored proposals are normalized; an identical-text proposal already on the table records the author's support rather than creating a duplicate (popularity accrues to one statement). New unique proposals are capped per round.
4. Expert briefings. De-duplicated evidence questions are answered by the panel (§9); each agent's bounded memory receives the briefing digests, which are injected into subsequent prompts so evidence can influence later votes and proposals.
5. Opinion analysis (§8) is computed each round; reflection (mock backend only) nudges agents toward their group centroid by their persuadability, modelling opinion movement over rounds.

8. Opinion analysis (Pol.is math)

Let M be the agents × statements vote matrix (entries in {+1,−1,0}, missing = NaN).

• Projection. Missing/pass entries are treated as 0; PCA reduces to 2 dimensions (the opinion map).
• Clustering. k-means is run for k = 2..5; the k maximizing the silhouette coefficient is selected (matching Pol.is's k-cap and selection rule).
• Per-group agreement. For group g and statement c, P(agree | g,c) = (1 + agrees) / (2 + (agrees + disagrees)) — a Beta(1,1)/Laplace (+1/+2) smoothing prior; passes do not count toward the denominator.
• Group-informed consensus. consensus(c) = Π_g P(agree | g,c). Because it is a product, one dissenting group drives the score toward zero — this is the bridging metric that rewards statements ~every group endorses and structurally resists tyranny-of-the-majority (verified equal to an independent recomputation, tests.py §3).
• Representativeness of a statement for a group uses the same smoothed in-group vs out-group agreement ratio.
• Semantic idea de-duplication. Agent-authored proposals are embedded with TF-IDF and greedily clustered by cosine similarity (threshold 0.45, seeded by most-voted); each cluster becomes one distinct idea whose support is the per-agent net stance aggregated across its phrasings. This corrects LLM mode-collapse (many paraphrases of one idea) so the report ranks genuinely distinct ideas (verified: tests.py §4). The exact Pol.is engine (red-dwarf) is a drop-in replacement for this module.

9. Expert panel and knowledge base

Evidence requests are cleaned (junk/bare-id removal, near-duplicate collapse, cap) and served knowledge-base-first:

• Lookup is keyword-based: a subject matches a stored briefing when keyword Jaccard ≥ 0.6 AND ≥ 2 shared keywords (exact slug = immediate hit). A hit costs zero model tokens (verified hit/miss behaviour, tests.py §7).
• Miss → the configured expert model writes a briefing from a fixed world-class, non-partisan prompt (TL;DR, strongest case for/against, costs & financing, named precedents with numbers, equity, success factors, uncertainties, sources-to-verify, follow-up questions). It is saved to a shared, persistent KB and reused by all future runs; the exact prompt is stored alongside for reproducibility/upgrade.
• Providers (--expert-provider): anthropic (Claude Opus, highest quality), openai-compatible, local (reuse the agents' model), manual (write paste-ready prompts for a human to run in a top model and ingest back), none. The agents' deliberation model and the expert model are decoupled so a cheap local model can deliberate while a frontier model writes evidence once.
• Follow-ups are stored as linked children, enabling progressive deepening without re-deriving the base briefing.

The panel design mirrors a real assembly's balanced learning phase (informants vs advocates) and explicitly fixes the Habermas Machine's acknowledged lack of a fact-finding phase.

10. Backends, determinism, transparency

• Mock backend (llm.py): a transparent generative model with no network — vote probability = σ(2.1·align + 1.4·issue_bias·sign(stmt_x)) where align is the normalized dot product of agent and statement latent vectors. It exists to test the full pipeline deterministically and to provide ablations; it is not a model of human opinion.
• OpenAI-compatible backend: any endpoint (local Ollama/vLLM, OpenRouter, hosted). Hardened for small models: JSON mode with graceful fallback, retries, tolerant parsing (markdown fences, prose wrapping, trailing commas, string-typed votes — verified tests.py §6), per-agent failure isolation, token accounting.
• Determinism. Roster, agents, routing, and mock votes are functions of explicit seeds; the roster and agent construction are bit-for-bit reproducible (verified tests.py §1). LLM backends are stochastic; reproducibility there is at the distribution level, supported by per-call logging.
• Transparency. Every run writes a timestamped folder with: structured events.jsonl (one record per round / expert call), human run.log, metadata.json (full config, package versions, timings, token usage, KB hits/misses), the roster (agents.json, personas.md), the vote matrix (votes.csv), statements.json, and the rendered report/map/dashboard. Nothing is hidden; any result is re-inspectable.

11. Software environment

Python 3.10+. Core: numpy, scikit-learn (PCA, k-means, silhouette, TF-IDF). Optional: openai (LLM backends), anthropic (Opus briefings), matplotlib (static map). Exact versions for any run are captured in that run's metadata.json.environment.packages. The verification suite is tests.py (python tests.py, exit 0 ⇔ all pass).

12. What changes at population scale (forward note)

Scaling toward millions of agents across languages/regions changes engineering, not the method: (i) sampling generalizes to any region given its marginals + affinities (or a real joint table + IPF); (ii) personas and prompts are language-agnostic given a localized template and a multilingual model; (iii) clustering must move from in-memory k-means to streaming/approximate methods, and routing to the real information-gain weighting; (iv) the KB becomes a shared multilingual evidence base with embedding-based lookup. The validity threats in VALIDITY.md do not shrink with scale — several (caricature of under-represented groups, language/cultural skew) grow — and must gate any scale-up. Scope for scaling is deferred until the present system is independently reviewed.

Neighbourhood-conditioned sampling

When a region datasheet declares a neighbourhoods array (each with a population and its own marginals), sampling is two-stage:

1. Allocation. The N agents are distributed across neighbourhoods in proportion to their population by the same largest-remainder quota used for marginals, so the per-neighbourhood head-counts are exact and deterministic.
2. Conditioned draw. Each neighbourhood's agents are drawn from its own marginals, overlaid on the citywide marginals (any variable a neighbourhood omits falls back to city). A neighbourhood's district fixes Profile.area. Blocks of ≥12 agents additionally get the targeted-swap correlation pass; smaller blocks keep exact marginals without it (too few people for a within-neighbourhood joint structure to be meaningful).

Each agent records its neighbourhood, which flows into the persona narrative, agents.json, and the dashboard. The citywide (single-stratum) path is unchanged when no neighbourhoods are declared. Construction and validity caveats: DATA_PROVENANCE.md (neighbourhood section).

Persona localization & standard geography codes

Place-coded persona prose (the place name, dwelling phrasings, commute framings, the "new here / watched it change" lines, and the Indigenous-relationship line) lives in a per-region locale block. The built-in default reproduces the Toronto wording exactly, so Toronto output is unchanged; a datasheet (e.g. ontario.json) overrides it with region-appropriate framings (rural/car-dependent, the North, smaller towns, Francophone Ontario), so a rural Ontario agent no longer references the TTC. Occupation lists are not yet locale-aware (a planned refinement).

Sub-areas follow standard nested geography. Each sub-area entry carries a level and an official code, and the geography block names the code_system — Statistics Canada's Standard Geographical Classification (Canada → province → census division → census subdivision → census tract → dissemination area), with City-administrative units (Toronto's 158 neighbourhoods) and electoral districts as parallel administrative options. Each agent records geo_level and geo_code, which is what lets regional maps nest/federate (DA→CT→CSD→CD→PR) and translate internationally (GADM admin 0–3 / OECD TL2–TL3).

Threats to validity and honest limitations

This document states, as plainly as possible, what the simulated assembly can and cannot support, and the threats that any reviewer should weigh. It is deliberately conservative. Read it before treating any number from this system as a finding.

0. What is and is not claimed

Claimed. The system is a rehearsal and hypothesis-generation instrument: it lets us (a) design and stress-test the agenda, framing, evidence, and process of a real deliberation cheaply and repeatably, and (b) produce testable hypotheses about where a population might find cross-group common ground. It is good at central tendencies and at surfacing bridging vs dividing framings.

Not claimed. It is not a measurement of public opinion; not a substitute for a real, human deliberation; and not evidence that any specific number (e.g., "80% support") would hold among real people. Simulated participants are not participants. Every artifact is labelled SIMULATED.

The deepest objection — that accurate simulation still does not deliver the representation real human-subjects deliberation exists to provide — we accept as correct (Agnew et al. 2024). The simulation informs and de-risks the real thing; it does not replace it.

1. Construct validity — do the agents represent humans?

1.1 Synthetic personas, not real interviews. Park et al. (2024) achieved high fidelity by grounding agents in two-hour interviews of the real individuals they simulate. Our personas are synthesized from demographics. This is the single largest fidelity gap. We match their architecture (narrative grounding + memory) so real qualitative material can be substituted with no interface change, but until then the agents are demographic archetypes, not people. (Roadmap §5.)

1.1b Census-missed populations are the least reliable of all. We deliberately include groups the census under-counts (homelessness, non-status migrants, precariously housed, precarious-status residents) so the simulated room is not limited to the well-counted — an inclusion improvement over census-only sampling. But their sizes are estimates with wide uncertainty, and the literature is clearest that LLMs misportray and flatten exactly such marginalized groups. These personas are crude, humble approximations that make the voices present; they are emphatically not a substitute for engaging real people from these communities ("nothing about us without us"). Provenance and ethics: DATA_PROVENANCE.md §2,5.

1.2 Algorithmic fidelity is unestablished and must be tested per model/task. "Silicon sampling" can reproduce some demographically-correlated response patterns (Argyle et al. 2023), but fidelity is not transferable across models, tasks, or time and must be demonstrated case-by-case. We have not validated fidelity against any real Toronto survey or assembly. Until we do, agreement levels are internal to the simulation.

2. Documented failure modes of simulating people with LLMs

These are established in the literature and apply directly:

• Variance collapse / artificial precision (Bisbee et al., Political Analysis 2024). Even when synthetic means track reality, synthetic responses are far less varied than real ones (reported synthetic SD ≈ half real), manufacturing false confidence and exaggerated polarization; conditional relationships can sign-flip. → Our cross-group agreement figures likely overstate within-group consensus.

• Caricature and flattening of identity groups (Wang, Morgenstern, Dickerson, Nature Machine Intelligence 2025). LLM personas reproduce out-group stereotypes and are systematically less internally diverse than real people — worst for minority and intersectional groups. → The parts of a population we most need to hear are the parts the model most distorts. This worsens at multilingual/global scale.

• Reliably mis-modelled groups and a default skew (Santurkar et al., ICML 2023, OpinionQA). Models poorly represent some groups (e.g., 65+, certain religious communities) and aligned models carry a measurable WEIRD / left-of-centre default that prompt-steering only partly corrects. → We observed exactly this: a local model's right-leaning agents still authored progressive-eco proposals.

• Alignment can reduce fidelity (Lyman et al., Sociological Methods & Research 2025). The RLHF that makes a model agreeable can make it worse at reproducing a subpopulation's true distribution. → "Better" assistant models are not necessarily better simulators.

• Single-model monoculture. All agents share one base model; its idiosyncrasies and blind spots are correlated across the whole "population." → Mitigate by ensembling ≥2 base models and reporting only what is robust across them (not yet default).

3. Internal validity (run-to-run, design artifacts)

• Mode collapse (mitigated, not eliminated). LLM agents tend to re-phrase one safe idea many times. We added TF-IDF semantic de-duplication (verified, tests.py §4), which collapses paraphrases into distinct ideas — but de-duplication is heuristic (lexical, threshold 0.45), and pre-dedup agreement counts are inflated.
• Vagueness without evidence. Running without the expert panel produced blander, more universally-agreeable (and less actionable) proposals, inflating apparent consensus. Expert briefings add specificity; their absence is itself a bias.
• Prompt/order/temperature sensitivity. LLM outputs vary with wording, statement order, and sampling temperature. Routing order is seeded but the model is stochastic.
• Simplifications in the reference build (each a known gap, flagged in code):
• Sampling matches marginals + declared pairwise correlations, not a real joint distribution (no true IPF against a StatCan custom cross-tab); several lifestyle marginals are illustrative, not census-exact.
• Comment routing is random/novelty, not Pol.is information-gain weighting.
• Reflection (opinion movement over rounds) is implemented for the mock backend only.
• In mock mode, emergent proposals are drawn from a built-in library and statement positions are keyword-estimated — mock is a plumbing/ablation tool, not a result.
• Researcher degrees of freedom. Agenda wording, seed statements, value/affinity tables, thresholds (silhouette k-cap, dedup 0.45, KB Jaccard 0.6) are choices that shape outputs. They are centralized and editable for transparency, but they are choices; results should be reported with the exact config (it is saved per run).

4. External validity (generalization)

• One city (Toronto), one default agenda, one model at a time. Generalization to other populations, topics, languages, or models is unestablished.
• The mock backend's structure is engineered, not learned from humans; its outputs carry no external meaning.
• At global/multilingual scale the validity threats in §2 grow (more under-represented groups, language and cultural skew, translation artifacts), so scale must not be read as increased reliability.

5. Statistical-conclusion validity

• Group-informed consensus is a product across groups and shrinks mechanically as the number of clusters grows; compare "floors" (minimum per-group support) across runs with equal k, not raw products. The metric is verified correct (tests.py §3) — the caution is interpretive.
• Small-N clustering is unstable; cluster count and membership vary with N, seed, and model. Use ≥100 agents and report cluster stability.
• No inferential statistics are reported (no CIs, no significance) because the data are synthetic; doing so would be the artificial-precision error (§2). Treat all numbers as descriptive of the simulation only.

6. Ethics and honesty safeguards

• Radical transparency: every model call, vote, and briefing is logged; the full config, prompts, and roster are saved; results are reproducible and re-inspectable.
• Mandatory SIMULATED labelling on all outputs.
• A hard line that the system informs and mobilizes real deliberation and never manufactures a "mandate." The endpoint is a real (or hybrid) human assembly.
• Briefings are evergreen reference documents that explicitly flag figures to re-verify; they are not authoritative data.

7. What would raise validity (mitigation roadmap, in priority order)

1. Calibration against ground truth — compare simulated agreement to a real Toronto survey and/or a real mini-public on the same items; report the gap. (Highest value.)
2. Interview-grounded personas — replace synthetic narratives with real consultation/oral-history transcripts (Park-style), closing §1.1.
3. Multi-model ensembles — run ≥2 base models; report only robust findings (compare_runs.py already diffs runs).
4. Within-group diversity repair — explicitly diversify personas and measure synthetic vs real variance to counter §2 variance collapse.
5. True IPF sampling against a commissioned StatCan joint cross-tabulation.
6. Real comment routing (information-gain) and a learned reflection model.
7. Pre-registration of agenda, config, and analysis before any externally-reported run.

8. Claims ledger

References

• Argyle et al. (2023). Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis 31(3). https://arxiv.org/abs/2209.06899
• Bisbee, Clinton, Dorff, Kenkel, Larson (2024). Synthetic Replacements for Human Survey Data? The Perils of LLMs. Political Analysis 32(4). https://www.cambridge.org/core/journals/political-analysis/article/B92267DC26195C7F36E63EA04A47D2FE
• Wang, Morgenstern, Dickerson (2025). Large language models that replace human participants can harmfully misportray and flatten identity groups. Nature Machine Intelligence. https://arxiv.org/abs/2402.01908
• Santurkar et al. (2023). Whose Opinions Do Language Models Reflect? ICML. https://arxiv.org/abs/2303.17548
• Lyman et al. (2025). (alignment degrades subpopulation fidelity). Sociological Methods & Research. https://journals.sagepub.com/doi/abs/10.1177/00491241251342008
• Agnew et al. (2024). The Illusion of Artificial Inclusion. CHI. https://arxiv.org/abs/2401.08572
• Park et al. (2024). Generative Agent Simulations of 1,000 People. https://arxiv.org/abs/2411.10109
• Park et al. (2023). Generative Agents: Interactive Simulacra of Human Behavior. UIST. https://arxiv.org/abs/2304.03442
• Tessler, Bakker et al. (2024). AI can help humans find common ground in democratic deliberation (Habermas Machine). Science 386:6719. https://www.science.org/doi/10.1126/science.adq2852
• Computational Democracy Project — Pol.is algorithms. https://compdemocracy.org/algorithms/ ; group-informed consensus https://compdemocracy.org/group-informed-consensus/
• OECD (2020). Innovative Citizen Participation and New Democratic Institutions: Catching the Deliberative Wave. https://www.oecd.org/en/publications/innovative-citizen-participation-and-new-democratic-institutions_339306da-en/full-report.html
• Flanigan et al. (2021). Fair algorithms for selecting citizens' assemblies. Nature. https://www.nature.com/articles/s41586-021-03788-6

Augment, do not replace (Part D framing)

The Pol.is team's own analysis of LLMs for deliberation (Small et al., Opportunities and Risks of LLMs for Scalable Deliberation with Polis, 2023) warns that using LLMs to replace participants "could be catastrophic for deliberation at scale." Accordingly this system is positioned as an augmentation instrument: it designs and de-risks a real Pol.is/assembly, stress-tests framings, and generates testable hypotheses about where bridges lie - and it can include census-undercounted voices the self-selected real tool misses - but it is not a substitute for a real deliberation with real people, and every artifact is labelled SIMULATED.