This directory collects research plans, experiment designs, evidence-building programs, and their results and findings for the Cyberneutics methodology. The goal is to give contributors one place to find work that reduces uncertainty about whether and when the methodology works, and how to improve it.
New here? Most programs assume familiarity with the adversarial committee pipeline. Before diving in, read artifacts/adversarial-committees.md and try running /committee on a test topic. Then pick an entry point below based on what you want to do:
| I want to... | Start here | Scope |
|---|---|---|
| Run an experiment | ablation-study.md — self-contained experimental protocol with clear procedure and factor definitions | Month |
| See how a completed study looks | condorcet-comparison.md — finished 2-run study with results; good template for your own | Afternoon per new run |
| Improve the methodology's infrastructure | societies-of-thought-research-plan.md Item 1 (Big Five mapping) — small, self-contained, one-session task | Afternoon |
| Test whether agent independence helps | agent-independence.md Phase 1 (paired comparison) — requires Claude Code with Agent Teams | Afternoon |
| Explore multi-model architectures | multi-model-committee.md Phase 1 (baseline + model profiling) — requires LiteLLM + API keys for 2+ providers | 1–2 weeks |
| Understand the evaluation framework | evaluation-schemes.md Sections I–III — the theoretical foundation for all empirical work here | Reading (no code) |
Each plan file should have a Contributing to this program block at the top with skills needed, scope estimate, and a specific entry point.
This checklist applies to formal research programs with defined protocols and results locations. If you have an exploratory idea that isn't ready for this level of structure, start at wild/ instead.
Check these before you commit to a track:
- Do you need a specific runtime or toolchain such as Claude Code Agent Teams, LiteLLM, or API keys?
- Is the task a reading task, a one-session infrastructure task, or a multi-run experiment?
- Does the plan already define a results location and status convention?
- Are you improving an existing study or starting a new run within it?
Where results go: Each study with its own plan file has a results location next to it. For example, the ablation study has ablation-study.md (plan) and ablation-study/results/ (run outputs: tables, summaries, raw data). The umbrella doc (e.g. evaluation-schemes) does not hold results; each extracted design has its own plan and results folder.
Recording completed studies: In the plan file, add a Status section: Not started | In progress | Completed; list Runs with date and link to results (e.g. 2026-03-15 Run 1 — results/2026-03-run1.md); optionally one-line finding. In the umbrella doc, add a column or sentence linking to the study’s results when available.
Archiving: Plans are not archived when a study completes; the plan stays in place and is updated with status and links to results. Archive only when a plan is superseded or retired (e.g. move to research-programs/archive/ and link from the new plan or this README). Completed runs live in the study’s results folder.
These programs are ordered by how much they reduce the main open uncertainties: Does the methodology outperform simpler approaches? When does it help? How do we know? (These primarily address the decision-quality claim. The inspectability claim — that the committee produces auditable reasoning records regardless of outcome quality — is treated as structural rather than empirical. A separate meta-level question — can LLM pipelines do reliable mathematics under structured guidance? — is outlined in wild/llm-mathematical-inquiry-outline.md.)
| Priority | Uncertainty reduced | Plan(s) |
|---|---|---|
| Highest | Does committee-based deliberation beat simpler prompting? When is it worth the cost? | evaluation-schemes.md — research design for empirical comparison; ablation-study.md — component contribution (Phase 1); condorcet-comparison.md — deliberative vs. CJT-style comparison (2 runs, completed); societies-of-thought-research-plan.md Item 10 (comparative effectiveness study) |
| High | Does committee implementation architecture matter? | committee-implementation-taxonomy.md — design space (model diversity × agent independence); agent-independence.md — Tier 1 (same model, separate agents); multi-model-committee.md — Tier 3 (different models via LiteLLM, 4 phases); reference for architectural patterns and model profiles |
| High | Does the methodology transfer across domains? When does roster composition matter? | societies-of-thought-research-plan.md Items 4, 5, 9 (transfer learning, domain variants, worked examples) |
| Medium | Why does it work? Can we formalize and cite it? | societies-of-thought-research-plan.md Items 1–3, 6–7 (personality/balance/reconciliation, information-theory essay, social scaling theory) |
| Medium | How do we implement it in a formal platform? | societies-of-thought-research-plan.md Item 8 (MOOLLM integration) |
| Reference | What docs and TODOs are still open? | meta/project-state.md — current architecture truths, open decisions, and active repo-level follow-ups |
| Plan | Summary |
|---|---|
| evaluating-deliberative-architectures.md | Black Swan Hindsight Framework: tests which deliberative architectures (single LLM, hub-and-spoke, peer-agent, deliberated choice) anticipate structural risks that simpler approaches miss, using historical cases with known outcomes + constructed scenarios (Glenda/Crock coercion, blast radius). Includes knowledge contamination mitigations, operationalized metrics (anticipation, epistemic humility, decision landscape topology), blind evaluation protocol, and dedicated test protocols for the Glenda/Crock and blast radius scenarios. Results: evaluating-deliberative-architectures/results/. |
| societies-of-thought-research-plan.md | Ten action items from the Societies of Thought paper: strengthen infrastructure (Big Five, balance metrics, reconciliation), test generalization (transfer, domain variants), formalize theory (information-theory essay, social scaling), expand evidence (worked examples, comparative study), MOOLLM integration. Phased roadmap. |
| evaluation-schemes.md | Research design for evaluating the methodology: core questions, why direct evaluation is hard, proposed dimensions (assumption coverage, trade-off explicitness, etc.), protocols. No ground truth required. |
| ablation-study.md | Component contribution and interaction effects (Design F). Full procedure, factor definitions, run budget, results tabulation. Results: ablation-study/results/. |
| committee-implementation-taxonomy.md | Design space for committee implementations: two axes (model diversity × agent independence), three tiers (built-in single-model subagents, built-in multi-model subagents, external orchestration via LiteLLM). Umbrella document linking the implementation-focused research programs. |
| agent-independence.md | Does architectural separation improve deliberation? Tests whether running committee characters as independent agent processes (e.g., Claude Code Agent Teams) produces better results than single-context roleplay. Tier 1 in the taxonomy. Fast, cheap precursor experiment (Phase 1 is an afternoon). |
| multi-model-committee.md | Using different LLMs for different committee characters via LiteLLM: experimental protocol (4 phases), evaluation framework, risks, recommendations. Tier 3 in the taxonomy. Clojure/pcrit-llm preferred for orchestration. Reference material (architectural patterns, model profiles, implementation code) in multi-model-committee/reference.md. Build-out plan (pcrit-llm orchestrator, phased implementation) in multi-model-committee/build-out-plan.md. Results: multi-model-committee/results/. |
These plans have been executed or implemented. They live in agent/archive/ and are linked here for pattern and history.
| Plan | Status | Location |
|---|---|---|
| Wild material incorporation | Executed and verified | agent/archive/wild-material-incorporation-plan.md |
| Cross-reference and synthesis | Executed (11/11 items) | agent/archive/cross-reference-synthesis-plan.md |
| Augmentation (deliberation structure, evaluation loop) | Implemented | agent/archive/augmentation-plan.md |
| Evaluation feedback loop | Merged into augmentation plan | agent/archive/evaluation-feedback-loop-plan.md |
| Condorcet comparison (deliberative vs. CJT-style) | Completed (2 runs, 2026-02-22) | condorcet-comparison.md — results in condorcet-comparison/results/ |
| Theme | Plans | Status |
|---|---|---|
| Validation / evidence | evaluation-schemes, ablation-study, condorcet-comparison, societies-of-thought Items 9–10 | Active / Completed |
| Theory / foundations | societies-of-thought Items 6–7 | Active |
| Infrastructure / process | societies-of-thought Items 1–3, 8; augmentation | Active / Implemented |
| Implementation architecture | committee-implementation-taxonomy, agent-independence, multi-model-committee | Active |
| Integration | wild-material-incorporation | Executed |
- meta/README.md — overview of the meta directory and link to this collection
- essays/societies-of-thought-synthesis.md — source analysis for the Societies of Thought research plan
- meta/project-state.md — canonical current-state doc for repo architecture, open decisions, and documentation follow-ups