From a promising idea
to a testable claim.
Our three research directions connect questions of AI safety, accountability and the distribution of benefits. This note explains the terms, the relevant literature and the studies we propose. These are research plans, not findings from Beyond.
Selected literature reviewed 24 October 2026. This is a focused review of the claims in our agenda, not an exhaustive systematic review. Sources include standards, peer-reviewed work and identified preprints. Citations do not imply affiliation or endorsement.
What we mean by safety and governance
We use AI safety to mean reducing the likelihood and severity of harm in a specified context of use. Governance concerns who sets objectives, has oversight, can challenge decisions and is accountable for outcomes. Decentralization distributes some of these functions; it does not automatically produce legitimate decisions or safer systems.
NIST’s AI Risk Management Framework connects governance with context-specific measurement and management of risk. Weidinger and colleagues distinguish model capabilities, human interaction and systemic impacts. Our programme draws on that broader sociotechnical perspective, with an emphasis on technical AI governance, participatory alignment and benefit-sharing. Our initial pilots address specific governance mechanisms; they do not establish control of advanced AI or prevention of catastrophic risk. [1] [2]
Human oversight, loss of control and AGI governance
Our long-term motivation is to help preserve meaningful human oversight as AI capabilities advance. Loss of control broadly concerns situations in which humans can no longer reliably direct, constrain or intervene in an AI system’s behaviour; definitions and severity thresholds differ across the literature. We use the term to describe a risk motivating governance research, not a failure mode our initial pilots have demonstrated that they can prevent. [14]
Frontier AI governance concerns oversight of the most capable systems; AGI governance also asks how institutions should prepare for possible broadly capable future systems. Relevance to those settings must be established rather than assumed. Our starting questions are narrower: what evidence supports an audit, whose input shapes a model specification, and which allocation rules resist manipulation?
Decentralized decision-making is an institutional design choice, not a safety guarantee. Distributing authority can also fragment responsibility or make coordination harder. We propose to compare governance arrangements by their auditability, accountability and resilience to capture, including trade-offs involving privacy and concentration of power. Work on compute governance illustrates why these trade-offs require explicit evaluation. [15]
01 / Data provenance and auditability
The distinction. Provenance describes the origins and history of an artefact. Data lineage tracks derivation and transformations within that history. Credit attribution names a source or contributor; training-data attribution estimates influence on a model or prediction. These are different tasks.
W3C PROV provides a vocabulary for entities, activities and agents. The Data Provenance Initiative shows why dataset sources and licensing records merit careful auditing. C2PA addresses signed provenance assertions for digital content; it is not a complete training-data lineage system. Its documentation explicitly separates tamper evidence from the truth of assertions. [3] [4] [5]
Proposed pilot. Build a small pipeline with versioned datasets, transformation records and model checkpoints. Use ordinary documentation as a baseline, then introduce known missing links, file changes and conflicting permission statements. Compare detection rates, false alarms, lineage coverage and time to investigate the same incidents.
Safety hypothesis. Better records may make it easier to identify and respond to data-related incidents. They do not detect every poisoned example, establish legal permission, prove consent or determine how much a training item influenced an output. A signed claim can still be false. Store only necessary metadata and test whether access controls avoid exposing sensitive contributor information.
02 / Participatory and pluralistic alignment
The distinction. Participation concerns who has input into objectives and evaluation. Pluralistic alignment concerns how systems represent or serve different values and perspectives. Preference elicitation gathers judgements; aggregation combines them. Evaluating those judgements is not the same as training a model to behave accordingly.
Collective Constitutional AI demonstrates a process that incorporates public input into principles, training and evaluation. The pluralistic alignment roadmap distinguishes representing a range of views, steering toward particular views and matching a population’s distribution of views. These are different objectives and require different tests. [6] [7]
More recent work makes the need for concrete evaluation especially clear. OVERTONBENCH measures coverage of viewpoints; that does not make it a general safety benchmark. A July 2026 roadmap argues for stronger empirical evidence of benefit and more explicit evaluation of trade-offs. The latter is a research position in a preprint, not settled consensus. [8] [9]
Proposed pilot. Choose one task and identify the affected population. Document participant recruitment, consent and underrepresented perspectives. Compare a pooled preference baseline with alternatives that retain disagreement. Evaluate on held-out participants and prompts, with uncertainty intervals and results by relevant groups where privacy permits. If we adapt a model, evaluate the resulting behaviour separately.
Safety hypothesis. The process may expose objectives or harms missed by a single aggregate score. Test harmful-response rates, unjustified refusals and usefulness alongside preference satisfaction. Specify safety constraints and an appeal process; do not treat a majority vote as permission to harm a minority. A bounded preference study cannot establish robustness under deployment shifts or solve the wider alignment problem. [10]
03 / Benefit-sharing and incentive design
The distinction. “Value redistribution” is an umbrella theme, not one established AI-safety method. Contributor compensation, funding public goods and redistributing benefits across society are separate problems. A contribution metric cannot by itself determine a just distribution.
Data Shapley estimates contribution to a chosen model-performance objective. Its values depend on the dataset, learning procedure and evaluation metric; they are not universal prices or measures of moral entitlement. Quadratic funding instead concerns allocating matching support to public goods. It should not be presented as a method for valuing individual training examples. [11] [12]
Proposed pilots. For compensation, compare equal shares and a documented contribution rule with approximate data valuations on a small, fixed task. For project funding, separately compare equal allocation, proportional matching and quadratic matching under the same budget. State the fairness objective rather than claiming a single objectively fair rule.
Stress-test duplication, collusion, strategic reporting and Sybil attacks, where one actor presents multiple identities. Report allocation concentration, sensitivity to assumptions, computational cost and the advantage an attacker can obtain. The classic Sybil result explains why counting accounts is not the same as counting independent participants. [13]
Safety hypothesis. Funding and compensation rules may affect incentives to disclose incidents, maintain useful infrastructure or capture decision-making. A simulation can test a mechanism under explicit assumptions; it cannot demonstrate real-world harm reduction. Empirical follow-up would be required before making that claim.
A common standard for proposed studies
Each project should state its question, threat model, baseline, data-access requirements and measurable outcome before evaluation. Report negative results, uncertainty and distributional effects. Human-participant work needs appropriate consent and ethical review; releases need a privacy and misuse assessment. Our initial output would be a protocol and a reproducible pilot, not a general safety guarantee.
Selected references
- NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023. Risk management framework; the 1.0 edition is the version cited here.
- Weidinger et al. Sociotechnical Safety Evaluation of Generative AI Systems, 2023. Framework paper, arXiv version.
- W3C. PROV-DM: The PROV Data Model, 2013. W3C Recommendation.
- Longpre et al. The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI, 2023. Dataset audit; cited arXiv version.
- C2PA. Content Credentials Explainer, specification 2.4. Technical guidance on provenance assertions and their limits.
- Huang et al. Collective Constitutional AI: Aligning a Language Model with Public Input, FAccT 2024.
- Sorensen et al. A Roadmap to Pluralistic Alignment, ICML 2024. Position paper.
- Poole-Dayan et al. Benchmarking Overton Pluralism in LLMs, ICLR 2026. Cited version revised March 2026.
- Poole-Dayan et al. A Roadmap to Impactful Pluralistic Alignment Research, July 2026. Preprint.
- Amodei et al. Concrete Problems in AI Safety, 2016. Research agenda covering objectives, supervision and learning-related risks.
- Ghorbani & Zou. Data Shapley: Equitable Valuation of Data for Machine Learning, ICML 2019.
- Buterin, Hitzig & Weyl. Liberal Radicalism: A Flexible Design for Philanthropic Matching Funds, 2018. Working-paper version.
- Douceur. The Sybil Attack, IPTPS 2002.
- Apollo Research. The Loss of Control Playbook, Chapter 1: A Taxonomy of Loss of Control, 2025. A proposed taxonomy and review of definitions, not a universal consensus.
- Sastry et al. Computing Power and the Governance of Artificial Intelligence, 2024. Governance mechanisms and trade-offs; not evidence that decentralization is inherently safer.