Contribution to the handbook
Priority · Assessed 2026-10-27
This is a consequential legislative-design question about assessing draft quality without replacing citizens’ policy choices. Its importance does not depend on assuming AI review is effective.
Does this challenge the goal?
In-scope gap
The question advances the existing goal by examining draft-rule review and reviewer disagreement. It does not contest the goal or prescribe who should hold final authority. The supplied material proposes separating policy value from drafting quality but leaves the review process and its authority open; it does not establish AI capability or a disagreement procedure.
The proposed distinction may be difficult to apply because clarity and interpretation can affect policy. The packet provides no empirical assessment of AI review capability.
Why this category?
- legislative design
- citizen authority and safeguards
- capability and process testing
The evidence items are interpretations of shared source material, not independent corroboration. No AI review capability, procedure, or method for resolving disagreement is established.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Making and interpreting laws →
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalAssess whether AI can appropriately review those drafting qualities, distinguishing demonstrated capability from the possibility raised.
What we can say so far
The supplied material does not demonstrate that AI can appropriately review draft rules for clarity, consistency, and ambiguity. It describes AI-generated discussion questions that people reviewed for clarity, among other qualities, and reports quality and clarity ratings for AI-generated group statements. These are adjacent activities, not evaluations of AI reviewing rule drafts. The material therefore raises the possibility of AI rule-quality review but does not establish its capability or appropriateness. That is a limit of the evidence, not evidence that AI cannot do it.
What remains open
- No supplied study tests AI review of draft rules for clarity, consistency, or ambiguity.
- No supplied results establish the accuracy, reliability, or appropriateness of such reviews.
Conditions and limits
- The discussion-question study describes people reviewing AI-generated questions; it does not report AI reviewing rules.
- Ratings of AI-generated statements concern those statements and do not establish performance on rule drafts.
- The distinction between drafting quality and policy value is a proposal; it does not establish that quality review is free of policy judgments.
- Absence of direct evidence here does not establish that AI is incapable of the task.
Objections to consider
- The study's review process involved people checking AI-generated questions, so it cannot establish that AI itself can review drafts for the specified qualities.
- The supplied interpretation raises the role as a possibility but explicitly supplies no evidence or procedure establishing it.
- The passage identifies barriers to review in some systems, but does not establish how those barriers bear on AI review of draft rules.
- The proposed separation leaves the appropriate review process and authority open and acknowledges that drafting judgments can affect policy.
- Reported option-order sensitivity in some prediction cases cautions against assuming reliability in another task, but does not determine rule-review performance.
- Reported consistency differed across models in a prediction task; the evidence does not show whether rule-quality reviews would be consistent.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
qualifies
Questions were generated and reviewed The authors used Chinchilla 70B to generate 10,000 discussion questions from 175 author-written examples. They reviewed the questions for safety, ethics, clarity, and whether they could support a range of views.
Specify how disagreements among reviewers should be resolved without transferring citizens’ policy choices to the reviewers.
What we can say so far
The supplied material does not specify a procedure for resolving disagreements among reviewers. It supports only a boundary: drafting quality should be assessed separately from policy value, and reviewers should not replace the policy choice citizens authorized. Because judgments about clarity and interpretation can affect policy, the material does not establish who should arbitrate disputed assessments or when a disagreement should be returned to citizens. A complete resolution procedure cannot be specified from this evidence.
What remains open
- No procedure is supplied for resolving conflicting reviewer assessments.
- The evidence does not say who has authority to decide whether a disagreement is merely about drafting or affects policy, or when such a dispute should return to citizens.
Conditions and limits
- The quality-versus-policy distinction is proposed, not established as a settled process.
- Clarity and interpretation judgments can affect policy, so the boundary between drafting review and policy choice remains important.
- The source interpretations are not independent new evidence.
Objections to consider
- The evidence itself leaves the appropriate quality-review process and its authority open, so it does not establish how reviewer disagreement should be resolved.
- The possibility that review empowers a small expert group is relevant to assigning dispute-resolution authority, but the supplied material does not provide safeguards or a resolution procedure.
- The reported revision mechanism does not specify what happens when critiques conflict and therefore cannot resolve the procedural gap.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
qualifies
Assess policy value separately from drafting quality Citizens’ votes could determine a rule’s value for citizens, while drafting quality—such as clarity, precision, and lack of ambiguity—could be assessed more systematically. Both dimensions matter.
Supports separating drafting quality from policy value, while explicitly leaving the process and its authority open.
No public source link is available for this evidence.
supports
Citizens authorize rules; AI acts within their mandate Citizens retain political authority by choosing rules. AI may interpret those rules, coordinate their implementation, and direct executive activity only within the authorized scope. This concerns governing decisions, not simply automating public services; how much implementation remains human is undecided.