Contribution to the handbook
Supporting · Assessed 2026-10-28
The evidence that non-fine-tuned models selected the first option in the tested setup and that a majority-vote ensemble did not outperform individual models helps identify limits in specific technical alternatives. The result is narrow and does not determine a governance design, but it is relevant context for testing rather than assuming these approaches improve performance.
Does this challenge the goal?
In-scope gap
The question compares specific model setups within the existing goal of developing and testing practical governance proposals; it does not contest that goal.
The reported outcomes concern particular models and settings and do not establish how other models or combinations would perform.
Why this category?
- comparison of alternatives
- observed limitation
- bounded applicability
The findings apply to the specific model settings described; they do not show that all simpler setups or all methods of combining predictions fail.
This is an AI assessment of contribution to the goal, separate from whether the question has an answer.
Authority, accountability and resilience →
Tests, stop conditions and unresolved work →
Read the exploration goalState whether the simpler model setups improved results and whether combining predictions improved on the relevant individual-model results.
What we can say so far
The reported simpler baselines did not improve the results: BERT SQuAD reached 50.83% accuracy, which the authors describe as not better than random choice, and the tested non-fine-tuned models selected the first option at zero temperature. Combining predictions by majority vote also did not outperform individual models. In a separate, narrow setup comparison, GPT-3.5 Turbo fine-tuned with LoRA performed better than the tested RAG method; that result does not establish that simpler setups generally help.
What remains open
- The supplied evidence does not explain why majority voting failed to improve results.
- It does not establish how these methods would perform in other tasks, populations, or model settings.
Conditions and limits
- These findings concern the specific models and settings tested; they do not establish that every simpler setup or every ensemble method will fail.
- The BERT accuracy was reported for a participant-held-out test, with 95% confidence intervals calculated using 100 bootstrap iterations.
- The fine-tuning-versus-RAG comparison used one model and reference documents selected according to participants’ favored candidates.
- The evidence is an interpretation of the study text, not independent new evidence.
Evidence and related ideas
Supporting evidence, qualifications and context play different roles in this answer.
context
Models in the comparison The comparison includes six language models and BERT SQuAD, a simpler benchmark. The models differ in what is known about their training: GPT 3.5 Turbo has proprietary architecture and data; LLaMA-2, LLaMA-3, Mistral, and Gemma are open-weight models trained on proprietary data; Falcon is open-weight and trained on public data; and BERT is a foundation model trained on public data.
Identifies BERT as the simpler benchmark and describes the models included in the study.
Large Language Models (LLMs) as Agents for Augmented Democracy
Read full source: Large Language Models (LLMs) as Agents for Augmented Democracy (opens in a new tab) ↗Read source discussion: Large Language Models (LLMs) as Agents for Augmented Democracy →