Complete underlying reasoning (99 claims and questions)
The main steps above are a reading path. These are all assessed contributions to this argument, including challenges and questions.
Answered · How do the authors describe uncertainty around the reported accuracy figures?
Uncertainty intervals inform interpretation of measured prediction quality but do not establish reliability across settings or populations.
How do the authors describe uncertainty around the reported accuracy figures?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered under conditions · Could the way a platform collects preferences change how well AI predicts them?
Platform-dependent performance bears on how far reported prediction results can be generalized beyond the studied setup.
Could the way a platform collects preferences change how well AI predicts them?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How sensitive were the reported results to the tested demographic details and language?
The tested variation bears on reliability, but the stated results are limited to particular models, categories, and language conditions.
How sensitive were the reported results to the tested demographic details and language?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How did Type II preference prediction compare with the study's other approach, called Type I B, and what explanation did the authors offer?
The comparison concerns reported prediction performance and must remain bounded to the particular model, data, and augmentation tests.
How did Type II preference prediction compare with the study's other approach, called Type I B, and what explanation did the authors offer? Here, augmentation means using a model to predict preferences.
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Did the models do better than a simple party-based prediction rule?
The question concerns a bounded prediction comparison, not general reliability or demonstrated civic representation.
Did the models do better than a simple party-based prediction rule?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · What do the authors infer from the individual prediction result?
This is an interpretation of a specific prediction comparison, not evidence of broad or reliable representation beyond the studied data.
What do the authors infer from the individual prediction result?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How do the authors measure agreement between predicted and recorded choices?
The measures are study-specific ways of evaluating prediction, relevant to the narrative’s caution that accuracy against recorded choices does not establish broader reliability.
How do the authors measure agreement between predicted and recorded choices?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How complete was the model and prompt testing?
The reported limits on model and prompt coverage qualify the scope of the study’s prediction comparisons, without establishing results for untested systems.
How complete was the model and prompt testing?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How did the authors define and test the rule based on party programs?
The benchmark is a particular comparison method for prediction performance, fitting the narrative’s emphasis on the specific models, measures, and comparisons used.
How did the authors define and test the rule based on party programs?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesPartly answered · Did the researchers examine how training-sample size relates to model results?
Training-sample size is a condition of the prediction tests; the evidence establishes that additional sizes were examined but does not establish their results.
Did the researchers examine how training-sample size relates to model results?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesPartly answered · Did models keep their choices when the options changed order, and how was this checked across models and samples?
Option-order stability is a reliability issue beyond accuracy against recorded choices, and the reported findings are limited to the tested models and sample settings.
Did models keep their choices when the options changed order, and how was this checked across models and samples?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Did the study's simpler model setups or combined predictions improve the results?
It concerns prediction performance in particular model comparisons.
Did the study's simpler model setups or combined predictions improve the results?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How accurately did the models predict individual choices in the reported test?
It concerns measured predictive performance in a particular study setting.
How accurately did the models predict individual choices in the reported test?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Did the researchers check accuracy with random samples of different sizes?
It concerns the sample conditions used in study-specific prediction-accuracy checks.
Did the researchers check accuracy with random samples of different sizes?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · What data and models did the authors study?
The question concerns which models underlie setting-specific prediction results.
What data and models did the authors study?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Which models were compared, and what distinguishes their training backgrounds?
Model identity and training-data conditions qualify the scope of reported prediction comparisons.
Which models were compared, and what distinguishes their training backgrounds?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Are the limitations described in the paper a complete list?
The caveat limits conclusions drawn from the specific prediction tests and listed shortcomings.
Are the limitations described in the paper a complete list?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · What does the study aim to contribute?
The benchmarks do not establish reliable real-world civic performance.
What does the study aim to contribute?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Which models do the authors test, and how do they use them?
Model choice and use define the scope of the reported prediction comparisons.
Which models do the authors test, and how do they use them?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Did the researchers compare the models' preference rankings?
Ranking comparisons are a particular measure of model prediction quality, and the missing values limit interpretation.
Did the researchers compare the models' preference rankings?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesOpen · Could adding information and further training help expand the data used by a model?
This is an open question about a proposed method for improving prediction-based data augmentation; the evidence reports no test of whether RAFT improves performance.
Could adding information and further training help expand the data used by a model?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Why did the experiment focus on pairs of proposals?
The paired-choice setup constrains the task on which reported prediction quality was evaluated.
Why did the experiment focus on pairs of proposals?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Can people understand why a model favors one proposal?
The question also concerns interpreting predicted choices, distinct from measuring accuracy against recorded choices.
Can people understand why a model favors one proposal?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Did adding model predictions for more people improve how closely the sample matched the study data?
This also concerns model prediction performance, with results varying by initial sample size and missing a separate 50% result in the supplied text.
Did adding model predictions for more people improve how closely the sample matched the study data?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Do the models give consistent answers when choices are presented differently?
Order sensitivity is directly relevant to reliability beyond accuracy against recorded choices; the evidence makes clear that inconsistency is not uniform across models.
Do the models give consistent answers when choices are presented differently?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Did prediction accuracy differ by political view or education in this study?
The specific group comparisons contextualize prediction accuracy across groups without establishing reliability beyond the studied participants and models.
Did prediction accuracy differ by political view or education in this study?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How did the models compare with the political-alignment rule for individual choices?
The comparison is limited to one baseline and does not settle what prediction accuracy establishes.
How did the models compare with the political-alignment rule for individual choices?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How do the researchers test whether a model predicts individual choices?
This asks about the prediction procedure underlying reported accuracy; the answer specifies the test setting and target.
How do the researchers test whether a model predicts individual choices?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · How did the models compare with the party-based rule and with predictions based only on political orientation?
The comparison concerns measured prediction performance; it does not prove reliable representation beyond the tested models, participants, and proposals.
How did the models compare with the party-based rule and with predictions based only on political orientation?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Could multiple AI agents perform differently from one model?
This identifies a boundary of the tested model configurations and prevents extending reported performance to untested multi-agent systems.
Could multiple AI agents perform differently from one model?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesAnswered · Why did the authors use a rule based on party programs for comparison?
The benchmark’s limited framing matters when interpreting what comparative prediction performance establishes.
Why did the authors use a rule based on party programs for comparison?
AI browsing question drawn from this source idea; not a quotation or a claim that the authors asked this question.
Inspect the contribution and sourcesImportant limit · Platform design may matter
The claim limits transfer of measured prediction results across platforms: the authors identify a possible design dependency but have not tested it comparatively.
The authors say model performance may depend on platform design. Brazucracia, the platform used in their study, ranks relevant proposals for people whose preferences may not fit one party perfectly; unlike some voting advice tools, it does not recommend a party or politician.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors identify the platform difference as a reason to study whether results depend on design. They do not test the effect by comparing platforms. Addresses: Effect of platform design
Inspect the contribution and sourcesStarting point · More ecological bases in the Amazon, though the source does not explain what they are
It contextualizes a policy option in the study’s comparison set. The unclear wording and proposal listing do not establish prediction quality on it or beyond the study.
One proposal would curb what the table calls “drug mining” and money laundering in the Amazon by increasing the number of ecological bases.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The phrase “drug mining” is unclear in the excerpt, so this summary retains it rather than guessing its intended meaning. The table attributes the proposal to Bolsonaro. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesStarting point · Meet the Paris climate targets
It contextualizes the policy choices used in the study, but does not establish prediction quality on this proposal or outside the tested setting.
One proposal would meet the country’s targets for reducing carbon emissions, agreed at the 2015 Paris conference.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The table attributes this proposal to Lula; it does not specify the actions or measures of progress. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesStarting point · Ways a language model can add predicted preferences
These are distinct prediction tasks, so success on one type of augmentation would not by itself establish reliability on the others.
The authors describe several ways to add predictions about preferences. Type I predicts preferences for additional people using demographic information, such as age. Type II A predicts more preferences for people already in the sample about proposals already included. Type II B predicts those people’s preferences about new proposals. Type II C combines those two Type II approaches. Type III adds predictions for both new people and new proposals.
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage distinguishes methods by whether they add people, preferences, or proposals. Type II C combines the two Type II methods; Type III combines predictions for new people and new proposals. Addresses: The methods for adding predicted preferences described in the study
Inspect the contribution and sourcesSupporting point · Individual choice predictions
This is a prediction-accuracy result against recorded choices, with benchmark and scope specified; it does not establish reliability under changed presentation or broad civic representation.
The authors report that the models predicted individual policy choices more accurately than a rule assigning people the proposals of the candidate matching their self-reported political orientation.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors use train-test cross-validation, which evaluates predictions on data held aside from model fitting. The comparison is with this specific rule, not every possible alternative.
Inspect the contribution and sourcesImportant limit · Results changed little in tested variations
The limited variations tested do not establish general reliability across all demographic characteristics, models, or languages.
The authors report that accuracy did not depend strongly on which demographic details were used. The tested categories were political ideology (liberal, conservative or centrist), location (city and state), sex (male or female) and education (non-college educated or college educated). The demographic tests used GPT-3.5 Turbo and Mistral 7B. They also report little change when prompts and consultation switched from English to Portuguese.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The report concerns particular demographic categories, two models and a change from English to Portuguese. It does not show that the method works equally well for all groups or languages. Addresses: Reported tests of changes in model inputs
Inspect the contribution and sourcesImportant limit · Why the experiment used pairs
Pairwise testing and the stated difficulty with larger choice sets limit generalization from these results to more complex choice settings.
The authors divided the 67 proposals into sets of two or five because ordering preferences across all 67 was challenging. They focused on pairs, citing earlier research that language models struggle to predict choices when shown three or more proposals.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The difficulty ordering all 67 proposals is a reason stated by the authors. The claim about model difficulty with larger sets comes from earlier research, not a new result from this study. Addresses: Experimental design
Inspect the contribution and sourcesStarting point · Restore damaged land
It provides context for the policy options included in the study’s comparisons, not evidence of accuracy or broad performance.
One proposal would restore land damaged by harmful activity and reforest devastated areas.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The table attributes this proposal to Bolsonaro and Lula; it does not specify locations, scale, or methods. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesImportant limit · Explore other data sources
The proposal is a possible extension of data augmentation; it does not establish improved predictive performance.
The authors suggest exploring ways to expand the data used by a model, including by using data collected in other ways, such as through voting advice applications.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors identify these data sources as an avenue for further study, without stating that they should improve the approach. Addresses: A proposed direction for future research
Inspect the contribution and sourcesStarting point · Sustainable farming practices
It contextualizes a type of policy choice in the prediction study’s comparison set, not the accuracy or generalizability of the models.
One proposal would encourage sustainable agricultural practices.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The table attributes this proposal to Bolsonaro and Lula; it does not define which practices count as sustainable. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesImportant limit · Recorded preferences may help
The proposed explanation is conjectural and does not establish general performance.
In tests with GPT-3.5 Turbo, the authors report that Type II A and B augmentation performed better than Type I B augmentation. Here, augmentation means using a model to predict preferences. The Type II tests used all demographic information and predicted additional preferences for people already in the sample. The authors suggest that these tests may benefit from using participants' recorded preferences as well as demographic details.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The reported comparison is for tests using all demographic information. The authors' explanation about the extra preference information is a possibility they suggest, not a cause tested separately. Addresses: Reported results and proposed explanation for Type II A and B augmentation
Inspect the contribution and sourcesImportant limit · The study develops tests for individual and group preferences
Preference benchmarks do not establish reliable performance in real civic decisions.
The authors say the study explores different ways of adding to preference data and develops tests for individual and group preferences.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. This is the authors’ stated aim, not a conclusion about how well the methods work. Addresses: The paper’s stated contribution
Inspect the contribution and sourcesImportant limit · How the proposals were collected
Prediction results are tied to this curated set of 67 proposals and its election context, rather than established for arbitrary policy choices.
The study used 67 policy proposals collected from Brazucracia.org and linked to three candidates in Brazil’s 2022 presidential election. A lawyer led the work of selecting and organizing the proposals.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. This describes where the proposals came from and how they were selected and organized, not a check that they were complete or fair representations of the candidates’ platforms. Addresses: Study materials
Inspect the contribution and sourcesImportant limit · Comparing participant groups
It identifies a subgroup dimension of the prediction evaluation, not a general reliability finding.
The authors plan to examine whether models predict choices differently across participants’ self-reported political ideology, age, education, and sex. For these comparisons, they balance the groups by downsampling the group with more participants; the supplied text does not include the results.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage introduces comparisons of accuracy across groups as a way to explore possible model bias. The figure caption names the groups and says the authors made the group sizes equal by reducing the larger group.
Inspect the contribution and sourcesImportant limit · Systems with several AI agents were not tested
This study’s single-model setup does not test the cited multi-agent findings.
The researchers did not use systems with multiple AI agents. They note that other studies have sometimes found such systems more accurate than one AI tool trained to work with language.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The study itself did not test systems with multiple AI agents; the comparison comes from other research cited by the authors. Addresses: Single-model study limitation
Inspect the contribution and sourcesImportant limit · Testing combined group preferences
The study-specific comparison does not establish reliability beyond the setting.
The researchers also compare whether ordinary random samples and samples expanded with model-generated choices can reproduce the combined preferences of all 267 participants.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The paper describes comparing ordinary random samples and samples expanded with model-generated choices against the combined preferences of all 267 participants.
Inspect the contribution and sourcesSupporting point · How the authors compared the recorded preferences
Identifies the specific data structure and comparison used, not a broader reliability guarantee.
The authors organize preference data in a matrix, a table of rows and columns. Each row represents a participant, and each column records a preference between a pair of policy proposals. They take samples and add model predictions to this data, then compare how well those samples reproduce the overall preferences in the full sample.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. This describes how the data are organized and how the authors checked whether a smaller sample, or one with added model predictions, matched the full participant data. Addresses: The structure of the preference data and how the authors checked the results
Inspect the contribution and sourcesImportant limit · Uncertainty around accuracy figures
The reported intervals are uncertainty information for particular accuracy and subgroup estimates, not evidence of performance beyond the tested data.
The authors report 95% confidence intervals, ranges used to show uncertainty around a result, based on 100 bootstrap iterations for individual-model accuracy. For comparisons between groups, they report 99% confidence intervals based on 100 iterations.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The passages specify different confidence-interval levels for the individual-model and group-comparison figures. Bootstrapping means repeatedly resampling data to estimate uncertainty.
Inspect the contribution and sourcesImportant limit · A limited set of comparisons
Results on this dataset do not demonstrate performance in other settings.
The dataset contains 8,719 choices between pairs of proposals from 267 participants. The choices cover 67 proposals, and the authors say the data are sparse, meaning they do not cover all possible comparisons.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors identify the limited set of recorded comparisons as a reason to study augmentation. Addresses: The study’s data coverage
Inspect the contribution and sourcesSupporting point · Testing individual predictions
The design evaluates predictions against recorded choices within the study.
The researchers train models on some participants’ choices. They test the predictions against choices from participants whose data were not used to train the models.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The researchers split participants into groups for training and testing. They compared the model’s predictions with choices from the test group.
Inspect the contribution and sourcesImportant limit · Adapting AI models to predict participants’ choices
The design is relevant to reliability under presentation changes but does not itself demonstrate order invariance.
The researchers give the AI models instructions and examples based on participants’ details and recorded choices. They include both orders of each proposal pair because a model’s choice may change depending on which proposal appears first. They use LoRA, a method that changes some of a model’s parameter values, to adapt about 12% of them while keeping the rest fixed.
Source coverage: Full readable text of the retrieved 37-page research draft, a research draft shared before formal publication, including references and appendices. Figures are not independently interpreted. The method gives the AI model instructions and examples using participant information and choices. Reversing proposal order is intended to address the possibility that the model’s choice changes depending on which option appears first. LoRA adapts a model by changing some of its parameter values without changing them all.
Inspect the contribution and sourcesImportant limit · Choices and option order
Directly illustrates that accuracy does not alone establish reliability under reversed presentation; findings remain limited to the tested models and samples.
The researchers defined a consistency rate as the share of choices between two options that stayed the same when the order of the two options was reversed. They reported rates across different models and sizes of samples selected using chance. GPT-3.5 Turbo and Falcon 7B were generally consistent in more than 70% of cases; the other models did not show this level of consistency despite similar accuracy. The excerpt gives no exact rates for each model.
Source coverage: Full readable text of the retrieved 37-page research draft shared before formal publication, including references and appendices. Figures are not independently interpreted. Passages 42–43 define the measure and report that results were presented across models and sizes of probabilistic samples. They give a broad comparison but not exact rates for each model. Addresses: Consistency under option-order changes
Inspect the contribution and sourcesStarting point · Six models adapted using volunteer data
The tested models and training-data context delimit which models the reported prediction findings concern; this setup alone is not evidence of broad performance.
The authors fine-tune six existing language models on the volunteer data: Llama-2 7B, LLaMA-3 8B, ChatGPT 3.5 Turbo, Mistral 7B, Falcon 7B, and Gemma 7B.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors adapt the models using collected policy-preference data, then use them to predict choices and group preferences.
Inspect the contribution and sourcesSupporting point · Accuracy across random samples
Provides context about the sample sizes used in accuracy checks, without establishing a reliability limitation or any substantive performance result.
The researchers compared model accuracy using random samples containing 5%, 25%, and 50% of the participants. The supplied passage names these sample sizes and says results were shown for five models, but does not give the accuracy values.
Source coverage: Full readable text of the retrieved 37-page research draft shared before formal publication, including references and appendices. Figures are not independently interpreted. Passage 42 says the researchers repeated an accuracy comparison using random samples of three sizes and identifies five models. The figure's results are not available in the supplied text. Addresses: Accuracy checks across sample sizes
Inspect the contribution and sourcesImportant limit · Defining the party-based rule
Clarifies the scope of one comparison used to evaluate prediction quality; results against this particular benchmark do not establish performance against every alternative.
The test rule predicted that a person who reported being a left-wing liberal would choose a proposal in Lula’s program, and that a person who reported being conservative would choose one in Bolsonaro’s. For liberals and conservatives separately, the authors compared choices of proposals from Lula’s or Bolsonaro’s program with choices of proposals from any other program. They left out pairs from the same candidate’s program in this comparison and examined those pairs separately.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The mapping and comparisons clarify what the test examined. Treating pairs from the same program separately defines what the first comparison covered.
Inspect the contribution and sourcesStarting point · More clean energy
It contextualizes the types of policy choices included in the prediction comparison, not model performance beyond the tested data.
One proposal would expand clean and renewable energy sources to strengthen the energy supply.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The table attributes this proposal to Bolsonaro and Lula; it does not specify technologies or targets. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesImportant limit · Earlier studies used models to imitate judgments and predict election choices
These prior findings concern different tasks and studies; they do not establish broad prediction quality for the current models or data.
Earlier studies cited by the authors reported similar moral judgments from language models and used detailed descriptions of people to predict election choices.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. These findings are attributed to earlier studies, not to the authors’ own experiment. Addresses: Background research on simulated participants
Inspect the contribution and sourcesImportant limit · How the study measured a match
Clarifies that a match on aggregate proposal win rates does not establish individual-level accuracy or broader reliability.
The study compares each proposal’s win rate in a sample with its win rate among all study participants. It then uses R², a score for how closely two patterns match, to measure how closely the sample reproduces the pattern for all participants.
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. This measures similarity to the full study sample; it does not directly test whether the preferences are correct for people outside the study. Addresses: The study’s measure of how closely a sample matches the full study
Inspect the contribution and sourcesStarting point · Portuguese prompt example
The prompt example specifies inputs used in a robustness check of prediction; it is methodological context, not evidence that predictions were accurate or robust.
For a robustness check, the researchers made Brazilian Portuguese prompts describing participant characteristics. The features included age (young or older/“third age”), political ideology (conservative, centrist, or left-wing), urban or rural zone, education (with or without higher education), sex (male or female), and city and state. One example described a 26-year-old liberal man with a master’s degree in Rio de Janeiro choosing a proposal to curb tax evasion over one to expand privatization of state-owned companies and infrastructure concessions.
Source coverage: Full readable text of the retrieved 37-page research draft shared before formal publication, including references and appendices. Figures are not independently interpreted. Passages 41–42 describe the Portuguese prompts, list their demographic features and values, and give an example pairing participant details with two policy proposals. They do not report whether this adaptation improved predictions. Addresses: Portuguese prompt design
Inspect the contribution and sourcesStarting point · Borrowing from investors for sustainable investments
It contextualizes a policy option in the prediction study’s comparison set; the listing does not show model accuracy on this option or beyond the study.
One proposal would offer bonds, a way to borrow money from investors, to fund investments considered sustainable, including in transport and energy.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The table lists transport and energy as examples and says there may be other areas; it does not define how projects qualify. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesImportant limit · Preparing Portuguese text for the BERT model
The prompt translation and embedding choices clarify the language-specific setup relevant to the reported language-variation test; they do not establish performance beyond that tested setup.
The researchers translated the instructions given to models into Brazilian Portuguese, using instructions provided in Appendix C. They said BERT’s prediction accuracy depends heavily on its embeddings: numerical representations of text that help it use context. They used BERTimbau embeddings, reportedly derived from brWaC, a Portuguese text collection with 3.53 million documents and 2.68 billion tokens. Tokens are the pieces into which a model divides text.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The appendix reports the translation procedure, the researchers’ reason for using embeddings (numerical representations of text), and the stated source and size of those representations. Addresses: Appendix C BERT methods
Inspect the contribution and sourcesImportant limit · Not all models or instructions were tested
The limited set of models and prompt variations narrows the scope of prediction-quality conclusions.
The authors say their exploration was limited: it did not include some newer models, such as GPT-4 and Claude 3, or test many versions of the instructions given to the models during training and use.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. These are stated limits of the authors’ exploration; the passage does not claim that newer models or different instructions would perform better. Addresses: Limits of model and instruction coverage
Inspect the contribution and sourcesSupporting point · Type III A combined two kinds of preference prediction
It reports a model-augmentation result, while its qualitative description and study-specific setting limit broader reliability conclusions.
The authors also report results for Type III A augmentation, which combines predictions for additional people and additional proposals. Here, augmentation means using a model to predict preferences. They describe its results as similar to the other augmentation tests; the figure caption specifies that this test used all demographic information.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage reports a general similarity to other tests, but gives no separate numerical result here. The Type III category combines predictions for new people and new proposals. Addresses: The reported Type III A preference-prediction result
Inspect the contribution and sourcesStarting point · A method adapted from urban planning
The paired-comparison method provides context for how the study elicits preference data; the analogy does not itself validate prediction performance.
The study adapts a paired-comparison approach from urban planning, where people compare options and computer systems use those choices to create a picture of which options people prefer.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors describe this as an analogy for their approach, not proof that the two applications have identical results. Addresses: The study’s methodological model
Inspect the contribution and sourcesStarting point · Farm-product industries and locally produced inputs
It identifies a kind of option in the study’s policy-choice comparison set, without establishing how well models predict choices on it.
One proposal would support industries that process farm products and produce inputs within the country. Inputs are things used to produce goods or provide services.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The table attributes this proposal to Bolsonaro and Lula; the entry continues across the page break. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesStarting point · Lower-priced transport services
As a comparison-set policy option, it helps contextualize the choices on which prediction quality is assessed, without establishing accuracy or performance beyond the study.
One proposal would improve transport services and reduce their prices by setting up concessions, which give providers the right to run services, and partnerships between the government and private businesses.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The table attributes this proposal to Bolsonaro; it does not describe the contract or partnership terms. Addresses: Table 1 policy proposals
Inspect the contribution and sourcesImportant limit · The authors report tests of a model setting called temperature
The absence of the test outcome limits what can be concluded about prediction robustness across settings.
The authors report that temperature sensitivity tests are presented in Appendix H. These passages do not state what those tests found.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. Passage 19 notes that the tests are presented in Appendix H but gives no outcome, so no conclusion about temperature's effect can be drawn here. Addresses: The reported tests of sensitivity to temperature, a model setting
Inspect the contribution and sourcesSupporting point · Predictions improved some sample matches
It reports specific augmentation performance while explicitly limiting generalization beyond the study setting.
In the Type I B test, the authors adjusted six models using the same preference data: GPT-3.5 Turbo, LLaMA-2 7B, Falcon 7B, Mistral 7B, LLaMA-3 8B, and Gemma 7B. They tested initial random samples of 5%, 25%, and 50%, adding predictions for 20% of the people who remained outside each sample. For the 5% sample, R², a score for how closely two patterns match, rose from about 30% to about 75%; for the 25% sample, the gain was about 7 to 10 percentage points. The supplied text gives no separate numerical result for the 50% sample.
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. These figures describe tests on the study’s data, with models adjusted using the same data and predictions added for a stated share of people outside the initial sample. The text identifies 50% as a tested sample size but gives no separate numerical result for it. These tests do not show how well the method would work in other settings. Addresses: The reported result for adding Type I B predictions
Inspect the contribution and sourcesImportant limit · Temperature, a setting that can affect model responses
This is a robustness qualification limited to the settings tested; it does not establish robustness across models or contexts.
The authors say changing the model’s temperature, a setting that can affect its responses, did not affect their results. They trained at temperature zero to make the process repeatable.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors report that other temperature settings did not affect results and point to supplementary material for details. Addresses: Training the models
Inspect the contribution and sourcesImportant limit · Keeping model architecture consistent
Holding architecture constant across training-set sizes helps define the scope of those comparisons, but does not control every difference or show performance beyond the tested models and data.
The researchers kept each model’s design the same across different training-set sizes to make comparisons more comparable.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage says the design of each language model was kept the same across training-set sizes; it does not say this controls every other difference. Addresses: Appendix C model-training methods
Inspect the contribution and sourcesSupporting point · Small samples gave better estimates in this test
It reports an improvement under a particular study measure and sample range; broader reliability remains outside the claim.
For samples below 30% of participants, the authors found that using models to improve estimates from the small sample gave a more accurate estimate of all participants’ preferences than estimates from random samples without model help.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors compare estimates of all participants’ preferences. They do not describe filling in individual participants’ missing responses. Addresses: Using models to improve estimates from a sample
Inspect the contribution and sourcesObjection · Changing option order could change answers
Order sensitivity challenges confidence in stable preference prediction, while its stated scope is limited to some cases.
The authors report that model predictions could be unclear and could change when the order of two options was reversed. In some cases, consistency was below 70%.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors report this result for some cases and point to a supplementary section; the passage does not say that all models or predictions had this level of inconsistency. Addresses: Consistency of predictions
Inspect the contribution and sourcesImportant limit · Two approaches did not improve results
Shows that performance depends on the tested model setup and combination method; conclusions should remain specific to these configurations.
In these tests, models that had not been adjusted using the study data always selected the first option when set to zero temperature, a setting that removes randomness from their choices. A method that chose the option supported by most models did not perform better than individual models.
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The findings concern the specific model settings described; they do not show that all models not adjusted using study data, or all ways of combining predictions, fail. Addresses: Reported results for models without this adjustment and a method that chose the option supported by most models
Inspect the contribution and sourcesImportant limit · Models can imitate some responses
The earlier tasks provide context for prediction research but do not establish accuracy for the current models, preference data, or settings.
Earlier studies cited by the authors found that language models could simulate some opinion-poll responses and predict votes in elections with two choices.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. These findings are attributed to earlier studies, not to the paper’s own experiment. Addresses: Prior research on simulated opinions
Inspect the contribution and sourcesStarting point · Six models tested on collected preferences
Identifies the models and prediction task underlying the reported performance comparisons.
The authors further trained six existing large language models, which are AI tools trained to work with language. They tested how well the models predicted preferences using detailed political preference data collected during Brazil’s 2022 presidential election.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The paper describes using detailed preference data collected during the election to study model predictions. This was not a test of a complete voting system. Addresses: Study method and data
Inspect the contribution and sourcesImportant limit · Fine-tuning did better than adding reference documents
The result is bounded by the single-model comparison and specified RAG documents, limiting broader claims about prediction methods.
In a comparison using GPT-3.5 Turbo, predictions from a model adjusted through fine-tuning with LoRA matched the full participants' preferences better than predictions made using RAG, especially with 25% and 50% samples. RAG, or retrieval-augmented generation, gives a model reference documents to use when answering. The reference document summarized proposals from the candidate favored by each participant: Bolsonaro for conservatives and Lula for liberals.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. This comparison used one model and reference documents selected according to participants' favored candidates. It does not establish that adjusting a model through fine-tuning is always better than giving it reference documents. Addresses: The study's comparison of fine-tuning with retrieval-augmented generation
Inspect the contribution and sourcesStarting point · Summarizing group preferences with proposal win rates
Specifies the win-rate measure used in group-level prediction comparisons; a match on this measure is not a complete reliability test.
To summarize policy preferences in a group, the authors define each proposal’s win rate as the share of recorded choices between two proposals in which it was selected. They then compare win rates from a sample with those from all participants in the study.
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. This approach summarizes choices even when participants did not compare every possible pair of proposals. Its conclusions concern how well a sample reproduces the results for all participants in the study, not necessarily the preferences of a wider public.
Inspect the contribution and sourcesImportant limit · Reasons for choices are unclear
Prediction accuracy does not by itself supply an explanation of why the model favors a proposal; interpretability remains a distinct limitation.
The authors say there is not yet a good way to explain why a language model chooses one policy proposal over another.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors identify this as a limitation of their approach. The passage does not propose or test a way to address it. Addresses: Explaining model predictions
Inspect the contribution and sourcesImportant limit · A proposal for AI helpers tailored to individuals
The evidence concerns prediction fit to this study’s recorded preferences, not general performance or real-world reliability.
The authors interpret the LoRA result as support for making AI helpers tailored to individuals by adjusting a model through fine-tuning. However, the reported comparison only tests how well predictions match the study participants' recorded preferences.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. A better match on this study's measure supports the authors' proposal for AI helpers tailored to individuals, but does not by itself show that such helpers work safely or well in real civic decisions. Addresses: The authors' interpretation and its limits as evidence
Inspect the contribution and sourcesSupporting point · Estimates of group preferences
Provides a group-level prediction result, but the absent magnitude and uncertainty leave the strength of the comparison unspecified.
Using cross-validation, a way to test a method on separate parts of the data, the authors report that a sample of people selected by chance, with each person's chance of selection known, augmented with model predictions estimated group policy preferences more accurately than the sample alone.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The paper estimates accuracy for individual choices and group preferences. The passages state the direction of the group-level result, but not its size or uncertainty.
Inspect the contribution and sourcesSupporting point · Reported stability in the sample
Provides a study-specific stability check for group preference estimates; it is distinct from individual prediction reliability.
The researchers estimated that two random samples of 53 participants each, about 20% of the study sample, would produce an R² of about 0.72 for proposal win rates. R² is a measure of how closely the rates in the two samples track each other. They reported an R² of 0.9538 when using the full sample.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The appendix presents the 53-person estimate and the full-sample repeatability result as findings from its check of the data, not as proof that participants represent the wider public. Addresses: Appendix B data adequacy checks
Inspect the contribution and sourcesImportant limit · Test added information and training
This proposed method could be evaluated for prediction quality, but the statement supplies no evidence that it performs better.
The authors suggest investigating whether combining added external information with further model training could improve data augmentation, or the way training data is expanded.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage identifies this as a possibility to explore, not a demonstrated improvement. Addresses: A proposed direction for future research
Inspect the contribution and sourcesImportant limit · Explore different viewpoints
This is a proposed model-evaluation direction, not evidence of prediction quality across viewpoints.
The authors suggest testing models guided toward different political viewpoints and then using other models to summarize those views.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage presents this as an avenue to explore and refers to earlier work; it does not report a finding from the proposed test. Addresses: A proposed direction for future research
Inspect the contribution and sourcesImportant limit · Did changing the setting affect model outputs?
The existence of sensitivity tests bears on model reliability, but the absent results cannot establish robustness or sensitivity.
The researchers changed the temperature setting, which affects variation in a model’s predictions, for GPT-3.5 Turbo and Mistral 7B to see whether it affected their outputs. The supplied passage contains no results from those tests.
Source coverage: Full readable text of the retrieved 37-page research draft shared before formal publication, including references and appendices. Figures are not independently interpreted. The appendix identifies two models tested, but only the figure caption is included and no results are described. Addresses: Temperature sensitivity checks
Inspect the contribution and sourcesImportant limit · Bias may differ across tasks
Cautions that model behavior in one task does not establish performance or bias in a different prediction task.
The authors distinguish bias in language tasks, such as assumptions linking jobs with gender, from bias in predicting civic preferences. They caution that a bias found in one task should not simply be assumed to operate in the other.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors make this distinction while comparing their work with earlier research. The passage does not deny that language-model bias may affect civic predictions; it warns against assuming the same pattern without evidence. Addresses: Limits on transferring bias findings across tasks
Inspect the contribution and sourcesImportant limit · Possible ways to draw out preferences
The proposed elicitation method does not itself establish reliable prediction or robustness beyond the tested choices.
The authors suggest that prompts or additional training might draw out preference patterns represented in language models’ training text.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors describe this as a possibility, not proof that models contain an accurate record of people’s views. Addresses: Possible ways to elicit preferences from models
Inspect the contribution and sourcesSupporting point · Individual prediction results
The reported accuracy figures and test setup directly concern prediction quality, while remaining limited to these models, participants, and data.
In a test trained on 50% of participants and evaluated on choices between two options from the other 50%, LLaMA-2 had 76.68% accuracy (reported uncertainty: ±0.0014), GPT 3.5 Turbo 70.4% (±0.0013), Mistral 7B 69.4% (±0.0017), Falcon 7B 69.3% (±0.0014), and BERT SQuAD 50.83% (±0.0015). The authors describe BERT SQuAD’s result as no better than random choice.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors report these accuracy figures for a test set made up of half the participants, after training on the other half. They estimate 95% confidence intervals, ranges used to show uncertainty around a result, with 100 bootstrap iterations. Bootstrapping means repeatedly resampling the data.
Inspect the contribution and sourcesSupporting point · Comparing proposal win rates in a sample and the full study
It specifies a measure used to assess reproduced group preference patterns, with the stated limits on what that measure shows.
The authors compare proposal win rates in a sample with the rates for all participants in the study. They use the squared Pearson correlation, a statistical score for how closely two sets of values follow the same pattern. The authors treat a value of 100% as identical win-rate patterns.
Source coverage: The full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. This describes the study’s chosen measure of similarity. A high score on this measure concerns the pattern of proposal win rates; it does not by itself establish that every individual’s choices or every other feature of the group is represented.
Inspect the contribution and sourcesImportant limit · Comparing preference rankings
It identifies a ranking comparison but withholds the results, so it cannot support a conclusion about how well rankings were reproduced.
The researchers report Kendall rank correlations for six models on a test set containing 50% of participants, but the supplied passage gives no correlation values.
Source coverage: Full readable text of the retrieved 37-page research draft shared before formal publication, including references and appendices. Figures are not independently interpreted. The appendix identifies the measure, models, and test-set size, but the figure's values are not supplied. Addresses: Kendall rank-correlation analysis
Inspect the contribution and sourcesPossible connection · Preferences beyond political labels
This interprets a particular comparative prediction result; it does not establish reliability beyond the tested data and comparison rule.
The authors interpret the individual prediction result as showing that models capture preference details beyond what the political-alignment rule predicts.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. This is the authors’ interpretation of models outperforming the rule based on self-reported political orientation. It does not show that models capture every aspect of a person’s views.
Inspect the contribution and sourcesImportant limit · Different training-sample sizes
The existence of other sample-size tests provides methodological context but, without their results, does not establish prediction reliability or its dependence on sample size.
The authors also report results for models trained on random samples of 5%, 25%, and 75% of participants, in addition to the 50% training sample used for the main test of individual predictions.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors say that results for models further trained on random samples of 5%, 25%, and 75% of participants appear in Supplementary Appendix E, an additional section of the paper.
Inspect the contribution and sourcesImportant limit · How group accuracy was checked
The method clarifies the scope of accuracy comparisons and cautions against treating the check as a complete reliability guarantee.
To compare prediction accuracy across groups, the authors trained models again on samples with equal numbers from each group when the training data had more people from one group. They then tested accuracy separately for each group.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. For example, their method matched the number of people from two groups by randomly selecting people from the larger group. This describes how they checked for differences, but does not establish that making group sizes equal removes every possible source of bias.
Inspect the contribution and sourcesSupporting point · Models beat a simple party rule
This is a comparative prediction result against one specified rule, not a general demonstration of reliable representation.
The models also predicted group preferences more accurately than a rule that assumes people always choose the policies of their preferred party or politician.
Source coverage: Full readable text of the 37-page research draft, including references and appendices. Figures are not independently interpreted. The authors interpret the comparison as evidence that their models captured preference details beyond a simple left-right political divide. It does not show what specific information the models used. Addresses: Comparison with party-based prediction
Inspect the contribution and sourcesImportant limit · Some reasoning tasks are difficult
The warning concerns task-specific capabilities, not the reported preference-prediction tests; it reinforces that performance evidence is task-bounded.
Language models may perform poorly on some reasoning tasks, including mathematics and reasoning about space. This could matter when a decision depends on those skills.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. This is a task-specific caution, not a claim that models fail at every policy task. Addresses: Limits on using models in decision systems
Inspect the contribution and sourcesSupporting point · Models usually beat a simple prediction rule in this study
The findings concern individual-choice prediction against a particular baseline and include stated exceptions, so they do not establish broad or universal accuracy.
The models were more accurate than the party-based rule in 21 of 24 comparisons. The exceptions were cases where liberals chose proposals from Bolsonaro’s program for Mistral, Falcon, and Gemma. For participants who identified as left-wing and chose proposals from Lula’s program, LLaMA-2 7B was 15 percentage points more accurate than the rule; all six models reached at least 80% accuracy, compared with 71% for the rule. For choices between proposals from the same candidate’s program, statistical tests found evidence that the models could predict choices, with 65% to 77% accuracy. Overall, the authors conclude that models adapted using LoRA, a method for training a model for a task, predicted individual preferences better than predictions based only on participants’ stated political orientation.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The results favor the models in most reported comparisons, but include specific exceptions. The authors also report significant predictive power for choices between proposals from the same candidate’s program. Their overall conclusion is limited to the study’s models, participants, proposals, and comparison methods.
Inspect the contribution and sourcesImportant limit · Model choices are hard to explain
Prediction results do not resolve the separate question of why the model favored a proposal.
The authors say there is no adequate way to explain why a language model selects one proposal over another.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage identifies the lack of understanding of model choices as a limitation of the proposed approach. Addresses: The authors’ stated limitation
Inspect the contribution and sourcesImportant limit · Measures of agreement
Specifies the particular measures and their relationship in the study, fitting the narrative’s qualification that reported prediction comparisons depend on particular measures and study data; it does not itself establish model quality.
The authors use accuracy and report Kendall’s tau, a measure of how well the order of choices agrees, as another way to compare choices. They say F1 is not a separate measure for these data. Each correct choice creates both a true positive and a true negative, while each incorrect choice creates both a false negative and a false positive. As a result, precision, recall, F1, and accuracy have the same value. Precision and recall are measures based on correct and incorrect predictions.
Source coverage: Full readable text of the retrieved 37-page research draft, including references and appendices. Figures are not independently interpreted. The passage describes accuracy and Kendall’s tau, a measure of agreement in the ordering of choices. It also explains how the paired-choice data link the positive and negative outcome counts used to calculate F1, a score based on precision and recall.
Inspect the contribution and sourcesImportant limit · Task-specific training may help
This comparison depends on the particular models and study setting, and its proposed further improvement is untested—both relevant limits on reliability inference.
In a comparison, the authors report that their smaller model, trained for this task, performed better than a prompted version of the larger Gemini 1.5 Pro model. They suggest that training a larger model for this task might improve results further.
Source coverage: Full readable text of the 10-page Science article copy, including references. Supplementary materials are not included. The comparison is a reported study result; the suggested improvement is a possibility, not a tested outcome. Addresses: The model comparison and proposed next step
Inspect the contribution and sources