Surveys that measure self-report are a workhorse in psychology and the social sciences, providing a vital window into beliefs, attitudes, and emotions, both at the level of groups and individuals. The validity of self-report data, however, is an enduring methodological concern, with self-reports vulnerable to a range of response biases, including (among others) the risk of social desirability bias in which, rather than responding honestly, participants answer questions in a way that they believe will be viewed favorably by others. One proposed solution to socially desirable responding is the so-called Bayesian Truth Serum (BTS), which aims to incentivize truthfulness by taking into account the relationship between an individual’s response and their belief about the dominant (or most likely) response given by other people and then assigning a high truthfulness score to answers that are surprisingly common (Prelec, 2004).

Although valid in theory (under a variety of assumptions), questions remain regarding the empirical utility of the BTS. One area of concern is participants’ uncertainty regarding incentives for truth-telling – if participants don’t understand the extent to which telling the truth is in their own interests (or they don’t believe that it matters) then the validity of the BTS is undermined. 

In the current study, Neville and Williams (2025) tested the role of clarifying incentives, particularly for addressing social desirability bias when answering sensitive questions. The authors administered an experimental survey design (N=877) including sensitive questions, curated from validated scales, that are relevant to current social attitudes and sensitivities (e.g., “Men are not particularly discriminated against”, “Younger people are usually more productive than older people at their jobs”). Three groups of participants completed the survey under different incentive conditions: the BTS delivered alone in a standard format, the BTS with an interim bonus payment that is awarded to participants (based on their BTS score) halfway through the survey to increase certainty in incentives, and a Regular Incentive control group in which participants receive payment without additional incentives.

The authors analyzed the effectiveness of the BTS through two registered hypotheses. First, the authors found that the BTS did not increase agreement with socially undesirable statements (compared to the control group), as theory would suggest, and even observed an opposite effect. This result, which could be confirmed by follow-up studies, raises some concerns about the robustness of the BTS method. Second, the authors conjectured that introducing an interim payment in the BTS mechanism would help reinforce its credibility in the eyes of the participants and would thus magnify its effect. However, the authors failed to detect a statistically significant difference between the standard BTS and interim-payment BTS mechanisms. Overall, the results of Neville and Williams (2025) call for some caution in the use of the BTS and for further work to better understand the contexts in which the BTS might be a useful tool to mitigate social desirability in surveys. 

This Stage 2 manuscript was evaluated over one round of in-depth review by two expert reviewers and a second round of review by the recommender. After the revisions, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.

Decision-making based on limited information is a common occurrence. Whether it is the possibility of a cheaper product elsewhere or the unknown qualifications of election candidates, people are regularly forced to make a decision under ignorance or uncertainty. In such situations, information about certain events is unavailable or too costly to acquire and people rely on subjective probability allocation to guide decision-making processes. This allocation seems to result in what is known as ignorance priors, i.e., decision-makers assigning equal probabilities to each possible outcome within a given set. How events are grouped or partitioned is often subjective and may influence probability judgments and subsequent decisions. In such cases, the way the choices within a choice set are presented may shape the perceived likelihood of different outcomes. Understanding the impact of partitioning on probability estimation is crucial for both psychological and economic theories of judgment and decision-making.

In the current work, Ding and Feldman (2025) conducted a replication study of one of the foundational works on the topic: Fox and Rottenstreich (2003). In the original work, the authors provided exploratory evidence indicating that the framing of a situation affects the way individuals perceive probabilities of possible outcomes. They showed that people assigned uniform probabilities to sets of events described in a problem, such that the way the events are described partly determined people’s partitioning of those events and evaluations of the probabilities of the possible outcomes. Additionally, this partitioned framing affected judgments both under conditions of ignorance (where individuals have no information and rely solely on uniform probability assignments) and uncertainty (where individuals have some information but still rely on heuristics influenced by partitioning). This suggests that priors resulting from the inference of available evidence are sometimes partly contaminated by partitioning bias, affecting both uninformed and partially informed decision-making processes. As a consequence, the partitioning of events into different subsets might lead to varying evaluations of a single situation, resulting in inconsistencies and poorly calibrated probability assessments.

Ding and Feldman (2025) conducted a replication work on Studies 1a, 1b, 3, and 4 from Fox and Rottenstreich (2003). Their close replication relies on original data (US participants, Prolific, N=603) with a large statistical power (>95%). The replication aimed to assess whether the partitioned framing affects prior formation under ignorance (Studies 1a, 1b, and 4) and uncertainty (Study 3). The authors also proposed an extension examining estimations of complementary events contrasting estimations of the probabilities of the events happening versus the probabilities of the events not happening.

Overall, the authors successfully replicated the original study based on their pre-registered evaluation criteria, finding support for partition dependence for most scenarios under scrutiny, yet with weaker effect sizes than the original studies. Out of the eleven Cohen’s h estimated by the replication study, one is consistent with the original study’s estimate (i.e., the original point estimate lies within the confidence interval of the replication), seven go in the same direction but are smaller (i.e., same sign for the estimated effect but the original point estimate is outside the CI of the replication), and two are not statistically different from zero (i.e., the CI of the replication includes zero).  

• Advances in Cognitive Psychology
• Collabra: Psychology
• Cortex
• Experimental Psychology
• F1000Research
• Journal of Cognition
• Peer Community Journal
• PeerJ
• Royal Society Open Science
• Studia Psychologica
• Swiss Psychology Open