Probability and randomness are foundational statistical concepts used not only throughout the sciences, but also in our daily lives to guide our behavior and make sense of the world. Their importance and widespread use may suggest that they are easy concepts to understand, yet that seems not to be the case. A classic article by Tversky and Kahneman (1971) on the “belief in the law of small numbers” revealed that professional psychologists tended to incorrectly perceive a small sample that is randomly drawn from a population as representative of that population. This finding has been hugely influential, inspiring myriad subsequent studies into error and bias when reasoning about probability. 

 

In the current study, Hong and Feldman (2023) propose a conceptual replication and extension of Tversky and Kahneman (1971). The original article was shockingly sparse on details regarding the method, sample, and findings, and, to our knowledge, has never been replicated. These facts are especially concerning given the foundational status that the article holds in the field. Hong and Feldman (2023) have developed a conceptual replication project, using the same approach and targeting the same claims from Tversky and Kahneman (1971), but modifying the wording of the stimuli for clarity and appropriateness for lay respondents. Although Tversky and Kahneman (1971) relied on professional psychologists as participants, many of their claims were not restricted to that population, but rather were generalized to all people—which is also how the findings have been subsequently applied. Thus, the change from professional to lay responders is entirely appropriate and the study will be diagnostic of the original claims.

 

Finally, Hong and Feldman (2023) extend the target study by manipulating the sample size indicated in the stimuli. Tversky and Kahneman (1971) relied on a single sample size in each scenario, leaving open the question as to how sample size might impact respondents’ reasoning. Accordingly, Hong and Feldman (2023) vary the sample size across the scenarios to determine whether participants answer differently as the sample size increases. 

 

The Stage 1 manuscript was evaluated over two rounds of in-depth review, the first round consisting of detailed comments from two reviewers and the second round consisting of a close read by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and was therefore awarded in-principle acceptance (IPA).

 

URL to the preregistered Stage 1 protocol: https://osf.io/fzbq7
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

• Advances in Cognitive Psychology
• Experimental Psychology
• F1000Research
• Journal of Cognition
• Meta-Psychology
• Peer Community Journal
• PeerJ
• Royal Society Open Science
• Swiss Psychology Open

 

 

1. Hong, C. K., & Feldman, G. (2023). Revisiting the “Belief in the law of small numbers”: Conceptual replication and extensions Registered Report of problems reviewed in Tversky and Kahneman (1971). In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/fzbq7

 

2. Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76(2), 105–110. https://doi.org/10.1037/h0031322