Recommendation
DOI or URL of the report: https://psyarxiv.com/tfp2c
, posted 28 Mar 2023, validated 28 Mar 2023Once again sorry for the far too long delay in getting back to you. In the end I asked 60 potential reviewers - and three responded who were experts in relevant areas, and I was very pleased those were the ones we now have. They are largely happy with your proposal, but have various comments concerning details of the protocol and requested clarifications.
, 06 Mar 2023In this article, the authors propose and aim to test a new framework for understanding when people will disclose information during an intelligence interview called the Disclosure Outcomes Management Model. This model frames the decision to provide information as a self-interest dilemma in which the interviewee must balance the potential benefits (e.g., community safety, upholding morality) of disclosing risky information with the potential harms to the self (e.g., retaliation from the group being reported on). They then describe the calculations of risk and benefit result in four categories, with low-stakes and guarded information (theoretically) being less likely to be disclosed, unguarded information being more likely, and high-stakes information being variable. The authors then report a preliminary study that supported these hypotheses, and then begin the registered report for the planned study.
Strengths:
-The theory is described well in the Introduction and Figure 1 displayed the model in a clear way.
-The authors use sophisticated multilevel models to account for random effects in their complex experimental design, which is important for their research design.
-In my opinion, the design has strong internal validity.
Concerns and Comments:
Overall, I think that this is a reasonable design with strong proposed analyses. I do not see any major issues. However, there are few aspects that I would like the authors to consider.
-First, on the theory-side, I found the definition of “self-interest” to be unusual (“broadly encompass[ing] any outcome an interviewee may want to achieve or avoid”). In social psychology, self-interest is typically defined as the motivation to achieve outcomes that benefit the individual and avoid those that do not (Miller, 1999; Gerbasi & Prentice, 2013). Additionally, many theories specifically state that if the outcome is intended to primarily benefit others, it is not self-interested (Cropanzano et al., 2005; Holley, 1999). Thus, I would argue that “act[ing] in the best interests of other associates” is not actually self-interest. Their definition is closer to a purely economic definition of self-interest (maximizing one’s gains, minimizing losses), but I am not sure that this fits either, and the interview situation is not a purely economic one. Could authors either provide further justification and citations for their definition of self-interest, or consider if another term would fit better?
-Could the authors more clearly state their exclusion criteria (e.g., how much missing data is too much or how many memory checks can be failed)? The authors could also consider reporting the results without exclusions in the Supplementary Materials. I would reccomend this especially given the large number of excluded participants in Study 1 (transperency would be best).
-In lieu of an a priori sample size analysis in Study 1, could authors perform a sensistivity power analysis ?
-For Study 2, given the large number of excluded participants in Study 1, could the authors describe the number of participants they will recruit (before exclusions) in order to meet the minimum sample size?
-In addition to the model fit stats (AIC), could the authors report effect sizes for their models (R^2 or f^2; or ICC for random effects)?
-I appreciated the authors' discussion of internal and external validity. However, I would note that concerns about external (and construct) validity go beyond just how interviews would have more psychological realism. In a real-life interrogation situation, the consequences go far beyond collecting points and competing to receive a monetary reward and could involve fear for one’s life, fear for loved ones, fear of implicating one’s self in a crime, etc. Thus, despite the efforts to vary the consequences and incorporating a choice structure with incentives, this design likely does not perfectly capture the construct of these situations in the real world. That being said, this artificiality is common in psychology research and is necessary for internal validity. This idea, though, could be added to the discussion on lower external validity.
I thank the authors for their efforts, and hope that this feedback is helpful.