Recommendation

How do public exposure and moral beliefs impact feelings of shame and guilt?

based on reviews by Roger Giner-Sorolla and Uriel Haran
A recommendation of:
toto

Revisiting the role of public exposure and moral beliefs on feelings of shame and guilt: Replication of Smith et al. (2002)’s Study 1

Abstract
Submitted: 16 February 2022, Recommended: 06 June 2022

Recommendation

Shame and guilt are powerful negative emotions that are notable for their external vs. internal focus: while shame is generally experienced in response to public scrutiny, guilt arises from a self-directed, private evaluation. In a formative study, Smith et al. (2002) asked whether the level of public exposure influenced levels of shame and guilt arising from one's transgressions, and found that, compared to private situations, public exposure was more strongly associated with shame than with guilt. Since then, these findings have had significant implications for theories and applications of moral psychology.
 
In the current study, Zhang et al.  propose to directly replicate Smith et al. (2002) in a large online sample. In particular, they will revisit the critical questions from Study 1, asking (a) whether public exposure affects the magnitude of shame and guilt over one’s misconduct, and (b) whether stronger moral belief increases guilt and shame over one’s misconduct.
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/j7kt2
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Smith, R. H., Webster, J. M., Parrott, W. G., & Eyre, H. L. (2002). The role of public exposure in moral and nonmoral shame and guilt. Journal of Personality and Social Psychology, 83, 138-159. https://doi.apa.org/doi/10.1037/0022-3514.83.1.138
 
2. Zhang, Y., Cheung, F. C., Wong, H.T., Yuen, L. Y., Sin, H. C., Chow, H. T. & Feldman, G. (2022). Revisiting the role of public exposure and moral beliefs on feelings of shame and guilt: Replication of Smith et al. (2002)’s Study 1. https://osf.io/j7kt2
Cite this recommendation as:
Chris Chambers (2022) How do public exposure and moral beliefs impact feelings of shame and guilt?. Peer Community in Registered Reports, . https://rr.peercommunityin.org/articles/rec?id=180

Evaluation round #1

DOI or URL of the report: https://osf.io/6bwsk/

Author's Reply, 17 Jun 2022

Download author's reply Download tracked changes file

Revised manuscript: https://osf.io/4nqy6/

All revised materials uploaded to: https://osf.io/j3ue4/, updated manuscript under sub-directory "PCIRR Stage 1\PCI-RR submission following R&R"

Decision by , 03 May 2022

Two expert reviewers have now assessed the Stage 1 manuscript. As you will see, the evaluations are broadly positive, with both reviewers praising the value of the replication and the methodological rigour of the proposal. There are, nevertheless, some conceptual and methodological issues to address in order to satisfy the Stage 1 criteria. Key concerns raised include the accuracy of the framing in the introduction, ensuring that the power analysis targets the appropriate (finest grained) level of the design, consideration of the validity of the attention checks, making clear all deviations from the original study design and ensuring they are scientifically valid, and clarification of the analysis plans.
 
In revising, please keep in mind that while it is perfectly fine to bring up new literature in the Stage 2 discussion section, it is generally not possible to add new literature the introduction section after in-principle acceptance. Therefore, in responding to the reviewers' comments, please ensure that the introduction is brought now into a state that will require minimal (if any) changes at Stage 2.

Reviewed by , 02 May 2022

In general there is very good practice here for highlighting original-replication comparisons and for ensuring that the final report follows the registration closely. Here are my comments for improvement.

  1. Abstract: "impacted differently by exposure" should be expanded for clarity, e.g. "... by appraisals of potential exposure to disapproving others"
  2. Authors should follow APA style when citing modern translators and commentators on ancient philosophers, see 7th ed. style manual, p. 325, example 36.
  3. The Introduction should position the public exposure/reputation view of shame vs. guilt more clearly against a dominant rival view, the Tangney interpretation of Lewis' Shame and Guilt in Neurosis that shame is distinct from guilt in being subjectively about the whole person rather than a single transgression. This omission is much missed, for example on p. 9 where the GASP is mischaracterized as being mainly about the public/private dimension, when actually it intentionally conflates that dimension with several others derived from the Tangney viewpoint. This addition will only strengthen the case for importance of this study.
  4. An important part of a case to replicate a specific article is that it has not been directly replicated already, and/or that conceptual replications/extensions are few - can the authors speak to this issue? 
  5. Power analysis should make clearer that main effects and not the interaction + simple effects (because the original did not find it significant) were the main hypotheses of the study. If testing the interaction is important then the study should be powered with regard to simple effects tests interpreting the interaction.
  6. Was an attempt made to contact the original authors for the missing disobeying scenarios?
  7. pp. 17-18 - if I understand correctly participants can only proceed after correctly answering or re-answering attention check questions. This is a deviation from the original that will likely increase effect size and should be questioned. It might be more defensible to not force this kind of learning and look at responses only with (better test of idea) or also without (better replication of original) a correct check response.
  8. Is the outlier exclusion a deviation from the original?
  9. I agree it is important to test the validity of the measures as face-wise several of the shame-related measures seem conceptually and empirically shaky (e.g. I usually find anger at self loads on guilt and not shame). Rather than simply testing correlations it might be better to test omnibus reliability and also discriminant validity (e.g., factor analysis of all items) of all the shame and guilt measures, both direct and indirect, together. This would let us assess, improving on the original, whether each "related" measure is correctly classified or not.
  10. Also, there is a discrepancy where the results test individual scale reliability but the analysis plan does not.
  11. If transforming data into long format, multilevel (hierarchical) analysis needs to be followed with participant as random factor in order to properly account for nonindependence of observations. However, treating shame/guilt as a within-participants factor while keeping wide format data would be simpler. It is not clear from the writing which analysis will be used.
  12. In general there are a few grammar and word errors noted that suggest further proofreading is needed.

signed, Roger Giner-Sorolla

Reviewed by , 26 Apr 2022

This report describes a replication plan for Study 1 of Smith et al.’s 2002 paper about the effect of public exposure of a wrongdoing on ratings of guilt and shame. The paper has been pretty influential, with over 600 citations on Google Scholar, and has been published in the Journal of Personality and Social Psychology a long time ago. Therefore, other than determine whether the study is worthy of replication (which it definitely is), there is not much to say about the merits of that research, the relevance of the original research question, the rigor of the original study design etc. As long as the authors follow the research protocol employed in the original study, they are not obliged to independently satisfy the evaluation criteria of that work.

The intended replication follows the original protocol pretty closely, including the experimental design, adjusted sample size and analytical approach. The authors had to write the stimuli of one of the conditions, which was not provided in the original article. The new text seems valid: it is close enough in length and in format to the texts in the other conditions. The authors are also adding a couple of attention check items, which is a legitimate addition, and standard in such studies in 2022. One difference that is noteworthy and should be discussed in the replication manuscript is the physical settings in which participants will complete the study. The original study was conducted in the lab with about 30 people in the room per session. The replication will be conducted online, with participants completing the study on their own electronic devices, presumably in private. I recommend also coding the type of device on which participants do the study, and if possible prevent people from completing it on their mobile phones, as research in information systems finds systematic differences in user attention between tasks performed on a PC or a tablet and tasks performed on mobile phones.

The only other comments I have are about the literature review. These comments are not relevant to the current stage of the submission, but the authors might want to take them into account later when they write the full manuscript.

1.     The abstract states that guilt and shame are similar in that they both are associated with negative evaluations of oneself, but this is not entirely accurate. Guilt, unlike shame, is associated with negative evaluations of one's behavior, separately from one’s view of one’s qualities and characteristics (i.e. “I did a bad thing” rather than “I am a bad person”). This externalization of the emotion-eliciting wrongdoing is what distinguishes guilt from shame. See Tangney & Dearing, 2002. 

2.     There is more modern research in psychology about guilt vs. shame in general (e.g., Cohen et al., 2011; Tangney et al., 2007) and particularly about the public-private dimension. Some of these works are cited later in the report (p. 9 Choice of replication), but it should be discussed earlier. 

3.    One wrinkle in the public-private distinction between guilt and shame is that guilt enhances interpersonal motives such as the desire to be loved and accepted by others (Baumeister et al., 1994). These motives might moderate the effect of public exposure of one’s behavior on one’s feelings of guilt and shame. Again, this does not affect your replication research but might be a point worth addressing in your discussion.  

 

In sum, I think the choice of study to replicate is clever and the replication plan is sound. Good luck with your research!

Uriel Haran

 

References:

Baumeister, R. F., Stillwell, A. M., & Heatherton, T. F. (1994). Guilt: An interpersonal approach. Psychological Bulletin, 115(2), 243–267.

Tangney, J. P., Stuewig, J., & Mashek, D. J. (2007). Moral emotions and moral behavior. Annual Review of Psychology, 58, 345–372. https://doi.org/10.1146/annurev.psych.56.091103.070145

 

User comments

No user comments yet