Why are there variations in perceptions of inequality?

The authors have addressed almost all of the issues I had raised in my original review -- nice job! I especially like the more detailed hypotheses regarding the moderating effect(s) of Justice Sensitivity in the preregistered main study (pp. 28-29).

That said, I still have trouble understanding (and, actually, reproducing) the results from the two pilot studies. What caught my attention was that, for instance, in Study 1, the framing x perceived size interaction effects were so highly significant (see Tables 2 and 3), yet the conditional ("simple") effects were not that different from each other, after all...

So I downloaded the raw data from the OSF website and tried to reproduce the results. And that left me with more questions than answers...

One thing I noticed was that the moderator variable ("perceived size"), was heavily skewed in both studies. In Study 1, no single participants chose 1 or 2 (on the 1-7 response scale), while 83% chose 6 or 7. In Study 2, the problem was even larger: Again, no one chose 1 or 2, but 89% chose 6 or 7. So, I doubt whether it makes sense to treat "perceived size" as a continuous moderator here... Dichotomizing this variable might be a solution, but even so, the question is whether it makes sense to treat "perceived size" as a moderator at all if the variance is so small.

Second, I could reproduce most of the findings reported in Tables 2 and 3, but not all of them. For instance, in Study 1, the PROCESS model I ran for the DV "legitimacy" was:

PROCESS vars = Treatment Perceived_Difference Legitimacy_scale SDO_scale
    /y=Legitimacy_scale
    /x=Treatment
    /m=Perceived_Difference
    /model=1
    /center=1.

One noteworthy difference was the (unconditional) effect of "perceived size", which was B=-.19 in "my" analysis (and -.39 in the authors' analysis; see Table 2). Also, the conditional ("simple") effects of "Treatment" (+/-1SD from the sample mean on "perceived size") differ substantially from the numbers reported in the Notes below Table 2. In a similar vein, the results for the DV "Intentions" differ (sometimes only slightly, sometimes more strongly, such as for the "perceived size" effects) from the numbers reported in Table 3. This, I think, needs to be double-checked and clarified, because it has important consequences for the interpretations!

Also, I was a bit worried about the fact that controlling for SDO in all of these models obviously had a strong impact on the pattern of findings... In Study 1, for instance, the treatment x perceived size interaction effect on both DVs disappears when SDO is *not* included in the respective models... I think this should be discussed more openly and explicitly in the paper.

Finally, I would be careful saying that "The results of the second study mainly replicated the results of the first study..." (p. 21) given that the treatment x perceived size interaction effect on the DV "legitimacy" was not significant in Study 2 (see Table 4).

So, all in all, I still do like the preregistered main study! But my doubts about the two pilot studies (and what they can tell us) have actually increased.

There is much to like about this Registered Report: the research question (i.e. how global inequality is mentally represented and whether this representation affects legitimacy appraisals and action intentions) is interesting and timely, the report is very well-written, the preliminary studies reported here have shown promising results, and the proposed study makes sense and is described in sufficient detail. In particular, I appreciate the detailed methods and results sections and the fact that the pilot data are openly available. 

At first, I was admittedly a bit skeptical about how robust "comparison framing" effects actually are, but after doing a bit of literature search (for recent findings on comparison framing effects; e.g., Inbar & Evers, 2021: https://doi.org/10.1037/xge0000804), I am convinced that these effects are robust and should be taken seriously. I also learned that theorizing about the psychology of framing effects is pretty advanced by now, and I think that some of these conceptual advancements deserve to be mentioned in the present paper, too. The authors of the present report seem to rely their reasoning exclusively on salience or figure/ground effects (e.g., page 6). An alternative interpretation is that perceivers draw inferences about a communicator's intentions (i.e., their "reference point") and values (e.g., McKenzie & Nelson, 2003: https://doi.org/10.3758/BF03196520; Sher & McKenzie, 2006: https://doi.org/10.1016/j.cognition.2005.11.001). Applied to the present research, this "information leakage" approach would explain the framing effects obtained in the two pilot studies more in terms of an implicit demand characteristic (such as "the researchers think that the fact that 'developing countries have a smaller share of global wealth' is problematic and that somebody should do something about it"). I would be interested to hear the authors' opinion on whether they think "information leakage" and the implicit demand it creates may be relevant for their own research. I don't think it is necessary to re-design their proposed study in order to test the "information leakage" account against a simple salience account -- but I think the authors may want to discuss "information leakage," the "reference-point hypothesis by McKenzie, and implicit demand as a potential alternative explanation in their General Discussion (if they share my impression that these issues are relevant here).

Besides this conceptual issue, I only have a couple minor (methodological) issues, which can easily be resolved in a revision:

(1) SDO as a covariate: On page 9, the authors write: "we propose that SDO is a relevant personal-level control variable that needs to be included when investigating framing effects on the perceived legitimacy of global inequality and individual action intentions." I wondered whether SDO could also be regarded a viable moderator variable: SDO has been conceptualized as a (dispositional) preference for inequality among social groups, so one could argue that people low in SDO should be more susceptible to a framing manipulation than people high in SDO (whose dispositional preference should have a stronger impact on their attitudes than contextual variations). Maybe the authors could discuss the plausibility of this reasoning in their paper and also test whether SDO moderated the effect of framing in the pilot studies. Also, related to this, I would like to see whether the pattern of results reported here changes when SDO is not included as a covariate into the models.

(2) Sample characteristics: As the authors discuss explicitly on page 26, the two pilot studies rely on student samples and are not representative of the general population in many respects (age, gender distribution, education level, probably also political attitudes). This is why I think a nationally representative study is actually warranted. I was a bit surprised that the authors think that "...this can be interpreted as an especially strong test of inequality-related framing effects" (p. 26), because it is possible that the framing effects are much smaller in a politically more diverse sample (especially when we assume that political attitudes covary with demand susceptibility, see the "information leakage" argument discussed above). Maybe the authors can clarify this?

(3) Power analyses: I missed a discussion of statistical power in the two pilot studies -- maybe the authors could at least report a sensitivity power analysis when they describe their samples. Later, when they determine the necessary sample size for the proposed study, they write that "...the size of the significant paths in our moderated mediations models of the pilot studies lay between .18 and .75" (p. 38). I could not find these estimates in the Results sections of the two pilot studies. I may have missed that, but if not, these estimates should be added to the results. Also, effect size estimates for the framing x perceived size interaction effects (i.e., the increase in R-square by adding the interaction term to the regression) should be explicitly reported. 

(4) Simple effects: Even though the framing x perceived size interaction effects are significant in both pilot studies, I would like to see tests for simple effects (or "conditional effects") to back up claims such as "Participants who perceived the economic inequality between low and high-income countries to be large were more strongly affected by the less (versus more) frame than participants who perceived the inequality to be small" (p. 16) or "stronger emotional reactions for participants who perceived the size of inequality to be large and were presented with the less (versus more) frame" (p. 17). As far as I can see, only conditional *indirect effects* are tested (see Table 8). By the way, looking at the conditional indirect effects in Table 8, it seems that the indirect effects among people high in PSI (perceived size of inequality) are not significant in Study 2. This needs to be mentioned and discussed in the text.

(5) Preregistered "Main" Study: As noted already, I do like the proposed study and I think it is necessary to make the paper sufficiently strong and convincing. Also, I do like the inclusion of Justice Sensitivity (JS) as a potential moderator variable. That said, I wondered why (a) all "other-oriented" JS perspectives will be aggregated into one score, (b) victim sensitivity will not be measured, and (c) the authors are so cautious regarding the potential moderator effects of JS. Let me quickly explain each of these three issues:

(a) It is likely that people high in beneficiary sensitivity will react more strongly towards a "more-than" frame than towards a "less-than" frame (given that German respondents are likely to identify with a "developed" or economically privileged country), but this should not be the case for observer- or perpetrator-sensitive people. Therefore, I would analyze the JS perspectives separately instead of aggregating across them.

(b) Even though it is likely that victim-sensitive individuals react less sensitively to framing manipulation than victim-insensitive individuals, being able to empirically demonstrate such an interaction may be worthwhile. Therefore, I suggest including the victim sensitivity subscale into the study.

(c) The authors write that "the role of justice sensitivity in framing research is still largely unclear, we investigate the moderating effect of this variable in an exploratory fashion" (p. 29). This is okay, but there are certainly interaction patterns including JS that are more plausible than others. For instance, I would expect that all three "other-oriented" JS perspectives should amplify (i.e., positively moderate) an effect of inequality size (i.e., 2-way interactions). Also, since framing effects appear to be driven more strongly by a "less-than" frame compared to a "more-than" frame (Inbar & Evers, 2021), JS should predict legitimacy appraisals and action tendencies more strongly in a "less-than" frame than in a "more-than" frame. I know that some effects are harder to predict, but at least the most plausible ones could and should be formulated as hypotheses here.

One minor issue: The authors refer to Schmitt et al. (2005) for the JS scales they want to use. I suggest they refer to the more recent version (Schmitt et al., 2010: https://doi.org/10.1007/s11211-010-0115-2). The German version of the scales can be found here: https://www.uni-landau.de/schmittmanfred/forschung/sbi/index.html.

Thank you for inviting me to review this report -- I really like this project and I wish the authors success with their proposed study!

Signed,
Mario Gollwitzer