Does ideological malleability moderate the effect of self-affirmation on prejudice?

Once again the authors have been very responsive to my review. As a result, I am happy with this revision.
 
My only comments concern some small suggested wording changes that will, I believe, make this well written draft even better. These changes do not need to be made at this stage – they can be made to the stage 2 draft. In the meantime, I wish the authors all the best with data collection.
 
Suggested changes:
 
p 14
“could seem conflicting with” > “could seem to conflict with”
“reminded with this value” > “reminded of this value”
 
p 17
“On the other hand, we expected …” > “On the other hand, we expect” (The rest of the paragraph is in the present tense)
 
p 23
“such as participants who self-affirm …” > “such that participants who self-affirm …”
 
p 24
“Finally, we examine … towards Muslims rather than …” > “Finally, we examine … towards Muslims than …”

I thank the Authors for addressing the issues that I raised during the first round of review. Overall, I believe the revised version has significantly improved from a theoretical standpoint, offering a stronger rationale for including ideological malleability.

In the revised version, I also note that the Authors have paid stronger attention to several methodological aspects (e.g., power analysis, manipulation checks, and assessment of different variables, including their DV). Thus, I am satisfied with the revision and wish all the best to the Authors for the upcoming data collection phase.   

I read the revised draft with great interest. The authors have clearly been very responsive to our initial reviews. As a result, most of my points are really comments rather than proposals for changes.
 
1 The interaction hypothesis.
I am still not as clear as I would like to be about why self-affirmation should increase prejudice when the individual affirms using a “new” representation of secularism, but I appreciate that the authors do attempt to explain this more clearly in this version. The explanation, as I understand it, is that “new” secularism enhances social hierarchy and, by making this representation salient during the value affirmation phase, the result is increased prejudice: “if the ideology’s representation enhances social hierarchy, self-affirmation can increase prejudice” (p. 14). I am afraid I do not really follow this reasoning, but I realise this is not really my zone of expertise.
 
2 The theoretical implications of the related-value condition.
The authors develop the study out of conflicting findings around the impact of self-affirmation on prejudice. However, one effect of the related-value condition that results from their theorising about potential moderators is to blur the distinction between self-affirmation processes per se (where the content of the value is irrelevant to the effects) and value-induction processes (where the content of the value is fundamental to the effects). This distinction isn’t really addressed in the current draft.
 
Interestingly, the inclusion of an additional self-affirmation manipulation, assessing an unrelated value (humour) in the revised design may help with the interpretation of this issue, so I turn to that next.
 
3 The additional experimental and modified control conditions.
I appreciate the authors being so responsive to my previous comment about affirming on secularism that they have added a manipulation in which the self-affirmation is unrelated to the threat. I am not sure that I would have gone that far, but seeing as they have – and that they therefore deem it worthwhile in terms of the extra recruitment this entails – it is worth examining some of the implications of this.
 
First, in order to guarantee that the chosen value is unrelated to the threat, the manipulation constrains the participant’s choice of value. This, of course, means that the value being affirmed (humour) may not be sufficiently important to the individual to offset the psychological threat. This is common to both self-affirmation conditions.
 
Second, having an unrelated value does potentially shed some light on the issue I raise above (point 2). If “pure” self-affirmation processes are involved, then the effects should be the same irrespective of the value. If the sort of additional cueing effects postulated by the researchers in proposing secularism as the moderator are involved, then the effects should differ between the two conditions. I assume this is what underlies the prediction that the effects of humour on prejudice will be weaker than the effects of secularism (p. 17, p. 30) – but the reasoning underlying that prediction is currently not spelt out.
 
The researchers have also amended the control condition, so that it now involves a value task and is closer to a control task typically used in self-affirmation research. I share their hunch that physical endurance (the control value) will be low in importance for most respondents. The new manipulation check will reveal whether this is the case. Indeed, much now depends on this manipulation check, so I turn to that next.
 
4 The new manipulation check for self-affirmation
I appreciate that the authors have introduced a manipulation check (p. 22) for self-affirmation at my suggestion, but I am concerned that the way they have chosen to do this will be too intrusive. I wonder if they might instead consider using a single item assessing value importance (as is used in many self-affirmation studies) and, if they really want to know about relative value importance to the individual respondent, measure the three items retrospectively? Let me explain.
 
First, to be clear, a value-importance measure as used in a standard self-affirmation study is essentially a fidelity check. That is, in experiments in which the control condition asks participants to choose their least important value it can be used to check that at least the value chosen by the experimental and control participants differs in importance. Whether that results in the former being self-affirmed and the latter not, of course, is another matter.
 
The analyses of the new manipulation checks (p. 25) test for relative value importance. It will certainly be useful to know whether physical endurance is less important than either secularism or humour, but I don’t see why we should expect secularism to be more important than humour? Indeed, it may be a problem for interpretation if it is. (As an aside, it is interesting to see if being asked to self-affirm on a value boosts that value’s importance, but that is a separate issue.)
 
Testing for value importance could, it seems to me, be achieved more simply by just adopting the more typical procedure of simply asking how important the value the participant was asked to consider (be it secularism, humour or physical endurance) is to the participant. (The researchers would, of course, need to bear in mind that this does not tell us whether someone is self-affirmed.)
 
If the researchers are really interested in knowing the relative within-person ratings of all three values, they could consider getting these ratings towards the end of the study, by asking them retrospectively
 
However, my main concern about doing it as currently described is that it may be too intrusive to ask participants to rate three values in between the manipulation and the dependent measures. All that is required, it seems to me, is to ask them how important to them is the value they used.
 
4 Distributive matrices
I appreciate that the changes to the dependent measures have been introduced in response to feedback from both reviewers, but I have some concerns about the extent to which the behaviour matrices will be sensitive to the self-affirmation manipulation. These matrices seem potentially quite complex and involved cognitively and I am not sure the manipulation, which is relatively subtle and potentially time limited, will be sufficiently robust to have an impact on the individual’s responses to it. I am not suggesting the researchers change this element of the design but they should bear this issue in mind when interpreting any null fin dings.
 
5 Other issues:
 
P 25-26 Do the authors mean “effective” rather than efficient?
 
 

Thank you for the opportunity to read and review the Registered Report entitled ‘Self-affirmation and negative attitudes towards minority groups: The role of ideological malleability.’ The main goal of this study is to test the effectiveness of a brief self-affirmation in mitigating affective prejudice towards Muslims in France. Additionally, the study aims to assess the moderating role of secularism malleability. I appreciate the Authors’ efforts to make the study as transparent as possible, including the results of a simulation study. I have a few comments that could help to strengthen their work.  

1) The Authors pointed out that ideologies (for example secularism and colorblindness) can be malleable. When reading the example of colorblindness, it came to my mind that this ideology can be applied in different domains. Individuals may be strongly endorse colorblindness in one domain (e.g., education) but be less in other spheres such as religion. Just like colorblindness, also secularism could be domain-specific, but not necessarily malleable. I think this aspect (malleable vs domain-specificity) could be explained in the introduction because as we know from the literature on intergroup ideologies that ideologies tend to stable and difficult to change.    

2) From reading the title of the manuscript, and throughout the introduction, I thought that the focus would be on attitudes towards minority groups (DV). However, as reported on page 18, the DV refers to affective prejudice towards two religious groups (Muslims and Christians) with prejudice towards Muslim being the main focus. Thus, I would encourage the Authors to be consistent throughout the manuscript and change the wording accordingly, e.g., affective prejudice towards Muslims.     

3) Could the Authors clarify if they will be recruiting participants regardless of the religious beliefs, or specifically targeting non-Muslim and non-Christian French participants? This important information is not reported in the participants section.     

4) It is stated that the calculation of the required sample size is based on an unpublished study (Nugier et al., 2023). I wonder if the Authors could explain why they chose this study. Also, I wonder if the expected effect size could be more in line with the literature on self-affirmation and prejudice reduction interventions.

5) I'm not quite sure about the manipulation check for the secularism display. If I understand correctly, two visions of secularism are presented, and then participants are presented with four items about the rights of religious groups. I wonder if filling out a scale about the rights of religious groups might already serve as a way of expressing prejudice against religious groups, in sum, as a manipulation, but not as a manipulation check. I would like to hear the Authors' thoughts on this concern.

6) I have also some concerns regarding the measure of affective prejudice towards Muslims. The plan is to use a differential measurement approach by comparing emotions towards (only) two groups, Muslims and Christians. Such differential prejudice measure may mischaracterize in-group favoritism as out-group bias. Some people may consider outgroup members as less good but may not necessarily harbor hostility against them. On a related note, I would encourage to make a stronger argument for why affective prejudice is the only DV of interest, and not for example cognitive prejudice (stereotypes) or behavioral prejudice (discrimination), or all three dimensions of prejudice.  

7) A final note on causal language. Although the present study proposed to carry on an experiment, I would be careful to use impact, cause, etc.

The paper presents an interesting study that will attempt to test a potential moderator of the effect of self-affirmation manipulations on prejudice reduction. The goal is to understand more about when self-affirmation can be expected to be effective as a means of reducing prejudice. The moderator is “ideological malleability”, operationalised as the participant’s representation of secularism.
 
My expertise is in self-affirmation and in issues such as the uptake of health-risk messages, rather than the reduction of prejudice. I therefore have some hesitation in evaluating the theoretical origins and operationalisation of the proposed moderator. Nevertheless, I felt the issue was set up well and clearly in the introduction and gave a persuasive account of the underlying rationale for choosing this moderator.
 
As I describe below, I had several queries and issues as I read through the draft manuscript. I must stress that these are not criticisms but as things to think about as the study is developed. It may well be that these issues have already been considered and the researchers are happy enough with their existing decisions. Indeed, several of the issues that bothered me while reading the paper I found were eventually dealt with somewhere later on. If this paper proceeds to stage 2 it would be useful if the authors indicate earlier in the paper that they are aware of these issues and will deal with them in due course. This includes the manipulation checks, especially for value importance in the self-affirmation condition, the impact of religiosity and own religion on the findings, and the risk of systematic effects involving exclusions (in some ways addressed by the fact that analyses will be reported both with and without exclusions).
 
Issues
 
1 Interaction hypothesis for self-affirmation: impact on those holding a “new” representation of secularism.
As I say, my expertise in self-affirmation is in issues such as the uptake of health-risk messages, rather than the reduction of prejudice. Perhaps as a result, I was not clear about the reasoning and evidence underlying the prediction that self-affirmation would increase prejudice among those with a “new” representation of secularism. This is asserted, rather than clearly explained, on p.10 and p. 11. I felt this prediction would benefit from more explicit derivation and explanation.
 
2 Manipulation of self-affirmation.
A central tenet of self-affirmation theory is that the self-attribute that forms the basis for affirming the self (e.g., the affirmed value) must be at least as important to self-integrity as the threat is detrimental to it. One issue that confronts anyone who constrains the individual’s choice of attribute – as is the case here with the focus on affirming secularism –is whether that meets this requirement. A second consideration is that there can be specific consequences when a chosen value is related, rather than unrelated, to the self-integrity threat, again as is the case here. At the very least, these considerations need to be borne in mind when interpreting the findings.
 
The wording of the self-affirmation instructions assumes that secularism is an important value (“Please explain why this vision of secularism is important for you personally”, p. 42) and it would be useful to be able to assess whether, and indeed how, important it is. In many studies researchers place a brief 7-point rating of value importance immediately after the manipulation, which can be used as a manipulation check, compared between conditions (not possible here, given the control condition), or used as a covariate or moderator in analyses.
 
Instead, the researchers opt to undertake a content analysis of the affirmation and code the responses (p. 45). (As I say above, it would be useful to mention this when describing the affirmation procedure in the paper at p. 17.) This is a good idea, but a lot will depend on what can be gleaned from the statements.
 
The researchers currently envisage using this to exclude participants. However, only those who receive a rating of 1 will be excluded (p. 45); yet arguably only values 4 and 5 of their coding scheme indicate the value is important to the individual. A rating of 3 (general justification) does not seem to me to necessarily indicate that the value is important personally.
 
At the risk of expanding the exploratory analyses, I wondered if the researchers had also considered exploratory analyses in which this rating is included as a covariate or even a moderator (split at 3 or lower v 4 and 5)?
 
Also, with this, as with all exclusions, it will be important to test whether these are systematically related to the conditions. However, analyses both with and without exclusions (p. 45) will help with that issue; again, however, it would be useful to mention this earlier in the paper.
 
3 No value control group.
I appreciate the difficulty the researchers have in finding an appropriate control, given the experimental condition focusses on secularism. However, the chosen control inevitably means that the conditions differ in both self-affirmation and value salience, which cannot be disentangled by this design. This will need to be borne in mind when interpreting the findings.
 
More mundanely, looking at the materials, the researchers do not provide the participants with any cover story (rationale) for such a task, which may therefore seem odd to the participant. Is there anything they can do to smooth this element of the procedure (e.g., by way of a cover story that enables the task to make some sense without undermining the experimental design)?
 
4 Sensitivity of the dependent measure
As I say, I am not an expert in prejudice, but I did wonder about the sensitivity of the dependent measure (6 items assessing emotional reactions). Clearly this measure has been used successfully in previous research, so my concerns may be misplaced, but the study has a fairly transparent design (questions about secularism > a manipulation > measures of emotional reactions to Muslims and Christians) and I wondered about the potential impact of socially desirable responding.
 
5 Own religion and religiosity
Again, like the manipulation check for self-affirmation, this issue is discussed rather late in the paper (pp. 24-25). It is highly likely, isn’t it, that one’s own religion will heavily influence responses to the dependent measure, to the potential detriment of the between-subjects manipulations? In the current draft this issue is touched on only briefly and again rather late on.
 
The proposed remedy is to run exploratory analyses with participants’ self-rated religiosity as a covariate (p. 25). I wondered if the researchers had considered also, or instead, running analyses controlling for, or assessing the impact of, own religion (especially Mulsim or Christian) as this may well be a powerful influence on responding? I appreciate this is not straightforward, but it does seem important.
 
Likewise, I wondered if own religion will be correlated with secularism (and therefore confounded with it)? It is not hard to imagine that people who practice some form of religion, perhaps particularly Muslims, may hold a more historical perspective.
 
Finally on this issue, I wondered why the researchers differentiated the type of Christian (Catholic, Protestant) but not any other religion, including (and especially) Muslim? (See comments on wording below.)
 
6 Attention checks
I have mixed feelings about attention checks. On the one hand, it is obviously good to detect inattentive responding, but on the other it could risk the questionnaire coming across as somewhat bizarre and potentially undermine engagement. On that front, it was good to see there was advanced warning in the introduction about the use of attention checks.
 
In this I think balance is key and I think I probably misunderstood the statement about this on p 18 – that there would be two attention items per prejudice measurement – to mean that each participant would see four attention check items. If I did misread it, it would be good to make clear exactly how many each participant saw. If there are four attention check items, this seems too many given there are only six emotion items.
 
7 Data quality
Several of the issues I have raised have implications for data quality, so it was good to see the researchers had comprehensive diagnostics in place to assess data quality (p. 25). It was, however, not always self-evident what each element was controlling for. Perhaps this section could be reworded to assist the reader in this regard. For example, “To control for problem X, we used options A, B and C …”
 
As I say, a related issue concerns systematic effects of exclusions that might undermine the randomisation, so it would be good top have an explicit description of how the researchers will assess this or control for it.
 
8 Power analysis
The power analysis is explicitly described on p 13 and p31. I was not sure, however, whether the fact that the analyses were derived from an unpublished study raised issues.
 
9 Analysis strategy
I was puzzled as to why there was a separate ANOVA to test the main interaction hypothesis concerning self-affirmation and secularism (p.19). Couldn’t this be picked up by tests of simple effects at the level of muslims following the test of this 2-way interaction as part of the three-way ANOVA? I appreciate the researchers are closer to this issue than am I and may have spotted a complication that I am missing.
 
10 Pre-test
In principle, pre-tests are a good idea. The draft is very clear about what will be done and the fact that the data from the 100 participants will not be analysed at the time of the pre-test but will nevertheless form part of the final sample considered for analysis (p. 15). However, I wondered whether there was a Plan B. What will the researchers do if they do not meet the requirements of the pre-test they set out on p. 14?
 
11 Manipulation checks
There are manipulation checks for secularism (measuring opposition to religious groups’ rights (p. 17) and for self-affirmation (a content analysis of the writings, mentioned in Table 1 but not in the main text).
 
The materials
When I looked at the materials themselves, I had some wording issues:
 
12  Definition of new secularism
I understand that the definitions employed here are based on previous studies (pp. 16-17), so there may be nothing the researchers feel they can do or wish to do about this, but as an old experimenter it did strike me that the descriptions were less equivalent than they might be. In particular, the wording of the new secularism condition, at least in its English version, came across as somewhat more strident than the historical one. It could easily be reworded so that the wording and structure of these two definitions more closely mirrored each other. For example, in English:
 
“Individuals are free to practice their religion in private but not in public. Citizens do not have the right to show their religious affiliation in the public sphere. The state is not totally neutral with regard to/must regulate* these religious practices.”
 
*either would do I think
 
I also wondered whether the wording of the experimental question could be reworded to avoid begging the question concerning whether this is an important personal value. As I say above, the current wording (“Please explain why this vision of secularism is important for you personally”, p. 42) assumes it is personally important. A more neutral question could ask:
 
“Is this vision of secularism important to you?” “If so, please explain why.”
Indeed, if there was a yes/no/uncertain answer available to the first question, this could be used as an index of value importance and manipulation check for comparison in the two conditions (in addition to the codings).
 
Or a self-rating could be used: “How important to you is this version of secularism” 7 points, not at all/extremely.
 
13 Funneling and debriefing
I wondered whether “How seriously have you answered this questionnaire?” might be less loaded than “Have you answered this questionnaire seriously?” (p. 43). Again, it might be something that is more of an issue in English than French.
 
14 Own religion
Was there some reason for differentiating the type of Christian (Catholic, Protestant) but not any other religion, including (and especially) Muslim?
 
Would it make more sense for the general question “To what extent do you practice a religion” to precede rather than follow the more specific question about which religion or beliefs you feel closest to?
 
15 Debrief
Will participants know what self-affirmation means in this context?
 
16 Some items by page number
 
P 17 I am not sure the fourth item, about French TV, is tapping into the same issue as the other three items. I could imagine holding an historical view of secularism but having no desire to see TV programmes made by or for religious people!
 
P 22 I was puzzled by the use of post-hoc tests when testing hypotheses (e.g., p. 22). Again, I may be missing some important subtlety here.