DOI or URL of the report: https://osf.io/weshm
Version of the report: 1
Dear Jesse & Yoel,
Thank-you so much for resubmitting your registered report to PCI:RR for consideration. I appreciate your patience whilst the two more detailed reviewers from the last round considered your responses and I re-read your work. As you may find below, these two reviews are now positive and consistently endorse your work for acceptance. One of the reviewers has some additional suggestions (mostly minor typographical, but one meaningful consideration about power), and I think it’d be of benefit for these comments to be addressed. I believe this is a really interesting project and so following your consideration and/or implementation of these comments, I will be very happy to ‘recommend’ your work for Stage 1 acceptance.
Take care,
Tom
Dr Thomas Rhys Evans
Dear Authors,
I finally had the chance to read the revised manuscript entitled “Do Right-Wing Authoritarianism and Social Dominance Orientation Predict Implicit Attitudes?”.
I am very pleased with your revision of the manuscript. You followed very carefully the suggestions the other reviewers and I gave you. Particularly, I find that the introduction makes it clear now what your objectives and hypotheses are, as well as the relevance of your work. The importance of investigating implicit associations has also become evident.
The method section also underwent significant and positive changes. The description of the methodological framework of the original study is much more comprehensive and easy to understand, and the analytical plan is well-developed.
As a result, I would only like to congratulate you on the great work you have done on this manuscript. I wish you success with this project!
Following are the last - minor - things I want to report.
Table 1 row 5. Consider indicating in the notes that “status quo” does not have a counterpart. It is explained only later in the manuscript.
Notes 2 on page 15. “A two-stage approach (Yuan & Lu, 2008) is preferable in for the Ideology 2.0…” Please, remove “in.”
Notes 3 on page 19. “the “SSpower()” function provided by the “semTools” package in R (CITE).” Insert proper citation.
The power analysis described on page 19 is not a sensitivity (what is the minimum detectable effect size given N, alpha, and power) but a post hoc analysis (what is the power to detect a specific effect size, with a given alpha and N). However, I suggest you perform a sensitivity analysis, as it could be more informative (Perugini et al., 2018 https://doi.org/10.5334/irsp.181)
Page 19, last row. “Miminum” is misspelled.
The authors have addressed my comments in a convincing manner. I now find it much easier to follow the rationale and planned analyses. I also appreciate that the authors included a strategy for type 1 error control and a novel approach for evaluating model fit. I am looking forward to reading the Stage 2 manuscript.
DOI or URL of the report: https://osf.io/kfm3u
Version of the report: 1
Dear Jesse & Yoel,
Thank-you so much for submitting your registered report to PCI:RR for consideration. I enjoyed reading this work, and solicited the views of a diverse range of researchers in the area. As you may find below, I secured the thoughts of four individuals who shared their time and expertise with us. Two reviews are very brief and provide somewhat sporadic thoughts about the work presented, whilst two others are substantive and detailed in their consideration of your proposal. As an RR where detail is everything, I based my assessment predominantly upon the latter. In sum, I agree with their assessment that the work is of interest to the academic community and there are some well-produced "open" dimensions of the project, yet there is insufficient information within the proposal to be able to clarify the contribution or procedure of the work proposed. In a revision, I would consider it most fruitful to see greater detail surrounding the methodology and analysis, and you will likely find it of benefit to reconsider your framing and coverage of the literature in the introduction to make the contribution of the current work clearer in context of the existing literature. As such, I encourage you to revise your proposal in-line with the feedback, using tracked changes and a comprehensive response to reviewer report where possible. Your work is really interesting so I look forward to the next iteration, and if there is anything that is unclear please don't hesitate to reach out and I'll be very happy to help!
Take care,
Tom
Dr Thomas Rhys Evans
To the Authors,
In the registered report entitled "Implicit Ideologies: Do Right-Wing Authoritarianism and Social Dominance Orientation Predict Implicit Attitudes?”, you describe a study that aims to investigate whether and how Right-Wing Authoritarianism and Social Dominance Orientation affect explicit and, crucially, implicit preferences for authority, tradition, and hierarchies.
Overall, I believe you aim to address an interesting and relevant research question. However, I also think there are improvement areas in both the introduction and methodology sections.
Theoretical Introduction
I like that you discussed implicit preferences by considering their limitations and critical issues. This is of extreme importance. However, I was left with the impression that it does not emerge why to care about implicit associations. In the paragraph “Implicit vs. Explicit Attitudes” (pp. 4-5), it would also be important to note why it is relevant to investigate implicit attitudes in addition to explicit ones. E.g., implicit measures are less susceptible to demand characteristics and can help measure responses that may be introspectively inaccessible. Further, even though implicit and explicit measures appear to assess the same underlying attitude in some research, they can still tap into more spontaneous evaluative reactions and more controlled responses. In other words, although the attitude might be the same, its expressions might differ.
In the next section (pp. 5-6), you stated, "Given the findings relating to the IAT and political orientation, we might expect RWA and SDO to predict implicit attitudes in a similar way to that of explicit attitudes. However, research employing RWA and SDO to predict implicit attitudes is relatively scarce and offers mixed findings.” The mixed results might be due to the heterogeneity of the attitudes examined. Thus, I do not feel it is a compelling reason to explore implicit attitudes. Perhaps it might be more effective to stress instead that much research has shown that RWA and SDO are significantly linked to various attitudes. If SDO and RWA are predictive of those attitudes, you can expect both to be related to the implicit attitudes of interest, which are closer and more relevant to the considered ideologies (which you noted in the abstract but not elsewhere in the RR).
Another major issue I see in the RR’s introduction is that it lacks a clear description of the research question and hypotheses. Indeed, from the “Present Study” section, it is hard to understand what the dependent variables are and how RWA and SDO are expected to relate to each of them (both in terms of direction and magnitude of the effect). For example, as you provided examples of IAT pairings that should be more relevant to RWA than SDO and vice versa, I find it puzzling that you did not speculate on differences in effects direction and size for each of them.
Related to the dependent variables, I learned later in the RR that you plan to analyze 13 concepts (implicitly and explicitly). However, it is unclear to me what criteria you followed to select them, as for some pairings, it might not be evident why they are relevant (e.g., 1950/2050 IAT pairings). Further, what pairings are relevant to either SDO or RWA and which are relevant to both?
Methodological Section
Although data collection and measurement details are described elsewhere, I believe this section should stand on its feet. For example, it should be clear what procedure was used to collect data (e.g., order of the measures, counterbalancing), data exclusion criteria (e.g., how to treat outliers), how variables will be computed (e.g., I guess explicit attitude will be the difference between the preference for x and y), and the sampling plan (which is briefly described in the Table under Study Design section) should be presented in more details. Further, it might also be helpful to understand the minimum effect size detectable given the available sample size (sensitivity power analysis).
Further, I believe the analysis plan should be enriched with relevant information, such as how the variables will be treated in the SEM model and the fit indices you will use to evaluate the goodness of fit. Further, do you plan to check whether the missing mechanism is as expected by design (in planned missingness design, missing data points are MCAR by definition; thus, you might use Little’s MCAR test to check this on your data)?
As I learned from the OSF documentation, data were collected from different countries (e.g., WEIRD and non-WEIRD countries). Thus, how do you plan to control the hierarchical structure of the data and measurement invariance, especially on SDO and RWA measures (there might be inconsistencies in the measurement models across cultures)?
In conclusion, I know you are addressing a relevant research question, and I see how important the results would be for understanding how ideologies relate to different attitudes measured at implicit and explicit levels. It is disappointing that all this potential has yet to emerge from the RR. I encourage you to provide a more detailed description of your objectives, to stress further the relevance of using implicit measures in addition to explicit measures, and to add the methodological details that would permit everyone to replicate your study. Good luck with this research!
Minor points
p. 5, row 3: Please, add a reference to IAT.
p. 5, row 7: Remove “toward” from the text.
p. 5, row 5-7: From the description, it should emerge that the implicit association measured with the IAT is relative in nature.
p. 6, row 14: “Emphasize” needs the s.
Sometimes words are spelled in British, and some others in American.
If these questions are address it will benefit and strengthen this study greatly.