Recommendation
based on reviews by Kate Button and 1 anonymous reviewer
DOI or URL of the report: https://osf.io/74vpt
Version of the report: v2
, posted 11 Jul 2022, validated 17 Oct 2022The two reviewers who assessed your initial submission kindly returned to evaluate the revised manuscript. As you will see, the reviewers agree that the manuscript is improved and significant progress has been made toward achieving Stage 1 in-principle acceptance (IPA). One reviewer is fully satisfied while the other asks for additional clarification of the rationale and use of terminology, as well as addressing a concern about measurement validity and the details of the hypothesis-tests.
Please attend to these comments in a thorough revision and response to the reviewer. Given the substantial progress made toward IPA, and the time constraints of your project, I will evaluate this revision at desk. Provided all matters are satsifactorily resolved, IPA should then be forthcoming without requiring further in-depth Stage 1 review.
I appreciate the effort the authors put into this revision of their submission. Several of my earlier points (overview of creativity research, assessing reappraisal ability, quality control of data) are now sufficiently addressed and make me confident in the study protocol. The authors' contraints on the time and financial aspects of the online assessment are understandable and I agree with their vote of confidence that a sample of n = 200, even if not able to test for small effects, may still provide a valuable contribution to literature. Therefore, I recommend that authors move on to stage 2.
Thanks for the opportunity of reviewing this stage 1 RR. The authors have clearly worked hard and made several changes to the manuscript. I have a few comments, however, which I hope they find useful.
The introduction and rationale is much clearer but I still have a few comments. The authors bring in a range of evidence linking creativity to depression but as they acknowledge creativity is a very broad concept which can be defined in several ways, and the evidence for an association is mixed. It would be helpful if more context were given to the individual studies cited to show how they measured creativity – as the authors are specifically looking at divergent thinking as a component of creativity then perhaps more weight should be given to findings from studies using similar measures. Indeed, I wonder whether the rationale could focus even more directly on divergent thinking.
I also think that issues of confounding haven’t been sufficiently considered in places. For example, rates of depression higher among art students, than science students – have the authors considered / or did the original work consider other differences between these two groups that could account for this difference?
The authors state “Creativity is usually regarded as a strength and an advantageous trait, but there may also be drawbacks to being creative. One potential drawback is that creativity may be associated with emotional instability and mental disorders.” My reading is that the authors predict that if an association between depression and creativity exists at all, it is not causal but instead is the result of confounding by rumination and reappraisal, so perhaps this could be more nuanced?
Methods:
The authors mention this is an epidemiological study, not a diagnostic one several times. I’m not sure what this means – we often use diagnosis in epidemiology to ascertain the absence/presence of disease. They might find this series in the BMJ helpful: https://www.bmj.com/about-bmj/resources-readers/publications/epidemiology-uninitiated/1-what-epidemiology . I understand the rationale for looking at the variation of depression scores in the general population.
A key thing in epidemiology is defining one’s population of interest. Another is to think about the generalisability of findings (to other populations). To do this it often helps to measure several participant characteristics at baseline (I suggested this previously). In this case medication status and whether they are currently receiving psychotherapy for depression seems really important to anyone reading this with an interest in depression. Employment status (student, employed, NEET) and profession (arts, science), as well as the usual age, gender, ethnicity/nationality ect. would also be useful for providing context - I’m not suggesting additional analyses but rather providing a detailed overview of the people recruited so that readers can decide whether the findings are generalisable to their population of interest. Also, if publishing data open these could be very useful for future meta-analyses.
The authors are only using a few items from each of the measures they use – is this justified? What will they do with the remaining items? Do they need to collect all of them?
For the analysis for hypothesis 3, is there a reason for including self-rumination in the model? This could be explained.
DOI or URL of the report: https://osf.io/74vpt/
, posted 09 Apr 2022I have now obtained two very detailed and constructive evaluations of your manuscript which are enclosed below for your consideration. As you will see, the reviews are thorough, raising concerns that span the full breadth of the Stage 1 criteria. I will not summarise every point raised, but since major revisions are required, I want to draw your attention to some of the key points.
Both reviewers noted the need for the introduction to include a more logical and comprehensive rationale for the study, which at present lacks sufficient cohesion and depth of scholarship. The reviewers also note the lack of sufficient plans to verify data quality, e.g. through the use of manipulations checks or positive controls (as well as steps to eliminate bots and repeat participation). Additional headline issues include the need for the design to include a measurement of reappraisal ability, deeper consideration of clinical implications, full elaboration of inclusion and exclusion criteria (and consideration of whether the inclusion criteria are appropriately calibrated), measuring state aspects of depression, inconsistencies in the study design plan, and need for a more detailed analysis plan. As noted, the effect size estimatation of your study is also overly optimistic, which means a larger sample size will be required to achieve sufficient power to provide an informative conclusion. For Stage 1 RRs, it is vital that you target the smallest effect size of interest. As noted in the section Evidence Thresholds in the author guidelines: “Since publication bias overinflates published estimates of effect size, power analysis should be based on the lowest available or meaningful estimate of the effect size.” (emphasis added).
In revising, please also confirm that the appropriate ethics approval has been granted.
The level of revisions required here is substantial: increasing the level of scholarship, modifying the study design, increasing the amount of methodological detail, and expanding the sample size. However, from an editorial perspective, the changes are not impossible and the issues you are facing are relatively common, especially for authors approaching RRs for the first same. For a regular manuscript describing a completed study, a set of reviews this critical would lead to outright rejection, but the advantage of the RR process is that it provides the opportunity to make critical design revisions before they become roadblocks. I am therefore happy to invite a comprehensive revision and response, addressing all points, which I will return to the reviewers for re-evaluation.
1A Scientific validity of the research questions:
The research question on the link between creativity and depression as well possible mediation through reappraisal and rumination is well derived. I agree that there is still a lot of discrepancy on whether creativity is associated with more or less depressive symptoms and real-life emotion regulation tendencies may play a major part in this relationship. The research questions proposed by this paper are definitely valid; however, I am not convinced that the authors did a thorough enough literature research to back up their research questions and hypotheses. Especially in the “Creativity” and the “Creativity and depression” paragraph, I am missing critical references for a) the categorization of creativity, b) the notion that divergent thinking leads to originality, c) that divergent thinking abilities are indicative of creative thinking and creative potential, and d) that anhedonia inhibits creativity. Adding more references would in my opinion boost the credibility of the research proposal.
1B. Logic, rationale, and plausibility of the proposed hypotheses, as applicable.
The hypotheses are solid and plausible. However, I have two major concerns, which also partly relates to the research questions and the methodology.
1) The authors state that they want to address “the inconsistencies in the current literature that blurs the distinction between reappraisal ability and reappraisal frequency”. In my opinion, this is only possible of this study investigates BOTH reappraisal ability and reappraisal frequency as a mediator between creativity and depression. To me, it is problematic that the authors build their hypotheses for reappraisal frequency on the idea that creative individuals have high reappraisal ability, yet there are only a handful of studies so far that propose this link for very specific measures of creativity and reappraisal (e.g., Weber et al., 2014; Fink et al., 2017). Accordingly, I strongly recommend that the authors include a measurement of reappraisal ability in their study as well. I elaborate on possibilities in 1C.
2) Further, another major factor that may influence links between creativity, depression, and emotion regulation strategies is gender. There is a plethora of research discussing gender differences in adaptive and maladaptive emotion regulation strategies like reappraisal and rumination and their link to depression, for example:
Nolen-Hoeksema, S. (2012). Emotion regulation and psychopathology: The role of gender. Annual review of clinical psychology, 8, 161-187.
Preston, T., Carr, D. C., Hajcak, G., Sheffler, J., & Sachs-Ericsson, N. (2021). Cognitive reappraisal, emotional suppression, and depressive and anxiety symptoms in later life: The moderating role of gender. Aging & Mental health, 1-9.
Perchtold, C. M., Papousek, I., Fink, A., Weber, H., Rominger, C., & Weiss, E. M. (2019). Gender differences in generating cognitive reappraisals for threatening situations: Reappraisal capacity shields against depressive symptoms in men, but not women. Frontiers in Psychology, 553.
Zlomke, K. R., & Hahn, K. S. (2010). Cognitive emotion regulation strategies: Gender differences and associations to worry. Personality and Individual Differences, 48(4), 408-413.
I think it is important to consider that according to these publications, women use both adaptive and maladaptive emotion regulation strategies more than men. Simultaneous use of both adaptive and maladaptive strategies may reduce the effect of adaptive strategies (e.g., reappraisal), which could also affect the link between creativity and depression in women. Thus, I think it would help clarify associations between creativity, depression, and emotion regulation if gender as additionally considered in all analyses (see 1C.)
1C. Soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis or alternative sampling plans where applicable).
Some information about the planned study sample and methodology is lacking:
a) Are there any exclusion criteria for study participation? What comes to mind is current psychiatric/neurological diagnosis, intake of psychoactive medication, and previous experience with the TTCT. It is also unclear whether the authors want to investigate a sample with normal variations in depression or whether individuals with high depressive symptoms will be included/excluded from the study.
b) What is the protocol if participants score very high on the Depression Scale (MSD-T)? The authors mentioned that they could not provide sufficient professional knowledge and advice regarding the CES-D, which leads me to believe that the authors planned to follow-up on individuals with high depression? How will this be accomplished?
c) Will participants be financially compensated for their participation?
d)There should be more clarity on which verbal subscales of the TTCT will be used. Additionally, how many raters will assess fluency, flexibility, and originality of ideas and how will the interrater-reliability be computed?
Like I stated above, I strongly recommend that the authors also measure reappraisal ability/reappraisal capacity with a maximum-performance test that capture participants’ objective reappraisal skills. Frequently used tests are the Reappraisal Inventiveness Test (RIT; Weber et al., 2014, which has also been used by Fink et al. 2017; Perchtold et al., 2019; Perchtold-Stefan et al., 2021), which assesses fluency, flexibility and ideational quality of reappraisals and has been shown to be a strong predicator of reappraisal success in daily life (Weber et al., 2014). In the RIT, participants are presented with an anger- or anxiety-eliciting situation and have three minutes to generates as many different reappraisals as possible. There is also the Script-based reappraisal Test (SRT) which is very similar to the Reappraisal Inventiveness Test (Zeier et al., 2020k, Cogn Emot; Zeier et al., 2021, Stress & Health). Alternatively, the authors could also have participants’ recall upsetting autobiographical memories and have them reappraise that memory as best as they can. Answers can then be rated for effectivity, plausibility, or even originality (see Rowlands et al., 2020; Cogn Emot). There is also the option of having participants’ rate their emotions with a specific situation prior and post reappraisal and then quantify the affective changes as indicated reappraisal ability scores.
In sum, for the analysis pipeline, I recommend:
1) Adding mediation analyses with an actual reappraisal ability measure
2) Considering gender as an additional factor in the analyses
1D. Clarity and degree of methodological detail
As states above (1C), more detail on the use and scoring of the TTCT should be provided. This also applies to any measure of reappraisal ability/capacity that the authors choose to implement.
1E. Outcome-neutral conditions/Data quality control
The authors briefly address their protocol for excluding participants from data analysis (blank responses to the creativity tasks, failure to complete all sections, etc.). Given that the study is conducted online via Amazon’s Mechanical Turk, I feel like more rigorous data quality control should be applied. For example, if participants are paid for study participation, are there any measures in place that prevent participants from filling in the surveys multiple times? Will TTCT answers be screened for plausibility? Will the overall test taking time be considered? (could point to arbitrary answers on the questionnaires). I would like the authors to elaborate on their security protocol to avoid that data from automated bots is treated at real participant data.