Integrative meta-analysis of anxiety-related traits and fear processing: bridging research to clinical application

The authors have provided a careful revision of their initial submission. They have convincingly addressed and incorporated my previous comments. I particularly appreciated that clear justifications were formulated to support the decisions made. I also think the inclusion of a z-curve approach is a nice addition. Overall, the revisions contributed to strengthening even further the manuscript and the planned meta-analysis. I look forward to seeing the results of this registered meta-analytic report and wish the authors good luck in this endeavor.

Signed,
Yoann Stussi

The authors have addressed each of my points comprehensively and satisfactorily. Their detailed explanations and the revisions made have resolved all my issues about the manuscript.

Therefore, I fully support the advancement of this article to the next stage of the review process.

No further comments

The authors of the registered report "Associations between anxiety-related traits and fear acquisition and extinction - an item-based content and meta-analysis" have addressed all the minor concerns I highlighted in the first review very well. In particular, I appreciated the more detailed information provided for the qualitative assessment of the included studies and the more careful use of the Q-test for assessing between-study heterogeneity.

The stage-1 registered report by Bruntsch et al. details the rationale and a protocol for conducting a meta-analysis of the associations between anxiety-related traits and threat acquisition and extinction during Pavlovian conditioning. The relevant studies will be identified with a systematic literature search and these associations will be assessed by means of a series of nested random-effects models using self-report anxiety-related questionnaires and psychophysiological and self-report measures of threat acquisition and extinction. Expected results are that there is a positive association between anxiety-related traits and threat acquisition and extinction. Moderator analyses will be also conducted to explore the effects of different anxiety-related questionnaires and methodological characteristics (e.g., reinforcement rate, clinical vs. non-clinical groups, type of outcome measures, etc.). Finally, an item-based content analysis will be performed to examine the overlap in content between anxiety-related questionnaires, with a limited content overlap being expected.

The registered report is very clear and well documented. The additional files and documents openly available on the associated Open Science Framework repository are clear and comprehensive. The main research question and secondary research questions are scientifically valid and well justified. Based on existing meta-analyses (e.g., Duits et al., 2015; Morriss et al., 2021), the hypothesis of a positive—albeit relatively modest—association between anxiety-related trait and threat acquisition and extinction is logic, well-motivated, and plausible. The minimal number of studies required to be included in the meta-analysis is clearly justified with an a priori power analysis based on related but more specific meta-analyses (Morriss et al., 2021; Sep et al., 2019). The methodology and analytic plan appear sound and feasible. The literature search and meta-analysis components are rigorously described with a high level of detail. The analytic pipeline additionally includes meta-analytic techniques to control for publication bias and quality checks are also considered, which contributes to increasing the chances of the meta-analysis to provide informative findings and calibrated interpretations. Overall, my evaluation is that this work has great potential to offer a highly informative and worthy contribution to the study of the links between individual differences in anxiety-related symptomatology and threat conditioning processes. I have a few comments that may be worth considering to strengthen this registered report even further. Specifically, I wonder about (a) the comprehensiveness of the meta-analytic methods used and decisions made to control for publication bias and (b) whether the nature of the conditioned (CS) and unconditioned (US) stimuli should be considered as another moderator in the moderator analyses. I describe these comments below along with some other minor points. I hope they will be constructive and helpful.

Primary comments

1)      It may be particularly beneficial to consider further statistical approaches—such as trim-and-fill, precision-effect test (PET), precision-effect estimate with standard error (PEESE), PET-PEESE, p-uniform—that aim to correct for potential publication bias in addition to the three-parameter selection model and the p-curve analysis. Because these various meta-analytic methods perform differently under various conditions (i.e., presence/absence of questionable research practices, publication bias, heterogeneity; see, e.g., the simulation work by Carter et al., 2019), the comprehensive inclusion of these techniques may contribute to more finely establishing the sensitivity of the associations between anxiety-related traits and threat acquisition and extinction. This could in turn allow for more calibrated and nuanced interpretations of the findings. If the authors prefer to exclusively use the three-parameter selection model and p-curve analysis, it would be important to provide a thorough justification of why these methods were selected over other available methods.

2)      Relatedly, I was unsure whether the benefits of the decision to only include published and peer-reviewed studies in the meta-analysis outweigh its costs. Whereas I understand that incorporating unpublished data (e.g., via calls to relevant societies and mailing lists as well as dissertations and theses databases such as ProQuest; https://www.proquest.com) would require a significant additional amount of work, this could potentially help provide a more accurate effect-size estimate of the associations between anxiety-related traits and threat learning and extinction that is less likely to be overinflated by publication bias. The inclusion of unpublished data was notably done in Duits et al.’s (2015) meta-analysis. If the authors still prefer to only include published and peer-reviewed studies, it would be extremely beneficial to provide a more elaborated cost-benefit analysis for this decision that would carefully balance the benefits associated with it (e.g., less time-consuming, no potential fluctuations of effect sizes) and the risks (e.g., risk of higher publication bias and overestimated effect sizes).

3)      The moderator analyses do not consider potential effects related to the CS and US nature. Nonetheless, these factors exert a powerful influence on threat acquisition and extinction (see, e.g., Rescorla & Wagner, 1972). For instance, threat-relevant or affective relevant CS have been shown to induce faster threat acquisition and enhanced resistance to extinction during Pavlovian threat conditioning (e.g., Öhman & Mineka, 2001; Stussi et al., 2018, 2021), while electro-tactile USs have been reported to elicit stronger physiological conditioned responses than loud screams (e.g., Ney et al., 2023). Although I’m not aware of any robust evidence that the CS and US nature moderate the associations between anxiety-related traits and threat acquisition and/or extinction, addressing this question may provide valuable information (provided there are enough studies investigating these aspects that can be included in the meta-analysis). Based on the proposal that threat-relevance (or “preparedness”) is a key factor in the development of phobias (Öhman & Mineka, 2001; Seligman, 1971), this may have especially relevant translational implications for clinical research and applications using threat conditioning as a model for the etiology and maintenance of anxiety-related conditions. Please note, however, that this is merely a suggestion and that I’m not requesting the the CS and the US nature to be included as moderators in the meta-analysis.

Secondary comments

4)      In the introduction (p. 4), I believe it could be worth referring to more recent work linking threat conditioning processes to the etiology and maintenance of anxiety-related conditions (e.g., Beckers et al., 2023; Zinbarg et al., 2022). This may contribute to further highlighting the translational relevance and timeliness of threat conditioning as a laboratory model for anxiety-related disorders, which might not necessarily be obvious to audiences outside of the field.

5)      I’m not sure I fully understood the decision to use the mean of reported effect size from two prior meta-analyses as an effect size estimate for the power analysis. Whereas I understand that meta-analyses often provide a good basis for effect-size estimates, it has been suggested that the lowest available or meaningful effect size be used to adjust for publication bias (see https://rr.peercommunityin.org/PCIRegisteredReports/about/full_policies#h_6720026472751613309075757, point 3.3). A more conservative approach could thus be to use the minimal effect size (g = 0.22) reported in Morriss et al. (2021) as an effect-size estimate for the power analysis. Based on the power curve plot (which is a great addition) and Duits et al.’s (2015) meta-analysis, I’m confident that the minimal number of studies (n = 23-24) that need be included to achieve sufficient statistical power to observe an effect size of g = 0.22 for the primary analysis will be reached.

6)      On page 16, “in association to anxiety-related traits” should be “in association with anxiety-related traits”.

7)      In the Study design table, under the column “Theory that could be shown wrong by the outcomes” and for the question “Do effects differ if different questionnaires have been used to assess anxiety-related traits?”, it would be important to condition the proposed interpretation on (a) the finding of an overall positive association between anxiety-related traits and threat acquisition and/or extinction, and (b) the confidence interval around the effect size for the moderator effect of the questionnaires allowing to determine whether the observed effects are consistent with an absence of effect (if the confidence interval is narrow) or a larger range of effects (if the confidence interval is wide). Without these conditions, I do not think it would be possible to unequivocally interpret the absence of statistically significant moderator effect of questionnaires as “support for an overall anxiety-related trait as potentially more explanatory of fear acquisition and extinction learning than any single questionnaire.”

Signed,
Yoann Stussi

REFERENCES

-          Beckers, T., Hermans, D., Lange, I., Luyten, L., Scheveneels, S., & Vervliet, B. (2023). Understanding clinical fear and anxiety through the lens of human fear conditioning. Nature Reviews Psychology, 2, 233-245. https://doi.org/10.1038/s44159-023-00156-1

-          Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019). Correcting for bias in psychology: A comparison of meta-analytic methods. Advances in Methods and Practices in Psychological Science, 2(2), 115-144. https://doi.org/10.1177/2515245919847196

-          Duits, P., Cath, D. C., Lissek, S., Hox, J. J., Hamm, A. O., Engelhard, I. M., van den Hout, M. A., & Baas, J. M. P. (2015). Updated meta-analysis of classical fear conditioning in the anxiety disorders. Depression and Anxiety, 32(4), 239-253. https://doi.org/10.1002/da.22353 

-          Morriss, J., Wake, S., Elizabeth, C., & van Reekum, C. M. (2021). I doubt it is safe: A meta-analysis of self-reported intolerance of uncertainty and threat extinction training. Biological Psychiatry Global Open Science, 1(3), 171-179. https://doi.org/10.1016/j.bpsgos.2021.0.011

-          Ney, L. J., Nichols, D. S., & Lipp, O. V. (2023). Fear conditioning depends on the nature of the unconditional stimulus and may be related to hair levels of endocannabinoids. Psychophysiology, 60(8), Article e14297. https://doi.org/10.1111/psyp.14297

-          Öhman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychological Review, 108(3), 483-522. https://doi.org/10.1037/0033-295X.108.3.483

-          Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prosky (Eds.), Classical conditioning II: Current research and theory (pp. 64-99). Appleton-Century-Crofts. 

-          Seligman, M. E. P. (1971). Phobias and preparedness. Behavior Therapy, 2(3), 307-320. https://doi.org/10.1016/S0005-7894(71)80064-3

-          Sep, M. S. C., Steenmeijer, A., & Kennis, M. (2019). The relation between anxious personality traits and fear generalization in healthy subjects: A systematic review and meta-analysis. Neuroscience and Biobehavioral Reviews, 107, 320-328. https://doi.org/10.1016/j.neubiorev.2019.09.029

-          Stussi, Y., Pourtois, G., Olsson, A., & Sander, D. (2021). Learning biases to angry and happy faces during Pavlovian aversive conditioning. Emotion, 21(4), 742-756. https://doi.org/10.1037/emo0000733

-          Stussi, Y., Pourtois, G., & Sander, D. (2018). Enhanced Pavlovian aversive conditioning to positive emotional stimuli. Journal of Experimental Psychology: General, 147(6), 905-923. https://doi.org/10.1037/xge0000424

-          Zinbarg, R. E., Williams, A. L., & Mineka, S. (2022). A current learning theory approach to the etiology and course of anxiety and related disorders. Annual Review of Clinical Psychology, 18, 233-258. https://doi.org/10.1146/annurev-clinpsy-072220-021010

The current registered report “Associations between anxiety-related traits and fear acquisition and extinction - an item-based content and meta-analysis” by authors Bruntsch et al. aims to investigate, through a meta-analysis of the existing literature, the association between fear acquisition and extinction, and anxiety related traits. Studies that used at least one among several different measures of fear conditioning (e.g., SCR, fear-potentiated startle) and one among different questionnaires to assess anxiety-related traits (e.g., STAI-T) are considered for the analysis. Several moderator and additional analyses are planned to be included in the manuscript.

The study is extremely well written, the hypotheses are clear and the planned methodology is explained very well and in detail. Importantly, the authors are careful in following the PRISMA guidelines for meta-analyses. Moreover, the research question is scientifically valid, and I believe the meta-analysis can interest a broad audience in several fields, including neuroscience, psychology and psychiatry, possibly become an important tool for future studies on fear conditioning and anxiety.

However, I have some minor concern that I believe should be addressed prior to support its progress to stage 2.

1)      Despite a previous meta-analysis on the relationship between fear extinction and anxiety-related traits was already published (Morriss et al., 2021), the authors want to focus part of this meta-analysis on such fear conditioning phase to give a more comprehensive understanding of the phenomenon, because the previous meta-analysis considered a limited number of variables. I appreciate this idea, but I believe authors should clarify which different results they expect from the new analysis, compared to Morriss and colleagues, and how such expected results may add novel insight into the relationship between extinction and anxiety related traits, in order to justify this new analysis.

2)      The authors are going to conduct the literature search on Web of Science and Pubmed. I suggest the authors to add Psycinfo to the other databases. I think such database can most fit with the query strings proposed by the authors. Moreover, this database overlaps only partially with the results found through Pubmed, whose articles are often selectively related to clinical aspects.

3)      The test of outlier will be conducted using two different methods, respectively based on the extreme effect size and on the “leave-one-out method”. I have some concern about this last method. Specifically, which are the criteria that the function InfluenceAnalysis compute to select the outliers? Please specify them, if possible.

4)      Another doubt is about the quality assessment of included studies. Authors state that they will assess it by quantifying some quality indicators: for each criterion, studies can receive a score of two points (lower numbers would correspond to higher quality, and higher number would indicate lower quality). The total score will be included between 0 and 14. Moreover, authors want to assess other descriptive variables, but they affirm that “these additional descriptive information will not be included in the quality assessment score”. So, it is not clear to me how they want to assess these information and eventually how they can be visualized with the Risk of bias tool if they are not quantitatively assessed.

5)      Authors mention the possibility to perform an additional moderator analysis on “the implementation of the experiment in an MRI scanner”. What does “additional” mean? In other words, how does this moderator differ from the others proposed?

The authors of the manuscript “Associations between anxiety-related traits and fear acquisition and extinction – an item-based content and meta-analysis” aim to investigate the relationship between implicit and explicit anxiety-related trait measures and classical fear conditioning acquisition and extinction. Moreover, the authors aim to assess the modulating effect of explicit anxiety-related traits at both questionnaires and questionnaire item levels.

This registered report demonstrates the potential to fill a gap in the literature concerning the relationship between anxiety-related traits and classical fear conditioning features. The manuscript appears to be very well-written and comprehensive in both theoretical and methodological aspects. However, some minor concerns should be addressed to help improve the quality of the manuscript:

1.      The search strategy initially focused on standardized procedure to reduce possible biases; however, at the end of the search process, the authors decided to add “fearfulness” and “harm avoidance” as search terms “to cover more aspects of anxiety-related traits”. This decision could risk weakening the carefully executed standardized search strategy. Could the authors provide a more detailed explanation of why new search terms were added? Additionally, “fearfulness” and “harm avoidance” didn’t appear in the query strings on OSF. Please add these two search terms.

2.      Although eligibility criteria were well explained; reporting them in list form could enhance readability.

3.      The authors reported that they will use τ2 parameter as one of the measures of between-study heterogeneity, which represents “the standard deviation of the true effect size”. Actually, τ2 represents the variance and not the standard deviation of the true effect size (Harrer et al., 2022). 

4.      τ2, I2, H2, and Q-test parameters will be used to assess the between-study heterogeneity. While τ2, I2, and H2 could provide different types of information regarding between-study heterogeneity, Q-test should be used carefully as it could produce biased results based on the statistical power of the meta-analysis (Harrer et al., 2022).

5.      Quality assessment of the included studies will be conducted by using a well-established protocol. However, it is not clear what will be the inclusion/exclusion criteria of the studies based on the protocol. Authors should clarify if there will be a cut-off in the rating or if other parameters will be used.