Recommendation

Replicable dynamic functional connectivity and cognitive correlates of cerebral small vessel disease in the Hamburg City Health Study

ORCID_LOGO based on reviews by 1 anonymous reviewer
A recommendation of:
toto

Functional MRI brain state occupancy in the presence of cerebral small vessel disease -- a pre-registered replication analysis of the Hamburg City Health Study

Abstract
Keywords
Submission: posted 17 October 2023
Recommendation: posted 01 February 2024, validated 05 February 2024
Cite this recommendation as:
McIntosh, R. (2024) Replicable dynamic functional connectivity and cognitive correlates of cerebral small vessel disease in the Hamburg City Health Study. Peer Community in Registered Reports, 100576. 10.24072/pci.rr.100576

Recommendation

In a previous analysis of data from 988 participants in the Hamburg City Health Study (HCHS), Schlemm and colleagues (2022) reported significant associations between the extent of cerebral small vessel disease (cSVD) and dynamic functional connectivity measures from resting state fMRI. Specifically, the volume of white matter hyperintensities of presumed vascular origin, a structural indicator of cSVD, was negatively related to the proportion of time (‘fractional occupancy’) spent in the two most occupied functional brain states. Reduced fractional occupancy was also associated with longer times to complete part B of the Trail Making Test.
 
In the present Registered Report, Ingwersen and colleagues (2023) successfully replicated these associations between structural, functional and cognitive measures in a sample of 1651 HCHS participants not included in the earlier study. An exploratory multiverse analysis found that the associations were generally robust to different brain parcellation and confound regression strategies. These replicable patterns reinforce the idea that cSVD may disrupt the brain’s ability to enter and maintain distinct functional modes, and that these changes in functional dynamics are predictive of cognitive impairment.
 
The Stage 2 manuscript was assessed over one round of in-depth review. The recommender judged that responses to reviewer comments were appropriate, and that the manuscript met the Stage 2 criteria for recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/9yhzc
 
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to Stage 1 in-principle acceptance, but the authors certify that they had not yet observed the key variables within the data that were used to answer the research question AND they took additional steps to maximise bias control and rigour.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Schlemm, E., Frey, B. M., Mayer, C., Petersen, M., Fiehler, J., Hanning, U., Kühn, S., Twerenbold, R., Gallinat, J., Gerloff, C., Thomalla, G. & Cheng, B. (2022). Equalization of brain state occupancy accompanies cognitive impairment in cerebral small vessel disease. Biological Psychiatry, 92, 592-602. https://doi.org/10.1016/j.biopsych.2022.03.019
 
2. Ingwersen, T., Mayer, C., Petersen, M., Frey, B. M., Fiehler, J., Hanning, U., Kühn, S., Gallinat, J., Twerenbold, R., Gerloff, C., Cheng, B., Thomalla, G. & Schlemm, E. (2023). Functional MRI brain state occupancy in the presence of cerebral small vessel disease -- a pre-registered replication analysis of the Hamburg City Health Study. Ecceptance of Version 2.01 by Peer Community in Registered Reports. https://github.com/csi-hamburg/HCHS-brain-states-RR/blob/f9d00adbbcf9593d8d191bf5b93912141b80ab1b/manuscript/build/main.pdf
 
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Evaluation round #1

DOI or URL of the report: https://github.com/csi-hamburg/HCHS-brain-states-RR/blob/d5a42f66c833cc023323ea9ee32448bb1ea71720/manuscript/build/main.pdf

Version of the report: v2.0.1

Author's Reply, 24 Jan 2024

Decision by ORCID_LOGO, posted 09 Jan 2024, validated 09 Jan 2024

Thank you for submitting your Stage 2 Registered Report. The manuscript has been assessed by one of the original (Stage 1) reviewers, who has provided some insightful comments, to which you should respond with appropriate revisions (or rebuttals).

Whilst you should do your best to fully consider reviewer comments, you should not change your Stage 1 material without further consultation, and you are not obliged to follow reviewer suggestions for additional exploratory analyses at this stage. That said, I think it would be acceptable to add information about achieved sample size into Figure 1.

I have been unable to obtain comments from a second reviewer and, given the time already elapsed, have decided not to wait any longer. I append some comments of my own, though these are more focused on RR requirements than on the specific scientific topic.

First, you have removed from the Stage 1 manuscript your previous pilot analysis, and also moved the timeline section (describing Stage 1 state of knowledge of the data) to the end of the manuscript. The timeline section should be reinstated within the Methods for correspondence with the approved Stage 1 plan. If you wish to remove the pilot analysis from the Stage 2 manuscript, you should explain your reasoning in your response, so that it can be evaluated, and you should at least add a footnote to the Stage 2 Methods to inform the reader that a pilot analysis included at Stage 1 has been omitted for brevity but can be found in the archived Stage 1 manuscript, providing a link to that document.

In passing, I note two very minor typographical/stylistic points: (1) please regularise 'subjects' to 'participants'; (2) there seems to be a word or two missing from the following: "network activation profiles were computed for brain states estimated Schaefer parcellations..."

Reviewed by anonymous reviewer 1, 15 Nov 2023

The authors present a phase 2 pre-registered replication study to examine associations between dynamic resting-state fMRI, small vessel disease (WMH), and cognition. The research question is scientifically valid, but the theoretical rationale requires some additional clarification and justification. The sample and methods are mostly appropriate, but I offer some suggestions for improved rigor. Results are presented clearly, but I offer some suggestions for additional transparency. My strongest critique is that the authors’ characterization of “robustness” in the behavioral association does not appear to be supported by the data.

1.     Figure 1 should be updated to report the achieved sample rather than expected.

2.     What is the justification for focusing on average fractional occupancy in either DMN+ or DMN- clusters? How is occupancy in these two clusters related across individuals? Are similar associations with WMH or cognition observed for DMN+ or DMN- occupancy individually?

3.     “49/81 (39/81) negative and 8/81 (0/81) associations of nominal statistical significance” I assume this sentence is missing the word “positive” after (0/81)?

4.     The authors acknowledge that the TMT-B results are “somewhat less robust” than the WMH results, but this wording seems too generous given that the effect nominally replicates in less than 20% of the analyses. At the very least, they should remove the word “somewhat” as this effect is clearly less robust than the WMH effect. It is also misleading to state that both effects are robust in line 301 and line 344 without qualification. Overall robustness should be assessed in a meta-analysis-like approach by calculating the average effect size and CI across the multiverse analyses. Is the average effect significantly different from 1? How does it compare to the observed effect size in the 2022 paper?

5.     Forest plots of the multiverse analyses for periventricular and deep WMH volumes should be provided as supplementary material.

6.     Figure 6. Do these spider plots align with network patterns identified in the 2022 paper?

7.     The spider plots do not characterize regional patterns of high vs. low activation in the clusters. Please provide brain images as well.

8.     It does not appear that the tests of additional cognitive relationships were corrected for multiple comparisons.

9.     “all reported associations were robust to additional, unplanned adjustments for DVARS, RMSD or mean 361 framewise displacement.” - please provide the details of these analyses as supplementary material.

User comments

No user comments yet