Understanding predictors of white matter lesions in the human brain
Investigating the impact of vascular risk factors on the progression of white matter lesions
Recommendation: posted 06 February 2023, validated 06 February 2023
URL to the preregistered Stage 1 protocol: https://osf.io/qkbgj
List of eligible PCI RR-friendly journals:
- Brain and Neuroscience Advances
- NeuroImage: Reports
- Peer Community Journal
- Royal Society Open Science
Chris Chambers (2023) Understanding predictors of white matter lesions in the human brain. Peer Community in Registered Reports, . https://rr.peercommunityin.org/articles/rec?id=320
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.
Reviewed by Max Elliott, 30 Jan 2023
The authors have sufficiently responded to my comments and the comments of the editor and other reviewers. The document is improved as a result and I have no further comments at this stage.
Reviewed by Isabel García-García, 24 Jan 2023
The authors have addressed all the previous points and the new report reads much clearer now. I do not have further suggestions.
Evaluation round #1
DOI or URL of the report: https://osf.io/ur9v4
Version of the report: v1
Author's Reply, 17 Jan 2023
Decision by Chris Chambers, posted 19 Dec 2022, validated 19 Dec 2022
I have now received two very detailed and constructive reviews of your submission. As you will see, the evaluations are broadly encouraging while also raising a variety of issues that will need careful attention to achieve Stage 1 in-principle acceptance (IPA). Most of the reviewers' comments seek more detailed clarification and justification of specific design features and analysis choices, but there are also some key points to consider concerning the study rationale.
In my own reading, I noted two additional points:
1. Bias control level
In the section "Existing data", you state: "At the time of this stage-1 protocol, we have access to the baseline anthropometric and medical data and have preprocessed and quality-controlled the imaging data of both time points. We have not gained access to the follow-up anthropometric, medical and cognitive data and have not explored any associations of these measures with WML volume beyond the baseline investigations cited above."
In Q8 of the submission checklist you selected Level 3 for the bias-control level: "At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence [Level 3]"
Level 3 requires that while authors have accessed the data, they have not observed it (in any way). Based on your description from the Method, however, it appears that you have observed some of the data and undertaken preprocessing. Therefore, I believe Level 2 is the most appropriate level:
"At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour (e.g. conservative statistical threshold; recruitment of a blinded analyst; robustness testing, multiverse/specification analysis, or other approach) [Level 2]"
At the same time, I also think the risk of additional bias resulting from your prior analyses is minimal, so I would set aside the normal requirement for Level 2 that you take take "additional steps to maximise bias control and rigour" on top of what you have already proposed.
If you agree with this assessment then when you submit your revision please alter the checklist entry for Q8 to select Level 2, and please state the assigned level in the "Existing data" section of the Method. If you disagree with my assessment then please include a rebuttal in your response letter.
2. Design table and page numbers
Please reduce the font size in the study design table and present it in landscape format so that it is more digestible to readers. Please also ensure that your revised submission includes page numbers.
Overall, I think your study is promising candidate for IPA and I look forward to receiving your revised submission and point-by-point response to the reviews in due course. Please note that PCI RR is currently closed to all submissions (including revisions) until 3rd January.
Reviewed by Max Elliott, 16 Dec 2022
The authors plan to investigate whether blood pressure and waste-to-hip ratio influence the progression of cerebral small vessel disease. Overall, the research question and hypothesis are clear and fit into the literature in the field. The analysis plan is generally thorough, and the power analysis is well-described and justifies the planned statistical tests. Below I describe several questions and limitations I found while evaluating the registered report
- In the introduction the authors mention that they will replicate findings and extend findings, however, it is unclear which aims, and hypotheses refer to replications versus extensions. It would be useful to make this explicit. What is a replication and what is an extension or novel aim?
Aims and Hypotheses
- H1, H2, and H3 seem to all be replications of previous research, is that right? If the primary goal of this project is to replicate previous results, then this is fine, but the researchers should make it more explicit in their introduction that the primary goal of this project is replication. However, based on the introduction it seems the researchers have more clear goals to ask novel questions about the influence of gender and Waist to hip ratio on longitudinal white matter lesions. If so, if so, I encourage the authors to explicitly state hypotheses about the waist-to-hip ratio and gender analyses.
- Why did the authors select these covariates for their hypotheses? These covariates seem reasonable, but I encourage the authors to justify each covariate and explain why its inclusion is important to each hypothesis/analysis.
- The methods are clear and logical. They are sufficiently detailed for replication and to limit analytical flexibility. I appreciate their thorough review of previous literature to justify parameters used in their power analyses
- How did the authors choose p<.033 for establishing statistical significance? Is this a field convention, or based on another justification?
- In the abstract and intro, sex/gender stratified analyses are discussed as a central focus of this project and these are included in the statistical models. However, I do not see a clear description of how sex/gender inferences will be made or evaluated from these models. Given the introduction, I encourage the authors to thoroughly describe their plans for evaluating and drawing inferences based on sex/gender.
- I would highly recommend using fewer acronyms. It is very hard to track what each acronym is, and this makes reading the text burdensome. A few acronyms for the most common and well-known terms are fine, but currently, there is an acronym in nearly every sentence. SBP is not defined as far as I can tell.
Reviewed by anonymous reviewer, 14 Dec 2022
In this proposal Beyer et al. have designed a study aimed at assessing the link between blood pressure and progression of white matter hyperintensity load, as well as examining the cognitive implications of white matter hyperintensity progression. Overall, the research questions are pertinent and valid, the methodology is robust, and the transparency of the research plan is satisfactory. I would like to have some clarifications on certain points that are outlined below.
1A. Scientific validity of the research question(s)
This proposal uses a longitudinal design to address the prospective effects of blood pressure and abdominal obesity on the progression of white matter hyperintensities. Additionally, it tests for the functional implication of increases in white matter hyperintensity load, since it also examines its associations with longitudinal decline in executive functions and in general cognition.
The research question is valid and it is clearly defined. The proposal includes both exploratory and hypothesis-based analyses.
1B. Logic, rationale, and plausibility of the proposed hypotheses
This proposal states three main hypothesis, which are logic and plausible.
1C. Soundness and feasibility of the methodology and analysis pipeline
Methodologically speaking, the current proposal is sound and feasible. The authors have performed a sophisticated statistical power analysis where they take into account previous results from the literature and they simulate their predictive power in their dataset, according to their baseline information.
I was missing some clarification regarding certain aspects of the proposal. Namely:
- Why do the authors focus on systolic blood pressure instead of diastolic blood pressure?
The longitudinal data has already been collected but the authors do not have access yet. From the information that the authors provide, I assume that the follow-up range should be somewhere between 3 and 10 years. Is this correct? Do the authors happen to have a mode accurate description of this?
- Regarding the confirmatory analyses, the authors will test 3 regression models that are aligned with the hypotheses. The summary provided in Table 7 highlights what predictors are expected to be significant regarding their hypotheses. I am a bit confused regarding the number of predictors and the highlights. The authors state somewhere earlier that the analyses will be divided by sex/gender. Then, why have they included sex as a predictor variable in the regression models?
- Interactions between variable of interest and age change. Could the authors state a bit clearer what is the rationale for including these interactions?
- Please correct me if I am wrong, in Table 7, M2 and M3 are missing the interaction term hyperintensity load x age change while this is stated in the text. Also, why is it that in M1 the highlighted predictor is the interaction term between systolic blood pressure x age change, while this is not the case for M2 and M3? If I am reading the hypothesis correctly, the predictor of interest in M1 would be systolic blood pressure.
- Regarding the dependent variable specified in models M1-M3, does it reflect the variable collected in the follow-up or is it a change score? Would the authors control for the scores obtained in these very same variables at baseline?
The authors have provided detailed methods. I do not have concerns regarding this point.
1E. Consideration of outcome-neutral conditions
This part has been addressed satisfactorily. The authors have detailed the thresholds at which they will consider that a certain variable is invalid for each participant.