Recommendation
The authors have sufficiently responded to my comments and the comments of the editor and other reviewers. The document is improved as a result and I have no further comments at this stage.
, 24 Jan 2023The authors have addressed all the previous points and the new report reads much clearer now. I do not have further suggestions.
DOI or URL of the report: https://osf.io/ur9v4
Version of the report: v1
, posted 19 Dec 2022, validated 19 Dec 2022I have now received two very detailed and constructive reviews of your submission. As you will see, the evaluations are broadly encouraging while also raising a variety of issues that will need careful attention to achieve Stage 1 in-principle acceptance (IPA). Most of the reviewers' comments seek more detailed clarification and justification of specific design features and analysis choices, but there are also some key points to consider concerning the study rationale.
In my own reading, I noted two additional points:
1. Bias control level
In the section "Existing data", you state: "At the time of this stage-1 protocol, we have access to the baseline anthropometric and medical data and have preprocessed and quality-controlled the imaging data of both time points. We have not gained access to the follow-up anthropometric, medical and cognitive data and have not explored any associations of these measures with WML volume beyond the baseline investigations cited above."
In Q8 of the submission checklist you selected Level 3 for the bias-control level: "At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence [Level 3]"
Level 3 requires that while authors have accessed the data, they have not observed it (in any way). Based on your description from the Method, however, it appears that you have observed some of the data and undertaken preprocessing. Therefore, I believe Level 2 is the most appropriate level:
"At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour (e.g. conservative statistical threshold; recruitment of a blinded analyst; robustness testing, multiverse/specification analysis, or other approach) [Level 2]"
At the same time, I also think the risk of additional bias resulting from your prior analyses is minimal, so I would set aside the normal requirement for Level 2 that you take take "additional steps to maximise bias control and rigour" on top of what you have already proposed.
If you agree with this assessment then when you submit your revision please alter the checklist entry for Q8 to select Level 2, and please state the assigned level in the "Existing data" section of the Method. If you disagree with my assessment then please include a rebuttal in your response letter.
2. Design table and page numbers
Please reduce the font size in the study design table and present it in landscape format so that it is more digestible to readers. Please also ensure that your revised submission includes page numbers.
Overall, I think your study is promising candidate for IPA and I look forward to receiving your revised submission and point-by-point response to the reviews in due course. Please note that PCI RR is currently closed to all submissions (including revisions) until 3rd January.
The authors plan to investigate whether blood pressure and waste-to-hip ratio influence the progression of cerebral small vessel disease. Overall, the research question and hypothesis are clear and fit into the literature in the field. The analysis plan is generally thorough, and the power analysis is well-described and justifies the planned statistical tests. Below I describe several questions and limitations I found while evaluating the registered report
Intro
- In the introduction the authors mention that they will replicate findings and extend findings, however, it is unclear which aims, and hypotheses refer to replications versus extensions. It would be useful to make this explicit. What is a replication and what is an extension or novel aim?
Aims and Hypotheses
- H1, H2, and H3 seem to all be replications of previous research, is that right? If the primary goal of this project is to replicate previous results, then this is fine, but the researchers should make it more explicit in their introduction that the primary goal of this project is replication. However, based on the introduction it seems the researchers have more clear goals to ask novel questions about the influence of gender and Waist to hip ratio on longitudinal white matter lesions. If so, if so, I encourage the authors to explicitly state hypotheses about the waist-to-hip ratio and gender analyses.
- Why did the authors select these covariates for their hypotheses? These covariates seem reasonable, but I encourage the authors to justify each covariate and explain why its inclusion is important to each hypothesis/analysis.
Methods
- The methods are clear and logical. They are sufficiently detailed for replication and to limit analytical flexibility. I appreciate their thorough review of previous literature to justify parameters used in their power analyses
- How did the authors choose p<.033 for establishing statistical significance? Is this a field convention, or based on another justification?
Statistical Analyses
- In the abstract and intro, sex/gender stratified analyses are discussed as a central focus of this project and these are included in the statistical models. However, I do not see a clear description of how sex/gender inferences will be made or evaluated from these models. Given the introduction, I encourage the authors to thoroughly describe their plans for evaluating and drawing inferences based on sex/gender.
Miscellaneous
- I would highly recommend using fewer acronyms. It is very hard to track what each acronym is, and this makes reading the text burdensome. A few acronyms for the most common and well-known terms are fine, but currently, there is an acronym in nearly every sentence. SBP is not defined as far as I can tell.
In this proposal Beyer et al. have designed a study aimed at assessing the link between blood pressure and progression of white matter hyperintensity load, as well as examining the cognitive implications of white matter hyperintensity progression. Overall, the research questions are pertinent and valid, the methodology is robust, and the transparency of the research plan is satisfactory. I would like to have some clarifications on certain points that are outlined below.
1A. Scientific validity of the research question(s)
This proposal uses a longitudinal design to address the prospective effects of blood pressure and abdominal obesity on the progression of white matter hyperintensities. Additionally, it tests for the functional implication of increases in white matter hyperintensity load, since it also examines its associations with longitudinal decline in executive functions and in general cognition.
The research question is valid and it is clearly defined. The proposal includes both exploratory and hypothesis-based analyses.
1B. Logic, rationale, and plausibility of the proposed hypotheses
This proposal states three main hypothesis, which are logic and plausible.
1C. Soundness and feasibility of the methodology and analysis pipeline
Methodologically speaking, the current proposal is sound and feasible. The authors have performed a sophisticated statistical power analysis where they take into account previous results from the literature and they simulate their predictive power in their dataset, according to their baseline information.
I was missing some clarification regarding certain aspects of the proposal. Namely:
1D. Replicability
The authors have provided detailed methods. I do not have concerns regarding this point.
1E. Consideration of outcome-neutral conditions
This part has been addressed satisfactorily. The authors have detailed the thresholds at which they will consider that a certain variable is invalid for each participant.