An fNIRS test of the neural correlates of the Cardinality Principle in typically-developing children

In the revised version of the Registered Report, the authors have significantly improved the quality of the manuscript and have well addressed my comments. I believe that the changes made to the experimental paradigm and analyses make the paper easier to follow and would provide a more in-depth exploration of the research question.

I have some outstanding points for the authors, which I think should be addressed before performing the study:

- The authors now include HHb within the analyses, but I think some information is still missing in relation to how this signal will be used in the GLM. For example: "The highest coefficient of HbO for each condition on each region of interest (ROIs; bilateral parietal regions) will be used to test whether CP-knowers exhibit higher bilateral parietal activation, defined by increased HbO/decreased HHb, specifically in the left parietal region, relative to subset-knowers (Hypothesis 1). " This analysis doesn't account for HHb, as one would need to select the lowest coefficient of HHb for each condition on each region of interest to examine whether a significant decrease in HHb is present. If this is correct, the authors should carefully check their analyses throughout.

- I think that the inclusion criteria of "a minimum of one clean channel per ROI and per participant for inclusion" is too lanient. I fully appreciate the challenges associated with testing infants and children, but it would be important to ensure that any activation found is a true reflection of a genuine response. I believe that in the developmental fNIRS literature, a common objective criterion is to exclude a participant from further analyses if > 30% of all channels had to be excluded (e.g. due to weak or noisy signal). It is also quite common for studies to not consider single isolated channels further in the analysis. This is because a channel is deemed reliably active if at least one spatially contiguous channel is also significant (see for example Lloyd-Fox's fNIRS studies). So I think it would be best to change the inclusion/exclusion criteria accordingly (or at the bare minimum consider any isolated channel response with caution). You might also want to make sure that each channel contains valid data in both conditions to be included in further analysis.

This is a preregistration study to study of the neural origins of number understanding in 3-4 year old children.  The study team plans to use functional near infrared spectroscopy (fNIRS) to measure brain responses in parietal and frontal regions in children who either understand counting principles (CP-knowers) or who do not yet understand counting principles (subset-knowers).  Knowledge of counting status will be assessed via the classic “give-a-number” task, and participants will be matched, as well as possible on other demographic and cognitive factors, especially non-verbal IQ and mean age.  

Brain responses in these children will then be measured via fNIRS while the children engage in an adaptation design: Children will be presented with a repeated auditory number word (“six”) and occasional deviants of “four” or “nine” (same ratio distance from the standard).  For children who do not yet know the counting principles, the word “nine” is expected to be outside their semantic understanding of numbers, while “four is within their semantic range.  For CP-knowers both numbers would be within their semantic range.    

This study addresses an important and timely question and the study team is highly expert in both numerical cognition and fNIRS.  Finally, the use of fNIRS makes it more feasible in the young (3-4 years old) children who would be tested here. The system the authors have chosen is a portable system (Brite, Artinis Medical Systems BV, The Netherlands). The use of fNIRS is an important choice, and it has several key advantages, specifically being more appropriate for kids in this age range due to its tolerance for motion, the ease of use, and lower cost.  The use of a portable system will increase their ability to record data in various locations including preschools and other school buildings, but at the expense of having fewer optodes/channels. fNIRS in general has poorer spatial resolution than fMRI, and the limited number of channels could be an important limitation for multivariate analyses (see below).

The analysis plan seems appropriate (but, see below), and involves traditional univariate analyses of signal change (Oxy-Hb/HbO and Deoxy-Hb/HHb), functional connectivity analyses and (unspecified) multivariate analyses.  Statistics will be carried out within a Bayesian framework, which will allow the authors to not only provide evidence in favor of differences, but also to measure the strength of evidence for the null hypothesis.      

However, I have many concerns about the study as currently proposed, some theoretical, and some more methodological.  As I see it, these concerns each make it more likely that the study team will fail to detect differences (Type II Errors), rather than increaseing the possibility of spurious positive results (Type I Errors). I present these concerns here in the hopes that the study team will address these concerns prior to carrying out the study, and therefore increase their likelihood of success.  

CP-Knower Status = Semantic Understanding?  

My first, conceptual/theoretical concern about this study is that the authors equate CP-knower status with semantic understanding of numbers.  Although it is clear that young children know the count sequence in a rote manner prior to semantic understanding, it is not clear to me that successful performance on the give-a-number task is the only (or even the best) indication of semantic understanding of auditory number words. Success on the give-a-number task indexes not only semantic number knowledge but also executive function skills (maintaining task set, inhibiting the response to simply continue giving objects; Chu et al., 2019 https://doi.org/10.1016/j.jecp.2019.104668; Chen et al., 2022 https://doi.org/10.1111/bjdp.12439).  Additionally, recent work has suggested that children may have “partial knowledge” (O’Rear et al., 2020 https://doi.org/10.1111/desc.12944) of number sequences, even prior to “full” success on the give-a-number task. These results suggest that children may have graded semantic representations of number words even prior to being coded as CP-knowers in the traditional analysis. For both of these reasons, semantic understanding that 9 is larger than 6 might be present, but weaker, even in children who are not yet CP-knowers.  If so, we would predict that this partial semantic knowledge would lead to more similar activation patterns between the two groups. Concerns makes it more likely to fail to detect differences between groups.  

Adaptation Paradigm 

I am very concerned about the design of the design of adaptation paradigm, in which only one deviant type is presented per block (that is, after adapting to “six” only “four” is presented for all the deviants in a block).  Given the presence of only one deviant type per block, even young children might recognize this constancy, and would presumably pay less attention to the deviants during the course of each run.  The authors argue that they have chosen this design to avoid task switching, which is difficult for children. However, there is no active task on the deviants (the only task is to detect occasional winks of the smiley face at fixation) so having different types of deviants does not introduce additional task switching demands. As adaptation effects reflect a mix of bottom-up and top-down processes (e.g., Summerfield et al., 2008 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2747248/; Larsson & Smith, 2012l https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3278317/) reducing the top-down attentional components that are present in other adaptation studies may reduce the ability to detect differences between the conditions. Concern makes it more likely to fail to detect differences between conditions.    

Sample Size/Power Analysis

I am concerned that the authors can only draw on one (quite different) study to estimate effect sizes for the power analysis.  The paradigm that the authors intend to use is based on Vogel et al., 2017, while the effect size estimate comes from Holloway et al., 2013.  Holloway et al. found an effect size (d) of approximately 0.73.  However, that study used fMRI to measure adaptation in bilingual (Chinese-English) adults, while the proposed study will use fNIRS to measure adaptation only to auditory number words in 3-4 year old children. Additionally, although the paradigms are both adaptation paradigms and both use at least some auditory stimuli, there is very little else that is similar between the paradigms.   

This raises several questions:

First, is it reasonable to expect similar effect sizes in children and adults?  The general pattern of weaker responses in children (including many studies by the co-authors of this proposal; e.g., Ansari & Dhital, 2006 https://pubmed.ncbi.nlm.nih.gov/17069473/) suggest that the answer here would be no.  

Second, it is reasonable to expect similar effect sizes for fMRI and fMRI? Again, the answer would appear to be no.  In a particularly relevant study, Cui et al. (2011 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3021967/) collected fMRI and fNIRS data simultaneously in a wide range of tasks targeting the exact parietal-frontal networks that will be targeted here.  On a positive note, Cui et al. found that activation measures were positively correlated across fMRI and fNIRS, especially when looking at the most strongly activated regions (voxels and channels, respectively).  However, they found that the contrast-to-noise ratio for fNIRS was less than half that of fMRI when looking at all locations, and still 50% higher for fMRI when examining the most strongly activated locations (see Figure 10 of Cui et al., 2011).  

Third, given the differences between the previous paradigms and the proposed paradigm, would we even expect the effect size to be similar between the two studies?   

Taken together, it seems that the ostensibly conservative “medium” effect size (d = .5) that the authors consider for their power analysis might, in fact, still be wildly inflated, which would lead to a significant underestimate of the sample size needed for a reasonable power.  Concern makes it more likely that the study would be underpowered, leading to a failure to detect differences between groups/conditions.

Predictions/Framework
Overall, I found the presentation of the specific empirical predictions to be confusing and poorly motivated. The key concern is that, under certain circumstances, increased signal is associated with greater skill/more mature performance, while under other circumstances, increased signal is associated with poorer skill/less mature performance.  Although these predictions are sometimes in opposite directions for different regions (frontal vs. parietal/right vs. left hemisphere) the integration of all these moving parts is lacking.  For example, the review of the functional connectivity fMRI literature on p.6 seems contradictory:

Emerson & Cantlon (2011) -> greater skill associated with greater fc in frontoparietal networks

Hyde (2021) -> younger children show greater frontoparietal connectivity, and then shifts to parietal (?)

Perhaps this just explained in a confusing way, and the authors need to present diagrams showing the developmental model that leads to their specific empirical predictions more explicitly.  Concern makes it more difficult to interpret any differences that are observed.

Multivariate Analyses/Channels 

The multivariate analyses are only mentioned in the abstract.  Perhaps this is something the study team changed their mind about, as we are told nothing about such multivariate analyses in the analysis portion of this pre-registration.  If the study team is still planning on carrying out multivariate analyses, they would need to specific a great deal more about the planned algorithms for multivariate analyses (SVM? Ridge regression? Penalized logistic regression? Will these be implemented in a Bayesian framework, and if so how?).  Additionally, I am concerned that the use of a limited number of fNIRS channels available in the portable system they have selected would lead to data with too low dimensionality for many multivariate approaches. Concern makes it so that we cannot properly evaluate the multivariate analysis plan.

The study proposed by Ivanova at al. aims to tackle an important question in the area of cognitive development, namely children’s acquisition of cardinality principle knowledge. 

The manuscript is well written and include enough methodological details to allow the replication of the experiment and analysis. The proposed study is feasible, and its rationale is supported by the needs to fill some gaps in developmental literature as summarized in the introduction. The task choice and the age range of the children is particularly very well explained and justified. The experimental procedure and the analysis that the authors intend to pursue are well described in the manuscript. However, I believe that additional clarifications are needed, particularly concerning some of the methodological details of the fNIRS experiment, that would improve the quality and structure of the manuscript and would make it easier to follow and understand. My comments and suggestions are listed below.

1.     Unless I misinterpreted the text, the authors made hypotheses only on HbO2 changes. Why are no predictions concerning deoxyHb? I think the hypothesis needs clarifying as the best practice would be to include results for both HbO2 and HHb (even if not significant). 

2.     Relatedly, how will the authors deal with situations where there is a positive result for oxy or deoxy but not both for a given ROI?

3.     I think quality checks are missing from the hypothesis and data analysis plan. The authors should include that they expect to find a significant differences in the hemodynamic response between stimuli and silence (which I believe is their baseline condition) over the parietal and frontal arrays. The results would show a significant increase in HbO2 and/or significant decrease in HHb in response to stimuli compared to baseline.

4.     I am not sure the fNIRS experimental paradigm is clearly presented. I think this might be due to terminology. Could the authors clarify that what they are contrasting is the hemodynamic response to the block (33.6 sec) vs jittered inter-block interval (mean of 16 sec)? 

5.     Do the authors expect significant differences in brain hemodynamics to the different comparisons in particular channels or across the whole fNIRS probes (within their 4 ROIs)?

6.     Procedure: are the authors planning to counterbalance the order of presentation of the tasks/test (IQ, verbal counting, etc)? If not, why so?

7.     I am not clear about the exclusion criteria for the study. For example, how will you ensure task compliance during the fNIRS experiment? Will you manually exclude trials in which the children are not attending to the smiley face and/or due to external (e.g. parental) interference? Importantly, how are you planning to monitor children's attentiveness is not clear to me. In page 12 it is mentioned that this will be done by using speakers to produce the sounds, but it's unclear how the use of speakers can ensure stimuli attendance. Additionally, will you further analyse the fNIRS data from participants with missing data points on the behavioural tests/tasks? What about signal quality problems (e.g. do you have an objective criterion for the % of channels being excluded)? I also assume another standard exclusion criteria pertains to experimental error. Unless I missed this information, please include a list of exclusion criteria in the manuscript.

8.     Relatedly, what’s the minimum number of trials required to carry out the GLM analyses?

9.     Are the authors planning to counterbalance gender in their sample?

10.  I am not sure I fully understand the approach of using the highest coefficient of HbO2 for each condition and on each of the 4 ROIs. Could the authors clarify the rationale behind their choice? I would have thought that with GLM you can obtain beta parameters for each of the regressors and for each child, which can then be used to calculate a contrast between the conditions of interest for each infant. 

11.  Probe locations, co-registration: A major weakness in the current proposal has to do with determining the spatial location of the probe. The authors propose to place their probes bilaterally over the parietal and frontal lobes and these locations are well-motivated by the literature. Do the authors intend to engage in a co-registration procedure themselves or somehow use the information from previous work? If the authors do not plan to do co-registration themselves, it's not clear how they will determine that the probe will be placed in particular cortical regions or even consistently across the children. This is both extremely important for data quality and interpretation and not trivial if a very specific protocol is not put in place. I think at minimum, the authors need a very clear procedure for placing the cap such that the channels can be localised to, say, 10-10 locations on the scalp and then a method of determining for each child whether the cap adhered to that protocol.