Diversifying our understanding of children’s word learning

Peer review of Stage 1 Revised Manuscript

Reading and vocabulary knowledge in English-Meetei Mayek biliterates

The revised Stage 1 manuscript is now titled Reading and vocabulary knowledge in English-Meetei Mayek biliterates which more accurately describes the study. The research study aims to answer two research questions: 1) Are the relationships among measures of word reading and vocabulary knowledge similar or different in English and Meetei Mayek? 2) Is there linguistic interdependence (English to Meetei Mayek) in word reading and vocabulary knowledge?

I commend the authors on the clarity of the revised stage 1 report, and for the detailed feedback on the reviewers’ comments. All my comments raised in the initial review have now been addressed and I am satisfied to support this stage 1 report. 

The literature review has been revised for clarity and new sources have been added that provide more context to the research problem. The research questions clearly follow from the presentation of the literature, and the proposed analytic choices can address the questions. The analytic choices are clearly defined, which enables replication of the choices made. The authors have also specified how they would interpret different results I relation to their hypotheses.

The sample size for each analysis is sufficient to provide informative results, based on their SESOI and associated power analyses. I was able to reproduce the power analyses. When replicating in R, I did get some warning messages that some functions in TOSTER had been deprecated and replaced with new function names. Nevertheless, I found the same results as included in the report. 

The pilot results and analysis of piloted instruments is presented in detail in the supplementary material. The results therein demonstrate that the authors have created valid and reliable instruments that can be used in the present study to answer the research questions. 

The authors have sufficiently addressed how they will check data quality, deal with missing data, and sensitivity analyses. The proposed study seems to be undertaken in line with ethical standards in the field, with ethical clearance from the university, and consent obtained from school, class teacher and students. The authors have also considered the political scenario and reduced the time requirements related to participating in the study. 

I am very much looking forward to the results of this research. The research will add to our understanding of reading development for multilingual readers in underexplored contexts. 

First of all, let me say that I am sorry to hear that your research has been impacted by the current political situation. Second, I found the manuscript much improved (and I like the new title), but still have some suggestions for improvement. 

-          At several points in the manuscript, it is stated that you will be recruiting children in Grades 3 and 4 (e.g., abstract, section at the end of the instruction), but in your response to reviewers, you stated that you decided to concentrate on Grade 3. Assuming the latter information is correct, please edit the manuscript to match your intended sampling.

-          Thank you for the changes that you made to the introduction; I found it much clearer.

-          Can you motivate better in the manuscript, why you decided to use three different measures for word reading? There are two timed measures (one for words, one for nonwords) for each language. In addition, there is also the untimed measure. Can you make clearer what the untimed measure is adding?

-          As I understand it, the only exclusion criterion is that participants have more than 30% missing data. Are there really no other exclusion criteria (e.g., uncorrected vision or hearing issues, diagnosed neurological difficulties)? 

-          Can you explain the averaging of the scores more? In one of the vocabulary tests, the maximum score is 17, in the other one it is 30. By simply taking the average of these two numbers, you would essentially give more weight to the test with the higher maximum number, correct? Or are you standardizing the scores first and then averaging them? Please elaborate.

-          For hypothesis 2, will Bayesian analyses only be run in case there is a null effect in the frequentist analyses or will they be run regardless? If the latter the case, one possibility could be to only use Bayesian analyses to start with for interpretation.

-          I am not familiar with some of the R packages that are being suggested for supplementary analyses (e.g., sensemakr) – would it be possible to include more detail as to what they are doing (given that, to my knowledge, these are not standard approaches)?

Minor comments:

-          “For the child population whose main literacy input is through school books, word reading and vocabulary development across ages (Newman et al., 2016) might be different from those living in a rich literacy resources environment, particularly when formal schooling does not begin in the first language (L1) but instead in a second language (L2) or even the third, and oral exposure to the dominant literacy language does not precede that of the written language.” This is a very long sentence, please split it in two for readability.

-          Incomplete sentence: “For English, N = 267 (< 354) and for Meetei Mayek, N = 1078 (> 354)”

-          “As explicitly stated in the DAGs” – what does this abbreviation stand for? It is not used anywhere else in the manuscript.

Review Stage 1:

Reading Two Languages Simultaneously: The Role of Academic Book Language

Thank you for the opportunity to review this manuscript. I believe the article addresses interesting research questions, but is sometimes hard to follow and is missing methodological and statistical detail. I hope the authors find my comments helpful.

Title:

After reading the manuscript, I am not sure the title relates to the questions actually addressed that well. I believe something along the lines of “The relationship between reading and vocabulary knowledge in English-Meetei Mayek biliterates” would be more appropriate. Based on the title, I would expect that participants are literally reading two languages at the same time or that academic book language is more investigated specifically.

Abstract:
The abstract in its current form does not provide enough information on the study. You may want to add information to address the following questions:

-          How many children will be recruited? What is their age (grade is not informative enough because ages per grade differ across countries)?

-          Please introduce the linguistic interdependence hypothesis earlier. What does this hypothesis predict? 

Introduction:

Overall, I had sometimes trouble following the arguments that were made in the introduction (especially in the beginning). A lot of different hypotheses were introduced without really elaborating how they related to each other (when talking about the relationship between reading and vocabulary acquisition). I imagine that some of this may be personal preference, but I do suggest editing the introduction to be clearer. Most of my comments below are about specific sentences that I had trouble understanding, but there are two more substantial comments about Research Questions 1 and 2 at the end (I made these cursive, so that they would be easier to spot).

Detailed comments:

-          P.5: I am not sure I agree with this sentence “Lexical diversity and quality in books are higher than in oral exposure (Hsiao & Nation, 2018; Montag & MacDonald, 2015).” Yes, written language tends to be more diverse and more complex (e.g., more use of passives) – but does that mean it’s “higher quality”? To me, they are just different/serve different purposes but are not necessarily better than one or the other.

-          P.5: I am not sure what this sentence means “With school books as the main input system, word reading and vocabulary development across age (Newman et al., 2016) is relatively less examined.” Do you mean that children mainly read school books (is this true? I guess it may be true for English in this population?)? Word reading and vocabulary development is examined less than what?

-          P.5: I had a hard time following this sentence too: “For children whose literacy and schooling experiences do not necessarily follow the first language (L1) and second language (L2) chronology in the order of oral preceding written language, how are meanings derived out of word reading or vice versa?” The phrase “chronology in the order of oral preceding written language” does not work for me (I do not understand it).

-          P.5: I also have trouble with this section: “The major merit of this hypothesis is that it allows testing of the distinction between reading experience from general language experience (Nation, 2017) for multifaceted word knowledge. Moreover, with orthographic transparency as the driver of division of cognitive labour, the influential triangle model of reading (Harm & Seidenberg, 2004) formalises the effects of both phonological and semantic pathways on reading acquisition (Monaghan, 2023)”. I do not understand the jump from the lexical quality hypothesis to the triangle model. Please unpack these ideas more, so that it is easier to follow.

-          P. 6: “With increase in experiences of schooling” = with more schooling?

-          P. 8: “Vocabulary threshold in L2 students is the minimal word knowledge they need for reading academic texts (Laufer & Ravenhorst-Kalovski, 2010). Broadly, it corresponds to vocabulary breadth (Grabe, 2004).” I do not understand what is meant by vocabulary threshold – does that mean that children need to know a certain number of L2 words to be able to read academic texts?

-          P. 8: “Given this premise, for years of schooling to yield more word acquisition, a prerequisite is constructing new concepts corresponding to new labels. The existing knowledge base of words in L1 (Jimenez & Hills, 2022) might act as the elicitors, if not the causes, of new words meaning in another language.” I had to read this more than once to understand, but I think it just means that word learning in a L2 will be easier if they correspond to existing L1 words (instead of completely novel concepts). Please edit the sentences for readability. 

-          P. 11: “Meetei mayek has 27 alphabets (Mapung Eeyek) serving as the main set” à Meetei mayek has 27 alphabets with Mapung Eeyek serving as the main set?

-          P. 12: “We expect that since the latter has a transparent orthography, children would learn to read it relatively easier than English. Therefore, it is possible that vocabulary knowledge in Meetei mayek would be less dependent on word reading because compared to English, most children would have basic exposure to the Meetei language in their everyday environment beyond the setting of formal schooling.” This section confounds transparency of the language with how language is being experienced. If I understand correctly, the idea is that for denser orthographies, the relationship between word reading and vocabulary should be reduced. But here, this does not apply because orthography denseness is confounded with when children actually see words? I.e. if you find that relationship we don’t know whether it’s because of the denseness or whether it’s simply because these children rarely read the other language (that is you may find these differences even if differences in denseness would not exist).

-          P. 12: Research Question 2 – how can you exclude the possibility of a third factor (beyond the three control variables age, phonological processing and IQ) contributing positively to both English reading/word knowledge and Meetei Mayek reading/word knowledge?

-          The acronyms LL1 and 1LL are both used – please be consistent.

-          The linguistic interdependence hypothesis as a term is first introduced in the research questions at the end of the introduction (besides the abstract). Given this is a central aspect of the manuscript, it should be probably introduced earlier and in more detail. 

Method: 

-          What are the approximate ages in year 3 and 4? This feels like it could be very language and culture dependent. 

-          How will the written consent look like for the children?

-          Can you say a bit more about the Meetei Mayek adaptations of the standardized tests? Were these developed for this project and/or have they been used before?

-          Will vocabulary knowledge (breadth/depth) be tested in both languages? If so, how?

-          Please be more specific about data exclusion criteria. Is the number 70% calculated across all data or for each task separately? Are there any other factors that may lead to a participants’ data being excluded?

Results:

-          P. 17: “Outliers will be defined based on the standard deviation, most likely in the common range of 3 SD above or below the mean.” This should be decided in advance.

-          P. 18: “For all the analyses, graphical representations of the correlations highlighting the confidence and credible intervals.” Incomplete sentence

-          Please be specific about which variables will be entered into the regression models for Research Question 2. For example, word reading is assessed with three subtests; how are these entered? Are they combined into one value?

-          What are the Bayesian analyses adding to Research Question 1? 

-          For Research Question 1, I am not getting a clear sense for what exact correlational patterns would be evidence of meaningful differences between English and Meetei Mayek. Likely, there will be some differences across the languages. How do we know which ones are meaningful or how many differences (specific ones?) need to exist for them to be meaningful?