Recommendation
based on reviews by Christian Nawroth and 1 anonymous reviewer
DOI or URL of the report: https://osf.io/uzgq4
Version of the report: 2
, posted 03 Feb 2025, validated 03 Feb 2025Thank you for your detailed replies and changes to the manuscript in response to my and the reviewers' comments. The clarifications and additional details help to understand what can be learned from your study and its relevance for other research. I did not send your manuscript back to the reviewers because it was clear how you addressed their minor comments. I only have a few remaining small suggestions:
- the changes in the methods and results now help to make it clear which analyses where originally planned and how these were modified. However, it is still slightly confusing that in the method section you separate the "registered model" from the "applied model", when the results talk about "registered comparisons". Maybe change the headers in the result section to "registered comparisons with the applied model" to make it clearer how it links to the different method section? In addition, I think it would be helpful to expand the first paragraph of the method section on the applied model with a more explicit statement that the "violations of certain assumptions, and issues with model convergence" mean that the outcomes of the originally registered model are meaningless or misleading. This is a requirement according to the PCI RR policy: section 2.9 of the Guide for Authors states that all registered analyses must be reported unless it is now clear that the originally planned analysis plan is now inappropriate. To strengthen this point, you might want to mention that the statistical inferences on which you based your decision to change the original models are presented in the supplementary material (and potentially add any statistical inferences which you based your decision on if they are currently not explicitly stated in the supplementary materials).
- you might want to add an additional column to your study design table to explain which inferences you are drawing about the three questions now that your study is concluded. Some of the recently accepted stage 2 manuscripts at PCI RR have done so in case you want examples. I think it can help the reader to quickly get a summary of the study.
- there was one comment from a reviewer about the species' ecological niche, where I think you could elaborate a bit more in the discussion to help readers who are not as familiar with these species or who are considering how your findings might translate to their species. My interpretation of the reviewer's comment asking for an example would be to more specifically refer to what exactly in the environment of these specific species might lead to such cognitive adaptations. I realise that you are being careful and do not want to speculate about inferences that are beyond the scope of this study. However, given that you specifically mention "certain ecological niches" and "context-specific factors", it might help the reader to see what you could mean by those and which kind of environmental conditions others might want to pay attention to when considering differences in detour performance.
DOI or URL of the report: https://osf.io/uzgq4
Version of the report: 1
, posted 07 Dec 2024, validated 10 Dec 2024Your article entitled “Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird Species.” has now been seen by the two reviewers who assessed the stage 1 report and the reviewers’ comments are appended below. I share the reviewers' view that this a well-written manuscript presenting solid new insights into the question of the role of signal detection on inhibitory response performance in a detour task. I also commend the authors on sticking to their registered methods of testing the birds, despite the unavoidable difficulties they encountered while working with animals.
However, I was confused about the changes to the statistical approach. On first reading, my impression was that you had made changes that were not preregistered. After checking the supplementary material, I now think these changes were mostly covered by the initial plans to change the statistical approach in case of "potential violations of model assumptions" (text included in the Stage 1 report). This does need to be clarified though in the manuscript. Currently, reading the methods of the Stage 2 manuscript can give the impression that you decided to change your analyses after you had seen the data to best fit your purposes. This would represent a major deviation. There are three potential issues here of having such a change appear in the Stage 2 manuscript. First, if these are additional analyses that were not peer reviewed, they would need to appear as exploratory analyses in the discussion, rather than as the main results. Second, it could affect the level of bias as originally this report had a level 6, but that would clearly be lower if decisions were made based on observing the data. Third, it is unclear whether you tried additional analyses that you did not report.
To avoid these issues, I therefore ask you to revise the method and result sections of your manuscript to make it clear how these deviations were developed, and how they fit within the original plan that was peer reviewed. Some of this information is in the supplementary material, but it is not always referenced and I am not sure whether all the information is included for how a particular outcome lead to a particular decision. I think it would be necessary to report in the manuscript the assessment of the primary analyses you performed to show that they are not reliable. In the methods, you should also preface each of the changes you did with whether it occurred according to the potential violations you mentioned in the Stage 1 report.
There are two changes you did to the statistical approach where I have more specific questions. The first is your following decision: "A minimum trial duration of zero seconds for persisting was assigned to the 483 trials (33.82% of the data set) in which birds did not enter the species-specific ’barrier zone of interest’." I assume these are birds who nevertheless detoured and interacted with the food bowl? Or does this include individuals who received a zero value because they never left the area around the start box?
The second is about including a temporal correlation structure in the model of detour latency. It is not clear how this affects your interpretation, because as far as I understand it your measure of time is the trial? Accordingly, you have trial in the model repeatedly? Does this temporal correlation structure reflect individual level differences? I was also confused because the supplementary materials appear to indicate that this correlation is negative - so when an individual performed above average in one trial, it performed below average in the next trial but at average several trials later? Why would there be a negative correlation, if anything I would expect a positive one?
I have one more question about the interpretation: your decision to exclude birds according to the different criteria means that your sample during Trial 1 is not the same as your sample during the final Trials. Do you think this could lead to selection biases that affect your results? In particular, could the increase in performance be slightly explained by birds who take longer initially being more likely to drop out? For example, if there are four birds of whom three take 5 seconds and one takes 10 second during the first trial, but the slowest bird later no longer participates, the average from Trial 1 to Trial 2 would drop from 6.25 seconds to 5 seconds. As far as I understand, you include an individual level offset for detour latency, but not individual level slopes of the change across the trials?
As I mention above, I think all of this can be clarified with more explanations in the manuscripts. I do think that this is a very strong registered report, but the adherence to the registered report format needs to be clear. I think you will also find the minor, detailled comments from the reviewers helpful to provide further information to the reader. As a note, wherever one of mine or one of the reviewers' comments could suggest an additional analysis, you should see whether you want to include this as an exploratory investigation in the discussion or leave it as a point for future analyses.
, 05 Dec 2024Kudos to the authors for an excellent piece of writing—I thoroughly enjoyed reading it!
The collected data allow the authors to test the proposed hypotheses, which remain unchanged from the Stage 1 submission. All deviations, though mostly minor, are clearly reported and well-justified. Deviations in the test protocol were implemented before data collection, and deviations in the statistical analysis were explained in a thoughtful and rational manner. The inclusion of footnotes to outline the rationale is particularly commendable, as these greatly enhance the reader's understanding. Similarly, the additional analyses are both justified and sound.
I only have a few minor queries and suggestions for clarification:
L618-619: Please explain why this demonstrates the value of analyzing two parameters.
L452-453: There is a missing closing bracket in “(corresponding with species-specific intercepts.”
L497: For consistency, “Barrier x Species x Trial” should read “Species x Barrier x Trial.”
L611: The phrase “inhibit their initial behavior” is somewhat vague. Could you clarify what is meant by “initial behavior” in this context?
L622: There should be no comma after “latency.”
L622: Could you specify what kind of subcomponents are being referred to?
L624: It would be helpful to include a brief example of how this might relate to the species’ ecological niche.
L654: A line break is needed here.
L662: Could you elaborate on how the exclusion criteria were specifically linked to the performance of the canaries?
L667: I fully agree with this statement. Perhaps you could additionally emphasize how this study particularly highlights its relevance for animal behaviour research.
Table 4: The text notes, “The first 60 individuals (58 for quails) that did not fail any exclusion criteria were selected for this study, ensuring a balanced design and minimizing group variation.” Could you provide a rationale for the substantially higher number of canaries included in the study (maybe as a footnote or similar)?
Table 10 and beyond: Although the number of frames is mentioned in the text, it may be helpful to explicitly include the unit of measurement in this and other tables.
Table 17: To improve comprehension, consider adding a note to clarify how, for example, only 3 quails came from a group of 6 individuals (e.g., accounting for dropouts).
Overall, this is a strong submission with only minor points to address. Well done!
This work presents an experiment evaluating the influence of stop-signal detection in the performance of 4 different bird species in a detour-barrier task, assessing whether results will be predicted according to the ecological niche of each species. This study is also a partial replication of 2 previous studies on the same issue (Regolin et al. 1994, Zucca et al. 2005), but presenting an improvement of some critical aspects raised in those studies. I reviewed previously the registered report of this study and the authors followed the registered experimental design, and tested the hypothesis registered à priori. The authors then found no evidence of an influence of the stop-signal detection nor the species ecological niche in their detour performance, contrary to previous studies. They found however evidences of learning for some species. This work highlights the importance of replication studies, with methodological and conceptual improvements that can validate results of more traditional works.
Overall, I enjoyed reading the manuscript and I think it highlights interesting views on the study of inhibitory control. the manuscript is well written with clear hypothesis stated and clear conclusions from results obtained. Most of the text was already reviewed according to comments from the preregistered stage 1 and thus I have very minor comments regarding this stage version. I hope the authors find them useful to include in the manuscript.
Lines 61-62: is this evidence in humans, or animals in general?
Line 261: how long is the initial indoor period?
Lines 327-328: food was given in a “coloured food bowl”. What colours were in this bowl? Could the different colour perceptions or sensitivities of the different species influence the level to which they reacted to the bowl?
Lines 347-348: I had this comment before and I do not think this is clarified yet. I understand clearly now that each individual performed only 3 trials overall (please correct me if I am missing something). Can you provide the reasoning of why only 3 trials were performed and whether this is representative to compare detour performance between species, when previous within species and comparative studies with detour tasks in birds considered a higher number of trials?
Lines 249-253: would it make sense to include lifespan as a covariate in models to control for this possible developmental effect?
