Dear David

I now have two expert reviews in, and I am very pleased with their thoughtful and constructive engagement, in the collaborative spirit that RRs help. Both reviewers are excited by an ambitious attempt to deal with a complex and important question. But both feel serious reworking is needed to enable the study to achieve its goals. The issues are partly methodological, whether the precise procedure is the best one to use; partly conceptual, thinking about what predictions are well motivated. Please give a detailed point by point response to each of their issues in your reply.

I very much look forward to seeing your revision,

best

Zoltan

Dear Dr Neequaye

Thank you for your revision. PCI RR has no power requirements; it is fine to calculate power for that minimally interesting effect which you have appropraitely identified, and indicate the power is too low for a non-signfiicant result to be taken as grounds for assserting Ho. That is just a matter of being explicit about what follows from a given inferential procedure. So do state the power which you do have for each effect of interest (if you are testing hypotheses).

Bear in mind that different journals, including the PCI RR friendly ones, that you may (or may not) wish to publish in afterwards sometimes do have power requirements, if power is used (though Royal Society Open Science does not, so that would be a possiblity).  But an alternative, as you say, is not to phrase your study in terms of hypothesis testing at all, but estimation. In that case your goals would not be stated as establishing whether or not there is an effect, and all such language would have to be carefully avoided; the goal is to estimate relevant effects with say a confidence or credibility interval (or similar).  The hypotheses could be phrased as implying a certain interaction etc is relevant (not present or absent, that language would be avoided). In fact, your phrasing is largely of the estimation type "to what extent", except for row 1 coloumn 1 of the Design table. You could change this, then the important results are, as I say, the confidence (etc) intervals for each effect of interest. In that case, power is not relevant, but you should indicate the expected precision, or width of the CI, that your N should provide.

Whichever way you go on that, thanks for adding the table; but each hypothesis is related in the table to a set of effects. This allows inferential flexibility. Can you identify a specific question with a specific effect?

In sum the changes needed for me to send out to review are minor, it is just a matter of being a bit clearer about your inferential goals and capabilities.

best

Zoltan

Dear Dr Neequaye

Thank you for your submission "Exploring How Members of Illicit Networks Navigate Investigative Interviews."  Before I send for review, there are a few issues to address:

1)  Include a  Study Design Template in the main manuscript. See here for details: https://rr.peercommunityin.org/help/guide_for_authors#h_27513965735331613309625021

This involves creating a table aligning specific theoretical questions with hypotheses with the specific statistical test that will test that hypothesis, and the conditions under which support will be inferred for the hypthesis, or rather the results will actively count against the hypothesis.

2) Make sure you remove all analytic flexibility from the analysis plan. Thus, be clear how convergence will be assessed, precisely when it will be decided to not have not occurred; precisely when parameters, like random slopes, will be dropped, etc.

3) Use an inferential procedure that can provide justification for the claim there was not an effect, i.e. the data counted against a hypothesis predicting an effect. For an overview, see https://psyarxiv.com/yc7s5/.  

4) In the light of 3), indicate how your N was determined, and what properties this gives your statistical procedure in terms of the likelihood of results counting for or against your hypothesis (e.g. power if you are using frequentist tests; see previously cited article for overview).

Let me know if there are any questions about how to proceed on any of these matters.

best

Zoltan