Examining the causal pathways from impulsivity to disordered gambling in online sports betting

Dear Dr. Kräplin,

  Thank you again for the opportunity to read your revised manuscript "Impulsivity and online sports betting behavior: Untangling the causal relationship". All my previous comments have been addressed very satisfactorily.

  During my final read of the whole manuscript, however, I noticed that some content in Table 1 has not been updated to match the revised manuscript. For instance, for the data analysis plan for Hypothesis 2, the method for determining the number of factors for risky gambling behavior seems to be based on an old text, and there is no mentioning of setting a maximum number of factors in the analysis. It would be good to resolve this inconsistency (and perhaps have a look at other cells in Table 1 as well), since once this Stage 1 manuscript is accepted, there will be strict limits on changing the accepted content.

  I would be happy to recommend this manuscript once this minor revision has been made. 

Kind regards,

Zhang Chen

Dear Dr. Kräplin,

Thank you for submitting your revised Stage 1 Registered Report, entitled "Impulsivity and online sports betting behavior: Untangling the causal relationship" to PCI RR. All previous comments have been addressed very satisfactorily. Both the procedure and data analysis plans are much clearer now. However, there are some final issues that still need to be further clarified:

The methods for determining the number of factors in risky betting behavior are now made concrete (starting from line 411 in the revised manuscript). However, in the Hypothesis testing section, starting from line 554: "If the eigenvalue of the first factor is significantly larger than those of the remaining factors, we will extract a single factor, resulting in a single risky gambling behavior score per participant. If inspection of the eigenvalues indicates several factors, we will run five separate models (5 predictors) for each of the factors." This method of determining the number of factors seems inconsistent from the section above (line 411). Please clarify this point.

I am afraid the statistical inference process for Hypothesis 2 is not entirely clear to me. If the factor analysis reveals multiple factors for risky betting behavior, say 3, I assume 15 models will be run in total for Hypothesis 2. Will a sub-hypothesis be considered to be supported if it correlates with at least one of the 3 factors on risky betting behavior? If that is the case, the test for a sub-hypothesis seems to become less stringent as the number of factors for risky betting behavior increases. I am wondering if you have considered alternatives, for instance, by requiring a predictor to be associated with at least N-1 factors (in case N > 1) of risky betting behavior, in order to say a certain sub-hypothesis is supported. I guess this will also be consistent with how you deal with the three facets of impulsive personality at the moment, by requiring at least two of them to correlate with outcomes of interest.

For Hypothesis 3, I wonder whether all models will be run, or only a subset of them will be examined depending on the results of Hypothesis 1 and 2. For instance, will the potential mediation effect of a certain factor of risky betting behavior still be examined, if the analysis in Hypothesis 2 revealed that it was not predicted by impulsivity?

Minor point: line 53: "an increase of 15,6%". You may want to use "15.6" to be consistent with the rest of the manuscript. 

Kind regards,

Zhang Chen

Dear Dr. Kräplin,

Thank you for submitting your Stage 1 Registered Report, entitled "Impulsivity and online sports betting behavior: Untangling the causal relationship" to PCI RR. I have now received comments from two expert reviewers, and have also read the manuscript myself. Combining player tracking data and survey data provides a unique opportunity to understand the etiology of gambling disorder, which is also a key strength of the proposed research. However, my major concern with the current manuscript is that certain methodological details do not seem to be fully specified, which can leave room for potential undisclosed flexibility in the procedures (see the review criteria 1C and 1D on the website of PCI RR). The reviewers and I have noted several aspects where more clarity is needed to reduce such potential flexibility. I hope these comments will be useful to you, and I look forward to seeing a revised version of the manuscript.

Please provide more information for the power analysis, such as: the exact statistical model used in the simulation, the assumed effect sizes (at the moment, the text only says "small associations" and "a small to medium association", without giving any specific values), the criterion for statistical inference in the simulation (an alpha level of .05?), and the observed power for each of the three hypotheses (I assume they will differ for the three hypotheses?). Such information will help readers better appreciate what effect sizes the target sample size will have a reasonable chance of detecting.

Both reviewers have concerns about whether the target sample size of 370 can be reached given the sample size at the first wave. This is of course an inherent difficulty with such a longitudinal design - one can never know in advance what dropout rate will eventually be observed. That the initial sample size at wave 1 slightly exceeded the planned sample size, and the dropout rate observed in a past study with a similar design to some extent support the feasibility of achieving the target sample size at wave 3. It would nevertheless be good to know if there are any contingency plans in place to address this potential issue.

Related to the previous point, since data collection for wave 1 is already finished, achieving a low dropout rate for waves 2 & 3 seems to be key here for reaching the target sample size. More methodological detail on the recruitment procedure would be helpful in this regard. For instance, how much time did/would participants have to participate in the task after getting an invitation? Did/would participants get reminders if they did not respond to the initial invitation, to potentially reduce the dropout rate for waves 2 and 3? In general, more details could be provided for the recruitment procedures for all three waves, to give a better picture of how the recruitment took place/will take place.

Please specify how the balanced integration score will be calculated. I think it will be more convenient to the readers, and also reduces potential flexibility in data analysis, if they do not need to look up another source to understand this method.

Please also specify how the scores for impulsive personality traits will be computed. For instance, will the sum score, average score, factor score (or an alternative score) be used for each of the three factors?

More clarity is needed for how the variables operationalizing riskier gambling behavior will be constructed. For instance, for "a higher variance (mean standard deviation) in the number of bets and stakes over 3 months", will the number of bets/stakes first computed per day, or per week, or per month, before the standard deviation is computed? For "betting at higher odds (mean weighted odds) over 3 months", how will the odds be weighted exactly?  For "no or low gambling limits set by the player over 3 months", what constitutes "low limits" in this case?

To promote transparency, please consider sharing the survey used in this project. Please also describe the procedure in the survey for all three waves (e.g., in which order do participants receive the different tasks and questionnaires etc.). Will the same plausible time limit (i.e., 10 minutes) also be used for waves 2 and 3?

Line 354: "we expect some individuals with too few data points to compute a reliable aggregated variance". What criterion will be used to say that a certain individual has "too few" data points?

As one reviewer pointed out, please specify how the number of factors will be determined for the variable risky gambling behavior. At the moment, the description (lines 372 - 377) seems to largely rely on subjective judgment, which leaves much room for flexibility in data analysis.

For H1, five separate models will be run. For H2 and H3, potentially more models will be run, depending on the number of factors obtained for the variable risky gambling behavior (which needs to be further specified, see above). This raises the question of how the results of these separate models will be used to infer whether a certain hypothesis is supported or not. For instance, will H1 only be considered to be supported when all 5 models reveal an effect of impulsivity on gambling disorder severity, or will the different facets of impulsivity be considered separately? If it's the latter case, the hypotheses will also need to be adjusted accordingly to reflect that the statistical inference will be made for each facet (instead of impulsivity as one construct). The same applies to H2 and H3.

Multiple indices will be used to assess model fit. Please specify how these indices will be used in the statistical inference. For instance, will the results from a model only be interpreted if all (or if any) of the indices show that there is adequate fit? Differences in model fit will also be computed, but it is not entirely clear to me which models will be compared, and what conclusions will be drawn from such model comparisons.

Line 280: "lack of consciousness" should be "lack of conscientiousness"?

Kind regards,

Zhang Chen