Complexity of Shakespeare’s Social Networks

This is a far more coherent proposed research plan with a clearer rationale as to why do the study.

The authors responses to the feedback and new plan, have from my perspective, addressed the key issues with the initial draft, and addressed methodological concerns. As before the analytical approach appears robust. I have made a few comments below, not all in relation to this stage of the process but as things to consider when interpreting findings and justifying the approach in their final write up.

• There has been a clarification of the “slices” and what these are, that makes the use of the analytical approach more comprehensible and justifiable.

• The response to my questioning of the justification of the scene as opposed to entrances and exits was very clear, and I think the response mentioning that using scenes is more feasible/ reproducible is actually a strong argument and could be used to justify this study further. As using entrances and exits will not necessarily produces the same results as scenes. Within that argument the strength of the scene is that it is not subject to individual interpretation of entrance and exits. However, I think their phrasing of “Please note that slicing a play based on exits and/or entrances automatically slices a play by scenes too” will ideally need unpacking in their final research as there would be potentially network implications.

• In relation to the use of speaking characters, there will at some point need to be a justification and consideration in the interpretation of the results in terms of what constitutes a speaking character. Some characters such as Livinia in Titus Andronicus will spend part of the play interacting with other characters (communicating) but not speaking or in some plays there could be scenes with significant onstage action and minimal dialogue.

• They could perhaps make more of the applicability to literary analysis and the application of the validation approach, however, this can be done in the final stages/ write up.  

Dear Authors,

I was very pleased to read the revised Stage 1 protocol and your responses to the initial feedback. I'd like to acknowledge the substantial improvements made to the paper. The introduction, including the study’s rationale, are now well-described. The methodology for the current four studies is sound and highly rigorous. Before suggesting in-principle acceptance, I’d like to request a few more edits and clarifications.

In several instances, the authors assume that the number of characters in theatre plays is mainly determined by recipients’ cognitive capacity. However, this upper limit for theatre plays might be influenced by other factors (e.g., historical and technical/pragmatic considerations). It might be worthwhile to briefly mention these alternative explanations.

A more accessible description of some of the concepts and methodology would benefit a broader readership. For instance, examples of how to apply Kolmogorov complexity (as a focal construct of the present study) in the context of theatre plays would be helpful. Specific examples applied to the context of theatre plays, along with brief insights into their potential implications, would also be very beneficial for Study 3. (Btw, I really liked your second operationalization of complexity described in Study 3.)

In Table 2, the authors state “If many plays lie above 3*IQR, they represent outliers of high complexity. If no or very few plays lie outside 3*IQR then Shakespeare’s plays are not particular with regard to complexity.” Please specify the exact number of plays that you consider as “very few.”

After reading your responses, I would appreciate a summary of the perceived/expected limitations and challenges mentioned in the paper. I’m not insisting on this, as it's a bit unusual request at Stage 1, but reflecting on this at this stage could greatly improve transparency and increase confidence in the methodological choices and results.

All the best,

Matúš Adamkovič

Apologies if this is a repeat of the review I submitted on the 21st Aug.

In brief this was an excellent and thorough proposed plan of research. The proposed research is interesting, and it would be good to see how the findings of Stiller et al (2003) fair across all plays. I particularly welcome revisiting the network analysis measures and distribution models. It would be interesting to hear how the researchers envisage this research being extended beyond Shakespeare. The following comments are in no way a negative criticism of the research proposal but instead highlighting areas where there is potential for more clarity and perhaps opportunity for a stronger connection with the source material. 

1) Overall, the proposal sounds like a thorough critical re-test of Stiller et al (2003), however, beyond critiquing Stiller et al (2003) it is not clear what the extension and application of this research is. Is it about providing a 'tidier' research approach, finding out more about Shakespeare (as a genre) or furthering our knowledge about studying cultural phenomena? 

2)      Slices: This could do with more justification as a rationale; There is need for a more easily replicable approach, but this does have significant issues. In table 1 it is unclear what options a and b are in the associated text and therefore slightly confusing on what they are trying to communicate. I may have misinterpreted but they have equated scenes as the same exits and entrances (which are not).
A more automated approach would remove selection error; however, the researchers from my interpretation have perhaps provided an over-simplification of the 'exits and entrances' as slices, suggesting a scene is the same. Across a scene you can get an idea of clique size, i.e. who a character can 'potentially' interact with and this would be easier in an automated analysis, however, you lose the dynamic information of the scene. A scene does not reflect the dynamics of the actual scene.  

Historically, the nature of Shakespeare's plays were that they would have limited cast playing multiple characters, and within one scene you could quite frequently get small part characters that only interact with one or two characters and leave before others join the scene and never meet more than one character (e.g. messengers) – this is a common plot device to make sure the audience can track the intentions of different characters and understand that some characters, even in the same scene, might not have the same knowledge. This was a big issue in the Stiller et al analysis for A Midsummers Night Dream, where characters in theory are on stage a lot of the time but actually asleep and not interacting. The judgement was made to treat this as an “exit” as from a cognitive perspective for tracking drama  they were not actively part of the scene. Therefore, there needs to be a clearer rationale on their term of “slice” and why a scene would be sufficient to capture such detail.  

The proposed method will not pick up on these finer points and they are important as many characters are not that well connected and there tends to be only a few key characters per play (see Stiller and Hudson paper). The use of the scene as the slice misses the detail of the on-stage groups and for some plays that can result in exceptionally high connectivity that does not exist. I would therefore disagree that the scene is equivalent to the entrances and exits as this is not looking at the cognitive complexity of the plays and raises the question of what the goal of the research is. However, I am intrigued to see what they find out as the entrances and exits can vary from different editions of the plays and is undeniably an issue of interpretation (Stiller et al, did this based on having more than one folio/ edition of each play analysed).

 
3)      Analysis: The analysis sounds interesting, and it would be good to see the network analysis in a more robust automated way, the original paper was largely calculated by hand as access to that network analysis software or free statistical/ modelling software was not available. The addition of Jaccard index and additional small world/ network analyses. IThe Latent Class Analysis sounds like a very good way for evaluating the distributions, however, if they are looking at doing a robust look at the similarities to naturally occurring networks/ groups it would be useful to have a clear overview of the data sources. 

4)      I woudl recommend avoid claiming Stiller et al (2003) stated facts (“As a criterion, Stiller et al. (2003) used the fact that “all the naturally occurring observations fall with the range of the ten plays, and within two standard deviations of their mean” (p. 400).”) this misrepresents the original researchers intentions. The paper was not fact but interpretation. 

5)      Clarify how the proposed research and replication will be of use to understanding the plays:
As the researchers touch on Dunbar’s number etc, if this is part of what is being evaluated then there needs to be a bit more contextualisation. One of the points the researchers make about the cognitive load of the plays could be contextualised more. Stiller and Dunbar were interested initially in tracking intentional stances (who knows what about someone), this can be complex and research such as Stiller and Dunbar (2006) have shown that people can struggle to do this in complex scenarios where multiple intentional stances need to be tracked. The structure of the play is not the only reason for the success of the plays, but it could play a part in making the unfamiliar appear more familiar. The small world network of the Shakespeare plays could provide a way to navigate this as the key characters, those that are most connected and act as "weak links" between scenes can be followed more confidently than less connected characters (obviously some poorly connected characters are essential to plot e.g. the messenger in Romeo and Juliet). Subsequent research by Stiller on the plays of Agatha Christie (where often characters have 100% connectivity) shows that by going away from a small world structure can allow for complex story telling e.g. in detective work and tracking complex intentions as this makes following viewpoints more cognitively demanding. 
 

The authors present a research aiming to link the overlap between human's capacity to follow on social relations and its presentation in cultural artefacts. The authors explain their intentions in a clear, concise, and transparent manner. The report provides sufficient jutifications for the objectives and for the methods planned to be used.

There are, however, a few comments/questions I would like to share:

- the authors largely introduce their research on the paradigm of Dunbar's number. The authors provide a short introduction to the paradigm and support it with seminal works. However, at this stage, the authors neglect a large research array of research standing in contrast to the paradigm of Dunbar's number. In future, it would be useful if the authors could briefly problematize the paradigm in the introduction/theory, and not just present it as granted.

- on p. 6, the auhors start describing the Stiller et al.'s (2003) reports on the cliquishness and the small world properties which Stiller et al. (2003) relate to "naturally observable human social network properties". I find it hard to follow on why it makes sense to analyze and interpret the connectivity within the play networks in which the ties between the characters represent not their social relations, but the occurence on the stage at the same time, in a similar manner than regular social connections between people would be analyzed and interpreted. Like the authors themselves write and rely on Latora & Marchiori (2001), in real-world social networks, connectedness usually denotes the ability to spread information. However, the presence of characters at the stage at the same time creates a very different type of ties. It would be useful if the authors could clearly define what the ties within the plays represent and how the structural network properties created by these ties will be interpreted.

- on line 244 of the code and further, the authors plan to drop the "All" speaking characters. If I understand this correctly, this is because at these moments, all characters are speaking at the same time? If yes, I probably understand why the authors would want to drop all those occurences, but I think it would be useful to explicitly describe that, because if I understand that correctly, it can influence the results of the analysis.

Overal, the report is comprehensive and the R code makes sense. My main suggestions for improvements revolve around the future interpretation of data since the authors will be dealing with a very different of type of ties and the frequently-used structural indices may have different meaning compared to, for example, interpersonal relationships.