Issue link: https://takingitglobal.uberflip.com/i/1542824
R. Adeboye, C. Flewelling,V. Ogbole, E. O'Sullivan 79 teachers - communicated through the annual teachers survey - that the program should support long-term goals like retention and graduation. Limitations This evaluation has six main limitations. They are described below, along with any strategies or steps that were taken to mitigate the risks they pose to the strength of the evaluation's conclusions. 1) Positivity bias: There are several potential sources of bias in the data that could overstate program strengths relative to weaknesses. These include self-selection bias (both for those who chose the answer the surveys and those who volunteered to be interviewed); as well as the fact that write-in survey questions were posed in a manner more likely to elicit positive responses (e.g., "How has Connected North been a catalyst to support learning in your classroom? 40 ") Nonetheless, the frank references to challenges in both interviews and surveys; the consistency of narratives across lines of evidence; and the strong enthusiasm expressed by participants suggest that there are no significant issues in the program that the evaluation has overlooked. That said, although challenges with session delivery appear to be relatively rare based on the available data, it is possible that positivity bias in that data may have contributed to an underestimation of the incidence of such challenges. 2) Subjectivity in qualitative coding and interpretation: Particularly given the large volume of data and coding categories, intra- and intercoder reliability is imperfect. For the content analysis, a procedure was established to validate responses and maximize reliability. In addition, counts of responses were ultimately collapsed into broad categories that would lessen the impact of coding inconsistencies. More generally, mitigation of this risk is largely achieved through transparency about the decision-making processes, illustrated in the evidence matrix and elaborated in the evaluation's detailed technical methodology report. 3) Limited information on student perspectives: Data for this evaluation came principally from the adult stakeholders in the Connected North program, owing to logistical constraints that prevented the evaluators from connecting directly with students. This omission is noted, particularly given that the limited student data that was available was not as overwhelmingly positive as the data gleaned from the adults. 4) Limited information on teacher professional development and other supports: Although the evaluation did include lines of inquiry related to teachers and their role in program delivery, it did not delve as deeply as it could have into the professional development and 40 A more neutral question, such as "In what ways, if any, has Connected North affected learning in your classroom" might elicit more neutral or mixed positive and negative answers; or questions could be asked such as "Can you describe any weaknesses you have noticed with the Connected North program?" to seek balance and reduce the risk of positivity bias.