Issue link: https://takingitglobal.uberflip.com/i/1542824
R. Adeboye, C. Flewelling,V. Ogbole, E. O'Sullivan 16 Next, the sentiment level of the responses was assessed, and the two classifications in concert determined how strong the evidence from that question was either for against the associated element of the Program Theory. The specific approach varied depending on the structure of the question, but Figure B below illustrates the approach for five point scales ranging from strongly positive to strongly negative, noting that the sentiment level is simply the average rating from 1-5, rounded to the nearest whole number. Figure B: Assessing Strength of Evidence for Survey Responses on a Five-Point Rating Scale Continuing with the same example, 95% of respondents at least somewhat agreed with this statement. The average sentiment rounds to 5 or strongly positive. Combined with moderate alignment with the associated outcome, this provides moderate evidence that Connected North is making a positive contribution to student motivation. While these labels are not always referenced directly in the findings, they underpin the evaluators' assessment of the overall strength and coherence of the evidence. Note that this approach was taken for univariate analyses of responses; the methodology for the multivariate analysis used to look for contextual effects is described in a later section along with the findings, for ease of reader reference. Content Analysis of Write-In Survey Questions Content analysis was applied to write-in responses using Microsoft Excel. Again, as a first step, write-in questions were assessed as potentially relevant to at least some of the PIAs and/or Outcomes, with the following four selected for further analysis: • How has Connected North been a catalyst to support learning in your classroom? (1154 teacher responses) • How has your experience with Connected North supported your personal and/or professional development? (827 teacher responses) • Is there a story you'd like to share with us about the impact a Connected North session had on a student? (403 teacher responses) with survey best practices; for example, the multifaceted nature of this question means it is "double-barrelled," which can lead to ambiguity in responses (Krosnick and Presser, 2010).