Program Strengths and Challenges
The Scholar Success program evolved over the course of five years, including responding to the COVID-19 pandemic, with a range of strengths as well as challenges.
Strengths
Over the five years of Scholar Success, the program had a number of strengths before and since the onset of the COVID-19 pandemic
Partnerships: Particularly before COVID-19, the program far exceeded its goals around community partnerships.
- Sunnycrest partnered with nine organizations in Year 1, five of which continued in Years 2 and 3. During the COVID-19 pandemic, the site continued to add new partners and two new partners joined in Year 4 and an additional two in Year 5.
- At Wildwood, the program had seven partners in Year 1, one of which continued into Years 2 and 3. They added nine new partners in Year 2, four of which continued in Year 3, in addition to the five new partners who joined in Year 3. They added one new partner for Years 4 and 5.
Adaptability and responsiveness: With the onset of the COVID-19 pandemic during Year 3 (2019-20) of the grant, the program was highly adaptable and responsive.
- The program’s structure was reinvented, shifting to a virtual format initially and then to a hybrid format once there was the opportunity for in-person programming. This allowed the program to serve more students and families with different preferences.
- As one Site Coordinator stated, “A few of our students' families wanted them to come home right after school over covid concerns, a couple have program alumni that have moved on to middle school, and one family changed districts to be able to access remote learning due to health conditions. By maintaining online and in person experiences we have been able to keep more scholars in the program and in touch with each other.”
- As one Site Coordinator stated, “A few of our students' families wanted them to come home right after school over covid concerns, a couple have program alumni that have moved on to middle school, and one family changed districts to be able to access remote learning due to health conditions. By maintaining online and in person experiences we have been able to keep more scholars in the program and in touch with each other.”
- The program responded to shifting needs of students and families throughout the COVID-19 pandemic.
- At the start of the pandemic, the program sent care packages home and worked with school staff to address families’ basic needs.
- Over time as the program shifted to a virtual format, the program was able to offer more frequent family events and more families were able to attend.
- When transportation was not available through the district due to driver shortages, the program arranged a charter bus to provide transportation home for the afterschool program.
While the overall number of student participants fell during the COVID-19 pandemic, the proportion who were regular attendees increased, particularly in the final year of the grant.
Data and measurement: The program’s adaptability extended to how it used data to understand program implementation and impact.
- Over time, the program adjusted its family surveys to better reflect the focus and intent of family activities. The survey questions were more relevant for families and provided more useful information for the program.
- Program and evaluation staff worked together to incorporate more qualitative data to complement quantitative measures needed to measure progress toward goals and objectives. In particular, coordinators provided narrative descriptions, including highlights and challenges, of each program session which helped to tell a fuller story of the program.
Challenges
Data and measurement: While the program had some strengths in data and measurement (per examples above), there were also challenges in using data to understand progress toward the program’s goals and objectives.
-
Measures for understanding academic achievement changed from year to year, which made it difficult to understand the impact of the program and progress over time.
-
Similarly, the changes to the family event survey – though better aligned to the intent and focus of the events – made it difficult to compare data over time to understand the impact of these events on families and the program’s progress and opportunities.