Appendix: Methods and Limitations

Methods

Data collected for the purposes of reporting and to inform continuous improvement came from several sources:

  • Washington 21st CCLC Data Portal: The Washington 21st CCLC Data Portal is a centralized data collection system for all 21st CCLC programs across the state. Site Coordinators enter data into the system throughout summer and school year sessions, and the 21st CCLC Evaluation Team accesses data from the portal for mid-year reporting (e.g., student program attendance) and for this year-end evaluation report. Data collected via the 21st CCLC data portal includes:
    • Program operations, activities, staffing, and community partnerships (collected once/session, summer and school year)
    • Youth enrollment and program attendance (collected monthly)
    • Family/adult family members attendance at family events (collected monthly)
    • Youth survey, for students in grades 4-12 (collected once/year)
    • Teacher survey, for school-day teachers of students in grades 1-5 (collected once/year)
    • SEL PQA data and program improvement plans: external and self-assessment scores and program improvements goals (collected once/year)
       
  • Additional data sources include:
    • Family event sign-in sheets and feedback surveys: Coordinators facilitated family events over the course of the program year and gathered sign-in sheets and feedback surveys when possible. All relevant files (sign-in sheets, completed feedback survey, event flyers) were provided to the evaluation team to supplement the adult participant data from the data portal.
    • Student demographic and academic data: In partnership with the Kent School District, a data sharing agreement was developed and used to obtain academic and demographic data for all Kent Community Success participants who attended one or more days of program, and for whom a parent/guardian consented to data sharing.
    • Additional program information: In partnership with Kent Community Success site coordinators and the program director, the evaluation team facilitated a year-end virtual conversation to obtain contextual information and site coordinators’ and the program director’s reflections on the year. This conversation covered several topics, including strengths and challenges from the year, family events, family input into programming, and hopes for the future.

 

Limitations

In the process of collecting and analyzing evaluation data for the Kent Community Success program, the following limitations were identified:

  • Incomplete data: The evaluation team analyzed all available data, which wasn’t always collected for all three sites. For example, family event surveys were provided for Mill Creek and River Ridge, but not for Neely O’Brien. 
  • Inconsistency in data sources across years: Some data varies by year, creating challenges in consistent “apples-to-apples” comparisons across years. For example, 2024-25 teacher survey data included River Ridge and Neely-O’Brien, whereas 2023-24 data included Neely-O’Brien only (Note: In 2023-24, youth surveys were optional; in 2024-25, this data collection activity was required for the statewide evaluation).
  • Data aligned to goals and objectives: The alignment between available data and the relevant goals and objectives has improved over the first three years of the Kent Community Success program. One area of continued need is more granular data for adult family member attendance, so that analysis is possible that is specific to family members of regular attendees as well as demographic analyses of adult family members attending family events. Additionally, there is an opportunity to refine data collection and analysis of family event survey data and feedback from site coordinators to inform a more detailed understanding of the program’s progress toward the objective that all family events reflect interests of families in the program as measured on surveys and CBO input.