Success For Each Child & Eliminate the Opportunity Gap by Leading with Racial Equity

Appendix: Methods and Limitations

Methods 

Data collected for the purposes of reporting and to inform continuous improvement came from several sources: 

  • Program attendance: Coordinators tracked and uploaded attendance to an Excel spreadsheet. These files were submitted to the evaluation team once a month. For students with data sharing consent from their parent/guardian, attendance data were also submitted to the 21st CCLC data portal maintained by the American Institutes for Research (AIR).  
  • Family attendance and feedback: Coordinators facilitated family events over the course of the program year. All relevant files (attendance, survey data, flyers) were provided to the evaluation team to supplement the family engagement analysis. For families that provided data sharing consent, family event attendance data were also submitted to the 21st CCLC data portal maintained by AIR. 
  • Student achievement and demographics: In partnership with the Kent School District, a data sharing agreement was developed and used to provide achievement and demographic data for all Kent Community Success participants who have attended one or more days of program, and for whom a parent/guardian consented to data sharing. 
  • Annual Performance Report (APR) data: In partnership with each of the Site Coordinators, a conversation was held over Zoom to obtain more contextual information regarding program. This was facilitated by the external evaluator and covered topics such as: partnerships, program operations, activities, family events, and successes/challenges. 

Limitations 

In the process of preparing this report, the following limitations were identified. These limitations are particularly related to data collection, analysis, and measures of programmatic progress. 

  • Data measures are misaligned with program objectives. Several program objectives (based on the grant proposal narrative) seek to examine content areas where necessary data elements were not available to the evaluator. As a result, several proxy measures were created. As the program continues in the coming years, the Kent Community Success team may consider revising measures or program objectives to reflect available data. 
  • Consistency of data. As several data elements require shared responsibility between Site Coordinators and the evaluation team, there are inherent limitations in the accuracy and reliability of some data sources. In particular, because Site Coordinators manage and update both family and student attendance data, the process for updating and reporting data may be worth examining, such as timelines and quality assurance processes during the program year. Another challenge related to consistency was the use of two different family event surveys in 2022-23. In future program years, Site Coordinators should use the same survey for all family events.