Impacts and Implementation of the i3-Funded Scale-Up of Success for All Archival Version


The Impacts and Implementation of the i3-Funded Scale-Up of Success for All study examined the implementation and impacts of a whole-school reform model, Success for All (SFA), which seeks to increase reading proficiency among students in elementary schools serving low-income children, as this model was scaled up under an Investing in Innovation (i3) grant from the United States Department of Education. The impact evaluation used a cluster random assignment design in which 37 schools in five school districts were selected at random either to implement SFA (the program group schools) or to put in place their business-as-usual reading program (the control group schools). This collection contains Parts 1 to 7 for public use; and Part 8 for restricted use only. In Part 1: Primary Student Sample data, children who began kindergarten in sampled schools and remained in them through second grade were assessed using standardized assessments of phonics skills, reading fluency, and comprehension. In Part 2: Auxiliary Sample, additional analyses examined third-year impacts on reading skills for all second graders, whether or not they remained continuously enrolled in the study schools, and for students in grades 3 through 5. The analysis also examined impacts for subgroups defined by ethnicity, early literacy skills measured upon entry into kindergarten, and other variables. Part 3: School Achievement Snapshot Data contain implementation data for analysis using an instrument created by SFA's developers that measures the extent to which program elements were put in place at each program group at the end of each year. This instrument was utilized to assess fidelity to the program model. The implementation analysis was also informed by principal (Part 5: Principal Survey) and teacher surveys (Part 4: Teacher Survey) in all schools, as well by logs (Part 6: Teacher Log) that asked teachers to document their instructional practices in early reading classes. These data, collected in all three study years, make it possible to measure implementation over the study period and to compare program group schools and control group schools on a variety of dimensions. Part 7: Scale-Up Sample Data provided data on the schools that adopted SFA (and those which were approached, but did not adopt), and schools that adopted before the Investing in Innovations grant. Part 8 is the restricted version of Part 7, which retains geographic location variables. MDRC employed a cluster random assignment research design to determine whether Success for All was as effective in its larger-scale version as it had been in earlier tests of the program. In this design, 37 schools located in five districts were assigned at random: 19 to receive the program and the remaining 18 to constitute a "business-as-usual" control group. The evaluation compared the reading outcomes of kindergartners in both sets of schools in the first year of the study, and of first-graders and second-graders in the next two years, using age-appropriate assessments that were administered individually to each child. During the second and third years of the study, students in third through fifth grades were also assessed, this time in a group setting. Student scores on state standardized tests served as additional measures of reading skills. The impact analysis also measured the program's effects on student attendance and grade promotion. Impacts were unlikely to occur unless schools were able to implement the program with reasonable fidelity. MDRC collected and analyzed data from a variety of sources -- teacher logs, principal and teacher surveys, and rating sheets completed by the coaches -- to examine the implementation experiences of the schools participating in the study. The analysis also compared the characteristics and implementation levels of the study schools with those of other schools that put Success for All in place as part of the i3 replication. Part 2, Auxiliary Student Sample Data, contains three weight variables: SCH26_GR3_WT, SCH26_GR4_WT, and SCH26_GR5_WT. In general, ICPSR provides unweighted data, but Part 3, School Achievement Snapshot Data, contains weighted variables of scores. Users are highly encouraged to refer to the User Guide for more information on weights. ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection: Checked for undocumented or out-of-range codes.. Presence of Common Scales: Woodcock-Johnson Letter Word Identifcation, Woodcock-Johnson Word Attack, Woodcock-Johnson Passage Comprehension, Test of Word Reading Efficiency, Gates-MacGinitie Reading Test-Vocabulary, Gates-MacGinitie Reading Test-Comprehension Response Rates: The response rate varied by instrument ranging from 85% to 100% 37 schools including their teachers, principles and students in five school districts in four states that agreed to participate in the study. Smallest Geographic Unit: District (anonymized) 37 schools within 5 pre-specified school districts in 4 states agreed to participate in the study. The schools within each district were randomly assigned to program or control conditions, which produced 19 program group schools and 18 control group schools. record abstracts, cognitive assessment test, paper and pencil interview (PAPI), on-site questionnairePart 8 is for restricted use only; it contains the same data as Part 7, with geographic location variables included: longitude and latitude. For more information about the Success for All study, please refer to the MDRC Web site.

Related Identifier DOI: 10.3886/ICPSR36387.v1
Metadata Access
Creator Quint, Janet
Publisher ICPSR - Interuniversity Consortium for Political and Social Research
Contributor United States Department of Education
Publication Year 2016
Rights Delivery;One or more files in this study are not available for download due to special restrictions; consult the study documentation to learn more on how to obtain the data.
Contact ICPSR - Interuniversity Consortium for Political and Social Research
Language English
Resource Type Dataset;administrative records data, event/transaction data, experimental data, survey data
Discipline Not stated
Spatial Coverage {"United States","2011 - 2014","Time period: 2011--2014","2011 - 2014","Collection date: 2011--2014"}
Temporal Coverage {"United States","2011 - 2014","Time period: 2011--2014","2011 - 2014","Collection date: 2011--2014"}