General Info | Resources | Research & Reports | Contacts & Connections | Curriculum
General Information
  A Brief Description of IMP
Inside IMP
IMP Success Brochure
IMP Strategies
Common Core Correlations
to IMP
Product & Pricing Info
Professional Development

Resources
  Publications & Articles
Research & Reports
Curriculum Resources
Especially for Parents

Contacts & Connections
  Contact IMP
NSF Funded Curriculum Projects

IMP® Students Score Higher than Traditional Peers

An analysis of SAT-9 math scores at Philadelphia’s High School for Girls showed that IMP students outperformed their traditionally taught counterparts. The study was conducted by Dr. Ned Wolff, of Beaver College, with the support of Steven Kramer, of the University of Maryland, and Atenssa Cheek, of La Salle University.

Three different scores were examined: the composite mathematics score and its two major components, the open-ended and multiple-choice scores. To account for differences in student backgrounds before enrolling in high school, the study took into consideration the students' eighth-grade test score on the Comprehensive Test of Basic Skills (CTBS). Using the statistical technique, Analysis of Covariance, it computed and compared the two lines (called regression lines) that best predict IMP and traditional students’ eleventh-grade SAT-9 scores from their eighth-grade CTBS scores.

All students were included in the study as long as they didn’t switch math programs and the Philadelphia School District was able to supply both eleventh-grade math SAT-9 scores and eighth-grade CTBS scores. The only exception was that one of the traditional students was dropped from the study. A so-called “outlier,” this student did very well on eighth-grade CTBS but so poorly on the SAT-9 that, if included in the study, would have appreciably lowered the overall performance of the traditional group. Remaining in the study were 27 IMP and 138 traditional students.


IMP vs. Traditional: Growth from CTBS to SAT-9

The graph above shows the regression lines for both the IMP and traditional students, where the composite math SAT-9 scores are predicted from the CTBS scores. The fact that the IMP line is higher means that if an IMP student and traditional student had the same CTBS score, the IMP student generally got a higher SAT-9 score. The IMP gains demonstrated in this study were statistically significant (p<.01).

The steeper slope of the IMP regression line suggests that IMP was especially beneficial for the best students, because the further to the right you go, the greater the gap between the predicted scores of IMP and traditional students. However, the difference in slopes of the two regression lines turned out not to be significant (perhaps due to the small number of IMP students). Therefore, the study next proceeded on the assumption that the lines were parallel. When the equations of the best-fit parallel lines were computed, the distance between these lines was found to be 9.73. That is, if an IMP and a traditional student both have the same eighth-grade CTBS math scores, the IMP student’s predicted eleventh-grade SAT-9 math composite score would be 9.73 points higher.


Best-fit Lines for Open-Ended Component of SAT-9.

To better understand the gains demonstrated by IMP students, the open-ended and multiple-choice components of the SAT-9 were examined separately. The graph above shows the best-fit lines for the open-ended component of the test. As suggested by the figure, IMP students significantly outscored (p<.01) the traditional students on this portion of the test by a comparatively large margin.

Indeed, given the same CTBS scores in eighth grade, an IMP student would be predicted to outscore the traditional student by 15.2 points. An examination of the multiple-choice scores showed a small gain of 4.27 points for IMP students, but this gain was not statistically significant.

The SAT-9 scores analyzed above were reported on a scale of 1 through 99. Based on these scores, the School District of Philadelphia characterizes student performance as being either below basic, basic, proficient, or advanced, with the last three categories considered as passing. An analysis of the scores of all the juniors at Girls’ High School (including those excluded from the above study because they did not take the CTBS in eighth grade) found that 80% of the IMP students versus 43.5% of the traditional students received passing scores. Also, although IMP students comprised only 13.6% of all juniors at Girls’ High, they accounted for 35.1% of the scores in the two highest (proficient and advanced) categories.

Evaluation Update Index


©2012 It's About Time Interactive. All rights reserved.