IMP® Students Score Higher than Traditional Peers
An analysis of SAT-9 math scores at Philadelphias High School for Girls showed that IMP
students outperformed their traditionally taught counterparts. The study was conducted by Dr. Ned
Wolff, of Beaver College, with the support of Steven Kramer, of the University of Maryland, and
Atenssa Cheek, of La Salle University.
Three different scores were examined: the composite mathematics score and its two major components,
the open-ended and multiple-choice scores. To account for differences in student backgrounds before
enrolling in high school, the study took into consideration the students' eighth-grade test score on
the Comprehensive Test of Basic Skills (CTBS). Using the statistical technique, Analysis of
Covariance, it computed and compared the two lines (called regression lines) that best predict IMP
and traditional students eleventh-grade SAT-9 scores from their eighth-grade CTBS scores.
All students were included in the study as long as they didnt switch math programs and the
Philadelphia School District was able to supply both eleventh-grade math SAT-9 scores and
eighth-grade CTBS scores. The only exception was that one of the traditional students was dropped
from the study. A so-called outlier, this student did very well on eighth-grade CTBS but
so poorly on the SAT-9 that, if included in the study, would have appreciably lowered the overall
performance of the traditional group. Remaining in the study were 27 IMP and 138 traditional
IMP vs. Traditional: Growth from CTBS to SAT-9
The graph above shows the regression lines for both the IMP and traditional students, where the
composite math SAT-9 scores are predicted from the CTBS scores. The fact that the IMP line is higher
means that if an IMP student and traditional student had the same CTBS score, the IMP student
generally got a higher SAT-9 score. The IMP gains demonstrated in this study were statistically
The steeper slope of the IMP regression line suggests that IMP was especially beneficial for the best
students, because the further to the right you go, the greater the gap between the predicted scores
of IMP and traditional students. However, the difference in slopes of the two regression lines turned
out not to be significant (perhaps due to the small number of IMP students). Therefore, the study
next proceeded on the assumption that the lines were parallel. When the equations of the best-fit
parallel lines were computed, the distance between these lines was found to be 9.73. That is, if an
IMP and a traditional student both have the same eighth-grade CTBS math scores, the IMP students
predicted eleventh-grade SAT-9 math composite score would be 9.73 points higher.
Best-fit Lines for Open-Ended Component of SAT-9.
To better understand the gains demonstrated by IMP students, the open-ended and multiple-choice
components of the SAT-9 were examined separately. The graph above shows the best-fit lines for the
open-ended component of the test. As suggested by the figure, IMP students significantly outscored (p<.01)
the traditional students on this portion of the test by a comparatively large margin.
Indeed, given the same CTBS scores in eighth grade, an IMP student would be predicted to outscore the
traditional student by 15.2 points. An examination of the multiple-choice scores showed a small gain
of 4.27 points for IMP students, but this gain was not statistically significant.
The SAT-9 scores analyzed above were reported on a scale of 1 through 99. Based on these scores, the
School District of Philadelphia characterizes student performance as being either below basic, basic,
proficient, or advanced, with the last three categories considered as passing. An analysis of the
scores of all the juniors at Girls High School (including those excluded from the above study
because they did not take the CTBS in eighth grade) found that 80% of the IMP students versus 43.5%
of the traditional students received passing scores. Also, although IMP students comprised only 13.6%
of all juniors at Girls High, they accounted for 35.1% of the scores in the two highest
(proficient and advanced) categories.
Evaluation Update Index