|General Info | Resources | Research & Reports | Contacts & Connections | Curriculum|
Evaluation of the Interactive Mathematics Program
By Diane Resek
This article was prepared for the American Educational Research Association (AERA).
General Characteristics of IMP®
The Interactive Mathematics Program (IMP) is a four-year, problem-based mathematics curriculum for secondary-school students that focuses on open-ended explorations of complex problems, yet covers the essential content of the traditional Algebra I-Geometry-Algebra II/Trigonometry-Precalculus sequence. The design of this integrated program began in 1989 and was conceived for non-college bound as well as college preparatory students to fulfill the mathematics standards developed by the National Council of Teachers of Mathematics. The program currently is used in roughly 250 high schools in 21 states.
Development of IMP was initially funded by the California Postsecondary Education Commission and later received major funding from the National Science Foundation for curriculum development, evaluation, and dissemination. Co-directors of the program are Professors Dan Fendel and Diane Resek of the Mathematics Department at San Francisco State University, and mathematics educators Lynne Alper and Sherry Fraser of Sonoma State University Academic Foundation.
Characteristics of the IMP curriculum include an emphasis on interactive group problem solving, an interdisciplinary approach, and the use of manipulatives and graphing calculators, as well as paper-and-pencil work. Teachers must learn new ways to manage a classroom and how to be effective facilitators and observers, since IMP does not rely on the traditional model of a teacher lecturing to students who then work on repetitive exercises.
IMP's approach to mathematics education is designed to prepare students for the workplace of today and the future. Today and tomorrow, if students continue with traditional mathematics preparation, they may be learning specific procedures that will be outmoded by the time they graduate. The challenge is to provide students with ways to learn new procedures on their own so they can adapt to changing skill requirements throughout their careers. Such learning includes, but is not limited to, knowledge of basic mathematics facts.
The student's background must also include a sound grounding in important mathematical ideas, as well as the ability to carry out independent inquiries, to reflect on one's own understanding of new ideas, and to contribute as a member of a working group.
In IMP classrooms, teachers establish performance criteria and evaluate student achievement using multiple sources: unit exams, performance tasks, portfolios, writing assignments, and oral presentations. A teaching priority is to determine whether the work of individual students meets specific expectations, rather than comparing and ranking students to measure achievement.
IMP Evaluation Overview
An extensive evaluation of IMP was conducted by Norman Webb at the Wisconsin Center for Education Research (WCER). He has collected retention data in mathematics courses, high school grades, and comparative data on student Scholastic Achievement Test (SAT) performance, as well as performance on activities involving probability, statistics, quantitative reasoning, and problem solving. Studies on student performance have been conducted at various IMP sites around the country. In addition, several studies were conducted by researchers outside IMP that compared IMP students to students in other programs on the basis of performance on tests and on attitudes.
Many of the studies comparing IMP students are summarized below. They are grouped in the following categories:
We conclude this paper with some research questions.
High School Grades and Retention
An analysis was done of transcripts of graduates from three high schools in 1993 (Webb and Dowling, 1996). Those high schools were the first to begin the IMP program and some students in each graduating class were in the first IMP classes. The study found that students who began the IMP program in 9th grade took more semesters of mathematics and had higher grade point averages than those who began the algebra I program. For the IMP group, compared to students in the algebra I sequence, a higher percentage of both female and male students and students of all ethnic groups completed at least three years of college-qualifying mathematics. A significantly higher percentage of IMP students (71%) took at least one semester of advanced mathematics beyond the basic three years than did students (52%) in the algebra sequence.
At one school, test scores from middle school were available for most students, so a comparison could be done using pre-measures. The groups of 62 students each were matched according to their Comprehensive Test of Basic Skills (CTBS) scores in mathematics in the seventh grade. The non-IMP group had virtually identical scores to the IMP group and also matched them in terms of gender and ethnic make-up. Of the traditional group, 76% took algebra I in ninth grade, and progressed, in whole or in part, through the traditional sequence of Algebra I, Geometry, Algebra II and Pre calculus or Calculus, and 24% were enrolled in Basic Mathematics courses. The IMP group was composed of those students who stayed in IMP's core curriculum sequence of integrated mathematics throughout their basic three year mathematics program.
The IMP students in the matched samples took significantly more semesters of math (7.79 versus 7.44), language arts (8.60 versus 7.21), and science (5.32 verses 4.94). The overall grade point average of the IMP students was significantly higher (3.03 versus 2.74). The study also examined the records of 58 students from IMP and 60 non-IMP students who were in the upper 25th percentile on the seventh grade CTBS test (using national norms). The two groups had virtually identical seventh-grade scores. It was found that IMP students in that group had significantly higher grade point averages in high school (3.11 versus 2.68), even after mathematics grades were discounted.
Student Performance on Standardized Tests
Many different schools in different parts of the country have compared IMP students to students in the traditional sequence using traditional standardized tests such as the Scholastic Assessment Test (SAT). In all of these studies, there has either been no significant difference in scores or the IMP students have scored significantly higher. The consistency of these studies shows that IMP students are holding their own on these traditional measures. Thus, they are not being harmed by using a non-traditional curriculum. This finding is especially meaningful in view of the fact that about 2030% of the IMP students' classroom time is spent learning important topics not covered on these standardized tests.
As an example of such a study, Eaglecrest School in Aurora, Colorado, compared ninth grade IMP students and ninth grade algebra I students at the beginning of their IMP and algebra I experience and at the end of the year (Interactive Mathematics Program, 1996). The IMP student raw SAT score (6.74) was below the algebra I mean (6.91) at the beginning of the year, but significantly above at the end of the year (9.66 versus 8.16).
In another example, the Stanford-9 Test was used to compare eleventh grade students at two magnet schools in Philadelphia (Wolff, April 1997 and August 1997). In one school, IMP students outperformed their traditional counterparts on 15 math-related scores, tied on two, and were outperformed on three. IMP students did better on all the cumulative scores. At the second school, IMP student outperformed traditional students on 26 Stanford categories and were outperformed in three.
In the larger matched pair study mentioned earlier (in the High School Grades and Retention section), after accounting for prior levels of achievement in 7th grade, there was no significant difference in SAT scores between the IMP and non-IMP group. When just the high-achieving students, who were in the top quartile in seventh grade (based on national norms), were compared, the IMP group average was higher (545) than the non-IMP group (531). but the difference was not significant.
There is also evidence that IMP's approach is helpful to students who score at the low end of the achievement spectrum. For example, in a study of high schools with low-income and low-achieving student populations, higher growth in achievement was observed among ninth grade students enrolled in the IMP courses than for those enrolled in the other curricula, including traditional algebra I (White, Gamoran, and Smithson, 1995). The IMP students began at a lower level than the other students in college-preparatory courses and finished at a higher level.
Performance Comparison on Other Tests
As mentioned earlier, a substantial amount of time in the IMP classroom, from 20% to 30%, is spent learning content that is not traditionally part of the high school curriculum. It seemed appropriate to verify the fact that IMP students were learning in those areas.
In Spring 1996, permission was sought from a prestigious eastern university to use a portion of the quantitative reasoning examination it gives to all entering freshmen. Passing this test or taking a special course is required for graduation. Permission was given for pilot testing for one year only, but not for release of the name of the university in discussing the data. With the use of a preliminary test, the test was shortened to ten items so that it could be administered within a single class period. This ten-item test was administered at a magnet high school on the east coast to eleventh grade IMP and algebra II students. Eighth grade standardized test scores were available for the students, so a matched group analysis was also done involving subsets of the two groups.
The mean of the full group of IMP students was significantly higher than that of the algebra II students, using analysis of variance to control for pre-test scores. Also, the IMP students in the matched group analysis significantly outperformed their counterparts (Webb and Dowling, 1997). Since permission was not given to use the test again, the study could not be replicated.
Two items from a performance test (developed by the state of Wisconsin) were administered to 10th grade IMP and geometry students at a magnet high school in the mid-west in Spring 1996. Eighth grade test scores were obtained for the students. Again, the mean of the full group of IMP students was significantly higher than that of the geometry students, using analysis of variance, and the IMP students in a matched group analysis significantly outperformed their counterparts (Webb and Dowling, 1997). The IMP students outperformed students taking Honors Geometry who had significantly higher pre-test scores, although this latter result was only significant when controlling for the difference in pre-test scores.
This study was replicated in Spring 1997 in three high schools in different parts of the country (Webb and Dowling, 1998). In two of these high schools the IMP students performed significantly better that their traditionally trained 10th grade counterparts. At the third school the curriculum had been enriched to include content that related directly to one of the problems. On that problem the traditional students outperformed the IMP students, although not significantly, but the IMP students scored significantly higher on the second problem and scored higher overall, although not significantly.
In Spring 1996, the statistics items from the Second International Math and Science (SIMS) international study were administered to ninth grade IMP and algebra I students at a suburban high school in California. The questions were given as open-response items rather than multiple choice items as they had appeared in the SIMS study. Eighth grade standardized test scores were again available for all students. The results followed the same pattern as in the other two studies, with IMP students significantly outperforming the algebra I students as a whole group and in the matched group analysis (Webb and Dowling, 1997).
This study was replicated in Spring 1997 in another California school, in a school in the East of the United States and another in the Rocky Mountains (Webb and Dowling, 1998). In the first two schools the IMP students again scored significantly higher than their matched pairs. In the third school, the algebra I curriculum was enriched with statistics curriculum relating directly to the questions, and the students from the enriched curriculum scored significantly higher than the IMP students.
Comparison of Attitudes
An outside study examined student attitudes towards mathematics, comparing 182 first-year IMP students with 217 algebra I students from the same or comparable schools (Clarke, Wallbridge, and Fraser, 1996). The study found that students who participated in the IMP program appeared to be more confident than their peers in conventional classes; to subscribe to a view of mathematics as having arisen to meet the needs of society, rather than as a set of arbitrary rules; to value communication in mathematics learning more highly than students in conventional classes; and to be more likely than their conventionally-taught peers to see a mathematical element in everyday activity.
Comparing Performance After Secondary School
When IMP was first conceived, it was felt that students who had had four years of IMP should be followed after high school, whether they went on to college or directly to the work force. It was hoped that their performance could be compared to that of students who had gone through a traditional mathematics program. Therefore, random samples were taken of IMP and non-IMP students graduating in 1994 from the first four high schools using the fourth-year curriculum.
About 200 of these students were interviewed during Spring 1994. However, these students no longer represented a random sample since not all students selected agreed to participate. In the non-IMP group there was an over representation from the suburban high school (59%) and an under representation from the inner-city high school (17%). In addition, a disproportionate number of respondents (27%) had been accelerated into geometry as 9th graders. None of the IMP students were from that accelerated group when they entered high school. Given the profile of the non-IMP students responding, that group could not be used as a valid control group to measure achievement in college.
This article concludes with some issues others may want to pursue:
Clarke, David, Margarita Wallbridge, and Sherry Fraser, "The Other Consequences of a Problem-Based Mathematics Curriculum, Research Report No. 3," Mathematics Teaching and Learning Centre, Australian Catholic University, Christ Campus, Victoria, Australia, 1996.
Interactive Mathematics Program, IMP Evaluation Update, No. 2, Fall 1996, Emeryville, CA, 1996.
Webb, Norman and Maritza Dowling, "Impact of the Interactive Mathematics Program on the Retention of Students: Cross-School Analysis of Transcripts for the Class of 1993 for Three High Schools," Wisconsin Center for Education Research, University of Wisconsin-Madison, July, 1996.
Webb, Norman and Maritza Dowling, "Comparison of IMP Students with Students Enrolled in Traditional Courses on Probability, Statistics, Problem Solving, and Reasoning," Wisconsin Center for Education Research, University of Wisconsin-Madison, April, 1997.
Webb, Norman and Maritza Dowling, "Replication Study of the Comparison of IMP Students with Students Enrolled in Traditional Courses on Probability, Statistics, Problem Solving, and Reasoning," Wisconsin Center for Education Research, University of Wisconsin-Madison, January, 1998.
White, Paula, Adam Gamoran, and John Smithson, "Math Innovations and Student Achievement in Seven High Schools in California and New York," Consortium for Policy Research in Education and the Wisconsin Center for Education Research, University of Wisconsin-Madison, January, 1995.
Wolff, Edward, "Summary of Matched-Sample Analysis Comparing IMP and Traditional Students at Philadelphia High School for Girls on Mathematics Portion of Stanford-9 Test," Mathematics Department, Beaver College, Glenside, PA, August 1997.
Wolff, Edward, "Summary of Matched-Sample Stanford 9 Analysis Comparing IMP and Traditional Students at Central High School, Philadelphia, PA," Mathematics Department, Beaver College, Glenside, PA, April 1997.