FINDINGS of the Modeling Workshop Project (1994-00) -- enlarged graphs
Teachers report that some graphs in the pdf document are too small to read. Thus the graphs are enlarged here, and explanations are included. (It was not possible to enlarge Figure 4.)
This is highlights of the data analysis, an excerpt of one section in the Final Report submitted to the National Science Foundation in fall 2000 for the Teacher Enhancement grant entitled Modeling Instruction in High School Physics. David Hestenes, Professor of Physics at Arizona State University, was Principal Investigator. Extensive information about the Project is at http://modeling.asu.edu.
We used a pre–posttest comparison design with matched students. Here we report highlights of the data analysis of Phase I, II, preliminary III, and preliminary Arizona Eisenhower FCI baseline, pretest and posttest data for the Modeling Instruction Project for considerably more than 10,000 high school students of about 170 teachers.
Š The average increase in student posttest FCI mean for Phase I teachers is 6 percentage points after two years of workshop participation (several teachers had previous exposure to aspects of Modeling Instruction and started out high). The average increase in student posttest FCI mean for Phase II, Phase III, and Arizona teachers is about 10 percentage points after only one year of participation. Figures 1a, 1b, and 1c present graphs of student FCI means of teachers in the Phase I, Phase II, and Phase III Modeling Workshops, ordered from lowest to highest baseline mean. (Phase III data are preliminary; not all have been analyzed yet.) Corresponding mean FCI posttest scores for students of matched Arizona physics teachers are 35.8% baseline in 1998 (507 students) and 54.5% in 2000 (572 students), a gain of 19 percentage points in 2 years!
Š When broken down by categories that reflect degree of implementation of Modeling Instruction, results show that teachers who report consistent implementation of most or all components of the Modeling Method have higher student posttest FCI means than their colleagues who report some implementation or little implementation. Figure 2 presents a bar graph that shows average student posttest FCI mean for 23 Phase I teachers for whom we have complete data across three years. The graph is clustered by the modeling-implementation categories.
The group that reported the most consistent use of components of the Modeling Method (on the 1996 experiences survey) also included some participants with some degree of prior exposure to reform courses. This group had an increase in average FCI posttest mean of over 10% (from 61% to 72%) and remained high after the following year (71%).
The second group, comprised of teachers reporting consistent use of some Modeling Method components, showed steady growth across the years, with a significant increase between 1995 (49%) and 1997 (61%). Many of these teachers reported consistent use of most or all components when resurveyed in 1997.
The third group, that reported the least degree of implementation of Modeling, showed no significant difference across years.
Š Students at similar pretest levels of conceptual knowledge, as measured by the FCI, demonstrate different degrees of pretest - posttest gain, depending on the degree of implementation of Modeling Instruction by their physics teacher. Figure 3 presents three categories of students, based on their pretest FCI performance (20% or below, 20-50%, and 50% or above). Each category of student is broken down by the degree of Modeling Method implementation by their teacher.
Students with teachers who consistently use most or all components of Modeling Instruction have the highest pretest-posttest gains.
Students with the lowest pretest scores show the highest degree of gain for the teachers in the highest Modeling implementation group.
Š When the matched student data are categorized into regular (algebra based and conceptual) and honors (trigonometry based) physics courses, the performance of “ordinary” Arizona teachers is like the groups of nationwide leaders. Figure 4 shows this result.
Š The program increases the achievement of underserved learners. Thirty-six Phase I teachers in our national modeling workshops reported their students’ FCI scores in their second year of using Modeling Instruction. The teachers also completed a 50-question survey of their classroom experiences during that academic year. The survey contains questions regarding the teacher’s perceptions of their understanding and implementation of the Modeling Method, their classroom practices and activities, and the socioeconomic status of their school. The teachers were found to be in two groups: those who were implementing all aspects of Modeling Instruction consistently (17 teachers), and those who were implementing some aspects of modeling consistently (19 teachers). Figure 5 shows that in disadvantaged/lower income schools, the mean normalized FCI gains of the 93 students of the two teachers who implemented the Modeling Method consistently were 25% higher than for the 335 students of the six teachers who were implementing less modeling. Students of both groups did better than in traditional courses; the FCI gains of the 93 students were double that under traditional instruction, and the FCI gains of the 335 students were 50% higher than under traditional instruction.
Š Gender differences are being explored where data are available. We find a significant gender gap under traditional instruction, with girls in regular courses attaining FCI (baseline) posttest scores averaging 9.5 percentage points less than boys, and girls in honors courses scoring 13 percentage points less than boys. (All other research studies in Western cultures find similar gender gaps in traditional physics courses.) For 900 Phase II students in regular physics courses in the first year of modeling implementation, Modeling Instruction dramatically increased the FCI gains both of boys (a mean increase of 14 percentage points compared to traditional instruction) and girls (a mean increase of 12 percentage points). For 666 students in honors courses, mean FCI gains improved by 3.5 percentage points for boys and 6.5 percentage points for girls relative to traditional instruction. Gender data from Phase I and III have yet to be analyzed.
Š Assessment of teacher understanding of the force concept was via the FCI and the Mechanics Baseline Test. These two tests were given at the start of each workshop. For teachers nationwide and in Arizona who participated in the series of workshops and taught physics in the intervening year, the average FCI score increased to 93% and the MBT score increased to about 78%. Arizona teachers started lower: their average FCI score increased by 9 percentage points, whereas national workshop teachers’ average FCI score increased by 6 percentage points. Figures 6a and 6b show these teacher FCI results for representative groups, ordered from lowest to highest baseline. The average MBT score of Arizona teachers increased by 14 percentage points and nationwide teachers by 6 percentage points. These improvements show that the workshop in mechanics not only resulted in improved teacher understanding of the force concept but also sharpened teachers’ problem solving skills, for that is the focus of the MBT.
The overall findings are that high school physics students of teachers in the Modeling Instruction in High School Physics Project demonstrate much greater gains on the FCI than a) physics students of the same teachers in the year before the teachers began the modeling workshop series, and b) a comparison group of 700 high school physics students a decade ago [D. Hestenes, M. Wells, and G. Swackhamer, Force Concept Inventory, The Physics Teacher 30: 141-158 (1992) and private communication].
The comparison groups are:
a) 3529 high school students of the same Phase II and III teachers just before the teachers began the Modeling Workshop series. Those FCI mean scores were: pretest: 26%, posttest: 43.1%.
b) 808 students of 17 Arizona teachers in 1998 just before the teachers began the Eisenhower Modeling Workshop series. Those FCI mean scores were: pretest: 25.7%, posttest: 41.3%.
c) Over 700 high school students of 17 teachers, mostly in Arizona, a decade ago just before they participated in pilot Modeling Workshops. Those Force Concept Inventory mean scores were: pretest: 28%, posttest: 46%.