FINDINGS of the Modeling Workshop Project (1994-00) -- enlarged graphs

Teachers report that some graphs in the pdf document are too small to read. Thus the graphs are enlarged here, and explanations are included. (It was not possible to enlarge Figure 4.)

This is highlights of the data
analysis, an excerpt of one section in the Final Report submitted to the
National Science Foundation in fall 2000 for the Teacher Enhancement grant
entitled *Modeling Instruction in High School Physics*. David
Hestenes, Professor of Physics at Arizona State University, was Principal
Investigator. Extensive information about the Project is at
http://modeling.asu.edu.

É

We used a pre–posttest comparison design with matched students. Here we report highlights of the data analysis of Phase I, II, preliminary III, and preliminary Arizona Eisenhower FCI baseline, pretest and posttest data for the Modeling Instruction Project for considerably more than 10,000 high school students of about 170 teachers.

á
The average increase in *student* posttest FCI mean for Phase I teachers is 6
percentage points after two years of workshop participation (several teachers
had previous exposure to aspects of Modeling Instruction and started out high).
The average increase in student posttest FCI mean for Phase II, Phase III, and
Arizona teachers is about 10 percentage points after only one year of
participation. **Figures 1a, 1b, and 1c** present graphs of student
FCI means of teachers in the Phase I, Phase II, and Phase III Modeling
Workshops, ordered from lowest to highest baseline mean. (Phase III data are
preliminary; not all have been analyzed yet.) Corresponding mean FCI posttest
scores for students of matched Arizona physics teachers are 35.8% baseline in
1998 (507 students) and 54.5% in 2000 (572 students), a gain of 19 percentage
points in 2 years!

á

Figure 1a

Figure 1a.

Figure 1b

Figure 1b.

Figure 1c

Figure 1c.

á
When broken down by categories that reflect degree of
implementation of Modeling Instruction, results show that teachers who report
consistent implementation of most or all components of the Modeling Method have
higher student posttest FCI means than their colleagues who report some
implementation or little implementation**. Figure 2** presents a bar
graph that shows average student posttest FCI mean for 23 Phase I teachers for
whom we have complete data across three years. The graph is clustered by the
modeling-implementation categories.

The group that reported the *most consistent* use of components of the Modeling Method (on the
1996 experiences survey) also included some participants with some degree of
prior exposure to reform courses. This group had an increase in average FCI
posttest mean of over 10% (from 61% to 72%) and remained high after the
following year (71%).

The second group, comprised of teachers reporting consistent use
of *some* Modeling Method components,
showed steady growth across the years, with a significant increase between 1995
(49%) and 1997 (61%). Many of these teachers reported consistent use of most or
all components when resurveyed in 1997.

The third group, that reported the *least* degree of implementation of Modeling, showed *no* *significant difference* across years.

Figure 2

Figure 2.

á
Students at similar pretest levels of conceptual
knowledge, as measured by the FCI, demonstrate different degrees of pretest -
posttest gain, depending on the* degree of implementation* of Modeling Instruction by their physics teacher. **Figure
3** presents three categories of students, based on their pretest FCI
performance (20% or below, 20-50%, and 50% or above). Each category of student
is broken down by the degree of Modeling Method implementation by their
teacher.

Students with
teachers who consistently use most or all components of Modeling Instruction
have the *highest* pretest-posttest gains.

Students with the *lowest
pretest scores* show the *highest
degree of gain* for the teachers in the *highest
*Modeling implementation group.

Figure 3

á
When the matched student data are categorized into
regular (algebra based and conceptual) and honors (trigonometry based) physics
courses, the performance of ÒordinaryÓ Arizona teachers is like the groups of
nationwide leaders. **Figure 4** shows this
result.

Figure 4

Figure 4.

á
The program increases the achievement of *underserved
learners*. Thirty-six Phase I teachers in
our national modeling workshops reported their studentsÕ FCI scores in their
second year of using Modeling Instruction. The teachers also completed a
50-question survey of their classroom experiences during that academic year.
The survey contains questions regarding the teacherÕs perceptions of their
understanding and implementation of the Modeling Method, their classroom
practices and activities, and the socioeconomic status of their school. The teachers were found to be in two
groups: those who were implementing all aspects of Modeling Instruction
consistently (17 teachers), and those who were implementing some aspects of
modeling consistently (19 teachers). **Figure 5** shows that in
disadvantaged/lower income schools, the mean normalized FCI gains of the 93
students of the two teachers who implemented the Modeling Method consistently
were 25% higher than for the 335 students of the six teachers who were
implementing less modeling. Students of both groups did better than in
traditional courses; the FCI gains of the 93 students were double that under
traditional instruction, and the FCI gains of the 335 students were 50% higher
than under traditional instruction.

Figure 5

á
*Gender differences*
are being explored where data are available. We find** **a
significant gender gap under traditional instruction, with girls in regular
courses attaining FCI (baseline) posttest scores averaging 9.5 percentage
points less than boys, and girls in honors courses scoring 13 percentage points
less than boys. (All other research studies in Western cultures find similar
gender gaps in traditional physics courses.) For 900 Phase II students in *regular* physics courses in the first year of modeling
implementation, Modeling Instruction dramatically increased the FCI gains both
of boys (a mean increase of 14 percentage points compared to traditional
instruction) and girls (a mean increase of 12 percentage points). For 666
students in *honors* courses, mean
FCI gains improved by 3.5 percentage points for boys and 6.5 percentage points
for girls relative to traditional instruction. Gender data from Phase I and III
have yet to be analyzed.

á
Assessment of *teacher* understanding of the force concept was via the FCI and the Mechanics
Baseline Test. These two tests were given at the start of each workshop. For
teachers nationwide and in Arizona who participated in the series of workshops
and taught physics in the intervening year, the average FCI score increased to
93% and the MBT score increased to about 78%. Arizona teachers started lower:
their average FCI score increased by 9 percentage points, whereas national
workshop teachersÕ average FCI score increased by 6 percentage points. **Figures
6a and 6b** show these teacher FCI results for representative groups,
ordered** **from lowest to highest
baseline. The average MBT score of Arizona teachers increased by 14 percentage
points and nationwide teachers by 6 percentage points. These improvements show
that *the workshop in mechanics not only resulted in improved teacher
understanding of the force concept but also sharpened teachersÕ problem solving
skills*, for that is the focus of the MBT.

Figure 6a

Figure 6b

The overall findings are that
high school physics students of teachers in the *Modeling Instruction in High
School Physics* Project demonstrate much
greater gains on the FCI than a) physics students of the same teachers in the
year before the teachers began the modeling workshop series, and b) a
comparison group of 700 high school physics students a decade ago [D. Hestenes,
M. Wells, and G. Swackhamer, Force Concept Inventory, *The Physics
Teacher* **30**: 141-158 (1992)
and private communication].

The comparison groups are:

a) 3529 high school students of the same Phase II and III teachers just before the teachers began the Modeling Workshop series. Those FCI mean scores were: pretest: 26%, posttest: 43.1%.

b) 808 students of 17 Arizona teachers in 1998 just before the teachers began the Eisenhower Modeling Workshop series. Those FCI mean scores were: pretest: 25.7%, posttest: 41.3%.

c) Over 700 high school students of 17 teachers, mostly in Arizona, a decade ago just before they participated in pilot Modeling Workshops. Those Force Concept Inventory mean scores were: pretest: 28%, posttest: 46%.

** **