8

Figure 2—enVisionMATH Students’ Math Performance at Pre- and Posttesting: GMADE, which focuses on math vocabulary.

120

# GMADE Scale Score (Percentile Rank)

100

80

60

40

20

0

94.9

98.7

108.9

Pretest (Fall 2007) Posttest (Spring 2008) Posttest (Spring 2009)

# Overall

enVisionMATH students also showed significant gains in math vocabulary (GMADE).

Learning gains experienced by enVisionMATH students can also be seen in growth of percentile ranks^{1 }from the norm-based assessments. It is a general rule of thumb that if a student makes a year’s growth for a year of instruction, then the percentile rank will remain the same. As shown in Figure 3, the percentile rank grew more than would be expected in a typical academic year for concepts and problem- solving, computation, and math vocabulary.

Figure 3—enVisionMATH Students’ Percentile Rankings at Pre- and Posttesting

100

# Percentile Rankings

80

60

40

20

67

64

74

60

62

67

37

47

73

Pretest (Fall 2007) Posttest (Spring 2008) Posttest (Spring 2009)

0

Concepts & Problem Solving

# Computation

Math Vocabulary

It is also useful to examine learning gains of enVisionMATH students as evidenced by grade equivalents. The 2nd to 3rd grade cohort of students saw gains of 3.8 grade equivalents in concepts and problem- solving, and 2.5 grade equivalents in computation. The 4th to 5th grade cohort of students saw gains of 3.3 grade equivalents in concepts and problem-solving, and 5.1 grade equivalents in computation. The average growth expected per year is 1 grade equivalent.

^{1}Percentile ranks indicate the percentage of students in the same grade in the norm (reference) groups who took the test at a comparable time and whose scores fall below a student’s score. Since percentile ranks do not represents equal units, and since their interpretation is limited to the reference group from which they were derived, they are best used for reporting scores when position in relation to the reference group is of primary interest.

Figure 4—enVisionMATH Study Performance at Pre- and Posttesting for 2nd to 3rd grade (left) and 4th to 5th grade (right): MAT8

MAT8 Scale Score (Corresponding Grade Equivalent)

670 650 630 610 590 570 550 530 510 490 470

656.5

## GE=6.3

599.8

## GE=3.1

583.7 GE=2.5

Concepts & Problem Solving

MAT8 Scale Score (Corresponding Grade Equivalent)

548.3

560

GE=3.7

540

Pretest (Fall 2007) Posttest (Spring 2008) Posttest (Spring 2009)

700 680 660 640 620 500 580

# Computation

520 500

604.9

## GE=3.7

648.1

## GE=4.8

685.7

689.7

GE=8.8

669.9 GE=9.2

645.4 GE=5.5

GE=7.4

671.7

GE=7.5

621.1

GE=4.1

Concepts & Problem Solving

Computation

Subgroup Results: enVisionMATH Students

Evaluators found that the enVisionMATH program worked just as well with 2nd–3rd, and 4th–5th graders, females and males, white and non-white students, special education and non-special education students, students receiving free/reduced lunch and those not receiving this aid, and students at various math levels. A greater rate of improvement was demonstrated for certain subgroups of enVisionMATH students, including special education, free/reduced lunch eligible, low math ability, and high math ability students. However, enVisionMATH students in all subpopulations showed significant learning gains on all assessment measures.

enVisionMATH vs. Other Math Programs

Evaluators conducted analyses comparing how enVisionMATH students performed in comparison to students using other math programs. Results showed positive effects of the enVisionMATH program. Elementary students who used enVisionMATH over a two-year period showed greater gains in math computation, math problem- solving and communication, and math communication as compared to students who used other math programs. These results can be seen in Figures 4–7.

The gains exhibited by enVisionMATH students as compared to students using other math programs were also noteworthy because the effect sizes obtained were higher than those observed during the first year of the study. The effect sizes for the enVisionMATH program on student math performance ranged from .25 to .46. These effect sizes are higher than those from the first year of the study (.20 to .24), suggesting that stronger effects were evident as students and teachers had more experience with enVisionMATH. Furthermore, last year there were significant differences on the MAT8 Computation (d=.21), GMADE (d=.24), and BAM (d=.20), also in favor of enVisionMATH. With the exception of the GMADE subtest, the consistency in positive effects obtained for the enVisionMATH program over the course of two years lends support to the conclusion that the enVisionMATH program has a positive impact on student performance.

9