Citation Information

Moon, T. R., Callahan, C. M., & Tomlinson, C. A., (2003, April 28). Effects of state testing programs on elementary schools with high concentrations of student poverty-good news or bad news? Current Issues in Education [On-line], 6(8). Available: http://cie.ed.asu.edu/volume6/number8/


Effects of State Testing Programs on Elementary Schools with High Concentrations of Student Poverty-Good News or Bad News?

Tonya R. Moon
University of Virginia

Carolyn M. Callahan
University of Virginia

Carol A. Tomlinson
University of Virginia



Abstract

In school districts across the country, accountability enacted through testing initiatives is beginning to play a more important role in documenting student attainment of high academic standards. Critics of the testing movement have argued that standardized testing is narrowing the curriculum resulting in instructional practices ineffective in promoting meaningful student learning. Using a nationally stratified random sample, public school teachers were surveyed on the effects of state testing programs on their practices. Results indicate that classroom practices were affected, most of these effects afflicted teachers in impoverished schools. Findings suggest that increases in test scores is not necessarily a result of student academic attainment but more of test preparation, calling into question the validity of such outcomes.


Table of Contents


Arrow Up

Introduction

In our continuing quest for better schools and high achieving students, testing has taken center stage. Society has begun to consider good test scores a major goal of schooling. Thus, there is great pressure, particularly in schools with large populations of low-achieving students, to demonstrate academic progress and success through improved test scores. Skeptics have begun to wonder if the effort to raise standards for all students through high-stakes testing initiatives has too steep a price, including a narrowing of the curriculum and a de-emphasis on curricular depth, an abandonment of constructivist-type activities that give meaning to learning, and a curtailment of extracurricular activities.

While standardized tests have become the driver of curriculum in many schools, resulting in less instruction in non-tested areas (Ascher, 1990) as well as less emphasis on higher-order thinking skills, some literature indicates that there is an even stronger influence of tests on the instructional process (Ascher, 1990; Cimbricz, 2002; Dorr-Bremme & Herman, 1986; Haertel, 1989; Darling-Hammond, 2003; Herman & Golan, 1993; Knapp & Associates, 1995; McNeil, 2000; McNeil & Valenzuela, 2000; Orfield & Kornhaber, 2001; Stecher, Barron, Kaganoff, & Goodwin, 1998) in schools with high concentrations of students from improvished environments (Hoffman, Assaf, & Paris, 2001). Schools with high concentrations of poverty, when compared to schools with low-poverty (a) rely more on basal readers and textbook-driven reading curriculum and less on literature and trade books; (b) do more reading aloud than silent reading; and (c) place less emphasis on analytic mathematical concepts using problem solving and word problems, or working on mathematical reasoning (Ascher, 1990; Knapp & Associates, 1995; Lomax, West, Harmon, Viator, & Madaus, 1996; Solomon, Battistich, & Hom, 1996). Evidence also suggests a greater emphasis on drill and practice and the use of worksheets in higher poverty schools (Ascher, 1990). In economically improvished schools teachers report spending more time instructing students directly about test-taking strategies (Herman & Golan, 1993; Monsaas & Engelhard, 1994; Stevens, 1984). Solomon et al. (1996) demonstrated that the greatest differences in terms of instruction between schools with low and high socioeconomic status students were the amount of student activity, interaction, and autonomy allowed and encouraged, all of which are in short supply in high poverty schools.

Lower teacher expectations may also impact the instructional methods used by teachers to prepare economically disadvantaged students. Several studies suggest that teachers tend to have lower expectations for students from improvished backgrounds and they often formulate these expectations before they have significant interaction with students. Teacher suppositions are based on their knowledge of students' test scores, tracking or ability placement, and previous teachers' comments (Gonder, 1991; Lumsden, 1997; Ornstein & Levine, 1989; Pajares, 1992; Solomon, et al., 1996). As a result, teachers are apt to deliver instruction based on a different, simplified curriculum (Ornstein & Levine, 1989), often reinforcing the drill-and-practice of basic skills while ignoring higher-order thinking skills that enable complex and meaningful learning to occur. In essence, teacher expectations of students from impoverished backgrounds devalue the educational opportunities provided to these students.

An increasing emphasis on state testing initiatives over the last five years may have lead to even more negative effects on students from impoverished environments. If increased testing pressures continue to create differential ways by which teachers respond to students in high and low-poverty schools, it can be expected that negative modes of test preparation in schools with high concentrations of student poverty will rematerialize in the near future.

This study sought to address the following question: Do teachers in high-versus low-poverty schools differentiate their instructional practices when faced with high-stakes tests given the population of students they teach?

To understand the actions of others one must consider insider perspectives (Eisenhart & Howe, 1990). In schools, teachers make decisions based on their interpretation of the cultural and societal norms of the school and community environment (Eisenhart & Howe, 1990). As teachers are pressured to produce better test scores, they make specific choices to accomplish this goal for the betterment or determent of sound instructional practices. Are their choices and actions influenced by a school culture related to poverty?


Arrow Up

Methodology

In order to describe the effects of student poverty on teachers instructional practices when faced with high-stakes tests, a 1% nationally stratified random sample of K-12 teachers, stratified on school metropolitan status and poverty level, were sent a survey designed specifically for this study.

Sampling Framework
Market Data Retrieval (MDR) provided teacher-level information for all public elementary schools in the United States. Based on the MDR information, 928,170 elementary teachers (defined as grades K-8) were employed in schools across the nation at the time of the sampling. It is important to note that approximately 15,000 elementary schools housed grades K-8. Because of the large numbers of public school elementary teachers, a 1% stratified random sample from each state at each level was drawn, stratifying on metropolitan status and poverty level indicators (as defined by the U.S. Census Bureau, 1990) associated with each school (n = 8,044 elementary).

The Survey
A 99-item teacher questionnaire was developed and piloted specifically for use in this study. The questionnaire asked about the perceived influence of state testing on curriculum and instruction, the pressure, if any, the teacher felt to improve test scores, the amount of time and attention given to test preparation, the perceived positive and negative effects of standardized testing, the teacher's perceptions of the consequences of testing, and teacher background data including geographic and school level poverty indicators.

Based on the pilot of the survey, principal component analyses were conducted to confirm the intended scales of the survey. Using a factor saliency criterion of +/- 0.50, 59-items were retained, accounting for 64% of the variance. Four factors emerged from the 59-items: Standardized tests influence on class time, teachers attention to tests during instruction, pressure felt to improve test results, and teacher attitudes about state testing. Coefficient alphas for each of the survey scales are presented in Table 1. For a complete description of the final questionnaire, see Appendix 1.

Table 1 - Final State Testing Questionnaire

Subscale No. of Items Range Alpha
STANDARDIZED TESTS INFLUENCE ON CLASS TIME
Class time preparation      
How much time do you spend in your classroom on various test preparation activities? 9 5=Regularly -> 0=None 0.93
How much time do you spend in your classroom on worksheets for test preparation? 3 5=Regularly -> 0=None 0.92
How much time do you spend in your classroom on various test preparation activities AFTER state testing? 4 5=Regularly -> 0=None 0.84
Teacher attention to tests during instruction      
How have state test results affected your instruction? 4 3=Strongly -> 1=Not At All 0.72
How frequently are various nontraditional item formats used in your classroom? 3 1=Yes -> 0=No 0.73
PRESSURE TO IMPROVE TEST SCORES
To what extent do you feel pressure to improve test scores from various groups? 9 5=Extremely High -> 0=No Pressure 0.84
How often during the year does your administration engage in various activities for improving test results? 5 3=Many Times -> 0=Not At All 0.85
What is the potential for various consequences to teachers whose students perform poorly on state tests? 5 1=Yes -> 0=No 0.76
If test scores have changed, how important are various classroom factors? 5 3=Major Factor -> 0=No Factor 0.81
Pressure on students and teachers 5 3=Strongly Agree -> 1=Strongly Disagree 0.82
Emphasis on state test outcomes 2 5=Extremely High -> 0=No Pressure 0.55
Teacher Attitudes About State Testing 5 3=Strongly Agree -> 1=Strongly Disagree 0.77

Questionnaire packets, including a brief study rationale and directions for completing the survey, were mailed to teachers. Teachers were provided postage-paid return envelopes for mailing the questionnaire back to the researchers. Data were received from 1,330 elementary teachers (return rate = 16%1 ).

Sample
The final sample included teachers from all 50 states and the District of Columbia. Approximately 15% of the teachers worked in schools with 0-5.9% poverty (n = 200), 35% worked in schools with 6.0-15.9% poverty (n = 465), 34% worked in schools with 16.0-29.9% poverty (n = 452), and 16% worked in schools with 30% or more poverty (n = 213 ). Seventeen percent of the teachers reported teaching first grade, 16% second grade, 21% third grade, 21% fourth grade, 19% fifth grade, and 6% sixth grade. The average number of years teaching was 16.7 years (sd = 9.6 yrs.).


Arrow Up

Data Analysis

Data were coded by state, metropolitan status, and poverty level of the school. Descriptive statistics were computed at the item level and reported by the categories verified by a factor analysis. One-way analyses of variance were used to examine differences among teachers' responses by school poverty level, adjusting alpha to control for multiple comparisons (Overall alpha was set at .05). Where overall significant differences were found, post hoc comparison procedures were used to determine which poverty level groups were significantly different.


Arrow Up

Findings - Effects of Testing on Teachers

Teachers reported spending substantial time preparing students for state-mandated tests. However, when compared to teachers in low poverty schools, teachers in schools with high concentrations of poverty reported spending more time on direct test preparation activities, such as instruction focused directly on test-taking strategies, reviewing and practicing state released test items, and giving students other practice opportunities on the types of item formats found on the state tests. Comparisons of mean responses for each item across poverty levels indicated, in general, statistical differences in the use of such practices both before and after testing and between teachers in schools with low poverty and schools with high poverty.

Table 2 - Use of Test Preparation Activities Reported by Teachers Across Poverty Levels (Means and Standard Deviations)

Item Available Range*** Poverty Level
A
(lowest)
B C D
(highest)
How much time do you spend in your classroom on the following test preparation activities during the first 1/3 of the year? 0-5; None to Regularly        
    Instruction on test-taking strategies*   2.8 (1.9) 2.8 (1.9) 3.0 (1.9) 3.4
(1.8)
    Review/practice using state-released test items**   2.1 (2.0) 2.1 (2.0) 2.3 (2.1) 2.8
(2.0)
    Student practice in the kinds of item formats that are on state test(s)**   2.8 (2.0) 2.9 (1.9) 3.0 (2.0) 3.6
(1.8)
           
How much time do you spend in your classroom on the following test preparation activities during the second 1/3 of the year? 0-5; None to Regularly        
    Instruction on test-taking strategies**   3.0 (1.8) 3.2 (1.8) 3.3 (1.8) 3.9
(1.7)
    Review/practice using state-released test items**   2.3 (2.0) 2.6 (2.0) 2.7 (2.0) 3.4
(1.9)
    Student practice in the kinds of item formats that are on state test(s)**   2.9 (1.9) 3.2 (1.8) 3.3 (1.9) 4.0
(1.6)
           
How much time do you spend in your classroom on the following test preparation activities during the month prior to state testing? 0-5; None to Regularly        
    Instruction on test-taking strategies**   3.3 (1.9) 3.7 (1.8) 3.8 (1.7) 4.3
(1.5)
    Review/practice using state-released test items*   2.8 (2.10 3.3 (2.0) 3.5 (2.0) 3.9
(1.8)
    Student practice in the kinds of item formats that are on state test(s) *   3.4 (1.9) 3.7 (1.8) 3.9 (1.8) 4.3
(1.5)
           
How much time do you spend in your classroom on the following test preparation activities after state testing? 0-5; None to Regularly        
    Instruction on test-taking strategies*   1.6 (1.9) 1.6 (1.9) 1.9 (2.0) 2.2
(2.2)
    Review/practice using state-released test items*   1.1 (1.7) 1.2 (1.8) 1.4 (1.9) 1.8
(2.0)
    Student practice in the kinds of item formats that are on state test(s)*   1.7 (2.0) 1.9 (2.0) 2.0 (2.0) 2.4
(2.1)

*: A & B group means differ significantly from group D mean at p < .05.
**: A, B, & C group means differ significantly from group D mean at p < .05.
***: 0 = none; 1 = 1 day; 2 = a few days; 3 = one week; 4 = one month; 5 = regularly

Poverty Levels: A (0-5.9%); B (6.0%-15.9%); C (16.0%-29.9%); D (30.0% or greater)

Table 3 displays the percentage of teachers across school poverty levels that reported regularly using the test preparation activities. As can be seen, across all time periods prior to the administration of state test(s), the percentage of teachers reporting the regular use of test preparation activities markedly increased prior to the administration of the test(s) with a sharp decline in use after the state test(s). In general, across all poverty levels, over half of the teachers reported regularly using the test preparation activities specified the month prior to state testing, with over three-fourths of teachers in schools with the highest concentration of poverty reporting regular usage. Surprisingly, after administration of the state test(s) almost one-third of the teachers in schools with high poverty levels reported the continued use of test preparation activities.

Table 3 - Percentage of Teachers Who Report Regular Use of Test Preparation Activities by Poverty Level

Item First Third of Year Second Third of Year Month Prior To Tests After Tests
How much time do you spend in your classroom on the following test preparation activities...
Instruction for students on test-taking strategies
  • 0-5.9% Poverty

  • 6.0-15.9% Poverty

  • 16.0-29.9% Poverty
  • 30% or more
34.5%
33.6%
36.9%
47.7%
35.5%
39.3%
42.6%
60.6%
50.8%
59.5%
63.7%
77.3%
16.8%
17.2%
21.1%
30.5%
Review/practice using state-released test items
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
23.6%
22.2%
27.0%
33.7%
25.6%
29.9%
34.2%
48.6%
43.5%
52.3%
59.8%
68.2%
8.8%
11.6%
14.7%
20.9%
Student practice in the kinds of item formats that are on tests
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
35.2%
35.1%
41.2%
53.4%
35.4%
39.5%
45.5%
64.4%
55.4%
62.8%
67.7%
79.0%
19.0%
20.7%
24.9%
31.8%

Teachers were asked the frequency with which certain non-traditional assessment formats were used in the classrooms (Table 4). Constructed response items, where short essays are required, were the most frequently used format reported across all poverty levels. Long-term projects were reported as the least frequently used format across all poverty levels. Statistically significant differences were found in the usage of long-term projects and performance-type items, with wealthier schools reporting more regular use of such formats than poverty-stricken schools.

Table 4 - Assessment Formats Used In Classrooms (Means and Standard Deviations)

Item Available Range Poverty Level
A
(lowest)
B C D
(highest)
How frequently are the following practices used in your classroom… 1-3; Rarely to Often        
    Use of constructed response items (short essays)   2.3 (.75) 2.2 (.75) 2.2 (.79) 2.1
(.78)
    Use of long-term projects (e.g. research project)*   2.0 (.69) 1.9 (.72) 1.8 (.75) 1.8
(.77)
    Use of performance-type items (e.g. experiment)*   2.2 (.65) 2.1 (.68) 2.1 (.69) 1.9
(.72)

*: A group mean differed significantly from D group mean at p < .05

Poverty Levels: A (0-5.9%); B (6.0%-15.9%); C (16.0%-29.9%); D (30.0% or greater)

The percentage of teachers in each poverty level responding to the frequency of use of the non-traditional assessment formats is presented in Table 5. The large proportion of teachers, across all poverty levels, indicated only occasionally using these assessment formats. Several interesting patterns were also found in the reported percentages. Regardless of poverty level, less than 25% of the teachers reported using constructed-response items or long-term projects often. Performance-type items were reported as the most often used/utilized assessment format, with over one-third of the teachers in schools in the lowest poverty level reporting using the format often. Less than one-fourth of the teachers in schools with the highest poverty level reported using performance-type items often.

Table 5 - Types of Assessment Formats Used Reported by Teachers

Item Often Sometimes Rarely
How frequently are the following practices used in your classroom...
Use of constructed response items (short essays)
  • 0-5.9% Poverty

  • 6.0-15.9% Poverty

  • 16.0-29.9% Poverty
  • 30% or more
17.2%
22.7%
22.8%
24.1%
35.4%
39.0%
34.2%
38.5%
47.4%
38.3%
43.0%
37.4%
Use of long-term projects (e.g., research or other type projects)
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
23.7%
20.0%
21.1%
20.5%
51.5%
47.2%
41.9%
35.8%
24.7%
32.8%
37.0%
43.8%
Use of performance type items (e.g., presentations, experiments)
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
34.4%
27.6%
28.7%
24.0%
53.3%
53.4%
51.3%
49.1%
12.3%
19.1%
20.0%
26.9%

Teachers indicated that their administrators engaged in various activities focused on improving test results several times per year (Table 6). Discussions of ways to improve test scores were the most frequently reported activity across all poverty levels. Significant differences were found between lower poverty schools regarding the types of discussions used to improve test scores, materials being provided to improve test scores, and the methods by which administrators monitored teachers. The schools with the three lowest levels of poverty differed from those with the highest poverty level while teachers from the poorest schools indicated this occurred with greater frequency. Results indicated that administrators in schools with the most poverty subjected teachers to activities to improve test scores consistently and most often. Interestingly, test preparation discussions were not found to be significantly more likely to occur in schools with lower poverty levels.

Table 6 - Pressure Felt from Administration (Means and Standard Deviations)

Item Available Range Poverty Level
A
(lowest)
B C D
(highest)
How often during the year does your administration engage in various activities for improving test results? 0-3; Not At All to Many Times        
    Reviews test scores at staff meetings   1.8 (.81) 1.8 (.75) 1.8 (.82) 2.0
(.78)
    Discusses ways to improve test scores*   2.0 (.90) 2.2 (.86) 2.1 (.96) 2.4
(.79)
    Provides materials to improve test scores*   1.8 (1.0) 1.8 (1.0) 1.9 (1.1) 2.2
(.83)
    Checks to see that teachers are emphasizing areas which showed weakness from past test results*   1.4 (1.2) 1.6 (1.1) 1.6 (1.1) 2.0
(1.6)
    Introduces or discusses important new instructional ideas   1.8 (1.1) 1.8 (1.0) 1.8 (1.8) 2.1
(.93)

*: A, B, & C group means differ significantly from group D mean at p < .05.

Poverty Levels: A (0-5.9%); B (6.0%-15.9%); C (16.0%-29.9%); D (30.0% or greater)

The percentage of teachers indicating that administrators led activities designed to improve test scores is shown in Table 7. For those teachers indicating that the activities occurred many times throughout the school year, a crescendo effect was seen. That is, as the poverty level of the school increased so did the percentage of teachers indicating the activity occurred. For example, approximately 52% of teachers in the highest poverty schools noted that discussions of ways to improve test scores occurred many times during the year compared with the lowest poverty level school where only 32% of the teachers reported the activity occurring many times throughout the year. Another interesting pattern was evident regarding how often administrative leaders checked to see that weak areas from past results were being emphasized. Forty-one percent of the teachers in the lowest poverty schools reported the activity occurring many times throughout the year compared to only approximately 20% of teachers reporting the activity occurring in the lowest poverty schools. In addition, 34% of teachers in the lowest poverty schools reported administrators did not check at all compared to only 16% of teachers in the highest poverty schools. The frequency that administrators introduced or discussed new instructional ideas was fairly consistent across each school poverty level.

Table 7 - Pressure Felt to Improve Test Scores

Item Many Times A Few Times Once Not at All
How often during the year does your administration engage in various activities for improving test results?
Reviews test scores
  • 0-5.9% Poverty

  • 6.0-15.9% Poverty

  • 16.0-29.9% Poverty
  • 30% or more
18.5%
17.2%
21.4%
24.3%
42.5%
51.7%
45.7%
51.9%
34.5%
27.5%
27.7%
19.3%
4.5%
3.5%
5.2%
4.4%
Discusses ways to improve test scores
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
32.2%
37.5%
40.5%
51.9%
45.7%
47.1%
39.2%
37.6%
13.6%
8.1%
9.7%
6.1%
8.5%
7.4%
10.6%
4.4%
Provides materials to improve test scores
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
26.8%
28.0%
31.6%
42.5%
43.4%
44.4%
39.7%
44.1%
13.6%
10.9%
10.4%
7.3%
16.2%
16.6%
18.3%
6.1%
Checks to see that teachers are emphasizing areas which showed weaknesses from past results
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
19.8%
24.5%
24.8%
41.3%
35.5%
38.2%
37.2%
36.9%
10.7%
12.9%
10.8%
5.6%
34.0%
24.5%
27.1%
16.2%
Introduces or discusses important new instructional ideas
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
29.6%
27.1%
29.2%
36.1%
44.4%
45.0%
39.8%
46.1%
6.1%
9.6%
10.8%
6.7%
19.9%
18.3%
20.2%
11.1%

Finally, teachers were asked to respond to the consequences that threatened them as professionals if students in their classrooms performed poorly on the state tests. As seen in Table 8, differences among poverty levels were found per consequence. While teachers indicated all of the consequences might impact them, pressure to change teaching strategies was the most frequently reported consequence across all poverty levels. A larger percentage of teachers in the schools with the highest level of poverty indicated potential loss of job, reassignment, or reprimand as consequences of poor student performance. The largest percentage of teachers who reported no consequences for posting poor test scores taught in schools with the least reported poverty levels (Table 9).

Table 8 - Consequences to Poor Student Performance on State Test(s) (Means and Standard Deviations)

Item Available Range Poverty Level
A
(lowest)
B C D
(highest)
What are the potential consequences to teachers whose students perform poorly on state test(s) in your school? 0-1; No - Yes        
    Potential loss of position*   0.13 (0.33) 0.18 (0.39) 0.20 (0.40) 0.26 (0.44)
    Reassignment of grade level or type of student taught**, ***   0.20 (0.40) 0.30 (0.46) 0.35 (0.48) 0.43 (0.50)
    Private reprimand*   0.24 (0.43) 0.30 (0.46) 0.32 (0.47) 0.38 (0.49)
    Pressure to change teaching strategies****   0.51 (0.50) 0.66 (0.48) 0.66 (0.48) 0.72 (0.45)
    No consequences****   0.35 (0.48) 0.24 (0.43) 0.23 (0.42) 0.14 (0.35)

*: A group mean differed significantly from D group mean at p < .05
**: A and B group means differed significantly from D group mean at p < .05
***: A group mean differed significantly from C group mean at p < .05
***: A group mean differed significantly from B, C, & D group means at p < .05

Poverty Levels: A (0-5.9%); B (6.0%-15.9%); C (16.0%-29.9%); D (30.0% or greater)


Table 9 - Types of Consequences Resulting From Poor Student Performance on State Tests Reported by Teachers across Different Poverty Levels

Item Yes
What are the potential consequences to teachers whose students perform poorly on state test(s) in your school?
Potential loss of position
  • 0-5.9% Poverty

  • 6.0-15.9% Poverty

  • 16.0-29.9% Poverty
  • 30% or more
12.5%
18.0%
19.8%
26.1%
Reassignment of grade level or type of student taught
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
19.5%
29.9%
35.4%
42.8%
Private reprimand
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
24.0%
30.3%
32.4%
37.8%
Pressure to change teaching strategies
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
28.6%
34.5%
23.6%
23.4%
No consequences
  • 0-5.9% Poverty
  • 6.0-15.9% Poverty
  • 16.0-29.9% Poverty
  • 30% or more
34.5%
23.6%
23.4%
13.9%


Arrow Up

Summary and Conclusions

These survey findings support previous studies' findings in that accountability, as implemented through testing initiatives, has tremendous effects on teachers and the teaching and learning process. Substantial amounts of time are spent in special test preparation activities prior to and after state test(s), administrators encourage attention to test scores by engaging in activities with teachers designed specifically to increase test scores (e.g., discussion of and provision of materials to improve test scores), and whether real or perceived, teachers associate potential negative consequences with poor student test performance (e.g., reassignment, loss of job). All of these effects are evidenced more prominently in schools with students from improvished backgrounds.

Whether the study findings are good news or bad news depends on one's philosophical ideas about testing initiatives. If one holds the view that the test(s) and what they are measuring are educationally valid then certainly these results are good news that testing initiatives are tools through which educational reform can be obtained. If, however, one holds the opposite view, that an increased emphasis on testing decreases the likelihood of meaningful systemic reform, then the results from this study indicate that testing initiatives negatively impact the instructional process by which children learn, distort the curriculum, and replace effective instructional practices by focusing on test preparation.

Several conclusions can be drawn based on this study's findings:

  1. The use of tests as an accountability mechanism appears to define the curriculum, oftentimes with substantial attention given to the form and format of the questions on the test(s). Regardless of the school's poverty level, a great deal of class time is devoted to reviewing and practicing for state tests, increasing in intensity until testing is completed. However, a larger percentage of teachers in schools with the highest concentration of poverty appear to maintain a more consistent focus on test preparation throughout the school year than teachers in schools with lower poverty levels.
  2. Teachers presume high-stakes decisions are associated with test results, and consequently align their instructional practices with state tests. One risk to this approach, which cannot be clearly determined from this study, is the de-emphasis of non-tested materials. This may narrow the curriculum and result in practices that may counter effective instruction, teaching for student understanding, self-direction and autonomy, and opportunities for interaction with other students.
  3. Because of the potential narrowing of curriculum and the tremendous amount of time devoted to test preparation activities, the importance of ensuring the validity of a testing initiative's ability to serve as an accurate indicator of student achievement and learning is paramount.

Are schools producing individuals who are ready to live in and make contributions to the complex society of today or are they producing individuals who excel at taking a test? We need to heed an observation made by E.G.A. Holmes in the first part of the 20th century in Britain:

...And in proportion as we tend to value the results of education for their measurableness, so we tend to undervalue and at last to ignore those results which are too intrinsically valuable to be measured. (Holmes, 1911, p. 128)

Unfortunately, in the minds of the teachers who responded to the survey, high test scores appear to have become a primary criterion against which an educational system's worth is judged. Students in poverty, who are most in need of developing high level skills which could prepare them for future careers, educational advancement, or opportunities needed to overcome the circumstances from which they come, are the most likely to be exposed to increased levels of test preparation activities.

While some will argue that the testing initiative's purpose is to ensure educational equity for all American students, regardless of economical circumstances, the reverse appears to be the reality. Based on this study, students from poverty are less likely to be exposed to challenging curricula and instructional methods. Results from this study would suggest that accountability through student testing is a vehicle to restrict educational opportunities from those who need opportunities most.


Arrow Up

Authors

Tonya R. Moon, Ph.D. is an assistant professor in the Curry School of Education at the University of Virginia and a principal investigator for the National Research Center on the Gifted and Talented. Her areas of expertise include educational measurement and assessment. Her professional publications include refereed journal articles, book chapters and monographs. She works both nationally and internationally with educators on appropriate use of assessment techniques for the improvement of student learning. Dr. Moon can be reached at University of Virginia, Curry School of Education, P.O. Box 400277, 405 Emmet Street S, Charlottesville, VA 22904-4277.
F: 434.924.1384. O: 434.924.0823
She may be contacted via e-mail at trm2k@virginia.edu.

Carolyn M. Callahan, Ph.D. is a professor in the Curry School of Education at the University of Virginia and is also associate director of the National Research Center on the Gifted and Talented. She teaches courses in the area of education of gifted and has authored more than 100 articles and 25 book chapters and monographs on the topics of creativity, the identification of gifted students, program evaluation, and the issues faced by gifted females.

Carol A. Tomlinson, Ed.D. is a professor in the Curry School of Education at the University of Virginia and a principal investigator for the National Research Center on the Gifted and Talented. She currently is president of the National Association for Gifted Children and has authored many journal articles, book chapters, and staff development materials. She works with educators nationally and internationally on issues related to teaching academically diverse students.


Arrow Up

End Notes

1. According to Krejcie and Morgan (1970), based on a population over 75,000 teachers, 382 returned surveys were needed for a representative sample.


Arrow Up

References

Ascher, C. (1990). Testing students in urban schools: Current problems and new directions. (Urban Diversity Series No. 100). New York: ERIC Clearinghouse on Urban Education. (ERIC Document Reproduction Service No. ED322283)

Cimbricz, S. (2002, January 9). State-mandated testing and teachers' beliefs and practice. Education Policy Analysis Archieves, 10(2). Retrieved February 28, 2003 from http://epaa.asu.edu/epaa/v10n2.html.

Darling-Hammond, L. (2003, February 16). Standards and assessments: Where we are and what we need. Teachers College Press. Retrieved February 28, 2003 from http://www.tcrecord.org.

Dorr-Bremme, D. W., & Herman, J. L. (1986). Assessing student achievement: A profile of classroom practices. (CSE Monograph Series in Evaluation No. 11). Los Angeles, CA: UCLA Graduate School of Education, Center for the Study of Evaluation.

Eisenhart, M. A., & Howe, K. R. (1990, April). Validity in qualitative research. Paper presented at the annual meeting of the American Educational Research Association, Boston, MA.

Gonder, P. O. (1991). Caught in the middle: How to unleash the potential of average students. Arlington, VA: American Association of School Administrators.

Haertel, W. M. (1989). Making sense of school testing. In B. R. Gifford (Ed.), Test policy and test performance: Education, language and culture. Boston: Kluwer Academic.

Herman, J. L., & Golan, S. (1993). The effects of standardized testing on teaching and schools. Educational Measurement: Issues and Practice, 12(4), 20-25, 41-42.

Hoffman, J. V., Assaf, L. C., & Paris, S. C. (2001). High-stakes testing in reading: Today in Texas, tomorrow? Reading Teacher, 54 (5), 482-92.

Holmes, E. G. A. (1911). What is and what might be: A study in education in general and elementary in particular. London: Constable.

Knapp, M. S. & Associates. (1995). Teaching for meaning in high-poverty classrooms. New York: Teachers College.

Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30 (3), 607-610.

Lomax, R. G., West, M. M., Harmon, M. C., Viator, K. A., & Madaus, G. F. (1996). The impact of mandated standardized testing on minority students. Journal of Negro Education, 64 (2), 171-185.

Lumsden, L. (1997). Expectations for students. Washington, D.C.: U. S. Department of Education, Office of Educational Research and Improvement. (ERIC Document Reproduction No. ED409609)

McNeil, L. (2000). Contradictions of school reform: Educational costs of standardized testing. New York: Routledge.

McNeil, L., & Valenzuela, A. (2000). The harmful impact of the TAAS system of testing in Texas: Beneath the accountability rhetoric. Retrieved July 2, 2001 from http://www.law.harvard.edu/groups/civilrights/conferences/testing98/drafts/

Monsaas, J. A., & Engelhard, G., Jr. (1994). Teachers' attitudes toward testing practices. The Journal of Psychology, 128, 469-477.

Orfield, G., & Kornhaber, M. L. (2001). Raising standards or raising barriers? Inequality and high-stakes testing in public education. New York: The Century Foundation Press.

Ornstein, A. C., & Levine, D. U. (1989). Social class, race, and school achievement: Problems prospects. Journal of Teacher Education, 40(5), 17-23.

Pajares, M. F. (1992). Teachers' beliefs and educational research: Clearning up a messy construct. Review of Educational Research, 62, 307-332.

Solomon, D., Battistich, V. & Hom, A. (1996). Teacher beliefs and practices in schools serving communities that differ in socioeconomic level. The Journal of Experimental Education, 64, 327-347.

Stecher, B. M., Barron, S., Kaganoff, T., & Goodwin, J. (1998). The effects of standards-based assessment on classroom practices: Results of the 1996-97 RAND survey of Kentucky teachers of mathematics and writing (CSE Technical Report 482). Los Angeles: University of Southern California at Los Angeles, Center for Research on Evaluation, Standards, and Student Testing.

Stevens, F. I. (1984). The effects of testing on teaching and curriculum in a large urban school district. (ERIC/TM Report 86). Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, & Evaluation. (ERIC Reproduction Service No. ED252581)

U.S. Bureau of the Census. (1990). Statistical abstracts of the United States: 1990. Washington, D.C.: Government Printing Office.


Arrow Up

Appendix 1

ELEMENTARY TEACHER QUESTIONNAIRE
State Testing Program

1. In what state do you teach? _________________

2. Grade(s) you currently teach:    K   1   2   3   4   5   6   7   8

3. Grade(s) you taught last year:    K   1   2   3   4   5   6   7   8

4. Years you've been teaching (including this year): ________ years

5. Years you've been teaching at this school (including this year): ________ years

6. Content areas you are responsible for teaching:

  • English / Language Arts
  • Mathematics
  • History / Social Studies
  • Science
  • Other: _________

7. Please indicate your estimate of your class's academic achievement/ability level.

Much above grade level Somewhat above grade level At grade level Somewhat below grade level Much below grade level

8. Approximately what percentage of your current students:

a. qualify or receive free/reduced lunch: ______%
b. are limited- or non-English proficient: _____%
c. receive special education services: _____%
d. are considered gifted/academically talented: _____%

9. How would you characterize the emphasis on the outcomes of state-level tests in your school during the past year?

_____ extremely high    _____ very high    _____ moderate
_____ low    _____ no emphasis

10. How has the emphasis on outcomes of state-level tests changed over the past 3 years?

  • increased greatly
  • increased slightly
  • remained about the same
  • decreased slightly
  • decreased greatly

11. To what extent to you feel pressure from the following groups to improve your students' standardized test scores?

  Extremely High Very High Moderate Little No Pressure
a. my principal 5 4 3 2 1
b. central office administrators 5 4 3 2 1
c. other teachers 5 4 3 2 1
d. local school board members 5 4 3 2 1
e. parents 5 4 3 2 1
f. local community including the press/media 5 4 3 2 1
g. professional organizations 5 4 3 2 1


12. Rank the order of the effectiveness of the following curriculum and instructional approaches for increasing student learning. (Assign a ranking of 1 to the most effective, 2 to the next effective, and so on. Please do not use the same ranking for any items.)

RANK

_____   Focusing clearly and consistently on the topics covered on the state test(s)
_____   Using an interdisciplinary curriculum
_____   Differentiating the curriculum (e.g. using tiered assignments, lessons targeted to student past achievements, accelerating pace of learning for some students)
_____   Doing hands-on work
_____   Using alternative (e.g., performance) assessments
_____   Providing broad coverage across many areas of basic knowledge
_____   Teaching which allows for in-depth exploration where one critical exemplar of a concept or principle can be understood as a basis to generalize to other exemplars.


13. How would you rank the order of the degree to which you balance the following curricula and instructional approaches as effective in increasing scores on state tests? (Assign a ranking of 1 to the most effective, 2 to the next effective, and so on. Please do not use the same ranking for any items.)

_____   Focusing clearly and consistently on the topics covered on the state test(s)
_____   Using an interdisciplinary curriculum
_____   Differentiating the curriculum (e.g. using tiered assignments, lessons targeted to student past achievements, accelerating pace of learning for some students)
_____   Doing hands-on work
_____   Using alternative (e.g., performance) assessments
_____   Providing broad coverage across many areas of basic knowledge
_____   Teaching which allows for in-depth exploration where one critical exemplar of a concept or principle can be understood as a basis to generalize to other exemplars.

For question 14 we would like you to consider each item TWICE. First, we ask you to RATE each item according to the scale noted. Second, we ask you to RANK the relative influence of each item. Assign a ranking of 1 to the most influential, 2 to the next most influential and so on. Please do not use the same ranking for any items.

14. To what extent do the following factors influence your instructional planning?

  Strongly To some extent Not at all RANKING
a. having academically advanced students in your class        
b. having struggling learners in your class        
c. having to prepare students for state test(s)        
d. making sure the content and skills covered on the state test(s) are reviewed prior to the test(s) administration        
e. my current students' most recent state test(s) results        
f. our school's overall test results        
g. test performance of the students I had last year        
h. a need for adjusting curriculum sequence based on the content coverage of the state test(s)        


15. How frequently are the following practices used in your classroom AND to what extent have their frequency changed as a result of the state testing program? (DK = Don't Know)

  FREQUENCY CHANGE
  Often Sometimes Rarely Increased Same Decreased DK
a. use of constructed response items (short essays)              
b. use of multiple-choice items              
c. use of long term projects (e.g., research or other projects requiring a week or more)              
d. use of performance type items (e.g., presentations, science experiments)              
e. test preparation (e.g., homework and classwork)              
f. making sure the content and skills covered on the state test(s) are reviewed prior to the test(s) administration              
g. adjusting the curriculum sequence based on the content coverage of the state test(s)              


16. How much attention are you able to give to the following curricular areas in your classroom AND to what extent has this changed as a result of the state testing program? (DK = Don't Know)

  ATTENTION CHANGE
  Often Sometimes Rarely Increased Same Decreased DK
a. higher-order thinking skills              
b. problem-solving skills              
c. topics which are not assessed on the state tests(s)              
d. the fine and performing arts (e.g. music, art, drama)              
e. basic skills (e.g., computations, grammar, vocabulary)              
f. factual knowledge              
g. enrichment or extension of the curriculum according to student interest and/or ability to delve deeper              


17. How much time do you spend in your classroom on the following test preparation activities...

... during the first 1/3 of the year?

  Regularly One
month
One
week
A few
days
One
day
None
a. student worksheets            
b. instruction for students on test-taking strategies            
c. review/practice using state-released test items            
d. student practice in the kinds of item formats that are on the state test(s)            


... during the second 1/3 of the year?

  Regularly One
month
One
week
A few
days
One
day
None
a. student worksheets            
b. instruction for students on test-taking strategies            
c. review/practice using state-released test items            
d. student practice in the kinds of item formats that are on the state test(s)            


... during the month prior to state testing?

  Regularly One
week
A few
days
One
day
None
a. student worksheets          
b. instruction for students on test-taking strategies          
c. review/practice using state-released test items          
d. student practice in the kinds of item formats that are on the state test(s)          


... during the period following state testing?

  Regularly One
month
One
week
A few
days
One
day
None
a. student worksheets            
b. instruction for students on test-taking strategies            
c. review/practice using state-released test items            
d. student practice in the kinds of item formats that are on the state test(s)            


18. How often during the year does your school administration engage in the following activities with teachers?

  Many times A few times Once Not at all
a. Reviews test scores at staff meetings        
b. Discusses ways to improve test scores        
c. Provides materials to improve test scores        
d. Checks to see that teachers are emphasizing areas which showed weakness from past test results        
e. Introduces or discusses important new instructional ideas        

19. How much of the feedback or evaluation you receive as a teacher from administrators is related to the test performance of your students? (check only one)

  • 100%
  • Three-quarters
  • About half
  • About one-quarter
  • Practically none
  • None at all
  • I never receive feedback

20. How has your school's performance on the state test(s) changed over the last three years? (check only one)

  • scores have increased
  • scores have decreased
  • scores have remained about the same (Go to question 22)
  • some grade level scores have increased while others have decreased
  • I don't know (Go to question 22)

21. If test scores have changed, how important are the following factors?

  Major Factor Moderate Factor Minor Factor No Factor
a. Changes in school population        
b. Alignment of instruction with state test content        
c. Attention to test-taking strategies        
d. Changes in instructional strategies        
e. Changes in assessment strategies        
f. Changes in teacher effectiveness        
g. Changes in school climate        
h. Changes in funding support        

22. How have state test results affected your instruction? (check all that apply)

  • I teach to the state test(s) more than I normally would.
  • I omit certain information because there is not enough time to fit it in because of state test(s).
  • I do not do certain things that look interesting or beneficial for students unless they are on the state tests.
  • I do not do anything differently because of the state tests.


23. Read each item and indicate your degree of agreement with it.

  Strongly Agree Neutral Strongly Disagree
a. State tests help clarify and specify learning goals      
b. Learning outcomes measured by state test(s) are the most important ones to measure      
c. Test results are an accurate picture of student learning      
d. Many of the students I teach are not capable of learning the material on the state test(s)      
e. State testing is helping schools improve      
f. My school is more interested in increasing student test scores than in improving overall student learning      
g. Teachers in my school feel there is discrepancy between what they think should be taught and what the state tests emphasize      
h. State tests give me important feedback about how well I am teaching the curricular area(s)      
i. Students are treated as test-takers rather than learners      
j. Students are under too much pressure to increase test scores      
k. Students see learning as a chore because of pressure from state testing programs      
l. Students feel badly if they do not have high test scores      
m. Teachers are under too much pressure to increase test scores      


24. What are the potential consequences to teachers whose students perform poorly on state test(s) in your school? (Check all that apply.)

  • Potential loss of position as a teacher in this school or school district
  • Reassignment of grade level or type of student taught
  • Private reprimand
  • Pressure to change teaching strategies
  • Other (please describe): ___________________________________________
  • No consequences

If you have any additional comments on the ways state testing is helping or hurting your instructional practices, please use the space below.




Arrow Up