The Honest Truth About Testing Bias and Manipulation

What is testing bias? Testing bias is real. Let’s discuss how tests can intentionally impact portions of our population.

What is Testing Bias?

Testing bias occurs when the process of testing or evaluation results in inaccurate or unfair results. This doesn’t have to be done with intention. Sometimes, when teachers are under pressure to make sure students understand one specific skill for the test, that’s all they have time to teach. Students perform better on that specific question type but might not understand anything that question connects to or any variation of the question type. That’s because student’s might not actually understand the question but only memorized the information.

I’ve seen this situation happen many times over, especially in areas of need.

What is Test Manipulation?

Test manipulation occurs when the process of testing or evaluation are skewed because of outside interference or because students were given material specific to the test.

Standardized tests scores have been the cornerstone of education evaluation for years, but is it possible that are biased? The answer is yes, and the implications are staggering.

1. Testing Bias: Teaching to the Test

In college I was in a debate class and one of the debates was on this idea of “teaching to the test”. I was about 25 at the time and had never really thought about the idea before.

What is “Teaching to the Test,” and Is it Bad?

“Teaching to the test,” means educators prioritize focus solely on material included on standardized tests. I would say teaching to the test is pretty bad. Let me explain.

When teachers have to teach to the test, we prioritize teaching the specific content and skills on the test, sacrificing a well-rounded education in the process. In 2018, the National Center for Education Statistics found that 55% of teachers felt pressure to teach to the test.

By focusing only on what is on the test, students may score well, but they may lack a deeper understanding of the subject matter. It is an unfortunate reality in many public schools. I know a teacher who had to put down an essay in the middle of final edits to begin 5-week intensive test prep (teaching to the test).

I think most adults understand that cramming generally results in a lack of retention. Students will likely soon forget the material as soon as testing is over. In addition, this type of teaching results in content being covered in “satellite” form, without context or connection to other material. Some students who may do well otherwise, do worse because they become disengaged.

2. Testing Bias: Teachers Feel Compelled to Cheat

Cheating is another way schools may manipulate test results. In 2019, the Georgia Department of Education investigated allegations of cheating on standardized tests in 53 schools. Teachers were found to have erased students’ incorrect answers and replaced them with correct ones, giving the impression of higher test scores.

This practice is not only dishonest but undermines the integrity of the entire education system. Thankfully, as standardized testing becomes digitized, it will be more difficult for one teacher to manipulate scores for their students. I guess it’s also possible we end up with larger scale cheating.

3. Testing Bias: Culturally Biased Questions

Income Disparities

The socio-economic status of students also influences test scores. A study by the National Assessment of Educational Progress found that in 2019, students from low-income families scored an average of 20 to 25 points lower on standardized tests than their more affluent peers.

I think I belabor this a lot but students from low-income families do not have access to the same resources as their peers. It is more difficult to attract high quality teachers to low income schools.

Many of my students face social and economic challenges that negatively impact their academic performance. In my middle school, students live with substance abuse, incarcerated parents, homelessness, physical/sexual/emotional abuse, and extreme poverty.

They have had little or no exposure to the types of situations within the stories or situations of the test. For example, I gave my middle school students an article on arctic foxes and they did not know where Alaska was or that it was cold. Having little context or reference point makes the test exceptionally difficult because they are trying to decipher more than just the question. This was middle school.

Race, ethnicity, and culture also impact test scores. It is difficult to explain to native English speakers, but immigrants and first-generation students have no reference point for much of the testing language. Many of my students can only operate in the most casual language register.

Students with Disabilities

I don’t think we talk about this enough. Students with disabilities get left behind in standardized testing. Testing bias affects them, too. Students with disabilities often have a different life experiences than other students and have different cultural reference points. Testing bias assumes students have the same type of shared experiences, and many of my Special Education students do not.

In addition, Special Education students require additional support. For example, one of my students required additional time and silence. They were placed in a separate room with different teachers. Unfortunately, without the people they knew, they rushed through the test and incorrectly answer questions just to exit the location.

Immigrants and First-Generation Students

Testing bias may not accurately reflect the knowledge and experiences of students from diverse backgrounds. People from other countries also have different life experiences. Culturally specific questions limit accessibility to the test.

While there are a lot of people in America who don’t think this is a huge problem, I would like to counter that our shared experiences are what makes America great. Differences elevate us, so we should create curriculum and tests that accurately reflect our diversity.

What types of questions are the most challenging?

Examples of testing bias include:

1. Questions that Assume Knowledge American Pop Culture

These include references to music, actors, or movies.

2. Questions that use language or references not commonly used in African American or Hispanic communities. These include colloquialisms and idioms.

3. Questions that assume a certain level of socio-economic status or privilege, such as referencing expensive activities or materials that may be less accessible to students from low-income families.

Testing bias affects my students. My students from low-income areas do not travel. My students live about 15 miles from the beach, and many have not seen the ocean. They have not seen snow. It is beyond their means.

Therefore, asking students to make inferences or assumptions about things they only barely comprehend is difficult for them. That doesn’t even address issues around poverty and language.

What Can We Do About Test Manipulation?

1. Diversify Assessment Methods:

Standardized tests often focus on multiple-choice questions, which may not effectively capture a student’s true abilities. By incorporating a variety of assessment methods, such as essays, presentations, or practical demonstrations, we can create a more comprehensive evaluation of students’ knowledge, critical thinking skills, and creativity.

2. Reduce High-Stakes Nature:

The high-stakes nature of standardized testing can create undue stress and pressure on students, potentially hindering their performance. Balancing the importance of standardized tests with other factors, such as class grades, projects, or portfolios, can provide a more holistic evaluation of a student’s capabilities.

3. Ensure Test Content Relevance:

Standardized tests should align closely with the curriculum, ensuring that they assess what students are expected to learn. Regular reviews and updates to test content can help address any discrepancies and maintain test relevancy.

4. Enhance Test Accessibility:

Efforts should be made to make standardized tests more accessible to all students, including those with disabilities or different learning needs. This may involve providing accommodations such alternative dates, formats, or assistive technologies to ensure equal opportunities for all test takers.

5. Promote Teacher and Student Engagement:

Encouraging teacher and student involvement in the test development process can help improve the quality and effectiveness of standardized tests. Collaborative efforts can lead to better alignment between teaching strategies and assessment methods, as well as increased buy-in and motivation from students.

Conclusion

I understand the need to evaluate student learning and also teaching methods, but I also think we need to do right by all of our students. Now that I am working in a low income area, it’s become glaringly obvious that my students are not set up for success.

Moreover, it can perpetuate social and economic inequalities by reinforcing the advantages of those who are already privileged. Results can also reinforce negative stereotypes about students who perform poorly.

Standardized tests measure student achievement and school performance when done correctly. However, we must ensure we create tests that allow all students to demonstrate their knowledge. As a society, we should address these issues and create a more equitable and comprehensive education system.