Global perspectives on educational testing : examining fairness, high-stakes and policy reform / Keena Arbuthnot.

Format:

Book

Author/Creator:

Arbuthnot, Keena, author.

Series:

Advances in education in diverse communities ; 1479-258X 13.

Advances in education in diverse communities: research, policy and praxis, 1479-358X ; v. 13

Language:

English

Subjects (All):

Educational tests and measurements--Evaluation.

Educational tests and measurements.

Fairness.

Multicultural education.

Physical Description:

1 online resource (210 pages).

Edition:

First edition.

Place of Publication:

Bingley : Emerald Publishing, 2017.

Summary:

This book provides a refined definition of standardized educational test fairness that can be utilized in multiple contexts to better understand the experiences and perspectives of diverse groups of test takers. Globally, there has been a significant influx in the use of and dependence on standardized tests to foster educational improvements. Standardized testing programs such as TIMSS (Trends in International Mathematics and Science Study), are designed to provide information about a country's global standing in multiple academic areas, ranking individuals and/or groups relative to the performance of others. The high-stakes nature of standardized tests has increased public concern and interest in issues related to test fairness. Since standardized test performance has a profound influence on multiple aspects of educational systems, it is imperative to better understand and examine issues of fairness.

Contents:

Intro

Global Perspectives on Educational Testing: Examining Fairness, High-Stakes and Policy Reform

Contents

List of Tables

Acknowledgments

Introduction

Chapter 1: International Assessments and Fairness Issues

TIMSS Assessment

International Assessments and Policy

Benchmarking Trends

Concerns and Shortcoming of Utilizing International Assessments

Test Fairness

Standards for Educational and Psychological Testing

Fairness in Treatment During the Testing Process

Fairness as Lack of Measurement Bias

Item Bias

Differential Item Functioning. One way to examine fairness issues is to identify items that seem to be biased toward one group in comparison with another. Traditionally, experts conduct differential item function (DIF) analyses to examine item bias. Doran

Fairness in Access to the Construct as Measured

Fairness as Validity of Individual Test Score Interpretations for the Intended Uses

Threats to Fair and Valid Interpretation of Test Scores

Test Content

Test Context

Test Response

Opportunity to Learn

Conclusion

Chapter 2: The Arbuthnot Assessment Fairness Framework

Four Phases of the TAAF Framework

Phase I: Country Selection

Phase II: Test and Subtest Performance

Test and Subtest

Test Wiseness

Phase III: Item-Level Performance

Identifying Item Bias

Sources of Item Bias

Contextual. The first, and typically the most common explanation of differences in item-level performance, is related to the context of the test item. In mathematics, this is typically the case when the item that is being analyzed is a word problem. Mathe.

Opportunity to Learn. Consequently, differences in curriculum coverage among countries is another factor that should be explored and examined as a reason or source of item-level differences. Curricular variations, particularly differences that are associa

Test Speededness. Test speededness, or the rate at which test takers finish an examination, should be investigated as a potential source of item-level differences between countries. Research has shown that certain groups of test takers (i.e., minorities i

Strategy Formation and Usage. The fourth area to explore in explaining item-level differences is related to variations in strategy formation and usage. Particularly in mathematics assessments, research has shown that multiple strategies can be used to ans

Distractor Choices. In an effort to understand test-taking behavior more thoroughly, and particularly at the item-level, it is important to understand how different groups choose distractors on multiple-choice items. To examine differences in distractor c

Omit Patterns. Another way to explore sources or explanations for the differences in item-level performance for different countries is to investigate the variations in test takers' omit patterns. Specifically, patterns of omission that deviate from the n

Phase IV: Cultural Impact

Cultural Context

Chapter 3: TAAF Framework Phase I: Country Selection

Country Selection

Criteria for Selection

Selected Countries

Educational History and Reform Efforts

Chinese Taipei. Chinese Taipei has been lauded as one of the most successful countries on many international assessments, especially in the area of mathematics and science. As previously discussed, Chinese Taipei is one of the many Asian countries that ha.

Finland. The Finnish educational system was included in this book because it has been praised as one of the most successful educational systems in the world. The Finnish have maintained high test performance on international assessments for several years.

United States. In the 1960s there was grave concern about the educational system in the United States. Specifically, these concerns were directly related to the United States' aspirations to be a world leader in technology and innovation. These apprehensi

Qatar. Before the beginning of the 21st century, Qatar's educational system was described as lacking vision and crippled by several challenges (Romanowski, Cherif, Al Ammari, &amp

Al Attiyah, 2013). This educational system was described as highly centralized

Factors Impacting Performance

Measures

Analysis

Results

Home and Background Factors

Teacher Factors

School and Instructional Practices

Student-Centered Factors

Assessments

Country Distinctions and Similarities

Country Distinctions

Country Similarities

Chaper 4: TAAF Framework Phase II: Test and Subtest Performance

Study I: Performance Patterns

Magnitude of the Differences between Countries

Step 1. Calculating Differences in p-Value. For each item, the difference in p-values between each pair of countries was calculated. The pairs of countries included: Chinese Taipei vs. Finland, Chinese Taipei vs. United States, Chinese Taipei vs. Qatar, F

Step 2. Calculating the Magnitude of the Difference in p-Values. The absolute value of the difference in p-values for each item was reported for each pair of countries.

Step 3. Calculating the Average Magnitude. For each pair of countries, the average of magnitude of the difference in p-values (see step 2) was calculated for each of the test items.

Step 4. Categorizing the Magnitude of Difference. For each pair of countries, those items that had a magnitude of the difference in p-values that was less than the average magnitude was categorized as small, while those items that had a magnitude of the d

Performance Differences

Magnitude of Differences

Chinese Taipei vs. Finland. When comparing the TIMSS mathematics assessment performance of 4th grade Taiwanese students in comparison with students from Finland, the results show that Taiwanese students outperformed Finnish students on a majority of the t

Chinese Taipei vs. United States. When comparing the TIMSS mathematics assessment performance of 4th grade Taiwanese students in comparison with students from the United States, the results show that Taiwanese students outperform American students on a la

Chinese Taipei vs. Qatar. When comparing the TIMSS mathematics assessment performance of 4th grade Taiwanese students in comparison with students from Qatar, the results show that Taiwanese students outperform Qatari students on all of the test items. The

Finland vs. United States. When comparing the TIMSS mathematics assessment performance of 4th grade Finnish students in comparison with students from the United States, the results show that Finnish students outperform American students on 56% of the test

Finland vs. Qatar. When comparing the TIMSS mathematics assessment performance of 4th grade Finnish students in comparison with students from Qatar, the results show that Finnish students outperform Qatari students on a large majority of the test items. S

United States vs. Qatar. When comparing the TIMSS mathematics assessment performance of 4th grade American students in comparison with students from Qatar, American students outperform Qatari students on all of the test items. The average difference in pe.

Chinese Taipei vs. Finland. When comparing the TIMSS mathematics assessment performance of 8th grade Taiwanese students in comparison with students from Finland, Taiwanese students outperform Finnish students on a large majority of the test items. The Fin

Chinese Taipei vs. United States. When comparing the TIMSS mathematics assessment performance of 8th grade Taiwanese students in comparison with students from the United States, the results showed that Taiwanese students outperform American students on a

Chinese Taipei vs. Qatar. When comparing the TIMSS mathematics assessment performance of 8th grade Taiwanese students in comparison with students from Qatar, Taiwanese students outperform Qatari students on all of the test items. The average difference in

Finland vs. United States. When comparing the TIMSS mathematics assessment performance of 8th grade Finnish students in comparison with students from the United States, Finnish students outperform American students on 55% of the test items, while American

Finland vs. Qatar. When comparing the TIMSS mathematics assessment performance of 8th grade Finnish students in comparison with students from Qatar, Finnish students outperform Qatari students on a large majority of the test items. Specifically, Qatari st

United States vs. Qatar. When comparing the TIMSS mathematics assessment performance of 8th grade American students in comparison with students from Qatar, American students outperform Qatari students on nearly all of the test items. The Qatari students o

Study II: Test-Wiseness Patterns

Test speededness

Study III: Opportunity to Learn

Chapter 5: TAAF Framework Phase III: Item-Level Performance

Identifying Item Bias.

Differentially Easier Test Items.

Notes:

Includes index.

Includes bibliographical references.

Print version record

ISBN:

9781787148963

1787148963

9781786354334

1786354330