How to Evaluate and Review a Test: Reflections of a Long-Time Mental Measurements Yearbook Reviewer.

Format:

Video

Series:

Academic Video Online

Evaluating Test Quality: Advice From Mental Measurements Yearbook Reviewers

Language:

English

Physical Description:

1 online resource (81 minutes)

Place of Publication:

[Place of publication not identified] : Buros Center for Testing, [date of publication not identified]

System Details:

video file

Summary:

This introductory-level webinar outlines thought processes that might be involved in completing a Mental Measurements Yearbook review of a test. Some parts of the review process present little difficulty. Other parts entail considerable challenge. Identifying the test's purpose and describing the test's materials and processes usually present little difficulty. Perhaps surprisingly, evaluating the standardization (the norms) is often straightforward. Four aspects of the review process frequently present special challenges. First, many test manuals present an overwhelming amount of data on reliability and validity. The reviewer must sort through all this information and formulate overall conclusions that can be conveyed in relatively succinct language. Second, many tests provide a host of scores. An important, and sometimes tricky, part of the review process is spotting all the scores (and suggested interpretations) and determining if adequate technical information is available for all of them. Third, fairness has emerged as a major concern, now ranking with the traditional "big three" of norms, reliability, and validity. Many tests have not yet adequately treated this subject. Some have covered it, but only in the most cursory manner. Fourth, in formulating an overall conclusion about the test, the reviewer faces the difficult task of deciding whether to comment exclusively about the test under review or to place it in the wider context of other tests in the same domain.

Notes:

Title from resource description page (viewed September 18, 2020).