Thirteen strategies to measure college teaching : a consumer's guide to rating scale construction, assessment, and decision making for faculty, administrators, and clinicians / Ronald A. Berk ; with a foreword by Mike Theall.

Format:

Book

Author/Creator:

Berk, Ronald A., author.

Contributor:

Theall, Mike, writer of foreword.

Language:

English

Subjects (All):

College teachers--Rating of.

College teachers.

Teacher effectiveness.

Physical Description:

1 online resource (319 pages)

Edition:

First edition.

Place of Publication:

Sterling, Virginia : Stylus, 2006.

Summary:

This book takes off from the premise that student ratings are a necessary, but not sufficient source of evidence for measuring teaching effectiveness.

Contents:

Cover

Half Title

Title

Copyright

Dedication

Contents

Acknowledgments

A Foreword (in Berkian Style)

Introduction

1 TOP 13 SOURCES OF EVIDENCE OF TEACHING EFFECTIVENESS

A Few Ground Rules

Teaching Effectiveness: Defining the Construct

National Standards

Beyond Student Ratings

A Unified Conceptualization

Thirteen Sources of Evidence

Student Ratings

Peer Ratings

External Expert Ratings

Self-Ratings

Videos

Student Interviews

Exit and Alumni Ratings

Employer Ratings

Administrator Ratings

Teaching Scholarship

Teaching Awards

Learning Outcome Measures

Teaching Portfolio

BONUS: 360° Multisource Assessment

Berk's Top Picks

Formative Decisions

Summative Decisions

Program Decisions

Decision Time

2 CREATING THE RATING SCALE STRUCTURE

Overview of the Scale Construction Process

Specifying the Purpose of the Scale

Delimiting What Is to Be Measured

Focus Groups

Interviews

Research Evidence

Determining How to Measure the "What"

Existing Scales

Item Banks

Commercially Published Student Rating Scales

Universe of Items

Structure of Rating Scale Items

Structured Items

Unstructured Items

3 GENERATING THE STATEMENTS

Preliminary Decisions

Domain Specifications

Number of Statements

Rules for Writing Statements

1. The statement should be clear and direct.

2. The statement should be brief and concise.

3. The statement should contain only one complete behavior, thought, or concept.

4. The statement should be a simple sentence.

5. The statement should be at the appropriate reading level.

6. The statement should be grammatically correct.

7. The statement should be worded strongly.

8. The statement should be congruent with the behavior it is intended to measure.

9. The statement should accurately measure a positive or negative behavior.

10. The statement should be applicable to all respondents.

11. The respondents should be in the best position to respond to the statement.

12. The statement should be interpretable in only one way.

13. The statement should NOT contain a double negative.

14. The statement should NOT contain universal or absolute terms.

15. The statement should NOT contain nonabsolute, warm-and-fuzzy terms.

16. The statement should NOT contain value-laden or inflammatory words.

17. The statement should NOT contain words, phrases, or abbreviations that would be unfamiliar to all respondents.

18. The statement should NOT tap a behavior appearing in any other statement.

19. The statement should NOT be factual or capable of being interpreted as factual.

20. The statement should NOT be endorsed or given one answer by almost all respondents or by almost none.

4 SELECTING THE ANCHORS

Types of Anchors

Intensity Anchors

Evaluation Anchors

Frequency Anchors

Quantity Anchors

Comparison Anchors

Rules for Selecting Anchors

1. The anchors should be consistent with the purpose of the rating scale.

2. The anchors should match the statements, phrases, or word topics.

3. The anchors should be logically appropriate with each statement.

4. The anchors should be grammatically consistent with each question.

5. The anchors should provide the most accurate and concrete responses possible.

6. The anchors should elicit a range of responses.

7. The anchors on bipolar scales should be balanced, not biased.

8. The anchors on unipolar scales should be graduated appropriately.

5 REFINING THE ITEM STRUCTURE

Preparing for Structural Changes

Issues in Scale Construction

1. What rating scale format is best?.

2. How many anchor points should be on the scale?

3. Should there be a designated midpoint position, such as "Neutral," "Uncertain," or "Undecided," on the scale?

4. How many anchors should be specified on the scale?

5. Should numbers be placed on the anchor scale?

6. Should a "Not Applicable" (NA) or "Not Observed" (NO) option be provided?

7. How can response set biases be minimized?

6 ASSEMBLING THE SCALE FOR ADMINISTRATION

Assembling the Scale

Identification Information

Purpose

Directions

Scale Administration

Paper-Based Administration

Online Administration

Comparability of Paper-Based and Online Ratings

Conclusions

7 FIELD TESTING AND ITEM ANALYSES

Preparing the Draft Scale for a Test Spin

Field Test Procedures

Mini-Field Test

Monster-Field Test

Item Analyses

Stage 1: Item Descriptive Statistics

Stage 2: Interitem and Item-Scale Correlations

Stage 3: Factor Analysis

8 COLLECTING EVIDENCE OF VALIDITY AND RELIABILITY

Validity Evidence

Evidence Based on Job Content Domain

Evidence Based on Response Processes

Evidence Based on Internal Scale Structure

Evidence Related to Other Measures of Teaching Effectiveness

Evidence Based on the Consequences of Ratings

Reliability Evidence

Classical Reliability Theory

Summated Rating Scale Theory

Methods for Estimating Reliability

Epilogue

9 REPORTING AND INTERPRETING SCALE RESULTS

Generic Levels of Score Reporting

Item Anchor

Item

Subscale

Total Scale

Department/Program Norms

Subject Matter/Program-Level State, Regional, and National Norms

Criterion-Referenced versus Norm-Referenced Score Interpretations

Score Range

Criterion-Referenced Interpretations

Norm-Referenced Interpretations

Formative, Summative, and Program Decisions.

Formative Decisions

References

Appendices

A. Sample "Home-Grown" Rating Scales

B. Sample 360° Assessment Rating Scales

C. Sample Reporting Formats

D. Commercially Published Student Rating Scale Systems

Index.

Notes:

Description based on print version record.

Includes bibliographical references and index.

ISBN:

9781620360507

1620360500

OCLC:

945135886