Brain Korea 21 phase II : a new evaluation model / Somi Seong, ... [and others].

Format:

Book

Contributor:

Sŏng, So-mi.

Language:

English

Subjects (All):

Brain Korea 21 (Project)--Evaluation.

Brain Korea 21 (Project).

Education, Higher--Korea (South)--Finance.

Education, Higher.

Government aid to higher education--Korea (South).

Government aid to higher education.

Finance.

Evaluation.

Korea (South).

Physical Description:

xxviii, 248 pages : illustrations ; 23 cm

Place of Publication:

Santa Monica, CA : RAND, 2008.

Summary:

In the late 1990s, the Korea Ministry of Education and Human Resources, in response to concern over the relatively low standing of the nation's universities and researchers, launched the Brain Korea 21 program (BK21). BK21 seeks to make Korean research universities globally competitive and to produce more high-quality researchers in Korea. It provides fellowship funding to graduate students and professors in research groups at top Korean universities. Concentrating on the second phase of BK21, which will continue until 2012, the authors develop quantitative and qualitative models and methods to evaluate how well BK21 is fulfilling its goals, suggest a database of measures needed for that evaluation, and make suggestions for the future management of the program. The measurement methods proposed in this book can assess not only the progress of the BK21 program but also the health and development of the Korean university system as a whole. Korean university research could be stimulated further through a combination of competitive awards for project funding, portable fellowships for graduate students who would select their universities, and university access to capital for construction of first-rate research facilities.

Contents:

BK21 Program Structure 1

Phase I and Phase II Comparison 3

Significance of the Program 4

Objective of the Study 5

Approach 6

Chapter 2 Implications of Phase I Program Evaluations for Phase II Evaluations 9

Previous Performance Evaluations of BK21 Phase I 9

Annual and Interim Evaluations 9

Program-Level Evaluations 11

Findings from Previous Evaluations 14

Changes Resulting from BK21 Phase I 14

Suggestions for Improving the Evaluation System 16

Limitations of Previous Evaluations 17

Implications for Designing the Phase II Evaluation Model 19

Chapter 3 A Logic Model for Evaluating BK21 21

Literature Review and Focus Group Interviews 21

Developing the Logic Model 23

Identifying the Goals and Missions 26

Missions 29

Performance Milestones 31

Goals and Missions Identified for the Logic Model 33

Program Inputs 34

BK21 Funding 34

Strategy 37

Unit of Support 40

Recipient Selection Criteria 41

Performance Evaluation Criteria 42

Other Operational Mechanisms 42

Incentives, Activities, Outputs, and Outcomes of the Program 44

Incentives 44

Activities 47

Outputs 48

Outcomes 54

Other Observations 55

Chapter 4 Quantitative Model to Assess BK21 Program Outcomes 59

Summary of Phase I Statistics 61

Potential Approaches 64

Unadjusted Comparisons of BK21 Winners and Losers 67

Ordinary Least Squares, Matching, and Propensity Score 69

Fixed-Effects and Difference-in-Differences Models 71

Regression Discontinuity 74

Other Strategies 76

Quantitative Model 77

Heterogeneous Funding 80

The Power to Identify Impacts 81

Data for Use in Validating the Model 81

ISI Data 81

Ministry of Education Data 83

Validation of the Model 83

Pre-Treatment Trends 84

Another Possible Comparison Group: Nonapplicants 88

Validation 90

Variations on the Specification 96

Quality 97

Social Science and Humanities 100

Other Aspects of the Logic Model 100

Simultaneous Equations Methods 102

International Competitiveness 104

Simulation: Ability to Precisely Estimate Effects at Phase II Funding Levels 105

Simulation: Outside Funding 107

Implications for Data Collection 109

Chapter 5 Metrics and Measurement for Program Evaluation 111

General Issues of Measurement 112

Reliability 113

Validity 114

Direct and Indirect Measures 114

Cost Versus Benefit 115

Strategy for Framing Evaluation Measures 116

Matching Measures to Goals 117

Selecting Among Candidate Measures 118

Adding to Korea's Research Manpower Pool 132

Highly Educated Workforce Employment 134

High-Quality Globally Competitive Research 135

Measuring Quality of Research Output 135

Consistent Measures of Quality Within Fields 138

Measures of Quality Across All Fields 144

Relative Research Quality Measures 146

Changes to University/Department Infrastructure 146

Fostering a Competitive Environment Within Korea 147

Other Measures of Prestigious and Competitive Graduate Research 150

Measuring Enhanced University-Industry Collaboration 154

Measuring Enhanced Regional University Standing and Industry Linkage 156

Industrial Research Quantity, Quality, and National Labor Productivity 159

Chapter 6 Database Design to Support BK21 Program Evaluation and Measurement 161

Priority Among Proposed Measures 161

Database Design and Content 166

Format of Database Design Entries 173

Chapter 7 Future Development of BK21 and University Research in Korea 177

A Good Model for an Allocation System 178

Applying the Model to Korea 179

Research Project Funding 179

Portable Student Fellowships 181

Facilities and Infrastructure Funding 182

Promotion of Specific Regions or Fields 183

BK21 Program Management 183

Requirements for University and Department Governance 183

Changing the Korean System 184

Appropriate Scale and Number of Research Universities 185

Analogy to the U.S. System 185

Applying the Proposed Measurement Tools 189

Directions for a Phase II Evaluation System 195

Phase II Program Goals and Characteristics Identified by the Logic Model 196

Quantitative Model and Data Requirements 196

Metrics, Measures, and Database to Assess Program Effects and to Monitor Progress 197

Assessing the Quality of Program Outputs and Outcomes 199

Quality of Research 199

Quality of the Highly Educated Workforce 200

Concentration Strategy and Competition Rule 201

Suggestions for Long-Term Changes in the Korean Scientific Research and Education System 202

Balancing Competition and Stability in Funding 202

Portable Student Fellowships 203

Infrastructure Funding 203

Improving University and Department Governance 204

Preconditions for the Proposed Changes 204

Enhancing Program Evaluation and Management Capabilities 205

A Performance Evaluation Metrics Used in Previous Evaluations of Phase I 207

B Interview Guides for Representatives of Participants, Policymakers, and Users Related to the University Research and Education System 209

C Selection Criteria for Applied Science Research Groups: Phase II 217

D Table with Lagged Outcomes 219

E Research Data Series 221

F Education Data Series 229

G Industry Data Series 237

H General Data Series 243.

Notes:

Includes bibliographical references (pages 245-248).

ISBN:

9780833043214

0833043218

OCLC:

191005802