Generation Next: Experimentation with AI / Gary Charness, Brian Jabarian, John A. List.

Format:

Book

Author/Creator:

Charness, Gary.

Contributor:

National Bureau of Economic Research.

Jabarian, Brian.

List, John A.

Series:

Working Paper Series (National Bureau of Economic Research) no. w31679.

NBER working paper series no. w31679

Language:

English

Physical Description:

1 online resource: illustrations (black and white);

Place of Publication:

Cambridge, Mass. National Bureau of Economic Research 2023.

Summary:

We investigate the potential for Large Language Models (LLMs) to enhance scientific practice within experimentation by identifying key areas, directions, and implications. First, we discuss how these models can improve experimental design, including improving the elicitation wording, coding experiments, and producing documentation. Second, we delve into the use of LLMs in experiment implementation, with an emphasis on bolstering causal inference through creating consistent experiences, improving instruction comprehension, and real-time monitoring of participant engagement. Third, we underscore the role of LLMs in analyzing experimental data, encompassing tasks like pre-processing, data cleaning, and assisting reviewers and replicators in examining studies. Each of these tasks improves the probability of reporting accurate findings. Lastly, we suggest a scientific governance framework that mitigates the potential risks of using LLMs in experimental research while amplifying their advantages. This could pave the way for open science opportunities and foster a culture of policy and industry experimentation at scale.

Notes:

Print version record

September 2023.