1 option
Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology / Nikhil Agarwal, Alex Moehring, Pranav Rajpurkar, Tobias Salz.
- Format:
- Book
- Author/Creator:
- Agarwal, Nikhil.
- Series:
- Working Paper Series (National Bureau of Economic Research) no. w31422.
- NBER working paper series no. w31422
- Language:
- English
- Physical Description:
- 1 online resource: illustrations (black and white);
- Place of Publication:
- Cambridge, Mass. National Bureau of Economic Research 2023.
- Summary:
- While Artificial Intelligence (AI) algorithms have achieved performance levels comparable to human experts on various predictive tasks, human experts can still access valuable contextual information not yet incorporated into AI predictions. Humans assisted by AI predictions could outperform both human-alone or AI-alone. We conduct an experiment with professional radiologists that varies the availability of AI assistance and contextual information to study the effectiveness of human-AI collaboration and to investigate how to optimize it. Our findings reveal that (i) providing AI predictions does not uniformly increase diagnostic quality, and (ii) providing contextual information does increase quality. Radiologists do not fully capitalize on the potential gains from AI assistance because of large deviations from the benchmark Bayesian model with correct belief updating. The observed errors in belief updating can be explained by radiologists' partially underweighting the AI's information relative to their own and not accounting for the correlation between their own information and AI predictions. In light of these biases, we design a collaborative system between radiologists and AI. Our results demonstrate that, unless the documented mistakes can be corrected, the optimal solution involves assigning cases either to humans or to AI, but rarely to a human assisted by AI.
- Notes:
- Print version record
- July 2023.
The Penn Libraries is committed to describing library materials using current, accurate, and responsible language. If you discover outdated or inaccurate language, please fill out this feedback form to report it and suggest alternative language.