Evaluating the effectiveness of artificial intelligence systems in intelligence analysis / Daniel Ish, Jared Ettinger, Christopher Ferris.

Format:

Book

Author/Creator:

Ish, Daniel, author.

Ettinger, Jared, author.

Ferris, C. (Christopher), 1813 or 1814-1887, author.

Series:

Research report (Rand Corporation) ; RR-A464-1.

[Research report] ; RR-A464-1

Language:

English

Subjects (All):

Artificial intelligence--Military applications--United States.

Artificial intelligence.

Military intelligence--United States--Methodology.

Military intelligence.

Methodology.

Artificial intelligence--Military applications.

United States.

Physical Description:

xx, 88 pages : illustrations (chiefly color) ; 23 cm.

Place of Publication:

Santa Monica, Calif. : RAND Corporation, [2021]

Summary:

The U.S. military and intelligence community have shown interest in developing and deploying artificial intelligence (AI) systems to support intelligence analysis, both as an opportunity to leverage new technology and as a solution for an ever-proliferating data glut. However, deploying AI systems in a national security context requires the ability to measure how well those systems will perform in the context of their mission. To address this issue, the authors begin by introducing a taxonomy of the roles that AI systems can play in supporting intelligence-namely, automated analysis, collection support, evaluation support, and information prioritization-and provide qualitative analyses of the drivers of the impact of system performance for each of these categories. The authors then single out information prioritization systems, which direct intelligence analysts' attention to useful information and allow them to pass over information that is not useful to them, for quantitative analysis. Developing a simple mathematical model that captures the consequences of errors on the part of such systems, the authors show that their efficacy depends not just on the properties of the system but also on how the system is used. Through this exercise, the authors show how both the calculated impact of an AI system and the metrics used to predict it can be used to characterize the system's performance in a way that can help decisionmakers understand its actual value to the intelligence mission.

Contents:

Introduction

Tracing Effectiveness from Mission to System

Measuring Performance and Effectiveness

Conclusions

APPENDIX: Derivations and Technical Details.

Notes:

Includes bibliographical references.

ISBN:

1977407250

9781977407252

OCLC:

1264711897