Opportunistic evaluation for compound ai systems Stephen Mell

Format:

Book

Thesis/Dissertation

Author/Creator:

Mell, Stephen, author.

Contributor:

University of Pennsylvania. Computer and Information Science., degree granting institution.

Language:

English

Subjects (All):

Computer science.

Computer engineering.

0984.

0464.

0800.

Local Subjects:

Computer science.

Computer engineering.

0984.

0464.

0800.

Genre:

Academic theses

Physical Description:

1 online resource (126 pages)

Contained In:

Dissertations Abstracts International 87-12B

Place of Publication:

Ann Arbor : ProQuest Dissertations and Theses, 2026

Language Note:

English

Summary:

AI systems have shown impressive performance across a wide range of tasks. Part of this has been driven by a shift toward compound AI systems, composed of one or more AI models and programmatic components. Latency is a key concern for these systems, and there are many opportunities to parallelize and stream model calls. However, manually doing so greatly increases code complexity, intertwining high-level algorithms with low-level performance optimizations. In this thesis, we propose providing parallelization and streaming automatically for high-level, sequential code via an approach called opportunistic evaluation. First, we develop its theory, introducing a novel calculus, λO, providing sound, out-of-order execution of general-purpose programs that make external calls, such as those to AI models. We then describe a language, Opal, which offers automatic parallelization and streaming, and we demonstrate its ability to express and optimize a wide range of compound AI systems, improving performance by up to 12.7×. Next, we build a system, PopPy, which automatically parallelizes Python programs, and we show that it can express 5 compound AI systems from the literature with minimal modifications, yielding speedups of up to 6.4×. Finally, we build an extensible language, Quasar, for code generated by AI models and show that it can provide latency, security, and reliability improvements, without accuracy degradation. Together, these show how opportunistic evaluation can provide significant benefits to compound AI systems written in general-purpose programming languages

Notes:

Source: Dissertations Abstracts International, Volume: 87-12, Section: B.

Advisors: Bastani, Osbert; Zdancewic, Steve Committee members: Alur, Rajeev; Liu, Vincent; Pierce, Benjamin C.; Cheung, Alvin

Ph.D. University of Pennsylvania 2026

Vendor supplied data

Local Notes:

School code: 0175

ISBN:

9798247973508

Access Restriction:

Restricted for use by site license