1 option
Compositional Generalization in Instruction Following Tasks / Soham Dan.
- Format:
- Book
- Thesis/Dissertation
- Author/Creator:
- Dan, Soham, author.
- Language:
- English
- Subjects (All):
- Computer science.
- Language.
- Computer and Information Science--Penn dissertations.
- Penn dissertations--Computer and Information Science.
- Local Subjects:
- Computer science.
- Language.
- Computer and Information Science--Penn dissertations.
- Penn dissertations--Computer and Information Science.
- Physical Description:
- 1 online resource (182 pages)
- Contained In:
- Dissertations Abstracts International 84-08A.
- Place of Publication:
- [Philadelphia, Pennsylvania] : University of Pennsylvania, 2022.
- Ann Arbor : ProQuest Dissertations & Theses, 2022
- Language Note:
- English
- Summary:
- Understanding instructions expressed in natural language is a fundamental task in artificial intelligence. A key feature of natural language that humans use while giving and following instructions is compositionality: the capacity to understand and produce a potentially infinite number of novel combinations from familiar components. This ability is instrumental in being able to learn from limited data and is crucial for instruction following robots to function in the real-world. This dissertation studies the compositional generalization abilities of machine learning models in various instruction following tasks. We study the various dimensions of compositionality for a diverse set of instruction following tasks of varying complexity: semantic parsing in synthetic languages, natural language instruction following in blocks world and vision-and-language navigation in complex indoor environments. We demonstrate empirically that existing systems for these tasks, while performant on the standard iid test set requiring mere interpolation, do not compositionally generalize, which requires extrapolation. We then present different strategies to induce compositionality, ranging from data augmentation, to auxiliary tasks, to a simple neurosymbolic algorithm. We present a compositional spatial representation language and discuss how using such a rich symbolic representation as auxiliary supervision can help generalization in complex, real-world, multi-modal instruction following tasks. Finally, we aim to develop a more foundational understanding of robust generalization by focusing on the task of learning regular languages, where we study the benefits of compositional models over end-to-end ones, from both theoretical and empirical perspectives.
- Notes:
- Source: Dissertations Abstracts International, Volume: 84-08, Section: A.
- Advisors: Roth, Dan; Committee members: Bastani, Osbert; Callison-Burch, Chris; Daniilidis, Kostas; Sha, Fei; Yatskar, Mark.
- Department: Computer and Information Science.
- Ph.D. University of Pennsylvania 2022.
- Local Notes:
- School code: 0175
- ISBN:
- 9798374412215
- Access Restriction:
- Restricted for use by site license.
- This item is not available from ProQuest Dissertations & Theses.
The Penn Libraries is committed to describing library materials using current, accurate, and responsible language. If you discover outdated or inaccurate language, please fill out this feedback form to report it and suggest alternative language.