Learning to compositionally reason over natural language / Nitish Gupta.

Format:

Book

Thesis/Dissertation

Author/Creator:

Gupta, Nitish, author.

Contributor:

Roth, Dan, degree supervisor.

University of Pennsylvania. Department of Computer and Information Science, degree granting institution.

Language:

English

Subjects (All):

Artificial intelligence.

Computer science.

Computer and Information Science--Penn dissertations.

Penn dissertations--Computer and Information Science.

Local Subjects:

Artificial intelligence.

Computer science.

Computer and Information Science--Penn dissertations.

Penn dissertations--Computer and Information Science.

Genre:

Academic theses.

Physical Description:

1 online resource (160 pages)

Contained In:

Dissertations Abstracts International 82-12B.

Place of Publication:

[Philadelphia, Pennsylvania] : University of Pennsylvania ; Ann Arbor : ProQuest Dissertations & Theses, 2021.

Language Note:

English

System Details:

Mode of access: World Wide Web.

text file

Summary:

The human ability to understand the world in terms of reusable "building blocks" allows us to generalize in near-infinite ways. Developing language understanding systems that can compositionally reason in a similar manner is crucial to achieve human-like capabilities. Designing such systems presents key challenges in the architectural design of machine learning models and the learning paradigm used to train them. This dissertation addresses aspects of both of these challenges by exploring compositional structured models that can be trained using end-task supervision.We believe that solving complex problems in a generalizable manner requires decomposition into sub-tasks, which in turn are solved using reasoning capabilities that can be reused in novel contexts. Motivated by this idea, we develop a neuro-symbolic model with a modular architecture for language understanding and focus on answering questions requiring multi-step reasoning against natural language text. We design an inventory of freely-composable, learnable neural modules for performing various atomic language understanding and symbolic reasoning tasks in a differentiable manner. The question guides how these modules are dynamically composed to yield an end-to-end differentiable model that performs compositional reasoning and can be trained using end-task supervision. However, we show that when trained using such supervision, having a compositional model structure is not sufficient to induce the intended problem decomposition in terms of the modules; Lack of supervision for the sub-tasks leads to modules that do not freely compose in novel ways, hurting generalization. To address this, we develop a new training paradigm that leverages paired examples--instances that share sub-tasks--to provide an additional training signal to that provided by individual examples. We show that this paradigm induces the intended compositional reasoning and leads to improved in- and out-of-distribution generalization.

Notes:

Source: Dissertations Abstracts International, Volume: 82-12, Section: B.

Advisors: Dan Roth; Committee members: Mitch Marcus; Lyle Ungar; Chris Callison-Burch; Luke Zettlemoyer.

Department: Computer and Information Science.

Ph.D. University of Pennsylvania 2021.

Local Notes:

School code: 0175

ISBN:

9798738618307

Access Restriction:

Restricted for use by site license.

This item must not be sold to any third party vendors.