Deep Learning for Natural Language Processing : A Gentle Introduction / Mihai Surdeanu and Marco Antonio Valenzuela-Escárcega.

Format:

Book

Author/Creator:

Surdeanu, Mihai, author.

Valenzuela-Escárcega, Marco Antonio, author.

Language:

English

Subjects (All):

Natural language processing (Computer science).

Deep learning (Machine learning).

Physical Description:

1 online resource (xviii, 325 pages) : digital, PDF file(s).

Edition:

First edition.

Place of Publication:

Cambridge : Cambridge University Press, [2024]

Summary:

Deep Learning is becoming increasingly important in a technology-dominated world. However, the building of computational models that accurately represent linguistic structures is complex, as it involves an in-depth knowledge of neural networks, and the understanding of advanced mathematical concepts such as calculus and statistics. This book makes these complexities accessible to those from a humanities and social sciences background, by providing a clear introduction to deep learning for natural language processing. It covers both theoretical and practical aspects, and assumes minimal knowledge of machine learning, explaining the theory behind natural language in an easy-to-read way. It includes pseudo code for the simpler algorithms discussed, and actual Python code for the more complicated architectures, using modern deep learning libraries such as PyTorch and Hugging Face. Providing the necessary theoretical foundation and practical tools, this book will enable readers to immediately begin building real-world, practical natural language processing systems

Contents:

Cover

Half-title page

Title page

Imprints page

Contents

List of Figures

List of Tables

Preface

1 Introduction

1.1 What This Book Covers

1.2 What This Book Does Not Cover

1.3 Deep Learning Is Not Perfect

1.4 Mathematical Notations

2 The Perceptron

2.1 Machine Learning Is Easy

2.2 Use Case: Text Classification

2.3 Evaluation Measures for Text Classification

2.4 The Perceptron

2.5 Voting Perceptron

2.6 Average Perceptron

2.7 Drawbacks of the Perceptron

2.8 Historical Background

2.9 References and Further Readings

2.10 Summary

3 Logistic Regression

3.1 The Logistic Regression Decision Function and Learning Algorithm

3.2 The Logistic Regression Cost Function

3.3 Gradient Descent

3.4 Deriving the Logistic Regression Update Rule

3.5 From Binary to Multiclass Classification

3.6 Evaluation Measures for Multiclass Text Classification

3.7 Drawbacks of Logistic Regression

3.8 Historical Background

3.9 References and Further Readings

3.10 Summary

4 Implementing Text Classification Using Perceptron and Logistic Regression

4.1 Binary Classification

4.2 Multiclass Classification

4.3 Summary

5 Feed-Forward Neural Networks

5.1 Architecture of Feed-Forward Neural Networks

5.2 Learning Algorithm for Neural Networks

5.3 The Equations of Backpropagation

5.4 Drawbacks of Neural Networks (So Far)

5.5 Historical Background

5.6 References and Further Readings

5.7 Summary

6 Best Practices in Deep Learning

6.1 Minibatching

6.2 Other Optimization Algorithms

6.3 Other Activation Functions

6.4 Cost Functions

6.5 Regularization

6.6 Dropout

6.7 Temporal Averaging

6.8 Parameter Initialization and Normalization

6.9 References and Further Readings

6.10 Summary

7 Implementing Text Classification with Feed-Forward Networks

7.1 Data

7.2 Fully Connected Neural Network

7.3 Training

7.4 Summary

8 Distributional Hypothesis and Representation Learning

8.1 Traditional Distributional Representations

8.2 Matrix Decompositions and Low-Rank Approximations

8.3 Drawbacks of Representation Learning Using Low-Rank Approximation

8.4 The Word2vec Algorithm

8.5 Drawbacks of the Word2vec Algorithm

8.6 Historical Background

8.7 References and Further Readings

8.8 Summary

9 Implementing Text Classification Using Word Embeddings

9.1 PretrainedWord Embeddings

9.2 Text Classification with PretrainedWord Embeddings

9.3 Summary

10 Recurrent Neural Networks

10.1 Vanilla Recurrent Neural Networks

10.2 Deep Recurrent Neural Networks

10.3 The Problem with Simple Recurrent Neural Networks: Vanishing Gradient

10.4 Long Short-Term Memory Networks

10.5 Conditional Random Fields

10.6 Drawbacks of Recurrent Neural Networks

10.7 Historical Background

10.8 References and Further Readings

10.9 Summary

Notes:

Title from publisher's bibliographic system (viewed on 02 Feb 2024).

Description based on print version record.

Includes bibliographical references and index.

ISBN:

9781009027809

1009027808

9781009027991

1009027999

9781009026222

1009026224