Hands-on transfer learning with Python : implement advanced deep learning and neural network models using tensorflow and keras / Dipanjan Sarkar, Raghav Bali, Tamoghna Ghosh.

Format:

Book

Author/Creator:

Sarkar, Dipanjan, author.

Contributor:

Bali, Raghav, editor.

Ghosh, Tamoghna, editor.

Language:

English

Subjects (All):

Python (Computer program language).

Machine learning.

Physical Description:

1 online resource (430 pages) : illustrations

Edition:

1st edition

Place of Publication:

Birmingham ; Mumbai : Packt, 2018.

System Details:

Mode of access: World Wide Web.

text file

Biography/History:

Sarkar Dipanjan: Dipanjan (DJ) Sarkar is a Data Scientist at Intel, leveraging data science, machine learning, and deep learning to build large-scale intelligent systems. He holds a master of technology degree with specializations in Data Science and Software Engineering. He has been an analytics practitioner for several years now, specializing in machine learning, NLP, statistical methods, and deep learning. He is passionate about education and also acts as a Data Science Mentor at various organizations like Springboard, helping people learn data science. He is also a key contributor and editor for Towards Data Science, a leading online journal on AI and Data Science. He has also authored several books on R, Python, machine learning, NLP, and deep learning. Ghosh Tamoghna: Tamoghna Ghosh is a machine learning engineer at Intel Corporation. He has overall 11 years of work experience including 4 years of core research experience at Microsoft Research (MSR) India. At MSR he worked as a research assistant in cryptanalysis of block ciphers. His technical expertise's are in big data, machine learning, NLP, information retrieval, data visualization and software development. He received M. Tech (Computer Science) degree from the Indian Statistical Institute, Kolkata and M. Sc (Mathematics) from University of Calcutta with specialization in functional analysis and mathematical modeling/dynamical systems. He is passionate about teaching and conducts internal training in data science for Intel at various levels.

Summary:

Deep learning simplified by taking supervised, unsupervised, and reinforcement learning to the next level using the Python ecosystem Key Features Build deep learning models with transfer learning principles in Python implement transfer learning to solve real-world research problems Perform complex operations such as image captioning neural style transfer Book Description Transfer learning is a machine learning (ML) technique where knowledge gained during training a set of problems can be used to solve other similar problems. The purpose of this book is two-fold; firstly, we focus on detailed coverage of deep learning (DL) and transfer learning, comparing and contrasting the two with easy-to-follow concepts and examples. The second area of focus is real-world examples and research problems using TensorFlow, Keras, and the Python ecosystem with hands-on examples. The book starts with the key essential concepts of ML and DL, followed by depiction and coverage of important DL architectures such as convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), long short-term memory (LSTM), and capsule networks. Our focus then shifts to transfer learning concepts, such as model freezing, fine-tuning, pre-trained models including VGG, inception, ResNet, and how these systems perform better than DL models with practical examples. In the concluding chapters, we will focus on a multitude of real-world case studies and problems associated with areas such as computer vision, audio analysis and natural language processing (NLP). By the end of this book, you will be able to implement both DL and transfer learning principles in your own systems. What you will learn Set up your own DL environment with graphics processing unit (GPU) and Cloud support Delve into transfer learning principles with ML and DL models Explore various DL architectures, including CNN, LSTM, and capsule networks Learn about data and network representation and loss functions Get to grips with models and strategies in transfer learning Walk through potential challenges in building complex transfer learning models from scratch Explore real-world research problems related to computer vision and audio analysis Understand how transfer learning can be leveraged in NLP Who this book is for Hands-On Transfer Learning with Python is for data scientists, machine learning engineers, analysts and developers with an interest in data and applying state-of-the-art...

Contents:

Cover

Title Page

Copyright and Credits

Dedication

Packt Upsell

Foreword

Contributors

Table of Contents

Preface

Chapter 1: Machine Learning Fundamentals

Why ML?

Formal definition

Shallow and deep learning

ML techniques

Supervised learning

Classification

Regression

Unsupervised learning

Clustering

Dimensionality reduction

Association rule mining

Anomaly detection

CRISP-DM

Business understanding

Data understanding

Data preparation

Modeling

Evaluation

Deployment

Standard ML workflow

Data retrieval

Exploratory data analysis

Data processing and wrangling

Feature engineering and extraction

Feature scaling and selection

Model evaluation and tuning

Model evaluation

Bias variance trade-off

Bias

Variance

Trade-off

Underfitting

Overfitting

Generalization

Model tuning

Deployment and monitoring

Feature extraction and engineering

Feature engineering strategies

Working with numerical data

Working with categorical data

Working with image data

Deep learning based automated feature extraction

Working with text data

Text preprocessing

Feature engineering

Feature selection

Summary

Chapter 2: Deep Learning Essentials

What is deep learning?

Deep learning frameworks

Setting up a cloud-based deep learning environment with GPU support

Choosing a cloud provider

Setting up your virtual server

Configuring your virtual server

Installing and updating deep learning dependencies

Accessing your deep learning cloud environment

Validating GPU-enablement on your deep learning environment

Setting up a robust, on-premise deep learning environment with GPU support

Neural network basics

A simple linear neuron.

Gradient-based optimization

The Jacobian and Hessian matrices

Chain rule of derivatives

Stochastic Gradient Descent

Non-linear neural units

Learning a simple non-linear unit - logistic unit

Loss functions

Data representations

Tensor examples

Tensor operations

Multilayered neural networks

Backprop - training deep neural networks

Challenges in neural network learning

Ill-conditioning

Local minima and saddle points

Cliffs and exploding gradients

Initialization - bad correspondence between the local and global structure of the objective

Inexact gradients

Initialization of model parameters

Initialization heuristics

Improvements of SGD

The momentum method

Nesterov momentum

Adaptive learning rate - separate for each connection

AdaGrad

RMSprop

Adam

Overfitting and underfitting in neural networks

Model capacity

How to avoid overfitting - regularization

Weight-sharing

Weight-decay

Early stopping

Dropout

Batch normalization

Do we need more data?

Hyperparameters of the neural network

Automatic hyperparameter tuning

Grid search

Chapter 3: Understanding Deep Learning Architectures

Neural network architecture

Why different architectures are needed

Various architectures

MLPs and deep neural networks

Autoencoder neural networks

Variational autoencoders

Generative Adversarial Networks

Text-to-image synthesis using the GAN architecture

CNNs

The convolution operator

Stride and padding mode in convolution

The convolution layer

LeNet architecture

AlexNet

ZFNet

GoogLeNet (inception network)

VGG

Residual Neural Networks

Capsule networks

Recurrent neural networks

LSTMs

Stacked LSTMs

Encoder-decoder - Neural Machine Translation

Gated Recurrent Units

Memory Neural Networks.

MemN2Ns

Neural Turing Machine

Selective attention

Read operation

Write operation

The attention-based neural network model

Chapter 4: Transfer Learning Fundamentals

Introduction to transfer learning

Advantages of transfer learning

Transfer learning strategies

Transfer learning and deep learning

Transfer learning methodologies

Feature-extraction

Fine-tuning

Pretrained models

Applications

Deep transfer learning types

Domain adaptation

Domain confusion

Multitask learning

One-shot learning

Zero-shot learning

Challenges of transfer learning

Negative transfer

Transfer bounds

Chapter 5: Unleashing the Power of Transfer Learning

The need for transfer learning

Formulating our real-world problem

Building our dataset

Formulating our approach

Building CNN models from scratch

Basic CNN model

CNN model with regularization

CNN model with image augmentation

Leveraging transfer learning with pretrained CNN models

Understanding the VGG-16 model

Pretrained CNN model as a feature extractor

Pretrained CNN model as a feature extractor with image augmentation

Pretrained CNN model with fine-tuning and image augmentation

Evaluating our deep learning models

Model predictions on a sample test image

Visualizing what a CNN model perceives

Evaluation model performance on test data

Chapter 6: Image Recognition and Classification

Deep learning-based image classification

Benchmarking datasets

State-of-the-art deep image classification models

Image classification and transfer learning

CIFAR-10

Building an image classifier

Transferring knowledge

Dog Breed Identification dataset

Exploratory analysis

Dog classifier using transfer learning

Summary.

Chapter 7: Text Document Categorization

Text categorization

Traditional text categorization

Shortcomings of BoW models

Benchmark datasets

Word representations

Word2vec model

Word2vec using gensim

GloVe model

CNN document model

Building a review sentiment classifier

What has embedding changed most?

Transfer learning - application to the IMDB dataset

Training on the full IMDB dataset with Word2vec embeddings

Creating document summaries with CNN model

Multiclass classification with the CNN model

Visualizing document embeddings

Chapter 8: Audio Event Identification and Classification

Understanding audio event classification

Exploratory analysis of audio events

Feature engineering and representation of audio events

Audio event classification with transfer learning

Building datasets from base features

Transfer learning for feature extraction

Building the classification model

Evaluating the classifier performance

Building a deep learning audio event identifier

Chapter 9: DeepDream

Introduction

Algorithmic pareidolia in computer vision

Visualizing feature maps

DeepDream

Examples

Chapter 10: Style Transfer

Understanding neural style transfer

Image preprocessing methodology

Building loss functions

Content loss

Style loss

Total variation loss

Overall loss function

Constructing a custom optimizer

Style transfer in action

Chapter 11: Automated Image Caption Generator

Understanding image captioning

Formulating our objective

Understanding the data

Approach to automated image captioning

Conceptual approach

Practical hands-on approach

Image feature extractor - DCNN model with transfer learning.

Text caption generator - sequence-based language model with LSTM

Encoder-decoder model

Image feature extraction with transfer learning

Building a vocabulary for our captions

Building an image caption dataset generator

Building our image language encoder-decoder deep learning model

Training our image captioning deep learning model

Evaluating our image captioning deep learning model

Loading up data and models

Understanding greedy and beam search

Implementing a beam search-based caption generator

Understanding and implementing BLEU scoring

Evaluating model performance on test data

Automated image captioning in action!

Captioning sample images from outdoor scenes

Captioning sample images from popular sports

Future scope for improvement

Chapter 12: Image Colorization

Problem statement

Color images

Color theory

Color models and color spaces

RGB

YUV

LAB

Problem statement revisited

Building a coloring deep neural network

Preprocessing

Standardization

Loss function

Encoder

Transfer learning - feature extraction

Fusion layer

Decoder

Postprocessing

Training and results

Challenges

Further improvements

Other Books You May Enjoy

Index.

Notes:

Includes index.

Includes bibliographical references and index.

Description based on print version record.

ISBN:

9781788839051

1788839056

OCLC:

1055555784