Big data analytics for large-scale multimedia search / edited by Stefanos Vrochidis [and three others].

Format:

Book

Contributor:

Vrochidis, Stefanos, 1975- editor.

Series:

THEi Wiley ebooks.

Language:

English

Subjects (All):

Multimedia data mining.

Big data.

Physical Description:

1 online resource (375 pages)

Edition:

1st ed.

Other Title:

Big data analytics for large scale multimedia search

Place of Publication:

Hoboken, N.J.: Wiley, 2019.

Hoboken, New Jersey ; Chichester, West Sussex, England : Wiley, [2019]

System Details:

Access using campus network via VPN at home (THEi Users Only).

Summary:

A timely overview of cutting edge technologies for multimedia retrieval with a special emphasis on scalability The amount of multimedia data available every day is enormous and is growing at an exponential rate, creating a great need for new and more efficient approaches for large scale multimedia search. This book addresses that need, covering the area of multimedia retrieval and placing a special emphasis on scalability. It reports the recent works in large scale multimedia search, including research methods and applications, and is structured so that readers with basic knowledge can grasp the core message while still allowing experts and specialists to drill further down into the analytical sections. Big Data Analytics for Large-Scale Multimedia Search covers: representation learning, concept and event-based video search in large collections; big data multimedia mining, large scale video understanding, big multimedia data fusion, large-scale social multimedia analysis, privacy and audiovisual content, data storage and management for big multimedia, large scale multimedia search, multimedia tagging using deep learning, interactive interfaces for big multimedia and medical decision support applications using large multimodal data. Addresses the area of multimedia retrieval and pays close attention to the issue of scalability Presents problem driven techniques with solutions that are demonstrated through realistic case studies and user scenarios Includes tables, illustrations, and figures Offers a Wiley-hosted BCS that features links to open source algorithms, data sets and tools Big Data Analytics for Large-Scale Multimedia Search is an excellent book for academics, industrial researchers, and developers interested in big multimedia data search retrieval. It will also appeal to consultants in computer science problems and professionals in the multimedia industry.

Contents:

Cover

Title Page

Copyright

Contents

Introduction

List of Contributors

About the Companion Website

Part I Feature Extraction from Big Multimedia Data

Chapter 1 Representation Learning on Large and Small Data

1.1 Introduction

1.2 Representative Deep CNNs

1.2.1 AlexNet

1.2.1.1 ReLU Nonlinearity

1.2.1.2 Data Augmentation

1.2.1.3 Dropout

1.2.2 Network in Network

1.2.2.1 MLP Convolutional Layer

1.2.2.2 Global Average Pooling

1.2.3 VGG

1.2.3.1 Very Small Convolutional Filters

1.2.3.2 Multi‐scale Training

1.2.4 GoogLeNet

1.2.4.1 Inception Modules

1.2.4.2 Dimension Reduction

1.2.5 ResNet

1.2.5.1 Residual Learning

1.2.5.2 Identity Mapping by Shortcuts

1.2.6 Observations and Remarks

1.3 Transfer Representation Learning

1.3.1 Method Specifications

1.3.2 Experimental Results and Discussion

1.3.2.1 Results of Transfer Representation Learning for OM

1.3.2.2 Results of Transfer Representation Learning for Melanoma

1.3.2.3 Qualitative Evaluation: Visualization

1.3.3 Observations and Remarks

1.4 Conclusions

References

Chapter 2 Concept‐Based and Event‐Based Video Search in Large Video Collections

2.1 Introduction

2.2 Video preprocessing and Machine Learning Essentials

2.2.1 Video Representation

2.2.2 Dimensionality Reduction

2.3 Methodology for Concept Detection and Concept‐Based Video Search

2.3.1 Related Work

2.3.2 Cascades for Combining Different Video Representations

2.3.2.1 Problem Definition and Search Space

2.3.2.2 Problem Solution

2.3.3 Multi‐Task Learning for Concept Detection and Concept‐Based Video Search

2.3.4 Exploiting Label Relations

2.3.5 Experimental Study

2.3.5.1 Dataset and Experimental Setup

2.3.5.2 Experimental Results

2.3.5.3 Computational Complexity.

2.4 Methods for Event Detection and Event‐Based Video Search

2.4.1 Related Work

2.4.2 Learning from Positive Examples

2.4.3 Learning Solely from Textual Descriptors: Zero‐Example Learning

2.4.4 Experimental Study

2.4.4.1 Dataset and Experimental Setup

2.4.4.2 Experimental Results: Learning from Positive Examples

2.4.4.3 Experimental Results: Zero‐Example Learning

2.5 Conclusions

2.6 Acknowledgments

Chapter 3 Big Data Multimedia Mining: Feature Extraction Facing Volume, Velocity, and Variety

3.1 Introduction

3.2 Scalability through Parallelization

3.2.1 Process Parallelization

3.2.2 Data Parallelization

3.3 Scalability through Feature Engineering

3.3.1 Feature Reduction through Spatial Transformations

3.3.2 Laplacian Matrix Representation

3.3.3 Parallel latent Dirichlet allocation and bag of words

3.4 Deep Learning‐Based Feature Learning

3.4.1 Adaptability that Conquers both Volume and Velocity

3.4.2 Convolutional Neural Networks

3.4.3 Recurrent Neural Networks

3.4.4 Modular Approach to Scalability

3.5 Benchmark Studies

3.5.1 Dataset

3.5.2 Spectrogram Creation

3.5.3 CNN‐Based Feature Extraction

3.5.4 Structure of the CNNs

3.5.5 Process Parallelization

3.5.6 Results

3.6 Closing Remarks

3.7 Acknowledgements

Part II Learning Algorithms for Large-Scale Multimedia

Chapter 4 Large‐Scale Video Understanding with Limited Training Labels

4.1 Introduction

4.2 Video Retrieval with Hashing

4.2.1 Overview

4.2.2 Unsupervised Multiple Feature Hashing

4.2.2.1 Framework

4.2.2.2 The Objective Function of MFH

4.2.2.3 Solution of MFH

4.2.2.3.1 Complexity Analysis

4.2.3 Submodular Video Hashing

4.2.3.1 Framework

4.2.3.2 Video Pooling

4.2.3.3 Submodular Video Hashing

4.2.4 Experiments.

4.2.4.1 Experiment Settings

4.2.4.1.1 Video Datasets

4.2.4.1.2 Visual Features

4.2.4.1.3 Algorithms for Comparison

4.2.4.2 Results

4.2.4.2.1 CC_WEB_VIDEO

4.2.4.2.2 Combined Dataset

4.2.4.3 Evaluation of SVH

4.2.4.3.1 Results

4.3 Graph‐Based Model for Video Understanding

4.3.1 Overview

4.3.2 Optimized Graph Learning for Video Annotation

4.3.2.1 Framework

4.3.2.2 OGL

4.3.2.2.1 Terms and Notations

4.3.2.2.2 Optimal Graph-Based SSL

4.3.2.2.3 Iterative Optimization

4.3.3 Context Association Model for Action Recognition

4.3.3.1 Context Memory

4.3.4 Graph‐based Event Video Summarization

4.3.4.1 Framework

4.3.4.2 Temporal Alignment

4.3.5 TGIF: A New Dataset and Benchmark on Animated GIF Description

4.3.5.1 Data Collection

4.3.5.2 Data Annotation

4.3.6 Experiments

4.3.6.1 Experimental Settings

4.3.6.2 Results

4.4 Conclusions and Future Work

Chapter 5 Multimodal Fusion of Big Multimedia Data

5.1 Multimodal Fusion in Multimedia Retrieval

5.1.1 Unsupervised Fusion in Multimedia Retrieval

5.1.1.1 Linear and Non‐linear Similarity Fusion

5.1.1.2 Cross‐modal Fusion of Similarities

5.1.1.3 Random Walks and Graph‐based Fusion

5.1.1.4 A Unifying Graph‐based Model

5.1.2 Partial Least Squares Regression

5.1.3 Experimental Comparison

5.1.3.1 Dataset Description

5.1.3.2 Settings

5.1.3.3 Results

5.1.4 Late Fusion of Multiple Multimedia Rankings

5.1.4.1 Score Fusion

5.1.4.2 Rank Fusion

5.1.4.2.1 Borda Count Fusion

5.1.4.2.2 Reciprocal Rank Fusion

5.1.4.2.3 Condorcet Fusion

5.2 Multimodal Fusion in Multimedia Classification

5.2.1 Related Literature

5.2.2 Problem Formulation

5.2.3 Probabilistic Fusion in Active Learning

5.2.3.1 If P(S&amp

equals

0|V,T)≠0:

5.2.3.2 If P(S&amp

0|V,T)&amp.

equals

0:

5.2.3.3 Incorporating Informativeness in the Selection (P(S|V))

5.2.3.4 Measuring Oracle's Confidence (P(S|T))

5.2.3.5 Re‐training

5.2.4 Experimental Comparison

5.2.4.1 Datasets

5.2.4.2 Settings

5.2.4.3 Results

5.2.4.3.1 Expanding with Positive, Negative or Both

5.2.4.3.2 Comparing with Sample Selection Approaches

5.2.4.3.3 Comparing with Fusion Approaches

5.2.4.3.4 Parameter Sensitivity Investigation

5.2.4.3.5 Comparing with Existing Methods

5.3 Conclusions

Chapter 6 Large‐Scale Social Multimedia Analysis

6.1 Social Multimedia in Social Media Streams

6.1.1 Social Multimedia

6.1.2 Social Multimedia Streams

6.1.3 Analysis of the Twitter Firehose

6.1.3.1 Dataset: Overview

6.1.3.2 Linked Resource Analysis

6.1.3.3 Image Content Analysis

6.1.3.4 Geographic Analysis

6.1.3.5 Textual Analysis

6.2 Large‐Scale Analysis of Social Multimedia

6.2.1 Large‐Scale Processing of Social Multimedia Analysis

6.2.1.1 Batch‐Processing Frameworks

6.2.1.2 Stream‐Processing Frameworks

6.2.1.3 Distributed Processing Frameworks

6.2.2 Analysis of Social Multimedia

6.2.2.1 Analysis of Visual Content

6.2.2.2 Analysis of Textual Content

6.2.2.3 Analysis of Geographical Content

6.2.2.4 Analysis of User Content

6.3 Large‐Scale Multimedia Opinion Mining System

6.3.1 System Overview

6.3.2 Implementation Details

6.3.2.1 Social Media Data Crawler

6.3.2.2 Social Multimedia Analysis

6.3.2.3 Analysis of Visual Content

6.3.3 Evaluations: Analysis of Visual Content

6.3.3.1 Filtering of Synthetic Images

6.3.3.2 Near‐Duplicate Detection

6.4 Conclusion

Chapter 7 Privacy and Audiovisual Content: Protecting Users as Big Multimedia Data Grows Bigger

7.1 Introduction

7.1.1 The Dark Side of Big Multimedia Data.

7.1.2 Defining Multimedia Privacy

7.2 Protecting User Privacy

7.2.1 What to Protect

7.2.2 How to Protect

7.2.3 Threat Models

7.3 Multimedia Privacy

7.3.1 Privacy and Multimedia Big Data

7.3.2 Privacy Threats of Multimedia Data

7.3.2.1 Audio Data

7.3.2.2 Visual Data

7.3.2.3 Multimodal Threats

7.4 Privacy‐Related Multimedia Analysis Research

7.4.1 Multimedia Analysis Filters

7.4.2 Multimedia Content Masking

7.5 The Larger Research Picture

7.5.1 Multimedia Security and Trust

7.5.2 Data Privacy

7.6 Outlook on Multimedia Privacy Challenges

7.6.1 Research Challenges

7.6.1.1 Multimedia Analysis

7.6.1.2 Data

7.6.1.3 Users

7.6.2 Research Reorientation

7.6.2.1 Professional Paranoia

7.6.2.2 Privacy as a Priority

7.6.2.3 Privacy in Parallel

Part III Scalability in Multimedia Access

Chapter 8 Data Storage and Management for Big Multimedia

8.1 Introduction

8.1.1 Multimedia Applications and Scale

8.1.2 Big Data Management

8.1.3 System Architecture Outline

8.1.4 Metadata Storage Architecture

8.1.4.1 Lambda Architecture

8.1.4.2 Storage Layer

8.1.4.3 Processing Layer

8.1.4.4 Serving Layer

8.1.4.5 Dynamic Data

8.1.5 Summary and Chapter Outline

8.2 Media Storage

8.2.1 Storage Hierarchy

8.2.1.1 Secondary Storage

8.2.1.2 The Five‐Minute Rule

8.2.1.3 Emerging Trends for Local Storage

8.2.2 Distributed Storage

8.2.2.1 Distributed Hash Tables

8.2.2.2 The CAP Theorem and the PACELC Formulation

8.2.2.3 The Hadoop Distributed File System

8.2.2.4 Ceph

8.2.3 Discussion

8.3 Processing Media

8.3.1 Metadata Extraction

8.3.2 Batch Processing

8.3.2.1 Map‐Reduce and Hadoop

8.3.2.2 Spark

8.3.2.3 Comparison

8.3.3 Stream Processing

8.4 Multimedia Delivery

8.4.1 Distributed In‐Memory Buffering.

8.4.1.1 Memcached and Redis.

Notes:

"with website"--Cover

Other editors: Benoit Huet, Edward Y. Chang, Ioannis Kompatsiaris

Includes bibliographical references and index

Includes bibliographical references and index.

Description based on print version record.

ISBN:

1-119-37699-8

1-119-37698-X

OCLC:

1048047326