Programming MapReduce with scalding : a practical guide to designing, testing, and implementing complex MapReduce applications in Scala / Antonios Chalkiopoulos.

Format:

Book

Author/Creator:

Chalkiopoulos, Antonios, author.

Series:

Community experience distilled.

Community Experience Distilled

Language:

English

Subjects (All):

Electronic data processing--Distributed processing.

Electronic data processing.

Physical Description:

1 online resource (148 p.)

Edition:

2nd ed.

Place of Publication:

Birmingham, [England] : Packt Publishing, 2014.

Language Note:

English

Summary:

This book is an easy-to-understand, practical guide to designing, testing, and implementing complex MapReduce applications in Scala using the Scalding framework. It is packed with examples featuring log-processing, ad-targeting, and machine learning. This book is for developers who are willing to discover how to effectively develop MapReduce applications. Prior knowledge of Hadoop or Scala is not required; however, investing some time on those topics would certainly be beneficial.

Contents:

Cover; Copyright; Credits; About the Author; About the Reviewers; www.PacktPub.com; Table of Contents; Preface; Introduction to MapReduce; The Hadoop platform; MapReduce; A MapReduce example; MapReduce abstractions; Introducing Cascading; What happens inside a pipe; Pipe assemblies; Cascading extensions; Summary; Get Ready for Scalding; Why Scala?; Scala basics; Scala build tools; Hello World in Scala; Development editors; Installing Hadoop in five minutes; Running our first Scalding job; Submit a Scalding job into Hadoop; Summary; Scalding by Example; Reading and writing files

Best practices for reading and writing files TextLine parsing; Executing in the local and Hadoop modes; Understanding the core capabilities of Scalding; Map-like operations; Join operations; Pipe operations; Grouping/reducing functions; Operations on groups; Composite operations; A simple example; Typed API; Summary; Intermediate Examples; Logfile analysis; Completing the implementation; Exploring ad targeting; Calculating daily points; Calculating historic points; Generating targeted ads; Summary; Scalding Design Patterns; The external operations pattern; The Dependency Injection pattern

The Late Bound Dependency pattern Summary; Testing and TDD; Introduction to testing; MapReduce testing challenges; Development lifecycle with testing strategy; TDD for Scalding developers; Implementing the TDD methodology; Decomposing the algorithm; Defining acceptance tests; Implementing integration tests; Implementing unit tests; Implementing the MapReduce logic; Defining and performing system tests; Black box testing; Summary; Running Scalding in Production; Executing Scalding in a Hadoop cluster; Scheduling execution; Coordinating job execution; Configuring using a property file

Configuring using Hadoop parameters Monitoring Scalding jobs; Using slim JAR files; Scalding execution throttling; Summary; Using External Data Stores; Interacting with external systems; SQL databases; NoSQL databases; Understanding HBase; Reading from HBase; Writing in HBase; Using advanced HBase features; Search platforms; Elastic Search; Summary; Matrix Calculations and Machine Learning; Text similarity using TF-IDF; Setting a similarity using the Jaccard index; K-Means using Mahout; Other libraries; Summary; Index

Notes:

Includes index.

Description based on online resource; title from PDF title page (ebrary, viewed July 10, 2014).

ISBN:

9781783287024

1783287020

OCLC:

882610623