Optimizing hadoop for MapReduce : learn how to configure your hadoop cluster to run optimal MapReduce jobs / Khaled Tannir ; cover image by Khaled Tannir.

Format:

Book

Author/Creator:

Tannir, Khaled.

Contributor:

Tannir, Khaled.

Series:

Community experience distilled

Language:

English

Subjects (All):

Apache Hadoop.

MapReduce (Computer file).

Electronic data processing--Distributed processing.

Electronic data processing.

Cluster analysis--Data processing.

Cluster analysis.

Open source software.

Physical Description:

1 online resource (120 p.)

Edition:

1st edition

Place of Publication:

Birmingham, England : Packt Publishing Ltd, 2014.

Language Note:

English

System Details:

text file

Summary:

This book is an example-based tutorial that deals with Optimizing Hadoop for MapReduce job performance.If you are a Hadoop administrator, developer, MapReduce user, or beginner, this book is the best choice available if you wish to optimize your clusters and applications. Having prior knowledge of creating MapReduce applications is not necessary, but will help you better understand the concepts and snippets of MapReduce class template code.

Contents:

Cover; Copyright; Credits; About the Author; Acknowledgments; About the Reviewers; www.PacktPub.com; Table of Contents; Preface; Chapter 1: Understanding Hadoop MapReduce; The MapReduce model; Overview of Hadoop MapReduce; Hadoop MapReduce internals; Factors affecting the performance of MapReduce; Summary; Chapter 2: An Overview of the Hadoop Parameters; Investigating the Hadoop parameters; The mapred-site.xml configuration file; The CPU-related parameters; The disk I/O related parameters; The memory-related parameters; The network-related parameters; The hdfs-site.xml configuration file

The core-site.xml configuration fileHadoop MapReduce metrics; Performance monitoring tools; Using Chukwa to monitor Hadoop; Using Ganglia to monitor Hadoop; Using Nagios to monitor Hadoop; Using Apache Ambari to monitor Hadoop; Summary; Chapter 3: Detecting System Bottlenecks; Performance tuning; Creating a performance baseline; Identifying resource bottlenecks; Identifying RAM bottlenecks; Identifying CPU bottlenecks; Identifying storage bottlenecks; Identifying network bandwidth bottlenecks; Summary; Chapter 4: Identifying Resource Weaknesses; Identifying cluster weakness

Checking the Hadoop cluster node's healthChecking the input data size; Checking massive I/O and network traffic; Checking for insufficient concurrent tasks; Checking for CPU contention; Sizing your Hadoop cluster; Configuring your cluster correctly; Summary; Chapter 5: Enhancement of Map and Reduce Tasks; Enhancing Map tasks; Input data and block size impact; Dealing with small and unsplittable files; Reducing spilled records during the Map phase; Calculating map tasks' throughput; Enhancing Reduce tasks; Calculating reduce task throughput; Improving Reduce execution phase

Tuning map and reduce parametersSummary; Chapter 6: Optimizing MapReduce Tasks; Using Combiners; Using compression; Using appropriate Writable types; Reusing types smartly; Optimizing mappers and reducers code; Summary; Chapter 7: Best Practices and Recommendations; Hardware tuning and OS recommendations; Hadoop cluster checklists; The Bios tuning checklist; OS configuration recommendations; Hadoop best practices and recommendations; Deploying Hadoop; Hadoop tuning recommendations; Using a MapReduce template class code; Summary; Index

Notes:

Includes index.

Description based on online resource; title from PDF title page (ebrary, viewed March 11, 2014).

ISBN:

9781783285662

1783285664

OCLC:

871189870