Hadoop.

Format:

Book

Author/Creator:

White, Tom (Tom E.)

Contributor:

Cutting, Doug, Contributor.

Series:

O'Reilly short cuts.

O'Reilly Short Cuts

Language:

English

Subjects (All):

Apache Hadoop (Computer file).

Apache hadoop (Computer file).

Cloud computing.

Electronic data processing--Distributed processing.

File organization (Computer science).

Local Subjects:

Apache hadoop (Computer file).

Cloud computing.

Electronic data processing--Distributed processing.

File organization (Computer science).

Physical Description:

1 online resource (628 p.)

Edition:

2nd ed.

Place of Publication:

Sebastopol : O'Reilly Media, 2010.

Language Note:

English

System Details:

text file

Summary:

Discover how Apache Hadoop can unleash the power of your data. This comprehensive resource shows you how to build and maintain reliable, scalable, distributed systems with the Hadoop framework -- an open source implementation of MapReduce, the algorithm on which Google built its empire. Programmers will find details for analyzing datasets of any size, and administrators will learn how to set up and run Hadoop clusters. This revised edition covers recent changes to Hadoop, including new features such as Hive, Sqoop, and Avro. It also provides illuminating case studies that illustr

Contents:

Table of Contents; Foreword; Preface; Administrative Notes; What's in This Book?; What's New in the Second Edition?; Conventions Used in This Book; Using Code Examples; Safari® Books Online; How to Contact Us; Acknowledgments; Chapter 1. Meet Hadoop; Data!; Data Storage and Analysis; Comparison with Other Systems; RDBMS; Grid Computing; Volunteer Computing; A Brief History of Hadoop; Apache Hadoop and the Hadoop Ecosystem; Chapter 2. MapReduce; A Weather Dataset; Data Format; Analyzing the Data with Unix Tools; Analyzing the Data with Hadoop; Map and Reduce; Java MapReduce; A test run

The new Java MapReduce APIScaling Out; Data Flow; Combiner Functions; Specifying a combiner function; Running a Distributed MapReduce Job; Hadoop Streaming; Ruby; Python; Hadoop Pipes; Compiling and Running; Chapter 3. The Hadoop Distributed Filesystem; The Design of HDFS; HDFS Concepts; Blocks; Namenodes and Datanodes; The Command-Line Interface; Basic Filesystem Operations; Hadoop Filesystems; Interfaces; Thrift; C; FUSE; WebDAV; Other HDFS Interfaces; The Java Interface; Reading Data from a Hadoop URL; Reading Data Using the FileSystem API; FSDataInputStream; Writing Data

FSDataOutputStreamDirectories; Querying the Filesystem; File metadata: FileStatus; Listing files; File patterns; PathFilter; Deleting Data; Data Flow; Anatomy of a File Read; Anatomy of a File Write; Coherency Model; Consequences for application design; Parallel Copying with distcp; Keeping an HDFS Cluster Balanced; Hadoop Archives; Using Hadoop Archives; Limitations; Chapter 4. Hadoop I/O; Data Integrity; Data Integrity in HDFS; LocalFileSystem; ChecksumFileSystem; Compression; Codecs; Compressing and decompressing streams with CompressionCodec

Inferring CompressionCodecs using CompressionCodecFactoryNative libraries; CodecPool; Compression and Input Splits; Using Compression in MapReduce; Compressing map output; Serialization; The Writable Interface; WritableComparable and comparators; Writable Classes; Writable wrappers for Java primitives; Text; Indexing; Unicode; Iteration; BytesWritable; Mutability; Resorting to String; NullWritable; ObjectWritable and GenericWritable; Writable collections; Implementing a Custom Writable; Implementing a RawComparator for speed; Custom comparators; Serialization Frameworks; Serialization IDL

AvroAvro data types and schemas; In-memory serialization and deserialization; Avro data files; Interoperability; Python API; C API; Schema resolution; Sort order; Avro MapReduce; File-Based Data Structures; SequenceFile; Writing a SequenceFile; Reading a SequenceFile; Displaying a SequenceFile with the command-line interface; Sorting and merging SequenceFiles; The SequenceFile format; MapFile; Writing a MapFile; Reading a MapFile; Converting a SequenceFile to a MapFile; Chapter 5. Developing a MapReduce Application; The Configuration API; Combining Resources; Variable Expansion

Configuring the Development Environment

Notes:

Description based upon print version of record.

Includes bibliographical references and index.

Description based on publisher supplied metadata and other sources.

ISBN:

9781449396893

1449396895

9781449396992

1449396992

OCLC:

780425302