Apache flume : distributed log collection for Hadoop : stream data to Hadoop using Apache Flume / Steve Hoffman.

Format:

Book

Author/Creator:

Hoffman, Steve.

Series:

Community experience distilled

Language:

English

Subjects (All):

Apache Hadoop.

Electronic data processing--Distributed processing.

Electronic data processing.

Physical Description:

1 online resource (108 p.)

Edition:

1st edition

Place of Publication:

Birmingham : Packt Pub., 2013.

Language Note:

English

System Details:

text file

Summary:

Stream data to Hadoop using Apache Flume Integrate Flume with your data sources Transcode your data en-route in Flume Route and separate your data using regular expression matching Configure failover paths and load-balancing to remove single points of failure Utilize Gzip Compression for files written to HDFS In Detail Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. Its main goal is to deliver data from applications to Apache Hadoop's HDFS. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with many failover and recovery mechanisms. Apache Flume: Distributed Log Collection for Hadoop covers problems with HDFS and streaming data/logs, and how Flume can resolve these problems. This book explains the generalized architecture of Flume, which includes moving data to/from databases, NO-SQL-ish data stores, as well as optimizing performance. This book includes real-world scenarios on Flume implementation. Apache Flume: Distributed Log Collection for Hadoop starts with an architectural overview of Flume and then discusses each component in detail. It guides you through the complete installation process and compilation of Flume. It will give you a heads-up on how to use channels and channel selectors. For each architectural component (Sources, Channels, Sinks, Channel Processors, Sink Groups, and so on) the various implementations will be covered in detail along with configuration options. You can use it to customize Flume to your specific needs. There are pointers given on writing custom implementations as well that would help you learn and implement them. By the end, you should be able to construct a series of Flume agents to transport your streaming data and logs from your systems into Hadoop in near real time.

Contents:

Apache Flume: Distributed Log Collection for Hadoop: If your role includes moving datasets into Hadoop, this book will help you do it more efficiently using Apache Flume. From installation to customization, it's a complete step-by-step guide on making the service work for you.

Notes:

Includes index.

ISBN:

9781782167921

1782167927

OCLC:

857078897