Mastering large datasets with Python : parallelize and distribute your Python code / J. T. Wolohan.

Format:

Book

Author/Creator:

Wolohan, J. T., author.

Language:

English

Subjects (All):

Big data.

Python (Computer program language).

Physical Description:

1 online resource (312 pages)

Edition:

1st edition

Place of Publication:

Shelter Island, New York : Manning, [2019]

System Details:

text file

Summary:

Mastering Large Datasets with Python teaches you to write code that can handle datasets of any size. You’ll start with laptop-sized datasets that teach you to parallelize data analysis by breaking large tasks into smaller ones that can run simultaneously. You’ll then scale those same programs to industrial-sized datasets on a cluster of cloud servers. With the map and reduce paradigm firmly in place, you’ll explore tools like Hadoop and PySpark to efficiently process massive distributed datasets, speed up decision-making with machine learning, and simplify your data storage with AWS S3.

Contents:

Introduction

Accelerating large dataset work: map and parallel computing

Function pipelines for mapping complex transformations

Processing large datasets with lazy workflows

Accumulation operations with reduce

Speeding up map and reduce with advanced parallelization

Processing truly big datasets with Hadoop and Spark

Best practices for large data with Apache Streaming and mrjob

PageRank with map and reduce in PySpark

Faster decision-making with machine learning and PySpark

Large datasets in the cloud with Amazon Web Services and S3

MapReduce in the cloud with Amazon's Elastic MapReduce.

Notes:

Description based on print version record.

Includes index.

ISBN:

9781638350361

1638350361

OCLC:

1257077453