Apache Spark machine learning blueprints : develop a range of cutting-edge machine learning projects with Apache Spark using this actionable guide / Alex Liu.

Format:

Book

Author/Creator:

Liu, Alex, author.

Series:

Community experience distilled.

Community experience distilled

Language:

English

Subjects (All):

Spark (Electronic resource : Apache Software Foundation).

Machine learning.

Big data.

Information retrieval.

Physical Description:

1 online resource (252 pages) : color illustrations.

Edition:

1st edition

Place of Publication:

Birmingham : Packt Publishing, 2016.

System Details:

text file

Biography/History:

Liu Alex: Alex Liu is an expert in research methods and data science. He is currently one of IBM's leading experts in Big Data analytics and also a lead data scientist, where he serves big corporations, develops Big Data analytics IPs, and speaks at industrial conferences such as STRATA, Insights, SMAC, and BigDataCamp. In the past, Alex served as chief or lead data scientist for a few companies, including Yapstone, RS, and TRG. Before this, he was a lead consultant and director at RMA, where he provided data analytics consultation and training to many well-known organizations, including the United Nations, Indymac, AOL, Ingram Micro, GEM, Farmers Insurance, Scripps Networks, Sears, and USAID. At the same time, he taught advanced research methods to PhD candidates at University of Southern California and University of California at Irvine. Before this, he worked as a managing director for CATE/GEC and as a research fellow for the Asia/Pacific Research Center at Stanford University. Alex has a Ph. D. in quantitative sociology and a master's degree of science in statistical computing from Stanford University.

Summary:

Develop a range of cutting-edge machine learning projects with Apache Spark using this actionable guide About This Book Customize Apache Spark and R to fit your analytical needs in customer research, fraud detection, risk analytics, and recommendation engine development Develop a set of practical Machine Learning applications that can be implemented in real-life projects A comprehensive, project-based guide to improve and refine your predictive models for practical implementation Who This Book Is For If you are a data scientist, a data analyst, or an R and SPSS user with a good understanding of machine learning concepts, algorithms, and techniques, then this is the book for you. Some basic understanding of Spark and its core elements and application is required. What You Will Learn Set up Apache Spark for machine learning and discover its impressive processing power Combine Spark and R to unlock detailed business insights essential for decision making Build machine learning systems with Spark that can detect fraud and analyze financial risks Build predictive models focusing on customer scoring and service ranking Build a recommendation systems using SPSS on Apache Spark Tackle parallel computing and find out how it can support your machine learning projects Turn open data and communication data into actionable insights by making use of various forms of machine learning In Detail There's a reason why Apache Spark has become one of the most popular tools in Machine Learning ? its ability to handle huge datasets at an impressive speed means you can be much more responsive to the data at your disposal. This book shows you Spark at its very best, demonstrating how to connect it with R and unlock maximum value not only from the tool but also from your data. Packed with a range of project "blueprints" that demonstrate some of the most interesting challenges that Spark can help you tackle, you'll find out how to use Spark notebooks and access, clean, and join different datasets before putting your knowledge into practice with some real-world projects, in which you will see how Spark Machine Learning can help you with everything from fraud detection to analyzing customer attrition. You'll also find out how to build a recommendation engine using Spark's parallel computing powers. Style and approach This book offers a step-by-step approach to setting up Apache Spark, and use other analytical tools with it to process Big Data and build machine learning pr...

Contents:

Cover

Copyright

Credits

About the Author

About the Reviewer

www.PacktPub.com

Table of Contents

Preface

Chapter 1: Spark for Machine Learning

Spark overview and Spark advantages

Spark overview

Spark advantages

Spark computing for machine learning

Machine learning algorithms

MLlib

Other ML libraries

Spark RDD and dataframes

Spark RDD

Spark dataframes

Dataframes API for R

ML frameworks, RM4Es and Spark computing

ML frameworks

RM4Es

The Spark computing framework

ML workflows and Spark pipelines

ML as a step-by-step workflow

ML workflow examples

Spark notebooks

Notebook approach for ML

Step 1: Getting the software ready

Step 2: Installing the Knitr package

Step 3: Creating a simple report

Summary

Chapter 2: Data Preparation for Spark ML

Accessing and loading datasets

Accessing publicly available datasets

Loading datasets into Spark

Exploring and visualizing datasets

Data cleaning

Dealing with data incompleteness

Data cleaning in Spark

Data cleaning made easy

Identity matching

Identity issues

Identity matching on Spark

Entity resolution

Short string comparison

Long string comparison

Record deduplication

Identity matching made better

Crowdsourced deduplication

Configuring the crowd

Using the crowd

Dataset reorganizing

Dataset reorganizing tasks

Dataset reorganizing with Spark SQL

Dataset reorganizing with R on Spark

Dataset joining

Dataset joining and its tool - the Spark SQL

Dataset joining in Spark

Dataset joining with the R data table package

Feature extraction

Feature development challenges

Feature development with Spark MLlib

Feature development with R

Repeatability and automation

Dataset preprocessing workflows.

Spark pipelines for dataset preprocessing

Dataset preprocessing automation

Chapter 3: A Holistic View on Spark

Spark for a holistic view

The use case

Fast and easy computing

Methods for a holistic view

Regression modeling

The SEM approach

Decision trees

Feature preparation

PCA

Grouping by category to use subject knowledge

Feature selection

Model estimation

MLlib implementation

The R notebooks' implementation

Model evaluation

Quick evaluations

RMSE

ROC curves

Results explanation

Impact assessments

Deployment

Dashboard

Rules

Chapter 4: Fraud Detection on Spark

Spark for fraud detection

Distributed computing

Methods for fraud detection

Random forest

Feature extraction from LogFile

Data merging

R notebooks implementation

A quick evaluation

Confusion matrix and false positive ratios

Big influencers and their impacts

Deploying fraud detection

Scoring

Chapter 5: Risk Scoring on Spark

Spark for risk scoring

Apache Spark notebooks

Methods of risk scoring

Logistic regression

Preparing coding in R

Random forest and decision trees

Preparing coding

Data and feature preparation

OpenRefine

The DataScientistWorkbench for R notebooks

Confusion matrix

ROC

Kolmogorov-Smirnov

Chapter 6: Churn Prediction on Spark

Spark for churn prediction

Spark computing

Methods for churn prediction

Regression models.

Decision trees and Random forest

Spark implementation with MLlib

Calculating the impact of interventions

Intervention recommendations

Chapter 7: Recommendations on Spark

Apache Spark for a recommendation engine

SPSS on Spark

Methods for recommendation

Collaborative filtering

Data treatment with SPSS

Missing data nodes on SPSS modeler

SPSS on Spark - the SPSS Analytics server

Recommendation deployment

Chapter 8: Learning Analytics on Spark

Spark for attrition prediction

Methods of attrition prediction

Regression models

About regression

Preparing for coding

Feature development

Principal components analysis

ML feature selection

Spark implementation with the Zeppelin notebook

The confusion matrix and error ratios

Calculating the impact of main causes

Chapter 9: City Analytics on Spark

Spark for service forecasting

Methods of service forecasting

Time series modeling

About time series

Spark implementation with the R notebook

Model evaluation.

RMSE calculation with MLlib

RMSE calculation with R

Explanations of the results

Biggest influencers

Visualizing trends

The rules of sending out alerts

Scores to rank city zones

Chapter 10: Learning Telco Data on Spark

Spark for using Telco Data

Methods for learning from Telco Data

Descriptive statistics and visualization

Linear and logistic regression models

Decision tree and random forest

Data and feature development

Data reorganizing

Feature development and selection

SPSS on Spark - SPSS Analytics Server

RMSE calculations with MLlib

RMSE calculations with R

Confusion matrix and error ratios with MLlib and R

Descriptive statistics and visualizations

Special insights

Model deployment

Rules to send out alerts

Scores subscribers for churn and for Call Center calls

Scores subscribers for purchase propensity

Chapter 11: Modeling Open Data on Spark

Spark for learning from open data

Methods for scoring and ranking

Cluster analysis

Principal component analysis

Score resembling

Comparing ranks

Rules for sending out alerts

Scores for ranking school districts

Index.

Notes:

Includes index.

Description based on online resource; title from PDF title page (ebrary, viewed March 7, 2017).

ISBN:

9781785887789

1785887785

OCLC:

951075461