Learn azure synapse data explorer : a guide to building real-time analytics solutions to unlock log and telemetry data / Pericles Rocha.

Format:

Book

Author/Creator:

Rocha, Pericles, author.

Language:

English

Subjects (All):

Big data.

Cloud computing.

Physical Description:

1 online resource (346 pages)

Edition:

1st ed.

Place of Publication:

Birmingham, England ; Mumbai : Packt Publishing, [2023]

System Details:

Mode of access: World Wide Web.

Summary:

A hands-on guide to working on use cases helping you ingest, analyze, and serve insightful data from IoT as well as telemetry data sources using Azure Synapse Data Explorer Free PDF included with this book Key Features Augment advanced analytics projects with your IoT and application data Expand your existing Azure Synapse environments with unstructured data Build industry-level projects on integration, experimentation, and dashboarding with Azure Synapse Book Description Large volumes of data are generated daily from applications, websites, IoT devices, and other free-text, semi-structured data sources. Azure Synapse Data Explorer helps you collect, store, and analyze such data, and work with other analytical engines, such as Apache Spark, to develop advanced data science projects and maximize the value you extract from data. This book offers a comprehensive view of Azure Synapse Data Explorer, exploring not only the core scenarios of Data Explorer but also how it integrates within Azure Synapse. From data ingestion to data visualization and advanced analytics, you'll learn to take an end-to-end approach to maximize the value of unstructured data and drive powerful insights using data science capabilities. With real-world usage scenarios, you'll discover how to identify key projects where Azure Synapse Data Explorer can help you achieve your business goals. Throughout the chapters, you'll also find out how to manage big data as part of a software as a service (SaaS) platform, as well as tune, secure, and serve data to end users. By the end of this book, you'll have mastered the big data life cycle and you'll be able to implement advanced analytical scenarios from raw telemetry and log data. What you will learn Integrate Data Explorer pools with all other Azure Synapse services Create Data Explorer pools with Azure Synapse Studio and Azure Portal Ingest, analyze, and serve data to users using Azure Synapse pipelines Integrate Power BI and visualize data with Synapse Studio Configure Azure Machine Learning integration in Azure Synapse Manage cost and troubleshoot Data Explorer pools in Synapse Analytics Secure Synapse workspaces and grant access to Data Explorer pools Who this book is for If you are a data engineer, data analyst, or business analyst working with unstructured data and looking to learn how to maximize the value of such data, this book is for you. If you already have experience working with Azure Synapse and want to incorporate unstructured data into your data science project, you'll also find plenty of useful information in this book. To maximize your learning experience, familiarity with data and performing simple queries using SQL or KQL is recommended. Basic knowledge of Python will help you get more from the examples.

Contents:

Cover

Title Page

Copyright and Credit

Dedicated

Contributors

Table of Contents

Preface

Part 1: Introduction to Azure Synapse Data Explorer

Chapter 1: Introducing Azure Synapse Data Explorer

Technical requirements

Understanding the lifecycle of data

Introducing the Team Data Science Process

Tooling and infrastructure

The need for a fast and highly scalable data exploration service

What is Azure Synapse?

Data integration

Enterprise data warehousing

Exploration on the data lake

Apache Spark

Log and telemetry analytics

Integrated business intelligence

Data governance

Broad support for ML

Security and Managed Virtual Network

Management interface

What is Azure Synapse Data Explorer?

Integrating Data Explorer pools with other Azure Synapse services

Query experience integrated into Azure Synapse Studio's query editor

Exploring, preparing, and modeling data with Apache Spark

Data ingestion made easy with pipelines

Unified management experience

Exploring the Data Explorer pool infrastructure and scalability

Data Explorer pool architecture

Scalability of compute resources

Managing data on distributed clusters

Mission-critical infrastructure

How much scale can Data Explorer handle?

What makes Azure Synapse Data Explorer unique?

When to use Azure Synapse Data Explorer

Summary

Chapter 2: Creating Your First Data Explorer Pool

Creating a free Azure account

Creating an Azure Synapse workspace

Basics tab

Security tab

Networking tab

Tags tab

Review + create tab

Finding your new workspace

Creating a Data Explorer pool using Azure Synapse Studio

Additional settings tab

Creating a Data Explorer pool using the Azure portal.

Creating a Data Explorer pool using the Azure CLI

Chapter 3: Exploring Azure Synapse Studio

Exploring the user interface of Azure Synapse Studio

Running your first query

Creating a database

Loading the data

Verifying whether your data has loaded successfully

Working with data in Azure Synapse notebooks

Saving your work and configuring source control

Managing and monitoring Data Explorer pools

Scaling Data Explorer pools

Pausing and resuming pools

Monitoring Data Explorer pools

Chapter 4: Real-World Usage Scenarios

Building a multi-purpose end-to-end analytics environment

Sources

Ingest

Store

Process

Enrich

Serve

User

Managing IoT data

Processing and analyzing geospatial data

Enabling real-time analytics with big data

Performing time series analytics

Part 2: Working with Data

Chapter 5: Ingesting Data into Data Explorer Pools

Understanding the data loading process

Defining a retention policy

Choosing a data load strategy

Streaming ingestion

Batching ingestion

Performing data ingestion

Using KQL control commands

Building an Azure Synapse pipeline

Implementing continuous ingestion

Using other data ingestion mechanisms

Chapter 6: Data Analysis and Exploration with KQL and Python

Analyzing data with KQL

Selecting data

Working with calculated columns

Plotting charts

Obtaining percentiles

Creating a time series

Detecting outliers

Using linear regression

Exploring Data Explorer pool data with Python

Creating an Apache Spark pool

Working with Azure Synapse notebooks

Reading data from Data Explorer pools

Plotting charts.

Performing data transformation tasks

Creating a lake database

Chapter 7: Data Visualization with Power BI

Introduction to the Power BI integration

Creating a Power BI report

Adding data sources to your Power BI report

Connecting Power BI with your Azure Synapse workspace

Authoring Power BI reports from Azure Synapse Studio

Chapter 8: Building Machine Learning Experiments

Understanding the application of ML

Introducing ML into your projects with AutoML

Creating an Azure Machine Learning workspace

Configuring the Azure Machine Learning integration

Finding the best model with AutoML

Exploring additional ML capabilities in Azure Synapse

Using pre-trained models with Cognitive Services

Finding patterns using KQL

Training models with Apache Spark MLlib

Building applications with SynapseML

Chapter 9: Exporting Data from Data Explorer Pools

Understanding data export scenarios

Exporting data with client tools

Using server-side export to pull data

Performing robust exports with server-side data push

Exporting to cloud storage

Exporting to SQL tables

Exporting to external tables

Configuring continuous data export

Part 3: Managing Azure Synapse Data Explorer

Chapter 10: System Monitoring and Diagnostics

Monitoring your environment

Checking your Data Explorer pool capacity

Monitoring query execution

Reviewing object metadata and changes

Setting up alerts

Creating action groups

Creating alert rules

Chapter 11: Tuning and Resource Management

Implementing resource governance with workload groups

Managing workload groups

Classifying user requests.

Queuing requests for delayed execution

Speeding up queries using cache policies

Chapter 12: Securing Your Environment

Security overview

Managing data encryption

Configuring data encryption at rest

Understanding data encryption in transit

Authenticating users

Configuring access to resources

Synapse RBAC roles

Reviewing role assignments

Assigning RBAC roles

Data Explorer database roles

Implementing network security

Using a managed virtual network

Managed private endpoint connection

Enabling data exfiltration protection

Controlling public network access

Protecting against external threats

Advanced Data Management

Managing extents

Extent tagging

Moving extents

Dropping extents

Purging personal data

Enabling purge on Data Explorer pools

Executing data purge operations

Monitoring data purge operations

Index

Other Books You May Enjoy.

Notes:

Description based upon print version of record.

Choosing a data load strategy

Includes bibliographical references and index.

Description based on print version record.

ISBN:

9781803239613

1803239611

OCLC:

1369649206