Data Science with . NET and Polyglot Notebooks : Programmer's Guide to Data Science Using ML. NET, OpenAI, and Semantic Kernel / Matt Eland.

Format:

Book

Author/Creator:

Eland, Matt, author.

Language:

English

Subjects (All):

Microsoft .NET Framework.

Application software--Development.

Application software.

Physical Description:

1 online resource (404 pages)

Edition:

First edition.

Place of Publication:

Birmingham, England : Packt Publishing, [2024]

Biography/History:

Eland Matt: Matt Eland is a senior software engineering and data science consultant at Leading EDJE in Columbus, Ohio. He loves sharing his journey by teaching software engineering, AI, and data science concepts in the most engaging ways possible. Matt has used machine learning to settle debates over whether certain movies are Christmas movies, reinforcement learning to create digital attack squirrels, data analytics to suggest improvements to his favorite TV show, and AI agents to play board games and create an AI agent with the personality of a dog. He is the author of multiple books and courses, helps organize a user group and regional conferences, holds a Master's of Science in Data Analytics, and is a 2x Microsoft MVP in AI and. NET.

Summary:

Expand your skillset by learning how to perform data science, machine learning, and generative AI experiments in .NET Interactive notebooks using a variety of languages, including C#, F#, SQL, and PowerShellKey FeaturesConduct a full range of data science experiments with clear explanations from start to finishLearn key concepts in data analytics, machine learning, and AI and apply them to solve real-world problemsAccess all of the code online as a notebook and interactive GitHub CodespacePurchase of the print or Kindle book includes a free PDF eBookBook DescriptionAs the fields of data science, machine learning, and artificial intelligence rapidly evolve, .NET developers are eager to leverage their expertise to dive into these exciting domains but are often unsure of how to do so. Data Science in .NET with Polyglot Notebooks is the practical guide you need to seamlessly bring your .NET skills into the world of analytics and AI. With Microsoft’s .NET platform now robustly supporting machine learning and AI tasks, the introduction of tools such as .NET Interactive kernels and Polyglot Notebooks has opened up a world of possibilities for .NET developers. This book empowers you to harness the full potential of these cutting-edge technologies, guiding you through hands-on experiments that illustrate key concepts and principles. Through a series of interactive notebooks, you’ll not only master technical processes but also discover how to integrate these new skills into your current role or pivot to exciting opportunities in the data science field. By the end of the book, you’ll have acquired the necessary knowledge and confidence to apply cutting-edge data science techniques and deliver impactful solutions within the .NET ecosystem.What you will learnLoad, analyze, and transform data using DataFrames, data visualization, and descriptive statisticsTrain machine learning models with ML.NET for classification and regression tasksCustomize ML.NET model training pipelines with AutoML, transforms, and model trainersApply best practices for deploying models and monitoring their performanceConnect to generative AI models using Polyglot NotebooksChain together complex AI tasks with AI orchestration, RAG, and Semantic KernelCreate interactive online documentation with Mermaid charts and GitHub CodespacesWho this book is forThis book is for experienced C# or F# developers who want to transition into data science and machine learning while leveraging their .NET expertise. It’s ideal for those looking to learn ML.NET and Semantic kernel and extend their .NET skills to data science, machine learning, and Generative AI Workflows.

Contents:

Cover

Title Page

Copyright

Dedication

Contributors

Table of Contents

Preface

Part 1: Data Analysis in Polyglot Notebooks

Chapter 1: Data Science, Notebooks, and Kernels

Exploring the field of data science

The rise of big data

Data analytics

Machine learning

Artificial intelligence

Data science notebooks and Project Jupyter

Extending notebooks with kernels

Polyglot Notebooks and .NET Interactive

Summary

Further reading

Chapter 2: Exploring Polyglot Notebooks

Technical requirements

Installing Polyglot Notebooks

Creating your first notebook

Executing notebook cells

Adding code cells

Working with variables

The Variables view

Markdown cells

Declaring classes and methods

Declaring methods

Declaring classes

Working with other languages

Sharing variables between languages

Exporting variables

Troubleshooting notebook execution

Resolving compiler errors

Problems with notebook execution

Diagnostic output for Polyglot Notebooks errors

Issues and the Polyglot Notebooks repository

Chapter 3: Getting Data and Code into Your Notebooks

Importing code and NuGet packages

Importing code files

Importing NuGet packages

Importing project files

Reading CSV data

Understanding CSV data

Reading CSV data into a DataFrame

Specialized CSV loading scenarios

Troubleshooting CSV loading errors

Loading TSV and other delimited file formats

Getting JSON data with PowerShell

Building DataFrames from objects

Connecting to databases with SQL

Connecting to a SQL database

Executing SQL from SQL kernels

Sharing SQL results with other kernels

Alternative ways of connecting to the Database

Querying Kusto clusters with KQL

Further reading.

Chapter 4: Working with Tabular Data and DataFrames

Understanding data cleaning and data wrangling

Where unclean data comes from

The impact of unclean data

Data cleaning and data wrangling

Working with DataFrames in C#

Viewing and sampling data

Rows

Getting and setting cell values

Iterating over rows

Working with columns

Columns

Analyzing columns

Removing columns

Renaming columns

Adding a new column

Handling missing values

Sorting, filtering, grouping, and merging data

Sorting DataFrames

Grouping and aggregating DataFrames

Merging DataFrames

Filtering DataFrames

DataFrames in other languages

Chapter 5: Visualizing Data

Understanding exploratory data analysis

Data visualization's role in exploratory data analysis

Descriptive statistics for EDA

Extracting insights with descriptive statistics

Using DataFrame.Description to generate descriptive statistics

Descriptive statistics with MathNet.Numerics

Creating a box plot with ScottPlot

Performing univariate analysis with Plotly.NET

Plotly and Plotly.NET

Box plots in Plotly.NET

Violin plots with Plotly.NET

Histograms with Plotly.NET

Chapter 6: Variable Correlations

Performing multivariate analysis with Plotly.NET

Loading data and dependencies

Multivariate analysis with box and violin plots

Plotting multiple values with scatter plots

Adding color to a scatter plot

3D scatter plots with Plotly.NET

Identifying variable correlations

Calculating variable correlations

Building feature correlation matrixes

Part 2: Machine Learning with Polyglot Notebooks and ML.NET.

Chapter 7: Classification Experiments with ML.NET AutoML

Understanding machine learning

Supervised learning

Classification and regression

Introducing ML.NET and AutoML

Understanding AutoML

AutoML and data pre-processing

Creating training and testing datasets

Training a classification model with ML.NET AutoML

Evaluating binary classification models

Evaluating our model

Calculating feature importance

Predicting values with binary classification models

Chapter 8: Regression Experiments with ML.NET AutoML

Understanding regression

Our regression task

Regression as a numerical formula

Our regression dataset

Performing a regression experiment

Understanding cross-validation

Interpreting cross-validation results

Evaluating regression metrics

Predicting values for outliers

Applying PFI to regression models

Applying a regression model

Chapter 9: Beyond AutoML: Pipelines, Trainers, and Transforms

Performing regression without AutoML

Features and pipelines

Creating an AutoML pipeline

Controlling AutoML pipelines

Customizing the Featurizer

Customizing the model trainer selector

Customizing hyperparameter tuning

Understanding the search space

Customizing the search space

Customizing the hyperparameter tuner

Scaling numeric columns

Selecting regression algorithms

Selecting binary classification algorithms

Chapter 10: Deploying Machine Learning Models

Introducing our multi-class classification model

Training our model

Evaluating multi-class classification models

Generating test predictions

Exporting ML.NET models.

Hosting ML.NET models in ASP.NET web applications

Configuring a PredictionEnginePool

Using the PredictionEnginePool

Understanding model performance, data drift, and MLOps

Detecting model drift

MLOps and updating models

Surveying additional ML.NET capabilities

ONNX and TensorFlow models in ML.NET

Part 3: Exploring Generative AI with Polyglot Notebooks

Chapter 11: Generative AI in Polyglot Notebooks

Understanding Generative AI

Deploying generative AI models on Azure

Creating an Azure OpenAI Service

Deploying models on Azure OpenAI Service

Getting access credentials for Azure OpenAI

Connecting to an Azure OpenAI Service

Chatting with a deployed model

Customizing model behavior with prompt engineering

Zero-shot, one-shot, and few-shot inferencing

Using text embeddings

Generating images with DALL-E

Chapter 12: AI Orchestration with Semantic Kernel

Understanding RAG and AI orchestration

Introducing Semantic Kernel

Chatting with Semantic Kernel functions

Building the Kernel

Creating a prompt function

Adding memory to Semantic Kernel

Defining complex functions

Creating functions from methods

Accepting KernelFunction parameters

Defining a memory function

Calling multiple functions using plugins

Examining FunctionResult objects

Azure OpenAI content filtering

Handling complex requests with planners

Knowing where to go from here

Part 4: Polyglot Notebooks in the Enterprise

Chapter 13: Enriching Documentation with Mermaid Diagrams

Introducing Mermaid diagrams

Communicating logic with flowcharts

Communicating structure with class diagrams.

Communicating data with Entity Relationship Diagrams

Communicating behavior with state diagrams

Communicating flow with sequence diagrams

Communicating workflow with Git graphs

Chapter 14: Extending Polyglot Notebooks

Understanding default formatting behavior

Default object formatting

Default collection formatting

Styling output with custom formatters

Exploring magic commands

Creating a Polyglot Notebook extension

Working with parameters

Invoking code on kernels

Chapter 15: Adopting and Deploying Polyglot Notebooks

Integrating Polyglot Notebooks into your day job

Enabling rapid experimentation

Supporting AI and analytics workloads

Assisting testing workloads

Training new team members with Polyglot Notebooks

Sharing Polyglot Notebooks with your team

Integrating Polyglot Notebooks into Jupyter or JupyterLab

Storing Notebooks in source control

Deploying Polyglot Notebooks to GitHub Codespaces

Configuring GitHub codespaces

Creating a codespace on GitHub

Advancing into machine learning and AI

Adding data science to your day job

Getting into data science

Succeeding in data science

Index

About PACKT

Other Books You May Enjoy.

Notes:

Includes index.

Description based on publisher supplied metadata and other sources.

Description based on print version record.

ISBN:

9781835882979

1835882978

OCLC:

1453914924