LLM Engineer's Handbook : Master the Art of Engineering Large Language Models from Concept to Production / Paul Iusztin, Maxime Labonne.

Format:

Book

Author/Creator:

Iusztin, Paul, author.

Labonne, Maxime, author.

Series:

Expert insight.

Expert insight

Language:

English

Subjects (All):

Natural language processing (Computer science).

Machine learning.

Artificial intelligence.

Physical Description:

1 online resource (300 pages) : illustrations

Edition:

First edition.

Place of Publication:

Birmingham, England : Packt Publishing, [2024]

Biography/History:

Iusztin Paul: Paul Iusztin is a senior ML and MLOps engineer at Metaphysic, a leading GenAI platform, serving as one of their core engineers in taking their deep learning products to production. Along with Metaphysic, with over seven years of experience, he built GenAI, Computer Vision and MLOps solutions for CoreAI, Everseen, and Continental. Paul's determined passion and mission are to build data-intensive AI/ML products that serve the world and educate others about the process. As the Founder of Decoding ML, a channel for battle-tested content on learning how to design, code, and deploy production-grade ML, Paul has significantly enriched the engineering and MLOps community. His weekly content on ML engineering and his open-source courses focusing on end-to-end ML life cycles, such as Hands-on LLMs and LLM Twin, testify to his valuable contributions. Labonne Maxime: Maxime Labonne is a Senior Staff Machine Learning Scientist at Liquid AI, serving as the head of post-training. He holds a Ph. D. in Machine Learning from the Polytechnic Institute of Paris and is recognized as a Google Developer Expert in AI/ML. An active blogger, he has made significant contributions to the open-source community, including the LLM Course on GitHub, tools such as LLM AutoEval, and several state-of-the-art models like NeuralBeagle and Phixtral. He is the author of the best-selling book Hands-On Graph Neural Networks Using Python, published by Packt.

Summary:

The field of Artificial Intelligence has undergone rapid advancements, and Large Language Models (LLMs) are at the forefront of this revolution. This LLM book provides practical insights into designing, training, and deploying LLMs in real-world scenarios by leveraging MLOps best practices. This comprehensive guide walks you through building an end-to-end LLM-powered technical content writer, by overcoming isolated Jupyter Notebooks and focusing on teaching how to build production-grade end-to-end LLM systems. Throughout this book, you will learn data engineering, supervised fine-tuning, and deployment . The hands-on approach, combined with detailed examples, helps you understand the implementation of MLOps components in your projects. The book also explores the cutting-edge advancements in the field, including inference optimization and real-time data processing, making it a vital resource for anyone looking to leverage LLMs in their projects. By the end of this book, you will be proficient in deploying robust large language models, leveraging them to solve practical problems, and maintaining low-latency and high-availability inference capabilities. Whether you are new to AI or an experienced practitioner, this book offers valuable insights and practical knowledge to enhance your expertise in LLMs.

Contents:

Cover

Copyright

Contributors

Table of Contents

Preface

Chapter 1: Understanding the LLM Twin Concept and Its Architecture

Understanding the LLM twin concept

What is an LLM twin?

Why building an LLM twin matters

Why not use ChatGPT (or another similar chatbot)?

Planning the MVP of the LLM twin product

What is an MVP?

Defining the LLM twin MVP

Building ML systems with feature/training/inference pipelines

The problem with building ML systems

The issue with previous solutions

The solution - ML pipelines for ML systems

The feature pipeline

The training pipeline

The inference pipeline

Benefits of the FTI architecture

Designing the system architecture of the LLM twin

Listing the technical details of the LLM twin architecture

How to design the LLM twin architecture using the FTI pipeline design

Data collection pipeline

Feature pipeline

Training pipeline

Inference pipeline

Final thoughts on the FTI design and the LLM twin architecture

Summary

References

Chapter 2: Tooling and Installation

Python ecosystem and project installation

Poetry: dependency and virtual environment management

Poe the Poet: task execution tool

MLOps and LLMOps tooling

Hugging Face: model registry

ZenML: orchestrator, artifacts, and metadata

Orchestrator

Artifacts and metadata

How to run and configure a ZenML pipeline

Comet ML: experiment tracker

Opik: prompt monitoring

Databases for storing unstructured and vector data

MongoDB: NoSQL database

Qdrant: vector database

Preparing for AWS

Setting up an AWS account, an access key, and the CLI

SageMaker: training and inference compute

Why AWS SageMaker?

Chapter 3: Data Engineering

Designing the LLM Twin's data collection pipeline.

Implementing the LLM Twin's data collection pipeline

ZenML pipeline and steps

The dispatcher: How do you instantiate the right crawler?

The crawlers

Base classes

GitHubCrawler class

CustomArticleCrawler class

MediumCrawler class

The NoSQL data warehouse documents

The ORM and ODM software patterns

Implementing the ODM class

Data categories and user document classes

Gathering raw data into the data warehouse

Troubleshooting

Selenium issues

Import our backed-up data

Chapter 4: RAG Feature Pipeline

Understanding RAG

Why use RAG?

Hallucinations

Old information

The vanilla RAG framework

Ingestion pipeline

Retrieval pipeline

Generation pipeline

What are embeddings?

Why embeddings are so powerful

How are embeddings created?

Applications of embeddings

More on vector DBs

How does a vector DB work?

Algorithms for creating the vector index

DB operations

An overview of advanced RAG

Pre-retrieval

Retrieval

Post-retrieval

Exploring the LLM Twin's RAG feature pipeline architecture

The problem we are solving

The feature store

Where does the raw data come from?

Designing the architecture of the RAG feature pipeline

Batch pipelines

Batch versus streaming pipelines

Core steps

Change data capture: syncing the data warehouse and feature store

Why is the data stored in two snapshots?

Orchestration

Implementing the LLM Twin's RAG feature pipeline

Settings

Querying the data warehouse

Cleaning the documents

Chunk and embed the cleaned documents

Loading the documents to the vector DB

Pydantic domain entities

OVM

The dispatcher layer

The handlers

The cleaning handlers

The chunking handlers

The embedding handlers

References.

Chapter 5: Supervised Fine-Tuning

Creating an instruction dataset

General framework

Data quantity

Data curation

Rule-based filtering

Data deduplication

Data decontamination

Data quality evaluation

Data exploration

Data generation

Data augmentation

Creating our own instruction dataset

Exploring SFT and its techniques

When to fine-tune

Instruction dataset formats

Chat templates

Parameter-efficient fine-tuning techniques

Full fine-tuning

LoRA

QLoRA

Training parameters

Learning rate and scheduler

Batch size

Maximum length and packing

Number of epochs

Optimizers

Weight decay

Gradient checkpointing

Fine-tuning in practice

Chapter 6: Fine-Tuning with Preference Alignment

Understanding preference datasets

Preference data

Data generation and evaluation

Generating preferences

Tips for data generation

Evaluating preferences

Creating our own preference dataset

Preference alignment

Reinforcement Learning from Human Feedback

Direct Preference Optimization

Implementing DPO

Chapter 7: Evaluating LLMs

Model evaluation

Comparing ML and LLM evaluation

General-purpose LLM evaluations

Domain-specific LLM evaluations

Task-specific LLM evaluations

RAG evaluation

Ragas

ARES

Evaluating TwinLlama-3.1-8B

Generating answers

Evaluating answers

Analyzing results

Chapter 8: Inference Optimization

Model optimization strategies

KV cache

Continuous batching

Speculative decoding

Optimized attention mechanisms

Model parallelism

Data parallelism

Pipeline parallelism

Tensor parallelism

Combining approaches

Model quantization

Introduction to quantization

Quantization with GGUF and llama.cpp.

Quantization with GPTQ and EXL2

Other quantization techniques

Chapter 9: RAG Inference Pipeline

Understanding the LLM twin's RAG inference pipeline

Exploring the LLM twin's advanced RAG techniques

Advanced RAG pre-retrieval optimizations: query expansion and self-querying

Query expansion

Self-querying

Advanced RAG retrieval optimization: filtered vector search

Advanced RAG post-retrieval optimization: reranking

Implementing the LLM twin's RAG inference pipeline

Implementing the retrieval module

Bringing everything together into the RAG inference pipeline

Chapter 10: Inference Pipeline Deployment

Criteria for choosing deployment types

Throughput and latency

Data

Understanding inference deployment types

Online real-time inference

Asynchronous inference

Offline batch transform

Monolithic versus microservices architecture in model serving

Monolithic architecture

Microservices architecture

Choosing between monolithic and microservices architectures

Exploring the LLM Twin's inference pipeline deployment strategy

The training versus the inference pipeline

Deploying the LLM Twin service

Implementing the LLM microservice using AWS SageMaker

What are Hugging Face's DLCs?

Configuring SageMaker roles

Deploying the LLM Twin model to AWS SageMaker

Calling the AWS SageMaker Inference endpoint

Building the business microservice using FastAPI

Autoscaling capabilities to handle spikes in usage

Registering a scalable target

Creating a scalable policy

Minimum and maximum scaling limits

Cooldown period

Chapter 11: MLOps and LLMOps

The path to LLMOps: Understanding its roots in DevOps and MLOps

DevOps

The DevOps lifecycle

The core DevOps concepts

MLOps.

MLOps core components

MLOps principles

ML vs. MLOps engineering

LLMOps

Human feedback

Guardrails

Prompt monitoring

Deploying the LLM Twin's pipelines to the cloud

Understanding the infrastructure

Setting up MongoDB

Setting up Qdrant

Setting up the ZenML cloud

Containerize the code using Docker

Run the pipelines on AWS

Troubleshooting the ResourceLimitExceeded error after running a ZenML pipeline on SageMaker

Adding LLMOps to the LLM Twin

LLM Twin's CI/CD pipeline flow

More on formatting errors

More on linting errors

Quick overview of GitHub Actions

The CI pipeline

GitHub Actions CI YAML file

The CD pipeline

Test out the CI/CD pipeline

The CT pipeline

Initial triggers

Trigger downstream pipelines

Alerting

Appendix: MLOps Principles

1. Automation or operationalization

2. Versioning

3. Experiment tracking

4. Testing

Test types

What do we test?

Test examples

5. Monitoring

Logs

Metrics

System metrics

Model metrics

Drifts

Monitoring vs. observability

Alerts

6. Reproducibility

Packt Page

Other Books You May Enjoy

Index.

Notes:

Description based upon print version of record.

The problem we are solving

Includes bibliographical references and index.

OCLC-licensed vendor bibliographic record.

Description based on print version record.

ISBN:

1-83620-006-4

1-83620-007-2

OCLC:

1456140393