Mastering Prometheus : Gain Expert Tips to Monitoring Your Infrastructure, Applications, and Services / William Hegedus.

Format:

Book

Author/Creator:

Hegedus, William, author.

Language:

English

Subjects (All):

Prometheus (Computer file).

Computer networks--Monitoring.

Computer networks.

Computer networks--Management.

Physical Description:

1 online resource (310 pages)

Edition:

First edition.

Place of Publication:

Birmingham, England : Packt Publishing, [2024]

Summary:

Learn how to effectively implement, manage, and optimize Prometheus for monitoring your systems Key Features Achieve high availability with Prometheus by using Thanos Integrate Prometheus into your broader observability stack with OpenTelemetry Tweak, tune, and debug Prometheus to reliably scale without limits Purchase of the print or Kindle book includes a free PDF eBook Book Description With an increased focus on observability and reliability, establishing a scalable and reliable monitoring environment is more important than ever. Over the last decade, Prometheus has emerged as the leading open-source, time-series based monitoring software catering to this demand. This book is your guide to scaling, operating, and extending Prometheus from small on-premises workloads to multi-cloud globally distributed workloads and everything in between. Starting with an introduction to Prometheus and its role in observability, the book provides a walkthrough of its deployment. You'll explore Prometheus's query language and TSDB data model, followed by dynamic service discovery for monitoring targets and refining alerting through custom templates and formatting. The book then demonstrates horizontal scaling of Prometheus via sharding and federation, while equipping you with debugging techniques and strategies to fine-tune data ingestion. Advancing through the chapters, you'll manage Prometheus at scale through CI validations and templating with Jsonnet, and integrate Prometheus with other projects such as OpenTelemetry, Thanos, VictoriaMetrics, and Mimir. By the end of this book, you'll have practical knowledge of Prometheus and its ecosystem, which will help you discern when, why, and how to scale it to meet your ever-growing needs. What you will learn Deploy Prometheus and Node Exporter to public clouds and Kubernetes Gain in-depth knowledge of how Prometheus's underlying code works Build your own custom service-discovery providers for Prometheus Debug Prometheus performance issues to identify cardinality issues in your environment Use VictoriaMetrics and/or Grafana Mimir for remote storage of Prometheus data Define and implement SLO-based alerting Who this book is for The book is for site reliability engineers (SREs), developers, and platform engineers involved in the monitoring and observability of their team or company's systems. A background in Prometheus is assumed, so the book dedicates minimal time to the basics of getting Prometheus up and running. Whether you aim to expand monitoring capabilities, streamline configuration management, or enhance integration with existing tools, this book will help you maximize the potential of your Prometheus monitoring stack.

Contents:

Cover

Copyright

Contributors

Table of Contents

Preface

Part 1: Fundamentals of Prometheus

Chapter 1: Observability, Monitoring, and Prometheus

A brief history of monitoring

Nagios

A word on SNMP

Enter the cloud

Introduction to observability concepts

Metrics

Logs

Traces

Other signals

Tying signals together

Getting data out of systems

Prometheus's role in observability

Alerting

Dashboarding

What Prometheus is not

Summary

Further reading

Chapter 2: Deploying Prometheus

Technical requirements

Components of a Prometheus stack

Prometheus

Alertmanager

Node Exporter

Grafana

Provisioning Kubernetes

Configuring the linode-cli

Creating a Kubernetes cluster

Deploying Prometheus

Prometheus Operator overview

Deploying kube-prometheus

Chapter 3: The Prometheus Data Model and PromQL

Prometheus's data model

Prometheus' TSDB

Head block

WAL

Blocks and chunks

Index

Compaction

PromQL basics

Syntax overview

Query operators

Query functions

Chapter 4: Using Service Discovery

Service discovery overview

Using service discovery

Relabeling

Using service discovery in a cloud provider

Linode service discovery

Custom service discovery endpoints with HTTP SD

Chapter 5: Effective Alerting with Prometheus

Alertmanager configuration and routing

Routing

Receivers

Inhibitions

Validating

Alertmanager templating

Configuring templates

Defining your own templates

Highly available (HA) alerting

Cluster sizing

Making robust alerts

Use logical/set binary operators

Use appropriate "for" durations.

Use _over_time functions

Anomaly detection

Unit-testing alerting rules

Part 2: Scaling Prometheus

Chapter 6: Advancing Prometheus - Sharding, Federation, and High Availability

Prometheus' limitations

Cardinality

Long-term storage

Sharding Prometheus

Sharding by service

Sharding with relabeling

Federating Prometheus

Achieving high availability (HA) in Prometheus

HA via the Prometheus Operator

Cleanup

Chapter 7: Optimizing and Debugging Prometheus

Controlling cardinality

Identifying cardinality issues

Remediating cardinality issues

Using limits

Recording rules

Recording rule conventions

Scrape jitter

Using pprof

Using promtool for pprof data

Query logging and limits

Query logging

Query limits

Tuning garbage collection

Using GOMEMLIMIT

Chapter 8: Enabling Systems Monitoring with the Node Exporter

Node Exporter overview

What is in an exporter?

Default collectors

conntrack

cpu

diskstats

filesystem

loadavg

meminfo

netdev

pressure

Others

The textfile collector

Troubleshooting the Node Exporter

Part 3: Extending Prometheus

Chapter 9: Utilizing Remote Storage Systems with Prometheus

Understanding remote write and remote read

Remote read

Remote write

Using VictoriaMetrics

Deployment methods

Deploying to Kubernetes

Using Grafana Mimir

Comparing to VictoriaMetrics

Chapter 10: Extending Prometheus Globally with Thanos

Overview of Thanos

Why use Thanos?

Thanos Sidecar.

Deploying Thanos Sidecar

Thanos Compactor

Vertical compaction

Downsampling

Deploying Thanos Compactor

Thanos Query

Deploying Thanos Query

Scaling Thanos Query

Thanos Query Frontend

Query sharding and splitting

Caching

Deploying Thanos Query Frontend

Thanos Store

Deploying Thanos Store

Scaling Thanos Store

Thanos Ruler

Stateless mode

Deploying Thanos Ruler

Thanos Receiver

Deploying Thanos Receiver

Thanos tools

Chapter 11: Jsonnet and Monitoring Mixins

Overview of Jsonnet

Syntax

Using Jsonnet

Generating files

Formatting and linting

Monitoring Mixins

Mixin structure

Using and extending mixins

Chapter 12: Utilizing Continuous Integration Pipelines with Prometheus

GitHub Actions

Validation in CI

Using promtool

Using amtool

Linting Prometheus rules with Pint

Configuring Pint

Integrating Pint with CI

Chapter 13: Defining and Alerting on SLOs

Understanding SLIs, SLOs, and SLAs

Why SLOs matter

Types of SLOs

Defining SLOs with Prometheus data

Window-based SLOs

Alerting on SLOs

Using Sloth and Pyrra for SLOs

Sloth

Pyrra

Chapter 14: Integrating Prometheus with OpenTelemetry

Introducing OpenTelemetry

OTel specification

OpenTelemetry line protocol

OpenTelemetry collector

Collecting Prometheus metrics with the OpenTelemetry collector

Sending metrics to Prometheus with the OpenTelemetry collector

Configuring Prometheus

Configuring OpenTelemetry collector

Chapter 15: Beyond Prometheus

Technical requirements.

Extending observability past Prometheus

Connecting the dots across observability systems

Logging with Loki

Tracing with Tempo

Other Books You May Enjoy.

Notes:

Includes index.

Description based on publisher supplied metadata and other sources.

Description based on print version record.

ISBN:

9781805125327

180512532X

OCLC:

1428132531