Ceph cookbook : practical recipes to design, implement, operate, and manage Ceph storage systems / Vikhyat Umrao, Michael Hackett, Karan Singh ; foreword by Ian R. Colle.

Format:

Book

Author/Creator:

Umrao, Vikhyat, author.

Hackett, Michael, author.

Singh, Karan, author.

Contributor:

Colle, Ian R., writer of foreword.

Language:

English

Subjects (All):

Information storage and retrieval systems.

Big data.

Physical Description:

1 online resource (1 volume) : illustrations

Edition:

Second edition.

Place of Publication:

Birmingham, England ; Mumbai, [India] : Packt, 2017.

System Details:

text file

Biography/History:

Singh Karan: Karan is an IT expert and passionate tech enthusiast living with his beautiful wife Monika in Finland. He holds Honours degree in Bachelor of Computer Science and a Master degree in System Engineering from BITS, Pilani. Apart from this, he is a certified professional for technologies like OpenStack, NetApp and Oracle Solaris. Karan is currently working as a System Specialist of Storage and Cloud Platform for CSC - IT Center for Science Ltd. focusing all his energies on providing IaaS cloud solutions based on OpenStack and Ceph and building economic multi-petabyte storage system using Ceph. Karan possesses a rich skill set and working experience of a variety of cloud technologies. He devotes a part of his time to R&D and learning technologies. He is also the author of the very first book on Ceph titled Learning Ceph, published in 2014. Karan possesses extensive system administration skills and has excellent working experience on a variety of Unix environments, backup, enterprise storage systems, and cloud platforms. When not working on Ceph and OpenStack, Karan can be found working with Configuration management, containers and devops related tools. He loves writing about technologies and is an avid blogger. You can reach him on Twitter. Hackett Michael: Michael Hackett is a storage and SAN expert in customer support. He has been working on Ceph and storage-related products for over 12 years. Apart from this, he holds several storage and SAN-based certifications and prides himself on his ability to troubleshoot and adapt to new complex issues. Michael is currently working at Red Hat, based in Massachusetts, where he is a principal software maintenance engineer for Red Hat Ceph and the technical product lead for the global Ceph team. Michael lives in Massachusetts with his wife, Nicole, his two sons, and their dog. He is an avid sports fan and enjoys time with his family. Umrao Vikhyat: Vikhyat Umrao has 9 years of experience with distributed storage products as a sustenance engineer and in the last couple of years, he has been working on software-defined storage technology, with specific expertise in Ceph Unified Storage. He has been working on Ceph for over 3 years now and in his current position at Red Hat, he focuses on the support and development of Ceph to solve Red Hat Ceph storage customer issues and upstream reported issues. He is based in the Greater Boston area, where he is a principal software maintenance engineer for Red Hat Ceph Storage. Vikhyat lives with his wife, Pratima, and he likes to explore new places.

Summary:

Over 100 effective recipes to help you design, implement, and troubleshoot manage the software-defined and massively scalable Ceph storage system. About This Book Implement a Ceph cluster successfully and learn to manage it. Recipe based approach in learning the most efficient software defined storage system Implement best practices on improving efficiency and security of your storage cluster Learn to troubleshoot common issues experienced in a Ceph cluster Who This Book Is For This book is targeted at storage and cloud engineers, system administrators, or anyone who is interested in building software defined storage, to power your cloud or virtual infrastructure. If you have basic knowledge of GNU/Linux and storage systems, with no experience of software defined storage solutions and Ceph, but eager to learn then this book is for you What You Will Learn Understand, install, configure, and manage the Ceph storage system Get to grips with performance tuning and benchmarking, and learn practical tips to help run Ceph in production Integrate Ceph with OpenStack Cinder, Glance, and Nova components Deep dive into Ceph object storage, including S3, Swift, and Keystone integration Configure a disaster recovery solution with a Ceph Multi-Site V2 gateway setup and RADOS Block Device mirroring Gain hands-on experience with Ceph Metrics and VSM for cluster monitoring Familiarize yourself with Ceph operations such as maintenance, monitoring, and troubleshooting Understand advanced topics including erasure-coding, CRUSH map, cache pool, and general Ceph cluster maintenance In Detail Ceph is a unified distributed storage system designed for reliability and scalability. This technology has been transforming the software-defined storage industry and is evolving rapidly as a leader with its wide range of support for popular cloud platforms such as OpenStack, and CloudStack, and also for virtualized platforms. Ceph is backed by Red Hat and has been developed by community of developers which has gained immense traction in recent years. This book will guide you right from the basics of Ceph , such as creating blocks, object storage, and filesystem access, to advanced concepts such as cloud integration solutions. The book will also cover practical and easy to implement recipes on CephFS, RGW, and RBD with respect to the major stable release of Ceph Jewel. Towards the end of the book, recipes based on troubleshooting and best practices will help you get to grips wi...

Contents:

Cover

Copyright

Credits

Disclaimer

Foreword

About the Authors

About the Reviewer

www.PacktPub.com

Customer Feedback

Table of Contents

Preface

Chapter 1: Ceph - Introduction and Beyond

Introduction

Ceph - the beginning of a new era

Software-defined storage - SDS

Cloud storage

Unified next-generation storage architecture

RAID - the end of an era

RAID rebuilds are painful

RAID spare disks increases TCO

RAID can be expensive and hardware dependent

The growing RAID group is a challenge

The RAID reliability model is no longer promising

Ceph - the architectural overview

Planning a Ceph deployment

Setting up a virtual infrastructure

Getting ready

How to do it...

Installing and configuring Ceph

Creating the Ceph cluster on ceph-node1

Scaling up your Ceph cluster

How to do it…

Using the Ceph cluster with a hands-on approach

Chapter 2: Working with Ceph Block Device

Configuring Ceph client

Creating Ceph Block Device

Mapping Ceph Block Device

Resizing Ceph RBD

Working with RBD snapshots

Working with RBD clones

Disaster recovery replication using RBD mirroring

Configuring pools for RBD mirroring with one way replication

Configuring image mirroring

Configuring two-way mirroring

See also

Recovering from a disaster!

Chapter 3: Working with Ceph and OpenStack

Ceph - the best match for OpenStack

Setting up OpenStack

Configuring OpenStack as Ceph clients

Configuring Glance for Ceph backend

How to do it….

Configuring Cinder for Ceph backend

Configuring Nova to boot instances from Ceph RBD

Configuring Nova to attach Ceph RBD

Chapter 4: Working with Ceph Object Storage

Understanding Ceph object storage

RADOS Gateway standard setup, installation, and configuration

Setting up the RADOS Gateway node

Installing and configuring the RADOS Gateway

Creating the radosgw user

See also…

Accessing the Ceph object storage using S3 API

Configuring DNS

Configuring the s3cmd client

Configure the S3 client (s3cmd) on client-node1

Accessing the Ceph object storage using the Swift API

Integrating RADOS Gateway with OpenStack Keystone

Integrating RADOS Gateway with Hadoop S3A plugin

Chapter 5: Working with Ceph Object Storage Multi-Site v2

Functional changes from Hammer federated configuration

RGW multi-site v2 requirement

Installing the Ceph RGW multi-site v2 environment

Configuring Ceph RGW multi-site v2

Configuring a master zone

Configuring a secondary zone

Checking the synchronization status

Testing user, bucket, and object sync between master and secondary sites

Chapter 6: Working with the Ceph Filesystem

Understanding the Ceph Filesystem and MDS

Deploying Ceph MDS

Accessing Ceph FS through kernel driver

Accessing Ceph FS through FUSE client

Exporting the Ceph Filesystem as NFS

Ceph FS - a drop-in replacement for HDFS

Chapter 7: Monitoring Ceph Clusters

Monitoring Ceph clusters - the classic way.

How to do it...

Checking the cluster's health

Monitoring cluster events

The cluster utilization statistics

Checking the cluster's status

The cluster authentication entries

Monitoring Ceph MON

Checking the MON status

Checking the MON quorum status

Monitoring Ceph OSDs

OSD tree view

OSD statistics

Checking the CRUSH map

Monitoring PGs

Monitoring Ceph MDS

Introducing Ceph Metrics and Grafana

collectd

Grafana

Installing and configuring Ceph Metrics with the Grafana dashboard

Monitoring Ceph clusters with Ceph Metrics with the Grafana dashboard

How to do it ...

Chapter 8: Operating and Managing a Ceph Cluster

Understanding Ceph service management

Managing the cluster configuration file

Adding monitor nodes to the Ceph configuration file

Adding an MDS node to the Ceph configuration file

Adding OSD nodes to the Ceph configuration file

Running Ceph with systemd

Starting and stopping all daemons

Querying systemd units on a node

Starting and stopping all daemons by type

Starting and stopping a specific daemon

Scale-up versus scale-out

Scaling out your Ceph cluster

Adding the Ceph OSD

Adding the Ceph MON

There's more...

Scaling down your Ceph cluster

Removing the Ceph OSD

Removing the Ceph MON

Replacing a failed disk in the Ceph cluster

Upgrading your Ceph cluster

Maintaining a Ceph cluster

How it works...

Throttle the backfill and recovery:

Chapter 9: Ceph under the Hood

Ceph scalability and high availability

Understanding the CRUSH mechanism

CRUSH map internals

How to do it.

How it works...

CRUSH tunables

The evolution of CRUSH tunables

Argonaut - legacy

Firefly - CRUSH_TUNABLES3

Hammer - CRUSH_V4

Jewel - CRUSH_TUNABLES5

Ceph and kernel versions that support given tunables

Warning when tunables are non-optimal

A few important points

Ceph cluster map

High availability monitors

Ceph authentication and authorization

Ceph authentication

Ceph authorization

I/O path from a Ceph client to a Ceph cluster

Ceph Placement Group

Placement Group states

Creating Ceph pools on specific OSDs

Chapter 10: Production Planning and Performance Tuning for Ceph

The dynamics of capacity, performance, and cost

Choosing hardware and software components for Ceph

Processor

Memory

Network

Disk

Partitioning the Ceph OSD journal

Partitioning Ceph OSD data

Operating system

OSD filesystem

Ceph recommendations and performance tuning

Tuning global clusters

Tuning Monitor

OSD tuning

OSD general settings

OSD journal settings

OSD filestore settings

OSD recovery settings

OSD backfilling settings

OSD scrubbing settings

Tuning the client

Tuning the operating system

Tuning the network

Sample tuning profile for OSD nodes

Ceph erasure-coding

Erasure code plugin

Creating an erasure-coded pool

Ceph cache tiering

Writeback mode

Read-only mode

Creating a pool for cache tiering

Creating a cache tier

Configuring a cache tier

Testing a cache tier

Cache tiering - possible dangers in production environments

Known good workloads

Known bad workloads

Chapter 11: The Virtual Storage Manager for Ceph

Introductionc.

Understanding the VSM architecture

The VSM controller

The VSM agent

Setting up the VSM environment

Getting ready for VSM

Installing VSM

Creating a Ceph cluster using VSM

Exploring the VSM dashboard

Upgrading the Ceph cluster using VSM

VSM roadmap

VSM resources

Chapter 12: More on Ceph

Disk performance baseline

Single disk write performance

Multiple disk write performance

Single disk read performance

Multiple disk read performance

Results

Baseline network performance

Ceph rados bench

RADOS load-gen

Benchmarking the Ceph Block Device

Benchmarking Ceph RBD using FIO

See Also

Ceph admin socket

Using the ceph tell command

Ceph REST API

Profiling Ceph memory

The ceph-objectstore-tool

Using ceph-medic

Deploying the experimental Ceph BlueStore

Chapter 13: An Introduction to Troubleshooting Ceph

Initial troubleshooting and logging

Troubleshooting network issues

Troubleshooting monitors

Troubleshooting OSDs

Troubleshooting placement groups

There's more…

Chapter 14: Upgrading Your Ceph Cluster from Hammer to Jewel

Upgrading your Ceph cluster from Hammer to Jewel.

How to do it.

Notes:

Includes index.

Description based on online resource; title from PDF title page (EBC, viewed December 23, 2017).

ISBN:

9781788392150

1788392159

OCLC:

1019901936