Linux shell scripting cookbook : over 110 incredibly effective recipes to solve real-world problems, automate tedious tasks, and take advantage of Linux's newest features / Clif Flynt, Sarath Lakshman, Shantanu Tushar.

Format:

Book

Author/Creator:

Flynt, Clif, author.

Tushar, Shantanu, author.

Lakshman, Sarath, author.

Language:

English

Subjects (All):

Linux.

Scripting languages (Computer science).

Operating systems (Computers).

Physical Description:

1 online resource (1 volume) : illustrations

Edition:

Third edition.

Place of Publication:

Birmingham, UK : Packt Publishing, 2017.

System Details:

text file

Biography/History:

Flynt Clif: Clif Flynt has been programming computers since 1970, administering Linux/Unix systems since 1985, and writing since he was 9 years old. He's active in the Tcl/Tk and Linux user communities. He speaks frequently at technical conferences and user groups. He owns and runs Noumena Corporation, where he develops custom software and delivers training sessions. His applications have been used by organizations ranging from one manstartups to the US Navy. These applications range from distributed simulation systems to tools to help fiction authors write better (Editomat). He has trained programmers on fourcontinents. When not working with computers, Clif plays guitar, writes fiction experiments with new technologies, and plays with his wife's cats. He's the author of Tcl/Tk: A Developer's Guide by Morgan Kauffman, 2012, as well as several papers, and magazine articles. His poetry and fiction have been published in smalljournals, including Write to Meow by Grey Wolfe Press, 2015. Lakshman Sarath: Sarath Lakshman is a 27 year old who was bitten by the Linux bug during his teenage years. He is a software engineer working in ZCloud engineering group at Zynga, India. He is a life hacker who loves to explore innovations. He is a GNU/Linux enthusiast and hactivist of free and open source software. He spends most of his time hacking with computers and having fun with his great friends. Sarath is well known as the developer of SLYNUX (2005) - a user friendly GNU/Linux distribution for Linux newbies. The free and open source software projects he has contributed to are PiTiVi Video editor, SLYNUX GNU/Linux distro, Swathantra Malayalam Computing, School-Admin, Istanbul, and the Pardus Project. He has authored many articles for the Linux For You magazine on various domains of FOSS technologies. He had made a contribution to several different open source projects during his multiple Google Summer of Code projects. Currently, he is exploring his passion about scalable distributed systems in his spare time. Sarath can be reached via his website. Tushar Shantanu: Shantanu Tushar is an advanced GNU/Linux user since his college days. He works as an application developer and contributes to the software in the KDE projects. Shantanu has been fascinated by computers since he was a child, and spent most of his high school time writing C code to perform daily activities. Since he started using GNU/Linux, he has been using shell scripts to make the computer do all the hard work for him. He also takes time to visit students at various colleges to introduce them to the power of Free Software, including its various tools. Shantanu is a well-known contributor in the KDE community and works on Calligra, Gluon and the Plasma subprojects. He looks after maintaining Calligra Active - KDE's offie document viewer for tablets, Plasma Media Center, and the Gluon Player. One day, he believes, programming will be so easy that everybody will love to write programs for their computers. Shantanu can be reached by e-mail on shantanu@kde. org, shantanutushar on Identi. ca/Twitter, or his website.

Summary:

Do amazing things with the shell About This Book Become an expert in creating powerful shell scripts and explore the full possibilities of the shell Automate any administrative task you could imagine, with shell scripts Packed with easy-to-follow recipes on new features on Linux, particularly, Debian-based, to help you accomplish even the most complex tasks with ease Who This Book Is For If you are a beginner or an intermediate Linux user who wants to master the skill of quickly writing scripts and automate tasks without reading the entire man pages, then this book is for you. You can start writing scripts and one-liners by simply looking at the relevant recipe and its descriptions without any working knowledge of shell scripting or Linux. Intermediate / advanced users, system administrators / developers, and programmers can use this book as a reference when they face problems while coding. What You Will Learn Interact with websites via scripts Write shell scripts to mine and process data from the Web Automate system backups and other repetitive tasks with crontab Create, compress, and encrypt archives of your critical data. Configure and monitor Ethernet and wireless networks Monitor and log network and system activity Tune your system for optimal performance Improve your system's security Identify resource hogs and network bottlenecks Extract audio from video files Create web photo albums Use git or fossil to manage revision control and interact with FOSS projects Create and maintain Linux containers and Virtual Machines Run a private Cloud server In Detail The shell is the most powerful tool your computer provides. Despite having it at their fingertips, many users are unaware of how much the shell can accomplish. Using the shell, you can generate databases and web pages from sets of files, automate monotonous admin tasks such as system backups, monitor your system's health and activity, identify network bottlenecks and system resource hogs, and more. This book will show you how to do all this and much more. This book, now in its third edition, describes the exciting new features in the newest Linux distributions to help you accomplish more than you imagine. It shows how to use simple commands to automate complex tasks, automate web interactions, download videos, set up containers and cloud servers, and even get free SSL certificates. Starting with the basics of the shell, you will learn simple commands and how to apply them to real-world is...

Contents:

Cover

Copyright

Credits

About the Authors

About the Reviewer

www.PacktPub.com

Customer Feedback

Table of Contents

Preface

Chapter 1: Shell Something Out

Introduction

Displaying output in a terminal

Getting ready

How to do it...

How it works...

There's more...

Escaping newline in echo

Printing a colored output

Using variables and environment variables

Finding the length of a string

Identifying the current shell

Checking for super user

Modifying the Bash prompt string (username@hostname:~)

Function to prepend to environment variables

Math with the shell

Playing with file descriptors and redirection

Redirection from a file to a command

Redirecting from a text block enclosed within a script

Custom file descriptors

Arrays and associative arrays

Defining associative arrays

Listing of array indexes

Visiting aliases

Escaping aliases

Listing aliases

Grabbing information about the terminal

Getting and setting dates and delays

Producing delays in a script

Debugging the script

Shebang hack

Functions and arguments

The recursive function

Reading the return value (status) of a command

Passing arguments to commands

Sending output from one command to another

Spawning a separate process with subshell.

Subshell quoting to preserve spacing and the newline character

Reading n characters without pressing the return key

Running a command until it succeeds

A faster approach

Adding a delay

Field separators and iterators

Comparisons and tests

Customizing bash with configuration files

Chapter 2: Have a Good Command

Concatenating with cat

Getting rid of extra blank lines

Displaying tabs as ^I

Line numbers

Recording and playing back terminal sessions

Finding files and file listing

Search based on name or regular expression match

Negating arguments

Searching based on the directory depth

Searching based on file type

Searching by file timestamp

Searching based on file size

Matching based on file permissions and ownership

Performing actions on files with find

Deleting based on file matches

Executing a command

Skipping specified directories when using the find command

Playing with xargs

Passing formatted arguments to a command by reading stdin

Using xargs with find

Counting the number of lines of C code in a source code directory

While and subshell trick with stdin

Translating with tr

Deleting characters using tr

Complementing character sets

Squeezing characters with tr

Character classes

Checksum and verification

There's more.

Checksum for directories

Cryptographic tools and hashes

Sorting unique and duplicate lines

Sorting according to keys or columns

uniq

Temporary file naming and random numbers

Splitting files and data

Specifying a filename prefix for the split files

Slicing filenames based on extensions

Renaming and moving files in bulk

Spell-checking and dictionary manipulation

Automating interactive input

Automating with expect

Making commands quicker by running parallel processes

Examining a directory, files and subdirectories in it

Generating a tree view of a directory.

Generating a summary of files and sub-directories

Chapter 3: File In, File Out

Generating files of any size

The intersection and set difference (A-B) on text files

Finding and deleting duplicate files

Working with file permissions, ownership, and the sticky bit

Changing ownership

Setting the sticky bit

Applying permissions recursively to files

Applying ownership recursively

Running an executable as a different user (setuid)

Making files immutable

Generating blank files in bulk

How to do it.

Finding symbolic links and their targets

Enumerating file type statistics

Using loopback files

Creating partitions inside loopback images

Mounting loopback disk images with partitions more quickly

Mounting ISO files as loopback

Flush changing immediately with sync

Creating ISO files and hybrid ISO

Hybrid ISO that boots off a flash drive or hard disk

Burning an ISO from the command line

Playing with the CD-ROM tray

Finding the difference between files, and patching

Generating difference against directories

Using head and tail for printing the last or first 10 lines

Listing only directories - alternative methods

Fast command-line navigation using pushd and popd

pushd and popd are useful when there are more than three directory paths used. However, when you use only two locations, there is an alternative and easier way, that is, cd -.

Counting the number of lines, words, and characters in a file

Printing the directory tree

HTML output for tree

Manipulating video and image files

Extracting Audio from a movie file (mp4)

Making a video from a set of still images

Creating a panned video from a still camera shot

Chapter 4: Texting and Driving

Using regular expressions

Position markers

Identifiers.

Count modifiers

Other

Treatment of special characters

Visualizing regular expressions

Searching and mining text inside a file with grep

Recursively searching many files

Ignoring case in patterns

grep by matching multiple patterns

Including and excluding files in a grep search

Using grep with xargs with the zero-byte suffix

Silent output for grep

Printing lines before and after text matches

Cutting a file column-wise with cut

There's more

Specifying the range of characters or bytes as fields

Using sed to perform text replacement

Removing blank lines

Performing replacement directly in the file

Matched string notation ()

Substring match notation (\1)

Combining multiple expressions

Quoting

Using awk for advanced text processing

Special variables

Passing an external variable to awk

Reading a line explicitly using getline

Filtering lines processed by awk with filter patterns

Setting delimiters for fields

Reading the command output from awk

Associative arrays in Awk

Using loop inside awk

String manipulation functions in awk

Finding the frequency of words used in a given file

See also

Compressing or decompressing JavaScript

Merging multiple files as columns

Printing the nth word or column in a file or line

Printing text between line numbers or patterns

Printing lines in the reverse order.

Getting ready.

Notes:

Description based on online resource; title from title page (viewed June 28, 2017).

ISBN:

9781785882388

1785882384

OCLC:

992147697