t.

me/DevTwitter
 About This E-Book
EPUB is an open, industry-standard format for e-books. However,
support for EPUB and its many features varies across reading devices and
applications. Use your device or app settings to customize the presentation
to your liking. Settings that you can customize often include font, font size,
single or double column, landscape or portrait mode, and figures that you
can click or tap to enlarge. For additional information about the settings and
features on your reading device or app, visit the device manufacturer’s Web
site.
Many titles include programming code or configuration examples. To
optimize the presentation of these elements, view the e-book in single-
column, landscape mode and adjust the font size to the smallest setting. In
addition to presenting code and configurations in the reflowable text format,
we have included images of the code that mimic the presentation found in
the print book; therefore, where the reflowable format may compromise the
presentation of the code listing, you will see a “Click here to view code
image“ link. Click the link to view the print-fidelity code image. To return
to the previous page viewed, click the Back button on your device or app.

t.me/DevTwitter
t.me/DevTwitter
 Linus for Developers
Jumpstart Your Linux Programming Skills

William “Bo” Rothwell

Boston • Columbus • Indianapolis • New York • San Francisco •

Amsterdam
Cape Town Dubai • London • Madrid • Milan • Munich • Paris • Montreal
Toronto • Delhi • Mexico City Sao Paulo • Sidney • Hong Kong • Seoul
Singapore • Taipei • Tokyo

t.me/DevTwitter
 Many of the designations used by manufacturers and sellers to distinguish
their products are claimed as trademarks. Where those designations appear
in this book, and the publisher was aware of a trademark claim, the
designations have been printed with initial capital letters or in all capitals.
The author and publisher have taken care in the preparation of this book,
but make no expressed or implied warranty of any kind and assume no
responsibility for errors or omissions. No liability is assumed for incidental
or consequential damages in connection with or arising out of the use of the
information or programs contained herein.
For information about buying this title in bulk quantities, or for special sales
opportunities (which may include electronic versions; custom cover
designs; and content particular to your business, training goals, marketing
focus, or branding interests), please contact our corporate sales department
at [email protected] or (800) 382-3419.
For government sales inquiries, please contact
[email protected].
For questions about sales outside the U.S., please contact
[email protected].
Visit us on the Web: informit.com/aw
Library of Congress Control Number: 2017932512
Copyright © 2017 Pearson Education, Inc.
All rights reserved. Printed in the United States of America. This
publication is protected by copyright, and permission must be obtained
from the publisher prior to any prohibited reproduction, storage in a
retrieval system, or transmission in any form or by any means, electronic,
mechanical, photocopying, recording, or likewise. For information
regarding permissions, request forms and the appropriate contacts within
the Pearson Education Global Rights & Permissions Department, please
visit www.pearsoned.com/permissions/.
ISBN-13: 978-0-13-465728-8
ISBN-10: 0-13-465728-4
1 17

t.me/DevTwitter
 Editor-in-Chief
Mark Taub
Executive Editor
Debra Williams Cauley
Development Editor
Chris Zahn
Managing Editor
Sandra Schroeder
Senior Project Editor
Lori Lyons
Project Manager
Dhayanidhi
Copy Editor
Paula Lowell
Indexer
Cheryl Lenser
Proofreader
SathishKumar
Technical Reviewers
Matthew Helmke
Keith Wright
Editorial Assistant
Kim Boedigheimer
Cover Designer
Chuti Prasertsith
Compositor
codeMantra

t.me/DevTwitter
 “A best friend is the only one that walks into your life when the world has
walked out.”
— Shannon L. Alder

Thank you, Sarah, my love and my wife, for walking into my life.

“Strong people don't put others down... They lift them up.”
— Michael P. Watson

Thank you, Mom and Dad, for being supportive.

“Fall seven times, stand up eight.”

— Japanese Proverb

Thank you, Julia, for being understanding.

t.me/DevTwitter
 Contents at a Glance
Preface
I: Open Source Software
1 Introduction to Open Source Software
II: Linux Essentials
2 Introduction to Linux
3 The Filesystem
4 Essential Commands
5 Text Editors
6 System Administration
III: Linux Programming Languages
7 Overview of Linux Programming Languages
8 BASH Shell Scripting
9 Perl Scripting
10 Python Scripting
11 C, C++, and Java
IV: Using Git
12 Git Essentials
13 Manage Files with Git
14 Manage Differences in Files
15 Advanced Git Features
Index

t.me/DevTwitter
 Contents
Preface
I: Open Source Software
1 Introduction to Open Source Software
Defining Source Code
Closed Source
Open Source
“Free” Software
Choosing Open Source Licensing
Options
Key Terms
Examples
Useful Links
Summary
II: Linux Essentials
2 Introduction to Linux
Accessing a Linux System
Choosing the Right Distribution
Logging In
Using the GUI
Basic Command-Line Execution
Command-Line Structure
Getting Help
Summary
3 The Filesystem
Understanding the Filesystem
Learning the Most Used Directories
Naming Considerations

t.me/DevTwitter
 Navigating the Filesystem
Managing the Filesystem
Managing Directories
Managing Files
Summary
4 Essential Commands
Command-Line Tools
Viewing Files
Finding Files
Comparing Files
Shell Features
Permissions
Developer Tools
File Compression Commands
The grep Command
Summary
5 Text Editors
The vi Editor
Why Learn vi?
What Is vim?
Essential vi Commands
Additional Editors
Emacs
gedit and kwrite
nano and joe
lime and bluefish
Summary
6 System Administration
Essential Tasks

t.me/DevTwitter
 Gaining Access to the Root Account
Displaying Disk Usage
Managing Software
Listing and Finding Software
Installing Software
User Accounts
Adding User Accounts
Modifying User Accounts
Deleting User Accounts
Understanding Groups
Managing Groups
Summary
III: Linux Programming Languages
7 Overview of Linux Programming Languages
Scripting Languages
BASH Shell Scripting
Perl Scripting
Python Scripting
Additional Scripting Languages
Compiled Languages
C Programming Basics
C++ Programming Basics
Java Programming Basics
IDEs
Summary
8 BASH Shell Scripting
Basics of BASH Scripting
Conditional Expressions
Integer Comparisons

t.me/DevTwitter
 File Test Comparisons
Flow Control Statements
The while loop
The for Loop
The case Statement
User Interaction
Additional Information
Summary
9 Perl Scripting
Basics of Perl Scripting
Executing Perl Code
Additional Perl Documentation
Variables and Values
Flow Control
Conditions
Additional Features
Summary
10 Python Scripting
Basics of Python Scripting
Executing Python Code
Additional Documentation
Variables and Values
Flow Control
Conditions
Additional Features
Summary
11 C, C++, and Java
Understanding System Libraries
Managing Shared Library Files

t.me/DevTwitter
 Viewing Shared Library Files
Building Packages
Building RPM Packages
Building Debian Packages
Exploring Java Installation and Basics
Summary
IV: Using Git
12 Git Essentials
Version Control Concepts
The First Generation
The Second Generation
The Third Generation
Installing Git
Git Concepts and Features
Git Stages
Choosing Your Git Repository Host
Configuring Git
Summary
13 Manage Files with Git
Basic Configuration
git status
Handling a Multiple Location Situation
Telling Git to Ignore a File
Removing Files
Handling Branches
Pushing Branches
Summary
14 Manage Differences in Files
Executing Diffs

t.me/DevTwitter
 Dealing with White Space
Comparing Branches
Merging Files
Summary
15 Advanced Git Features
Managing Repositories
Getting Content from the Remote Server
Connecting via SSH
Patching
Summary
Index

t.me/DevTwitter
 Preface
When I envisioned this book, I thought of it as the beginning of a journey.
Your exact starting point on this journey may differ from that of others, but
the purpose of this book is to provide you with what you need to know to
start developing software on a Linux Operating System.
Some readers will already have software developing experience on
Windows-based platforms. For those folks, this book should serve as a
guide to how software development differs on Linux from the platform you
are used to developing on.
Perhaps you already work in Linux, but want to start writing code. Again,
this book will provide you with an excellent starting point for that journey.
The book is organized into four parts.
Part I, “Open Source Software,” contains a single chapter—Chapter 1,
“Introduction to Open Source Software.” You learn about open source
software, including its advantages over closed source software, as well
as some essentials regarding software licenses.
In Part II, “Linux Essentials,” you are introduced to the Linux
operating system. The goal of this part is to provide you with the key
knowledge that software developers need to know while working in
Linux. This includes both end user and administration topics. The
chapters in this part include the following:
Chapter 2, “Introduction to Linux.” In this chapter you learn the
basics of Linux, including how to access a Linux system, how to use a
Linux-based GUI, and basic command-line execution.
Chapter 3, “The Filesystem.” This chapter focuses on how the files
are organized in Linux. You are introduced to filesystem concepts,
learn how to navigate the filesystem, and learn how to manage the
filesystem.
Chapter 4, “Essential Commands.” In this chapter you learn a variety
of Linux commands that are critical for any developer to know.
Chapter 5, “Text Editors.” As a developer, you need to know how to
edit files. This chapter focuses on the vi editor, a common text editor
in both Linux and Unix. You are also introduced to additional Linux
editors.

t.me/DevTwitter
 Chapter 6, “System Administration.” It is helpful to know how to
perform system administration tasks, even as a developer. You learn
how to add software and administer users in this chapter.
Part III, “Linux Programming Languages,” provides an overview of
different programming languages available in Linux. You have a lot of
choices here! The goal here isn't to teach you everything about each
language but to introduce you to many of them so you can decide which
language will work best for you.
Chapter 7, “Overview of Programming Languages.” This chapter
provides an overview of programming languages with the focus on
distinguishing between scripting languages and structured (or
compiled) languages.
Chapter 8, “BASH Shell Scripting.” In this chapter you are
introduced to the BASH shell language. You learn how to create code
that interacts with users as well as other features of the BASH shell
programming language.
Chapter 9, “Perl Scripting.” The focus of this chapter is how to
program in the Perl scripting language. Topics include flow control
and variable usage.
Chapter 10, “Python Scripting.” You learn the basics of Python
scripting in this chapter, including the large variety of Python variable
types, and how to reuse code and flow control.
Chapter 11, “C, C++, and Java.” In this chapter you learn the
essentials to create C, C++, and Java code on Linux-based systems.
Lastly, Part IV, “Using GIT,” covers a very popular software revision
control product called Git. Using a revision control product, especially
when working with teams of developers, can save a lot of time, money,
and effort.
Chapter 12, “GIT Essentials.” In this chapter you learn the concepts
of GIT. Topics include revision control concepts, GIT installation, and
GIT features.
Chapter 13, “Manage Files with GIT.” In this chapter you learn how
to use GIT features such as staging, committing, and branches.
Chapter 14, “Manage Differences in Files.” The focus of this chapter
is on how to deal with different versions of files. You learn how to

t.me/DevTwitter
 execute diffs (differences in files) and merge files.
Chapter 15, “Advanced GIT Features.” You learn how to manage GIT
repositories and perform patching in this chapter.
Good luck in your journey!

—William “Bo” Rothwell

December 22, 2016

Register your copy of Linux for Developers at informit.com for convenient

access to downloads, updates, and corrections as they become available. To
start the registration process, go to informit.com/register and log in or create
an account. Enter the product ISBN 9780134657288 and click Submit.
Once the process is complete, you will find any available bonus content
under “Registered Products”.

t.me/DevTwitter
 Acknowledgments
Thanks to all who helped me put this book together. As any decent author
will tell you, there are many, many hours put into a publication like this,
which were the results of other people’s efforts and dedication.
Keith Wright and Matthew Helmke, thank your technical reviews. There is
no doubt this is a much better book than my original effort because of your
feedback.
Chris Zahn: I couldn't have asked for a better editor. You make me look like
I can put together a coherent sentence—no small feat!
Debra Williams Cauley, thanks for seeing the value and providing a guiding
hand throughout this process.

t.me/DevTwitter
 About the Author
At the impressionable age of 14, William “Bo” Rothwell crossed paths
with a TRS-80 Micro Computer System (affectionately known as a “Trash
80”). Soon after, the adults responsible for Bo made the mistake of leaving
him alone with the TSR-80. He immediately dismantled it and held his first
computer class, showing his friends what made this “computer thing” work.
Since this experience, Bo’s passion for understanding how computers work
and sharing this knowledge with others has resulted in a rewarding career in
IT training. His experience includes Linux, Unix, and programming
languages such as Perl, Python, Tcl, and BASH. He is the founder and lead
instructor of One Course Source, an IT training organization.

t.me/DevTwitter
 I. Open Source Software
One of the most important questions you need to answer when creating
software is, “Under what type of license will this software be released?”
Arriving at that answer can be a difficult journey.
You must determine what sort of protection you want to place on your code
as well as what you will allow others to do with the software that you
create. This part includes just one chapter, focusing on helping you decide
how to license your software. In this chapter, you learn the following:
The difference between closed and open source software
Open source protection concepts
The difference between primary open source licenses

t.me/DevTwitter
 1. Introduction to Open Source Software
You have created an awesome program, and now you want to make it
available to the public. Now comes an important decision to make: what
license to apply to your software.
This decision will have several important impacts, including the following:
How users can use your software
Whether the code is visible to others or “hidden” from plain sight
Whether other developers can use the code to create their own programs
Whether others can sell (or resell) the program

Disclaimer
License issues can be complex and have a significant impact on
how your software is used. The discussion in this book is
designed to provide you with a basic understanding of different
licenses but is not intended to provide any legal advice. The
author of this book does not intend to provide any legal advice.
Always consider consulting proper legal advice before making a
decision regarding software licenses.

Defining Source Code

Most likely the first question you need to answer is, “Will this be closed
source software or open source software?” To answer this question, you first
need to understand what source code is.
Software consists of a collection of instructions that are written in a
programming language. Dozens of different languages exist, including C,
C++, Java, Perl, Python, and many others. This collection of instructions is
referred to as source code. See Figure 1.1 for a demonstration of source code
written in C.

t.me/DevTwitter
 Figure 1.1 Source code written in C
Typically, you can’t use this source code directly to run your program. Most
languages require a compile process in which the source code is converted
into instructions that the operating system can understand. The result is data
that looks like garbage to a human but makes sense to the operating system.
See Figure 1.2 for a demonstration of source code that is converted into
compiled code.

Figure 1.2 Example of source code converted to compiled code

If you choose to license your software as closed source, then you only
provide the compiled code to customers. Software open source licenses
include access to the original source code.

Closed Source
Also called proprietary software, the purpose of closed source software is
to keep the source code a closely guarded secret. The idea is that if others
can see the source code, then the source code may be copied and illegally
used. As a result, competitors of the organization that created the software

t.me/DevTwitter
 might end up having a negative impact on the financials of the organization
that created the software. As you can imagine, copying someone else’s
software is much cheaper than creating your own software.
Often the term closed source is used in place of the term commercial
software, but this isn’t accurate. Commercial software must be purchased in
order to use it. Both closed and open source software can be
commercialized; the specific license determines whether the software is
commercial or “free.”1
Examples of closed source software include:
Microsoft Windows
Adobe Photoshop
Apple macOS

Open Source
Software is considered open source when both the compiled and source
versions of the code are made available.2 The software license from the
copyright holder grants certain rights to view, modify, and distribute the
software. A variety of open source licenses are available that enable you to
pick which rights are granted.
Although some open source software is free, in the economic sense, that
isn’t a requirement for open source software to be considered open source.
Open source refers to the capability to access the source code, not how the
software can be used or any costs associated with the software.
Common examples of open source software include:
Linux3
Apache HTTP Server
Firefox
Git
Openoffice.org

“Free” Software
The concept of free in regards to software is not necessarily agreed upon by
all. Some people might consider free software to be software without cost. In
other words, it doesn’t cost you anything to obtain and use the software.

t.me/DevTwitter
 However, what does it mean to use software? Does this mean that the
software can be used in any way that the user wants, or are there some
restrictions? Can this software be used anywhere in the world, or are there
geographic limitations? Are you free to modify the software and distribute
free copies of the modified format, or are these actions prohibited? As you
can see, free in terms of software isn’t so clear cut.
One way of defining the term free is to use the definition created by Richard
Stallman and published by Free Software Foundation (FSF):
“The word ‘free’ in our name does not refer to price; it refers to freedom.
First, the freedom to copy a program and redistribute it to your neighbors, so
that they can use it as well as you. Second, the freedom to change a program,
so that you can control it instead of it controlling you; for this, the source
code must be made available to you.”
Note that the preceding definition mandates that free software also be open
source software. Not everyone agrees with this, and you will find some
closed source freeware on the market.
Another way to define the term free is by the Four Freedoms, as defined by
the Free Software Foundation:
Freedom 0: “The freedom to run the program as you wish, for any
purpose.”
Freedom 1: “The freedom to study how the program works and change
it so it does your computing as you wish. Access to the source code is a
precondition for this.”
Freedom 2: “The freedom to redistribute copies so you can help your
neighbor.”
Freedom 3: “The freedom to distribute copies of your modified
versions to others. By doing this you can give the whole community a
chance to benefit from your changes. Access to the source code is a
precondition for this.”

These Four Freedoms are at the heart of what is referred to as FOSS (Free
and Open Source Software).4 FOSS attempts to address the challenge of
defining what constitutes “free software.” Note that the definition highlights
the fact that not all free software is open source. Conversely, not all open
source software is licensed with the Four Freedoms.

t.me/DevTwitter
 Understanding the complex world of open source software and what part
“freedom” plays takes some time. See Figure 1.3 for a graphic that includes
the various components of open source software.

Figure 1.3 Visualize the Open Source community

The graphic in Figure 1.3 highlights the complex combination of elements
that make up open source software. You can see that the developers write the
source code, create documentation, and provide support. However, the users
of the software are often major components in this process as well. In fact,
some open source software includes little or no support or documentation
from the developer, but rather relies on a strong user base (the community)
to provide these critical elements.
Note that the freedoms to modify, distribute, and utilize are also depicted in
the diagram in Figure 1.3. Additionally, the software license is what provides
these freedoms.

Choosing Open Source Licensing

Ultimately you need to decide whether to license your software as closed
source or open source. Part of that decision will be based on specifics
regarding licenses, a topic that this chapter covers further. However, you
should consider some general benefits to creating open source software:
Open source tends to invoke trust. The reason is that others can see
exactly what the software does by looking at the source code.
Open source can result in better code and less development time.
With other developers reviewing your code and providing feedback, bug

t.me/DevTwitter
 fixes, and improvements are created more rapidly, often at no cost to
you.
Open source that is “free” can increase the scope of the user base.
More users are willing to try software that is free rather than paying
money to test out new software.
You can still make money from “free” open source software. Other
available sources of revenue include training, support contracts, and
additional services.

Options
Dozens of standard open source licenses and a large number of custom
licenses are available. Typically, they fall into one of the following four
categories:
Standard—Regular licenses that are often reused for other software
products. Typically, these are specific to a country and many of them
center around United States or European laws.
International—Regular licenses that are often reused for other software
products. Unlike the standard licenses, these licenses are designed to be
used throughout the world.
Special purpose—Licenses that are written for specific cases.
Nonreusable—Licenses that are not permitted to be used for any
software product besides the product the license was written for.

Key Terms
Regarding open source software licenses, you should understand a couple of
key terms. One term is copyleft, which ensures intellectual property (IP) can
be copied or distributed as open source software. The two forms of copyleft
are
Strong—All derived works must maintain original copyleft.
Weak—Derived works don’t need to follow the original restrictions.

Another important open source license term is permissiveness. This term is

related to derived works and whether mixed licenses are allowed. The two
forms of permissiveness are

t.me/DevTwitter
 Strict—Limited mixed licenses (no closed source or more permissive
licenses)
Permissive—Allows mixed licenses

Examples
The following list describes some of the more popular open source licenses:
Apache License 2.0:
Very permissive
Non-copyleft
Use for any purpose
Distribute and modify
Allows derived works
MIT License:
Also known as X11 license
Similar to Apache license 2.0
Very permissive
Non-copyleft
Use for any purpose
Must keep copyright message
User must agree that no warranty is provided
GNU General Public License (GPL):
Strong copyleft
Strict permissiveness
All derived works must be GPL
Two versions: v2 and v3
BSD License:
Very permissive
Non-copyleft
Three types:
Two-clause—Same as MIT
Three-clause—Derived work not endorsed by original holder

t.me/DevTwitter
 Four-clause—Advertising must acknowledge original holder

Useful Links
Hopefully, you now have a basic understanding of open source software and
licenses. Clearly, the topic is not a simple one, and spending additional time
researching which license is the best for your project and organization is
important. In addition to consulting a legal expert, you should find the
following URL resources helpful:
http://choosealicense.com—This tool uses a series of questions to help
you determine which license works best for your situation. It provides a
good start, but you should also consult a legal expert before making a
final decision.
http://fsf.org—The website for the Free Software Foundation offers a
great deal of useful information regarding open source software and
licenses.
http://opensource.org—This is another great resource to learn more
about open source licenses.

Open Source Humor

Open source: free as in “free speech,” not as in “free beer.” —
Anonymous

Summary
In this chapter you learned the differences between closed and open source
software. The concept of free software was also explored. Lastly, you
learned about some of the basics of open source licenses. At this point you
should be able to start the process of determining what type of license under
which to release your software. However, remember that you want to put a
lot of time, effort, and thought into this decision before releasing your
software because these licenses can have a powerful effect on how your
software will be used by the community.

1 There is a reason why I put free in quotes. As you will soon read, the term
free must be legally defined in regards to software use.

t.me/DevTwitter
 2 Actually, some open source projects only provide the source code and
leave it to the people who use the software to compile it. Additionally,
some languages don’t have a compile process, so a program written in
such a language would only include the source code.
3 Technically, Linux refers to the Linux kernel, which is the heart of the
Linux operating system. Most of the software found on Linux operating
systems is also open source, but this isn’t a requirement for inclusion in
the OS.
4 Often used interchangeably with the term FLOSS (Free/Libre and Open
Source Software).

t.me/DevTwitter
 II. Linux Essentials
If you are going to develop software on a Linux-based operating system
(OS), then it will be important for you to know how to interact with and
manage the OS. The next five chapters are devoted to providing you with a
solid foundation in the tools and features of Linux.
These chapters focus specifically on what you should know about Linux as
a developer. Linux itself is a huge topic with large volumes devoted to
exploring the OS. The goal here is to provide you with what you need to
know as a developer, not as a regular end user or administrator.1
The next five chapters cover the following:
The core concepts of the Linux operating system
What Linux distributions are
How to manage the Linux filesystem
Critical Linux commands that all software developers should know
Essential system administration tasks that will be useful for you to
know as a developer

1 Certainly, the topics presented will also be useful to these sets of Linux
users.

t.me/DevTwitter
 2. Introduction to Linux
What exactly is Linux? The answer can be a complex one. Technically,
Linux is a piece of software called the kernel. The kernel handles tasks such
as booting the system and interacting with hardware devices. By itself, the
kernel doesn’t really provide users with any functionality. The rest of the
operating system (OS), consisting of the filesystem and a large number of
commands, is what provides users with useful features.
Although Linux is technically just the kernel, many people refer to the entire
OS as Linux. In reality, the collection of software that makes up the OS is
known as a Linux distribution (also called a distro). Many distros are
available to choose from, which often results in some confusion for novice
Linux users.

What Is a Distribution?
A distribution is a collection of Linux software that is
maintained by an organization. Each distribution has features
that make it unique. Some distributions are designed for general
use whereas others are designed for a very specific use case,
like a firewall or a web server.
Picking the distro that works best for you might take some time.
The website distrowatch.com can aid you in this endeavor.
Additional information is also provided later in this chapter.

Accessing a Linux System

Before you can access a Linux system, you need to have one installed.
Because each distro has a different installation program, the steps to
installing a distro are not included in this book. However, the following
should provide you with enough information to successfully install a distro:
First consider which distro you want to use. This topic is explored in
more detail later in this chapter.
Consider installing the distro on a Virtual Machine (VM). By using a
VM, you can install multiple distros and still use your host operating
system. Several VM software choices are available, including

t.me/DevTwitter
 VirtualBox, VMware, and Parallels Desktop (for Mac users). Some of
these products have free versions, whereas others might charge a license
fee.
During the installation, consider accepting the defaults. Typically, the
defaults provide you with the software that you need as a developer. You
can always reinstall the distro in the future or add additional software
using the tools described in Chapter 6, “System Administration.”

Choosing the Right Distribution

Often an organization spends a great deal of time to ensure that it chooses
the Linux distribution that best fits the company’s needs. Each distro is
different, and there isn’t a “one distro fits all scenarios” solution. There are
many features to take into consideration, including:
Cost—Some distros are entirely free, whereas some charge for support
and access to updates.
Features—Some distros provide limited access to software based on the
objective of the distro. For example, a security-hardened distro only
provides software that has met rigorous security benchmarks.
Function—Some distros are specifically tailored to meet a specific
function. For example, a distro might be designed to host database
applications.
Support—The organization that creates the distro might provide
support, or the support might be completely community sponsored. For
most distros used in a corporate environment, the organization that
creates and maintains the distro provides support.

Although these might not be all the features you or your organization
consider when choosing a distro, the preceding list provides you with an idea
of what features are normally considered. Because literally hundreds of
distros exist, I recommend exploring http://distrowatch.com, a website
devoted to monitoring these distros, their features, and their popularity.
Linux distributions are generally split into three primary families:2
Debian—Known to be a favorite of the hard-core Linux geeks, Debian
was one of the first Linux distributions. Popular variations of Debian
include Ubuntu and Mint.

t.me/DevTwitter
 Red Hat:—Designed to be a commercial distro, Red Hat Linux was
introduced shortly after Debian. It is now called Red Hat Enterprise
Linux (RHEL). Because it is a commercial distro, payment of a license
fee (for support and updates) is required to use RHEL. However, several
free Red Hat–based distros exist, including Fedora and CENTOS.
Slackware/SUSE—Slackware was the original “founding father” of
this family of distros, but SUSE is the most popular now. SUSE is like
RHEL in that it is designed to be a commercial distro. OpenSUSE is a
free variation.

Regarding the information provided in this book, in most cases which distro
you use does not matter. However, for some topics the distro does matter.
When these situations arise, I provide detailed information regarding the
differences between the distros. Otherwise, the examples make use of
different distros to highlight that core functionality is largely the same across
Linux distributions.3

Logging In
Typically, there are three ways to log in to a Linux system:
Via the GUI—On laptops, desktops, and some servers, a GUI-based
login appears by default.
Via the command line—Administrators commonly do not install the
GUI software on servers because that software is a huge hardware hog
(CPU, RAM, and so on). On these servers, you typically see a
command-line login.
Via the network—A network-based login could either be via command
line (very common) or GUI (not as common). The machine that you log
into must have special software installed and enabled to allow this sort
of access.

Logging In via the GUI

The software that allows you to log in to a Linux system is called a display
manager. Several different display manager software programs are
available, resulting in a different look and feel for the login screen. In some
ways, this is dependent on the distro, because the organization that supports
the distro tends to favor one display manager or another. However, an

t.me/DevTwitter
 administrator can choose to install a different display manager or configure
the appearance of the existing display manager. See Figure 2.1 for an
illustration of display managers.

Figure 2.1 The default display manager on CentOS (left) and MintOS
(right)
The good news is that regardless of which display manager you use, the
basic operation shouldn’t change. You either select your user name from a
list or type your user name in the dialog box provided. Then you are
prompted for your password. As you become more experienced, try
exploring other options provided by your display manager, such as shutting
down your system.

Logging In via the Command Line

In most cases, you only log in via the command line when the server you are
working on has no GUI. However, if your system has a GUI, logging in via a
command line is still possible (even if you are working on a virtual
machine). Holding down the Ctrl+Alt buttons on your keyboard and pressing
the F2 key4 should provide a command-line login. See Figure 2.2 for an
example of a command-line login screen.

t.me/DevTwitter
 Figure 2.2 A command-line login screen for CentOS
To log out of a command-line environment, type exit and then press the
Enter key. To return to the GUI screen, press either Ctrl+Alt+F1 or
Ctrl+Alt+F2.

Logging In via the Network

You can use a few techniques to log in to a remote system via the network.
The techniques that are available depend on two factors: what sort of system
you are logging in from and whether you want to log in to a command-line
interface (CLI) or GUI:
Logging in to a Linux system from a Microsoft Windows system:
You will likely need some additional software because Microsoft
Windows doesn’t normally provide the software needed to log in to a
Linux system. If you want to log in through a CLI, install a Secure Shell
(SSH) client program. If you want to log in to a GUI, install a virtual
network computing (VNC) client program.5
Logging in to a Linux system from a Macintosh system: At its heart,
the Macintosh OS is Unix-based, so some client tools should already be
available. For example, you should be able to open a terminal window
and use the ssh command to log in to a Linux system via the CLI. You
could also install VNC client software to log in to a Linux system via the
GUI.

t.me/DevTwitter
 Logging in to a Linux system from a Linux system: You should be
able to use the ssh command to log in. A VNC client might also be
installed, but if it isn’t then contact the administrator.6

In the following example, the “bob” user on the local machine logged in to
the machine named “remote” using the “student” account (Note: Instead of a
machine name, an IP address can be used):
bob@ubuntu:~$ ssh student@remote
The authenticity of host 'remote' can't be established.
ECDSA key fingerprint is
8a:d9:88:b0:e8:05:d6:2b:85:df:53:10:54:66:5f:0f.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'remote' (ECDSA) to the list of known
hosts.
student@remote's password:
Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.16.0-30-generic x86_64)
student@remote:~$

Note the message about the key fingerprint. This is used in the future to
verify that you are logging in to the correct system and not some fake
machine that has assumed the identity of the remote system. This question
only appears the first time you log in to this system.
To return to the local system, execute the exit command.

Using the GUI

The heading “Using the GUI” is a bit misleading because there isn’t just one
GUI. With Linux, you have your choice of several different desktops, each
of which provides the same essential functions in different ways in addition
to having unique features. A small sample of available desktops includes the
following:
GNOME
KDE
Unity+
Cinnamon
Xfce
MATE

t.me/DevTwitter
 With so many desktops available, you might ask yourself “Which one should
I use?” To some extent, that question might be answered for you when you
choose your Linux distribution. Most distros have a “favored default”
desktop which will be installed and used automatically. Although other
desktops might be available for the distro, you typically need to install them
separately.
In some cases, the developers of the distro might provide you the option of
choosing your desktop before you install the distro. For example, consider
Figure 2.3, which displays the download links for the Mint distribution.

Figure 2.3 Mint download links

Figure 2.3 shows the different install options provided, primarily divided by
the type of desktop. Additionally, a user with administrative access can
install multiple desktops on a single system. If there are multiple desktops,

t.me/DevTwitter
 you could choose which desktop you want to use before you log in, as
demonstrated in Figure 2.4.

Figure 2.4 Desktop choices at login

After you successfully log in, a desktop appears. The overall look and feel of
the desktop varies. However, figuring out where things are on a particular
desktop shouldn’t take too long. For example, look at Figure 2.5, which
displays the default desktops for MintOS.

t.me/DevTwitter
 Figure 2.5 Cinnamon versus MATE desktops
As you can see, the Cinnamon and MATE desktops look very similar. Both
provide a menu (bottom left) that enables you to access additional programs
and features. Both have quick launch icons and both provide system
information (bottom right) such as date/time. The choice comes down to
exploring the desktop that you are using rather than memorizing where
features reside for a specific desktop.

Basic Command-Line Execution

Although the GUI interface makes using Linux easy, most users and
administrators use the command-line environment for system tasks. If you
log in to a system via the GUI, you can access the command-line
environment by opening a terminal window. The terminal window is a GUI-
based program that launches a shell, a program that enables you to enter
commands. See Figure 2.6 for an example of a terminal window.

t.me/DevTwitter
 Figure 2.6 Typical terminal window
The most common shell program in Linux is the BASH7 shell. This book
shows BASH shell–based examples.

Command-Line Structure
A command has three components:
Command name—This is just the name of the command.
Option(s)—An option (also referred to as a flag) is a predefined value
that changes the behavior of the command. The format of options can
vary. In some cases, the option begins with a single hyphen followed by
a single character; for example: ls -a. In other cases, the option begins
with two hyphens followed by a word; for example: ls --all. In some
rare cases, the option is just a single character with no hyphen. The
format depends on the command because some commands support the
single-hyphen method whereas others support the double-hyphen
method (and some support both methods).
Arguments—Arguments are used to provide additional information to
the command. This information could be things like a filename, user

t.me/DevTwitter
 name, or host name.

See Figure 2.7 for a visual example of the command-line components.

Figure 2.7 Components of a command line

Getting Help
You might be wondering how to tell what options and arguments a command
will accept. Several ways exist to display documentation that can help you
use a command. One method you could use is the help command:
bo@mintos:~ > help alias
alias: alias [-p] [name[=value] ... ]
Define or display aliases.

Without arguments, 'alias' prints the list of aliases in the

reusable
form 'alias NAME=VALUE' on standard output.

Otherwise, an alias is defined for each NAME whose VALUE is

given.
A trailing space in VALUE causes the next word to be checked for
alias substitution when the alias is expanded.

Options:
-p Print all defined aliases in a reusable format

Exit Status:
alias returns true unless a NAME is supplied for which no alias
has
been defined.

As you can see from the output of the help alias command, the command
accepts the -p option. Because the option is enclosed within square brackets,
the option is not required for the command to run. Additionally, you can
include a name or name=value argument with the command. Again, the square

t.me/DevTwitter
 brackets indicate that these arguments are not required and that name does not
require an =value argument.
One drawback of the help command is that is only works on built-in shell
commands. These commands are part of the BASH shell, not a separate
executable.8 The majority of commands that you will execute are not built-
in shell commands, so the help command won’t be that helpful.
However, almost all commands have documentation available via a
command called the man command (man is short for manual). To view a man
page for a command, execute man cmd, replacing cmd with the command
name. For example, to view the man page for the cal command, execute man
cal.

Note
You will eventually discover that not only commands have man
pages. Configuration files and other things have man pages, too.

The man page for a given command9 can be quite large. To facilitate the
process of reading the man page, you can use several keys to move through
the document:
<spacebar>—Move down one screen.
<ENTER>—Move down one line.
b—Move back one screen.
/term—Search for term.
h—Display help screen.
q—Quit displaying man page.

Note that these are just a few of the commands that you can use to display a
man page. The h key provides a full list of commands.

t.me/DevTwitter
 Helpful Suggestion
When you are first learning Linux, you might find using man
pages to be difficult. The format and syntax of the output can be
a challenge to follow. I highly suggest you get as much practice
as you can reading man pages. You can do this by looking at the
man page for each new command that you learn. Try to find a
new feature or option for the new command. Test your ability to
run the command with different options based on what you
learn from the man page. Learning how to read man pages just
comes down to practice.

A typical man page has several different sections. The output begins with a
line like the following:
CAL(1) BSD General Commands
Manual CAL(1)

This indicates the command and the category. This chapter covers man page
categories a bit later.
The next section provides a brief description of the command like the
following:
NAME
cal, ncal — displays a calendar and the date of Easter

After this description, you see a summary of how you can execute the
command:
SYNOPSIS
cal [-3hjy] [-A number] [-B number] [[month] year]
cal [-3hj] [-A number] [-B number] -m month [year]
ncal [-3bhjJpwySM] [-A number] [-B number] [-s country_code]
[[month]
year]
ncal [-3bhJeoSM] [-A number] [-B number] [year]
ncal [-CN] [-H yyyy-mm-dd] [-d yyyy-mm]

Recall that square brackets indicate something that isn’t required but which
is a valid option. For example, you could just run the cal command with no
options or arguments:
bo@mintos:~ > cal
July 2016

t.me/DevTwitter
 Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

You could also use an option, such as -3:

bo@mintos:~ > cal -3
June 2016 July 2016 August 2016
Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa
1 2 3 4 1 2 1 2 3 4 5 6
5 6 7 8 9 10 11 3 4 5 6 7 8 9 7 8 9 10 11 12 13
12 13 14 15 16 17 18 10 11 12 13 14 15 16 14 15 16 17 18 19 20
19 20 21 22 23 24 25 17 18 19 20 21 22 23 21 22 23 24 25 26 27
26 27 28 29 30 24 25 26 27 28 29 30 28 29 30 31
31

However, only options specified in the Synopsis are allowed:

bo@mintos:~ > cal -2
cal: invalid option -- '2'
Usage: cal [general options] [-hjy] [[month] year]
cal [general options] [-hj] [-m month] [year]
ncal [general options] [-bhJjpwySM] [-s country_code]
[[month] year]
ncal [general options] [-bhJeoSM] [year]
General options: [-NC31] [-A months] [-B months]
For debug the highlighting: [-H yyyy-mm-dd] [-d yyyy-mm]

Additionally, you can specify a month and year:

bo@mintos:~ > cal 3 1968
March 1968
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

The next section provides a greater level of detail regarding how the
command and its options work:
DESCRIPTION
The cal utility displays a simple calendar in traditional
format and
ncal offers an alternative layout, more options and the date of

t.me/DevTwitter
 Easter.
The new format is a little cramped but it makes a year fit on a
25x80 terminal.
If arguments are not specified, the current month is displayed.
The options are as follows:
-h Turns off highlighting of today.

Some commands also provide a See Also section that includes a list of
related commands:
SEE ALSO
calendar(3), strftime(3)

Depending on the man page that you are viewing, you might see additional
sections. However, the critical ones have been described here.

Man Page Categories

Many different types of man pages exist. For example, there are man pages
for commands that regular users execute and man pages for commands that
administrators execute. There are also man pages for libraries (programs
used by other programs) and system configuration files. These different
types of man pages are broken into categories,10 as shown in the man page
for the man command:
bo@mintos:~ > man man
{output omitted}
The table below shows the section numbers of the manual
followed by
the types of pages they contain.
1 Executable programs or shell commands
2 System calls (functions provided by the kernel)
3 Library calls (functions within program libraries)
4 Special files (usually found in /dev)
5 File formats and conventions eg /etc/passwd
6 Games
7 Miscellaneous (including macro packages and conventi
ons),
e.g. man(7), groff(7)
8 System administration commands (usually only for root)
9 Kernel routines [Non standard]

In most cases, you don’t have to worry much about these categories. When
you execute the man command, it first searches category 1 for the man page.
If it doesn’t find it, it then searches the next category.11 Eventually it finds
and displays the man page or displays an error if it can’t find the man page
in any category:

t.me/DevTwitter
 bo@mintos:~ > man nope
No manual entry for nope

In some cases you must specify the category. For example, there is a user
command named passwd (category 1) and a file format named passwd
(category 5). If you execute the man passwd command, the passwd command
man page displays. To view the man page for the passwd file, you must
execute the man 5 passwd command.

Tip
Most distributions have a GUI-based man page viewer that you
might find easier to use. Just type xman to launch it.

Info Documentation
In addition to man pages, you might find info documentation helpful. Not all
commands and files have info documentation, but for those that do, using it
can be easier than using man pages. To use an info document, execute the
info command followed by the command to display; for example, info ls.
The documentation found in info pages tends to be more verbose than in
man pages. The sections of info pages are also organized differently than
man pages. Instead of one long document of text, info pages appear in
hyperlinked sections. For example, if you scroll down the document for the
ls command (use the down-arrow key on your keyboard), you see this:

* Menu:
* Which files are listed::
* What information is listed::
* Sorting the output::
* Details about version sort::
* General output formatting::
* Formatting file timestamps::
* Formatting the file names::

This is a menu of subcategories you can jump to by placing your cursor on a

menu item and pressing the Enter key. After you’re in a subcategory, type
the letter u to go back up one level. To see additional commands, press the H
or ? key.

t.me/DevTwitter
 Note
In the help section, press l to exit help and return to the info
page. Use q to quit the info document and return to the
command line.

Tip
You can learn a lot from info documentation. Try executing the
info command with no arguments. Then scroll down to the
menu section, pick a category, press the Enter key, and start
exploring.

Additional Documentation
In addition to man pages and info documentation, you might find the files in
the /usr/share/doc directory to be useful. Chapter 3, “The Filesystem,”
covers how to navigate the filesystem, which will allow you to access the
documentation in the /usr/share/doc directory. For now, just realize that
additional documentation exists in this directory (typically it is geared more
toward administrators, but some end user documentation exists in this
directory, too).

Linux Humor
Ever wonder whether the computer is actually watching you as
you are working? Type the xeyes command in a terminal
window (this only works in a GUI environment) to get the
answer to this question.

Summary
In this chapter you learned how to log in to an existing Linux distribution via
the command line, a graphical user interface, or across the network. You also
learned about different desktops, which provide the look and feel of a GUI.
Lastly, you learned how to get additional help using man pages and info
documentation.

t.me/DevTwitter
 2 Other families of distros exist, but these three are by far the most often
used. If you want to see a “family tree” (somewhat overwhelming), I
suggest viewing the following graphic:
https://upload.wikimedia.org/wikipedia/commons/1/1b/Linux_Distributio
n_Timeline.svg
3 As a developer, you shouldn’t get too hung up on which distro to use.
This is often the choice of the company system administrator. Knowing
the differences and which ones can affect a developer is what you should
focus on (and that is the approach for this book).
4 If you are working in a virtual machine, this keystroke combination might
vary.
5 Note that normally VNC requires installing a VNC server on the Linux
box and configuring it.
6 Unless you have access to root privileges. See Chapter 6 for details
regarding software.
7 BASH stands for Bourne Again SHell. It is based on an older Unix shell
named the Bourne Shell.
8 Executable is a term that essentially means program. If a file can be run
like a program, it should have the execute permission set.
9 Not only commands have man pages. Configuration files and other items
have man pages, too.
10 I call them categories, but the man page calls them sections. I think this
term is confusing because sections is also used to refer to different pages
within a single man page. So, I call these collections of man pages
categories to avoid confusion.
11 The following line in the man configuration file determines the order in
which man page sections are searched:
SECTION 1 1p 8 2 3 3p 4 5 6 7 9 0p n l p o 1x 2x 3x 4x 5x 6x 7x 8x

t.me/DevTwitter
 3. The Filesystem
Regardless of what you plan to use the Linux distribution for, you need to
know how to navigate the filesystem. The filesystem is how the files are
structurally organized into directories. Understanding this structure and how
to manage the files is critical to using Linux.

Understanding the Filesystem

Typically, new Linux users have some experience in another operating
system, such as Microsoft Windows. One of the challenges of using the
Linux filesystem is understanding that the layout is likely to be different than
what you are used to.
For example, in Microsoft Windows, physical drives are assigned letters,
such as C: or F:. They may be visible under the My Computer icon. Linux
doesn’t use drive letters or a My Computer icon. Instead, all drives,
including network drives and removable media, are located under the root
directory.
The root directory is symbolized by the / character. This character is also
used to separate directory and filenames in a path. Think of the path as
directions to get to a file or directory. For example, the path
/home/bob/sample.txt refers to a file named sample.txt that is in the bob
directory. The bob directory is under the home directory, which in turn is
under the root directory. See Figure 3.1 for a small example of a Linux
filesystem.

t.me/DevTwitter
 Figure 3.1 Part of a Linux filesystem

Learning the Most Used Directories

Thousands of directories are in a typical Linux filesystem. You should not
worry about learning about all of these directories when you first start
learning Linux, but some are important enough to learn now:
/home—A user’s home directory; each user has a directory under the
/home directory where she can store her files. This is one of the few
places where the user should always have the right to create and manage
files.
/root—Root user’s home directory; the system administrator account on
the system is called the root user. The home directory for the root user
isn’t under the /home directory; instead it is under the /root directory.1
/bin—Executables (programs); most of the commands you execute as a
regular user are placed here or in the /usr/bin directory.
/usr/bin—Executables (programs); most of the commands you execute
as a regular user are placed here or in the /bin directory.
/sbin—Executables (programs) for system administrators; most of the
commands you execute as a system administrator are placed here or in
the /usr/sbin directory.
/usr/sbin—Executables (programs) for system administrators; most of
the commands you execute as a system administrators are placed here or
in the /sbin directory.

t.me/DevTwitter
 /media—Removable media (could also be /run/media); this is where
you find the files for removable devices, such as CD-ROMs and USB
drives.
/tmp—Temporary files; typically, programs store files in this directory
rather than placing files in a user’s home directory.

Naming Considerations
When you create a file or directory, you should take into consideration the
following guidelines:
Files and directories have same name rules.
Names are case sensitive. This means that a file named Data.txt is not
the same as a file named data.txt.
Special characters are permitted. However, you should avoid using
whitespace (space and tab characters) and certain special characters
called metacharacters (*, ?, [, ], !, $, &, and so on). Metacharacters are
special characters to BASH, and they can create problems when used in
a file or directory name.
The / character is used to separate files and directory names:
/usr/share/doc. As a result, you cannot use the / character in file or
directory names.
Extensions (.txt, .cvs, and so on) are permitted but normally have no
special meaning to BASH. In some rare cases they may be required for a
specific command, but Linux and the BASH shell typically don’t require
specific extensions for files. However, extensions are useful for users
because understanding the purpose of a file is easier if the extension is
provided (some GUI interfaces also make use of extensions to determine
which GUI-based program to launch).
There are some special predefined directory names:
~ —Represents the current user’s home directory
. —Represents the current working directory (the directory you are
working in when using a command-line shell)
.. —Represents one level above the current working directory

Navigating the Filesystem

t.me/DevTwitter
 When using the command-line environment, you often need to refer to a
directory structure to access a file or subdirectory. For example, you may
want to view a file in a specific directory.
When you first open a shell, you are automatically placed in your home
directory. The directory you are in is referred to as your working directory or
current directory. A common task is to switch the working directory to
another directory, a process called change directory.
A path or pathname is how you refer to a file or directory in the directory
structure. The two types of pathnames are
Absolute—A path that always starts from the root directory; for
example, /home/bob/sample.txt
Relative—A path that starts from the current directory; for example, if
you are in the /home directory and want to access the sample.txt file that
is under the bob directory (which is under the /home directory), you use
bob/sample.txt.

Note
Absolute paths always start with a / character but relative paths
never start with a / character.

To help understand the difference between an absolute and relative path,

consider Figure 3.2.

t.me/DevTwitter
 Figure 3.2 Using a pathname, example #1
In this example you are in the /home/julia directory and want to switch to
the /usr/bin directory. To switch directories, you can use the cd command,
and to view the current directory you can use the pwd1 directory. The
following example demonstrates using an absolute path:
julia@mintos:~ > pwd
/home/julia
julia@mintos:~ > cd /usr/bin
julia@mintos:/usr/bin > pwd
/usr/bin

Note that the prompt (julia@mintos:/usr/bin) indicates the current

directory, so the pwd command isn’t always necessary. This isn’t always the
case because the prompt is customizable.
The next example demonstrates using a relative path to move from the
/home/julia directory to the /usr/bin directory. Note that the cd command,
when given no arguments, returns you to your home directory:
julia@mintos:/usr/bin > cd
julia@mintos:~ > pwd
/home/julia
julia@mintos:~ > cd ../../usr/bin
julia@mintos:/usr/bin > pwd
/usr/bin

Why use ..? Because a relative path requires giving directions from the
current directory, and you have to tell the cd command to “go up” two levels
before going down to the usr and bin directories.
Clearly, in this case, the absolute path was easier, but that isn’t always the
case. For example, now consider Figure 3.3.

t.me/DevTwitter
 Figure 3.3 Using a pathname, example #2
Now you want to move from the /home/julia directory to the /home/sarah
directory. The first example uses an absolute path:
julia@mintos:/usr/bin > cd
julia@mintos:~ > cd /home/sarah
julia@mintos:/home/sarah > pwd
/home/sarah

Compare the previous example with the following:

julia@mintos:/home/sarah > cd
julia@mintos:~ > cd ../sarah
julia@mintos:/home/sarah > pwd
/home/sarah

The relative method is easier to use in this case. Consider if you were 20
levels deep in the directory structure and you just wanted to move up one
level and down to another subdirectory. Using an absolute path would result
in a lot of typing whereas a relative path would be much less work. The
lesson here is to know both methods because one is going to be easier than
the other most of the time.

Managing the Filesystem

Now that you know how to move from one directory to another, you will
want to see what is inside the directories. The ls command lists files in a
directory:

t.me/DevTwitter
 julia@mintos:~ > cd /etc/sound/events
julia@mintos:/etc/sound/events > ls
mate-battstat_applet.soundlist

By default the ls command displays all files in the current directory except
hidden files. A hidden file has a . character at the beginning of the filename.
To see all files, including hidden files, use the -a option to the ls command:
julia@mintos:/etc/sound/events > ls -a
. .. mate-battstat_applet.soundlist

Recall that . represents the current directory and .. represents the directory
above the current directories. You will always see these two hidden files3
regardless of which directory you are in.
To understand why some files are hidden, look at the files in a typical user’s
home directory:
julia@mintos:/etc/sound/events > ls -a ~
. .bash_history .bashrc .config .gimp-
2.8 .local .profile
.. .bash_logout .cache .face.icon .kde .mozilla

Each of the hidden files (in some cases they are directories) that you see in
this output contain information that modifies the user’s environment. For
example, .bashrc and .profile modify how the BASH shell works for the
current user. The .mozilla directory contains configuration settings for
Firefox, a web browser provided by the Mozilla Foundation.
How could you tell if .bashrc was a file or .mozilla was a directory? Use
the -l option:
julia@mintos:/etc/sound/events > ls -a -l ~
total 48
drwxr-xr-x 8 julia julia 4096 Jul 14 17:26 .
drwxr-xr-x 5 root root 4096 Jul 14 17:26 ..
-rw------- 1 julia julia 72 Jul 14 17:26 .bash_history
-rw-r--r-- 1 julia julia 220 Apr 8 2014 .bash_logout
-rw-r--r-- 1 julia julia 1452 Jan 5 2016 .bashrc
drwx------ 3 julia julia 4096 Jul 14 17:12 .cache
drwxr-xr-x 6 julia julia 4096 Jun 5 15:59 .config
lrwxrwxrwx 1 julia julia 5 Jun 5 14:40 .face.icon -> .face
drwxr-xr-x 2 julia julia 4096 Jan 5 2016 .gimp-2.8
drwxr-xr-x 3 julia julia 4096 Jan 5 2016 .kde
drwxr-xr-x 3 julia julia 4096 Jan 5 2016 .local
drwxr-xr-x 3 julia julia 4096 Jan 5 2016 .mozilla
-rw-r--r-- 1 julia julia 675 Apr 8 2014 .profile

t.me/DevTwitter
 When you use the -l option, each line describes detailed information for a
file. See Figure 3.4 for a demonstration of the details that are provided.

Figure 3.4 Details of the ls -l command

The details of the information provided by the ls -l command:
File type—d means this is a directory whereas - means it is a plain file.
There are additional file types, but these are the primary two you should
know.4
Permissions—These are used to allow or disallow access to the file.
Chapter 4, “Essential Commands” covers permissions.
Hard link count—A more advanced topic normally reserved for system
administrators, this is a file that is hard linked and shares the same data
block space with another filename.
User owner—The user who owns the file has special access to the file.
For example, only the user owner (and the root user) can change the
permissions of a file.
Group owner—Group owners have special access to the file via
permissions.
File size—The size of the file in bytes.5
Modification timestamp—The date and time the file was last
modified.6
Filename—The name of the file.

t.me/DevTwitter
 Note
You can use many other options with the ls command. Recall
the suggestion from Chapter 2, “Introduction to Linux,”
regarding viewing the man page for each new command that
you learn. This would be an excellent time to make use of that
suggestion!

Managing Directories
To create a new directory, use the mkdir command:
julia@mintos:~ > ls
julia@mintos:~ > mkdir data
julia@mintos:~ > ls -l
total 4
drwxrwxr-x 2 julia julia 4096 Jul 15 09:34 data

Note a situation in which this could fail:

julia@mintos:~ > ls
data
julia@mintos:~ > mkdir test/samples
mkdir: cannot create directory 'test/samples': No such file or
directory

This failure occurs because to make the samples directory in the test
directory, the test directory must exist. To make both the samples and test
directory, use the -p option to the mkdir command:
julia@mintos:~ > ls
data
julia@mintos:~ > mkdir -p test/samples
julia@mintos:~ > ls
data test
julia@mintos:~ > ls test
samples

To delete a directory that is empty, use the rmdir command:

julia@mintos:~ > ls
data test
julia@mintos:~ > rmdir data
julia@mintos:~ > ls
test

The rmdir command only works on empty directories:

t.me/DevTwitter
 julia@mintos:~ > ls
test
julia@mintos:~ > rmdir test
rmdir: failed to remove 'test': Directory not empty

To delete an entire directory structure, including all the files and

subdirectories, use the rm command with the -r option:
julia@mintos:~ > ls
test
julia@mintos:~ > rm -r test
julia@mintos:~ > ls
julia@mintos:~ >

Note
The rm command is normally used to delete files. With the -r
option, it can delete directories and files.

t.me/DevTwitter
 Be careful when using the rm -r command. You could accidentally
delete files that you really need to keep. Consider using the -i option
when executing the rm -r command because this enables you to pick
which files to delete. When prompted, answer y for yes and n for no:7
julia@mintos:~ > rm -ri events
rm: descend into directory 'events'? y
rm: remove regular file 'events/mate-
battstat_applet.soundlist'? y
rm: remove directory 'events'? n

Managing Files
To copy a file, use the cp command. You should have two arguments:
what file to copy and where to copy it:
julia@mintos:~ > ls
events
julia@mintos:~ > cp /etc/hosts.
julia@mintos:~ > ls
events hosts

Recall that . represents the current directory.

Be careful when using the cp command because you can accidentally
overwrite an existing file. This happens when the destination (where to
copy the file) contains a file with the exact same name as an existing
file:
ulia@mintos:~ > ls -l hosts
-rw-r--r-- 1 julia julia 221 Jul 15 10:47 hosts
julia@mintos:~ > cp /etc/passwd hosts
julia@mintos:~ > ls -l hosts
-rw-r--r-- 1 julia julia 2074 Jul 15 10:52 hosts

You can tell that the original file was overwritten because the file size
changed (221 bytes to 2074 bytes) and the modification timestamp
changed. To avoid overwriting an existing file, use the -i option:
julia@mintos:~ > ls -l hosts
-rw-r--r-- 1 julia julia 221 Jul 15 10:56 hosts
julia@mintos:~ > cp -i /etc/passwd hosts
cp: overwrite 'hosts'? n
julia@mintos:~ > ls -l hosts
-rw-r--r-- 1 julia julia 221 Jul 15 10:56 hosts

t.me/DevTwitter
 The -i option stands for “interactive” mode and prompts you if the cp
command will end up overwriting an existing file.
Some additional useful commands to manage files include the
following:
mv—Used to move files or directories
rm—Used to delete files
touch—Creates an empty file or updates the modification
timestamp of an existing file

Wildcards
Suppose you want to copy all the files that end in .conf from the /etc
directory to a directory called config in your home directory. You look
in the /etc directory and realize that there are about 20 of these files.
You don’t want to have to type in the name of each file. This is a case
in which you want to use wildcards.
With wildcards, you can make use of special characters to match file
or directory names. For example, you can list all the files in the /etc
directory that end in .conf by executing the following command:8
julia@mintos:~ > ls -d /etc/*.conf
/etc/adduser.conf /etc/insserv.conf /etc/pam.conf
/etc/apg.conf /etc/inxi.conf /etc/pnm2ppa.c
onf
/etc/avserver.conf /etc/kernel-img.conf /etc/request-
key.conf
/etc/blkid.conf /etc/kerneloops.conf /etc/resolv.co
nf
/etc/brltty.conf /etc/ld.so.conf /etc/rsyslog.c
onf
/etc/ca-
certificates.conf /etc/libao.conf /etc/sensors3.conf
/etc/casper.conf /etc/libaudit.conf /etc/sysctl.co
nf
/etc/colord.conf /etc/logrotate.conf /etc/ts.conf
/etc/debconf.conf /etc/ltrace.conf /etc/ucf.conf
/etc/deluser.conf /etc/mke2fs.conf /etc/uniconf.c
onf
/etc/fuse.conf /etc/mtools.conf /etc/updatedb.
conf
/etc/gai.conf /etc/netscsid.conf /etc/usb_modes
witch.conf
/etc/hdparm.conf /etc/nsswitch.conf /etc/wodim.con
f

t.me/DevTwitter
 /etc/host.conf /etc/ntp.conf /etc/wvdial.co
nf

The * character represents “zero or more characters in a filename.” So,

you are asking to see files in the /etc directory that begin with “zero or
more characters, followed by .conf.” Using the * character, you can
copy these files into a directory under your home directory (don’t
worry if you get an error message for some of the files):1
julia@mintos:~ > mkdir config
julia@mintos:~ > cp /etc/*.conf config
cp: cannot open '/etc/fuse.conf' for reading: Permission denied
cp: cannot open '/etc/wvdial.conf' for reading: Permission
denied
julia@mintos:~ > ls config
adduser.conf gai.conf logrotate.conf resolv
.conf
apg.conf hdparm.conf ltrace.conf rsyslo
g.conf
avserver.conf host.conf mke2fs.conf sensor
s3.conf
blkid.conf insserv.conf mtools.conf sysctl
.conf
brltty.conf inxi.conf netscsid.conf ts.con
f
ca-certificates.conf kernel-
img.conf nsswitch.conf ucf.conf
casper.conf kerneloops.conf ntp.conf unicon
f.conf
colord.conf ld.so.conf pam.conf update
db.conf
debconf.conf libao.conf pnm2ppa.conf usb_mo
deswitch.conf
deluser.conf libaudit.conf request-
key.conf wodim.conf

Use the ? character to represent a single character. So, to display any

files in the /etc directory that have filenames that are exactly four
characters in length, execute the following command:
julia@mintos:~ > ls -d /etc/????
/etc/acpi /etc/dkms /etc/init /etc/mono /etc/perl /etc/ude
v
/etc/cups /etc/dpkg /etc/kde4 /etc/mtab /etc/sgml /etc/xrd
b
/etc/dhcp /etc/gimp /etc/ldap /etc/newt /etc/skel

t.me/DevTwitter
 The ? character will match any single character. If you want to match a
specific character, you can use a set of square brackets, []. For
example, to match a file in the /etc directory that begins with an a, b,
or c character, execute the following command:
julia@mintos:~ > ls -d /etc/[abc]*
/etc/acpi /etc/avserver.conf /etc/chatscript
s
/etc/adduser.conf /etc/bash.bashrc /etc/chromium-
browser
/etc/adjtime /etc/bash_completion /etc/colord.con
f
/etc/akonadi /etc/bash_completion.d /etc/compizconf
ig
/etc/alternatives /etc/bindresvport.blacklist /etc/console
/etc/anacrontab /etc/blkid.conf /etc/console-
setup
/etc/apg.conf /etc/blkid.tab /etc/cracklib
/etc/apm /etc/bluetooth /etc/cron.d
/etc/apparmor /etc/brlapi.key /etc/cron.daily
/etc/apparmor.d /etc/brltty /etc/cron.hourl
y
/etc/apport /etc/brltty.conf /etc/cron.month
ly
/etc/apt /etc/ca-certificates /etc/crontab
/etc/at.deny /etc/ca-
certificates.conf /etc/cron.weekly
/etc/at-spi2 /etc/calendar /etc/cups
/etc/avahi /etc/casper.conf /etc/cupshelper
s

Using a Range in [ ]
Note that [abc] is the same as [a-c]. Using a - enables
you to specify a range of permitted characters. Just be
sure it is a valid range according to the ASCII text table.
You can view this table by executing the command man
ascii.

The wildcard characters are not specific to any particular command.

Instead, they are part of the BASH shell. This is important to note
because it means you can use wildcard characters with any command.
The BASH shell interprets the wildcard characters before the
command runs.

t.me/DevTwitter
 In fact, the command doesn’t even know that wildcard characters are
being used. For example, consider the following command:
julia@mintos:~ > ls -d /etc/[xyz]*
/etc/xdg /etc/xemacs21 /etc/xml /etc/xrdb /etc/zsh

The argument /etc/[xyz]* isn’t passed into the ls command. BASH

first converts the wildcard pattern into the filename(s) that match. So,
if you execute the ls -d /etc/[xyz]* command, the command that is
really executed is the ls -d /etc/xdg /etc/xemacs21 /etc/xml
/etc/xrdb /etc/zsh command.
This is also why when you use wildcards with the ls command, you
should use the -d option. When the ls command runs and is passed a
directory name as an argument, the contents of the directory are listed:
julia@mintos:~ > ls /etc/xdg
autostart menus Trolltech.conf user-dirs.conf user-
dirs.defaults

When you use wildcard characters, some of the matching files could
actually be directories. This produces output that you likely don’t want
to receive:
julia@mintos:~ > ls /etc/[xyz]*
/etc/zsh

/etc/xdg:
autostart menus Trolltech.conf user-dirs.conf user-
dirs.defaults

/etc/xemacs21:
site-start.d

/etc/xml:
catalog docbook-xml.xml.old rarian-compat.xml xml-
core.xml
catalog.old docbook-xsl.xml sgml-data.xml xml-
core.xml.old
docbook-xml.xml docbook-xsl.xml.old sgml-data.xml.old

/etc/xrdb:
Editres.ad Emacs.ad General.ad Motif.ad Tk.ad Xaw.ad

To avoid having the contents of all of these directories, use the -d

option to the ls command. The -d option tells the ls command “if any

t.me/DevTwitter
 argument happens to be a directory, don’t display what is in that
directory, just show the directory name.”

Redirection
Suppose you run a command and you decide that you want to store the
output of the command into a file for future use. In a situation like this,
you can use a process called redirection. The idea is to redirect the
output of a command into a file or another process. You can also
redirect the input to a command from a file.
Each command has three data streams:
Standard input (stdin)—The data that is sent into the command.
This is not an argument but rather additional information that is
being sent into a command. Typically, this data comes from a user
who is executing the command. The user provides this data via the
keyboard. This input could be redirected from a file or another
process.
Standard output (stdout)—The data that is sent from the
command when all goes well. Typically, this appears on the
screen, but it can be sent to a file or to another command.
Standard error (stderr)—The data that is sent from the
command when an error occurs. Typically, this appears on the
screen, but it can be sent to a file or to another command.

To redirect stdout, use the > character after the command:

julia@mintos:~ > cal 12 2000 > mycal
julia@mintos:~ > cat mycal
December 2000
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

Note
You use the cat command to display small files; it is
covered in Chapter 4.

t.me/DevTwitter
 You use the > character when you want to create a new file or
overwrite the contents of an existing file. If you want to append to an
existing file, use two > characters:
julia@mintos:~ > cat mycal
December 2000
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31
julia@mintos:~ > date >> mycal
julia@mintos:~ > cat mycal
December 2000
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31
Sun Jul 17 07:24:51 PDT 2016

The output of the cal and date commands in the previous examples is
considered standard output because the command ran successfully.
Notice that if a command fails for some reason (such as improper
options or arguments), the command’s output is not redirected to a file
when you use the > or >> characters:
julia@mintos:~ > cal -5 12 2000 > mycal
cal: invalid option -- '5'
Usage: cal [general options] [-hjy] [[month] year]
cal [general options] [-hj] [-m month] [year]
ncal [general options] [-bhJjpwySM] [-s country_code]
[[month] year]
ncal [general options] [-bhJeoSM] [year]
General options: [-NC31] [-A months] [-B months]
For debug the highlighting: [-H yyyy-mm-dd] [-d yyyy-mm]
julia@mintos:~ > cat mycal
julia@mintos:~ >

When an error occurs, the command sends output to stderr. You can
redirect this output by using the 2> characters:10
julia@mintos:~ > cal -5 12 2000 > mycal 2> error
julia@mintos:~ > cat error

t.me/DevTwitter
 cal: invalid option -- '5'
Usage: cal [general options] [-hjy] [[month] year]
cal [general options] [-hj] [-m month] [year]
ncal [general options] [-bhJjpwySM] [-s country_code]
[[month] year]
ncal [general options] [-bhJeoSM] [year]
General options: [-NC31] [-A months] [-B months]
For debug the highlighting: [-H yyyy-mm-dd] [-d yyyy-mm]

Important notes regarding redirecting stderr include the following:

The 2> characters either create a new file or overwrite the contents
of an existing file. To append stderr messages to an existing file,
use 2>>.
To send all output, both stdout and stderr, to a single file, use the
following:11
cmd > file 2>&1
Sometimes you will want to run a command without seeing any of
the error messages. To discard the output of a command, send it to
the file. This file is called the “bit bucket” or “black hole” because
whatever you send to the file will be discarded.

Redirecting stdout and stderr is a fairly common practice. Conversely,

redirecting stdin (standard input) is much rarer. Before demonstrating
redirecting stdin, consider the following exercise, which demonstrates
where stdin comes from by default. Start by executing the following
command:
julia@mintos:~ > tr 'a-z' 'A-Z'

It looks like this program hangs, but it is just waiting for stdin. You
can provide stdin from the keyboard. For example, type a sentence and
then press the Enter key:
julia@mintos:~ > tr 'a-z' 'A-Z'
today is a good day to learn linux
TODAY IS A GOOD DAY TO LEARN LINUX

You can see what the tr command does with the input. It translates all
lowercase letters to uppercase letters. Clearly, performing this task on
a file, not input from the keyboard, would be much more useful.
Unfortunately, the tr command doesn’t allow you to provide a

t.me/DevTwitter
 filename (By the way, to stop the current tr command, hold down the
Ctrl key and press the C key. This is normally written as Ctrl+C or just
^C):
julia@mintos:~ > tr 'a-z' 'A-Z' mycal
tr: extra operand 'mycal'
Try 'tr --help' for more information.

The tr command only accepts input from stdin. So, you need to tell
the BASH shell to pull stdin data from a file rather than from the
keyboard. To do this, use the < character:
julia@mintos:~ > tr 'a-z' 'A-Z' < mycal
DECEMBER 2000
SU MO TU WE TH FR SA
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

Note that the output of the tr command is sent to the screen via stdout.
Essentially, it is now lost, but you can also run the command and
redirect stdout to a file (just make sure it is a different file than the
original):
julia@mintos:~ > tr 'a-z' 'A-Z' < mycal > mynewcal
julia@mintos:~ > cat mynewcal
DECEMBER 2000
SU MO TU WE TH FR SA
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

Most commands that require input accept a file as an argument, so you

won’t redirect stdin nearly as often as you will redirect stdout or stderr.
However, some advanced scenarios exist in which knowing how to
redirect stdin is very useful.
You can also redirect stdout into another command instead of a file.
This is useful because many of the commands on the system perform
filtering or paging functions.12

t.me/DevTwitter
 For example, execute the following command: ls -l /etc. Typically,
the /etc directory contains hundreds of files, so the output of this ls
command will quickly scroll by on the screen. This can make viewing
large amounts of information difficult. The solution is to send the
output of the ls command into another command that will display one
page of data at a time: ls -l /etc | more.
The | character is used to redirect stdout to another command. It
becomes the stdin of the command to the right of the | character, in
this case the more command, which displays data one page at a time.
Chapter 4 covers the more command in greater detail. For now, use the
spacebar to scroll one page at a time and the q character to end
displaying the output and return to a prompt.

This process, called piping, is useful when you work in a command-

line environment. Understanding it might be a bit difficult if this is
your first exposure, so a couple of diagrams might be helpful. First
consider Figure 3.5, which shows how the ls command functions
when piping is not used.

Figure 3.5 The ls -l command without piping

Now compare this to Figure 3.6 in which the stdout from the ls
command is piped into the more command.

t.me/DevTwitter
 Figure 3.6 The ls -l command with piping
Notice in Figure 3.6 that only stdout is redirected when using piping.
The command’s stderr will still be displayed on the screen directly.

Linux Humor
There is no place like ~

Summary
In this chapter you learned how to manage the Linux filesystem, including
how to handle files and directories. The concept of using wildcards to match
file and directory names was introduced. In addition, you learned how to
redirect the output and input of commands into either files or other
commands.

1 I realize this part might be a bit confusing because there are really three
“roots” in Linux. The /directory is called the root directory because it is
the start of the filesystem. The system administrator is called the root user
and the home directory for the root user is the /root directory. Linux users
often refer to the root directory as the “slash” directory to avoid confusion
with /root and the root user.

t.me/DevTwitter
 2 pwd stands for print working directory. In early Unix times, monitors were
rare, so output was often sent to an actual printer. Because Unix is the
precursor to Linux, many of the commands are named the same.
3 You may be wondering if I accidently called . and .. a file. In Linux,
everything is considered a file, including directories. A directory is just a
special file that holds other directories. Normally I would refer to these as
directories, but I wanted to take the opportunity to make this point as it
could prove important as you dive deeper into Linux.
4 You may also want to know about l, which stands for symbolic link. A
symbolic link is a file that points to another file. If you have experience in
Windows, symbolic links are akin to shortcuts on your desktop.
5 File sizes in bytes can be difficult to comprehend, especially for large
files. Consider including the -h option to show file sizes in “human
readable” sizes.
6 If the file was last modified within the past six months, the timestamp
includes a month, day, and time. If the file was last modified more than
six months ago, the time is replaced with the year.
7 The rm-ri command is the same as the rm -ir command, which is the
same as the rm -r -i command. In most cases, single character options
can be combined and order doesn’t matter.
8 You may be wondering why I used the -d option with the ls command. Be
patient; I will explain this soon.
9 The cp command in this example generates some error messages because
of file permission issues. Chapter 4 covers file permissions. For now,
don’t worry if your command generates a few error messages like these.
10 It might seem strange that you use > to redirect stdout and 2> to redirect
stderr. However, the official way you redirect stdout is by using the 1>
characters. Because stdout is more commonly redirected than stderr, the
BASH shell permits you to drop the 1 before the > character.
11 You can also use the following syntax in the BASH shell: cmd &> file
12 Paging is displaying one page or screen of data at a time.

t.me/DevTwitter
 4. Essential Commands
A major component of working on a Linux system is the command-line
environment. Often referred to as the command-line interface (CLI), this
component of Linux provides a huge variety of tools. As a developer, you
don’t have to learn how to use all of these tools, but knowing key command-
line tools makes the task of developing code a much easier one.
This chapter focuses on the essential Linux commands that all developers
should know. This chapter builds on what you learned in Chapter 2 (basic
command-line execution) and Chapter 3 (filesystem management
commands) and provides you with a solid foundation for working in a Linux
command-line shell environment.

Command-Line Tools
At this point you might be wondering “Why command-line tools?” If your
experience is primarily with GUI-based systems, such as Microsoft
Windows, you might consider the CLI something that belongs in the dark
ages of computing. However, good reasons exist for why command-line
tools have their place on modern operating systems:
Stability: Many Linux commands were derived from Unix and are
essentially decades old. This stability means Linux developers can focus
on making more tools rather than reinventing features that already
exist.1
Speed of development: Developing command-line tools takes much
less time than developing GUI-based tools. As a result, the developers
who create Linux tools can create command-line tools faster than GUI-
based tools.
Scripting: Suppose you want to execute a set of instructions each day.
With a GUI-based tool, you would have to do this manually every day.
With command-line tools, you can create a script, which is a collection
of command-line tools. You will learn more about this in Chapters 7 and
8.
Speed of use: Although you might not believe this initially, you can
actually perform tasks quicker on the command line (especially if you

t.me/DevTwitter
 are good at typing on a keyboard). Normally, GUI-based tools require
both mouse and keyboard input (imagine doing a “save as” of a
document). This slows you down as you have to take your hands off the
keyboard to use the mouse (or vice versa). Additionally, in Linux you
can quickly re-execute previous commands as well as bring up previous
commands, edit them, and execute them. After you get used to all of
this, you can accomplish system tasks more quickly.
Power: You can combine commands to do things the original creator
never conceived of doing and complete tasks in a much more elegant,
efficient, and useful way.

How Many Commands Are There?

As an instructor I have often been asked by students, “Can you
provide a full list of all the Linux commands?” I often wonder
whether astronomers are ever asked the similar question, “Can
you provide a full list of all the stars in the sky?”
Although there are not 100 billion Linux commands, there are
many more than you want to commit to a single list. A typical
“small” installation with just the basic software will result in at
least a couple thousand commands. It isn’t unusual to have
more than 10,000 commands on a system that has many of the
optional software packages installed.
My advice: Don’t worry about learning about all the commands.
Focus on the ones that help you do your job (in this case, those
that help you develop code).

Viewing Files
A good number of the files on a Linux filesystem are text files. As a result, a
lot of commands exist to view the contents of text files. This section
introduces many of these files.

The file Command

Before you attempt to view the contents of a file, first make sure the contents
are in text format and not some other format. Linux supports many file types
besides just text files, including compressed files, files that contain

t.me/DevTwitter
 executable code, and database formatted files. To determine what type of
contents a file contains, execute the following file command:
[student@localhost ~]$ file /usr/share/dict/linux.words
/usr/share/dict/linux.words: ASCII text
[student@localhost ~]$ file /bin/ls
/bin/ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.32,
BuildID[sha1]=aa7ff68f13de25936a098016243ce57c3c982e06, stripped
[student@localhost ~]$ file /usr/share/doc/sed-4.2.2/sedfaq.txt.gz
/usr/share/doc/sed-4.2.2/sedfaq.txt.gz: gzip compressed data, was
"sedfaq.txt", from Unix, last modified: Mon Feb 10 09:11:16 2014,
max compression

If the output of the file command includes “text”, such as with the command
file /usr/share/dict/linux.words, then you can use the commands
described in this section to view its contents. However, you don’t want to
use these commands to view ELF 64-bit, gzip compressed data, or other
non-text file types. In most cases, viewing these files will result in “garbage”
being displayed on your screen. In some cases it can even mess up your
terminal window.2

The cat Command

For the times when you want to view the contents of a small file, the cat
command (short for concatenate) works well:
[student@localhost ~]$ cat /etc/cgrules.conf
# /etc/cgrules.conf
#The format of this file is described in cgrules.conf(5)
#manual page.
#
# Example:
#<user> <controllers> <destination>
#@student cpu,memory usergroup/student/
#peter cpu test1/
#% memory test2/
# End of file
*:iscsid net_prio cgdcb-4-3260

A useful cat option for developers is the -n option, which is used to number
lines. This can be helpful when viewing source code scripts that execute
with an error message, as shown in Listing 4.1.3

Listing 4.1 The cat -n command

t.me/DevTwitter
 Click here to view code image
[student@localhost ~]$ ./display.sh
Report of current contents of /etc:
./display.sh: line 5: [-d: command not found
[student@localhost ~]$ cat -n display.sh
1 #!/usr/bin/bash
2
3 echo "Report of current contents of /etc:"
4
5 if [-d /etc]
6 then
7 echo -n "Number of directories: "
8 ls -l /etc | grep "^d" | wc -l
9 echo -n "Number of links: "
10 ls -l /etc | grep "^l" | wc -l
11 echo -n "Number of regular files: "
12 ls -l /etc | grep "^-" | wc -l
13 fi

The more and less Commands

The problem with the cat command comes about when trying to display
large files. You will discover that it doesn’t pause at any point during display
of the file, but rather it scrolls through the file as if you had some superhero
speed-reading skill.
To pause the display while displaying the contents of large files, use the more
or the less commands:
[student@localhost ~]$ more /usr/share/dict/linux.words
[student@localhost ~]$ less /usr/share/dict/linux.words

t.me/DevTwitter
 Why Both more and less?
Why two commands that do essentially the same thing? The
more command is the original and the less command is an
“improved version” of the more command (hence giving rise to
the joke “less does more than more”).4
In reality, the extra features provided by the less command are
less-often used features, at least for most Linux users. The more
command is also useful because it is on every Linux (and Unix,
MacOS, and Windows) system in the world. The less command
is part of an optional software package and not available by
default on many systems.

The more and less commands are also useful for pausing the display of a
command that produces a large amount of output. Use the pipe character that
was covered in Chapter 3 to send the output of a command to the more
command:
[student@localhost ~]$ ls -l /etc | more

While viewing a file with the more or less command, you can use commands
to control the display. For example, press the spacebar to scroll down one
screen of data. Use the Enter key to move down one line at a time.
See the following for useful commands to control the display while using the
more or less commands:
Spacebar Scroll down one screen
Enter Scroll down one line
h Displays help screen (summary of commands)
q Exit
/{pattern} Search for {pattern}
n Find next occurrence of previous {pattern}
:f Displays filename and current line number

The head and tail Commands

Sometimes you might want to display only the top or bottom part of a file.
For example, you might want to look at the comments at the top of a source

t.me/DevTwitter
 code file. Or, you want to display recent entries of a log file, which are
normally placed at the bottom of the file. For these situations, use the head
and tail commands.
By default, these commands display ten lines. For example Listing 4.2
demonstrates displaying the top ten lines of the
/usr/share/dict/linux.words file.

Listing 4.2 The head command

Click here to view code image
[student@localhost ~]$ head /usr/share/dict/linux.words
1080
10-point
10th
11-point
12-point
16-point
18-point
1st
20-point

Use the -n option to specify how many lines to display. For example, the
command tail -n 5 /etc/passwd displays the last five lines of the
/etc/passwd file.

The wc command
To display statistical information about a file, including the number of lines,
words, and characters in the file, use the wc command:
[student@localhost ~]$ wc display.sh
13 59 291 display.sh

The output displayed is the number of lines (13), the number of words (59),
and number of bytes (291) that are in the display.sh file. Because
display.sh is a text file, the number of bytes is actually the number of
characters (1 character = 1 byte).
You can limit or modify the output of the wc command by using the
following options:
-c Display number of bytes

t.me/DevTwitter
 -m Display number of characters (different than number of bytes for
non-text files)
-l Display number of lines
-w Display the number of words

Finding Files
There is bound to be a time when you have misplaced a file or just cannot
remember where a file is stored. In these cases, you can turn to the locate or
find commands to search the system for the missing file.

The locate Command

Early each morning a database is created that contains a list of all files and
directories on the system. The locate command is used to search this
database. For example, to find the linux.words file, execute the following
command:
[student@localhost ~]$ locate linux.words
/usr/share/dict/linux.words

The locate command searches for any file that contains the pattern
“linux.words,” which might result in more output than expected:
[student@localhost ~]$ locate words | head
/etc/libreport/forbidden_words.conf
/etc/libreport/ignored_words.conf
/usr/include/bits/wordsize.h
/usr/lib64/perl5/CORE/keywords.h
/usr/lib64/perl5/bits/wordsize.ph

The find Command

The locate command is useful, but it does have a couple of drawbacks. One
drawback is that it searches a database that was created earlier in the day. So,
if you lose a file that you created today, the locate command won’t be able
to find this file.
The find command searches the live filesystem, which takes more time than
using the locate command (searching databases is much faster), but it does
find files that are currently on the filesystem. The syntax for the find
command is as follows:
find [starting location] [option/arguments]

t.me/DevTwitter
 For example, to search for the linux.words file, execute the following
command:
[student@localhost ~]$ find /usr -name linux.words
find: '/usr/lib/firewalld': Permission denied
find: '/usr/lib64/Pegasus': Permission denied
/usr/share/dict/linux.words
find: '/usr/share/Pegasus/scripts': Permission denied
find: '/usr/share/polkit-1/rules.d': Permission denied
find: '/usr/libexec/initscripts/legacy-actions/auditd': Permission
denied

Note the error messages that appear are because directories existed that the
current user was not allowed to search. This is one of the reasons why you
want to start your search in a subdirectory, not in the / directory. Another
reason is because a search of the entire filesystem, starting from the root
directory, might take a lot of time.
You can prevent these error messages by using the redirection method
discussed in Chapter 3:
[student@localhost ~]$ find /usr -name linux.words 2> /dev/null
/usr/share/dict/linux.words

Another advantage of the find command over the locate command is that
the find command can search using a variety of different file attributes. For
example, you can search for files that are owned by specific users:
find /home -user student

Commonly used find options for specifying what to search for include the
following:
-mmin n—Display files that were modified n minutes ago. Use -mmin +n
to specify “more than n minutes ago” or -mmin -n to specify “less than n
minutes ago.”
-mtime n—Display files that were modified n days ago (technically n*24
hours ago). Can use +n and -n like the -mmin option.
-group groupname—Display files owned by groupname.
-size n—Display files of a given size represented by n. Follow the n
value with a character to represent a unit of space. For example, -size
+10M would display files that were 10 megabytes or larger.

t.me/DevTwitter
 You can change what the find command does when it finds a file. For
example, the -ls option can provide detailed information about each file
found:
[student@localhost ~]$ find /usr -name linux.words -ls 2> /dev/null
22096370 4840 -rw-r--r-- 1 root root 4953680
Jun 9 2014 ➥/usr/share/dict/linux.words

Commonly used find options for specifying what to do with the files that are
found:
-delete Deletes the file.
-ls Provides a long display listing of the files that were found (like the
ls -l command).
-exec { } \; Executes a command on that file that was found. For
example:
find /home/student -name sample.txt -exec more {} \;

Note
I know that the syntax here is very strange. In a nutshell, the
find command generates a series of commands like this: more
file1; more file2; more file3. The {} represents where to
place the filename that was found and \; tells the find
command “put a semicolon between each command to treat
them as separate commands.”

Comparing Files
As a developer, you are going to have different versions of files as you
improve and bug-fix existing programs. This can cause confusion because
determining whether two files are the same or somehow different is
sometimes hard. In these cases, you should use the cmp and diff commands.

The cmp Command

If you only want to determine whether two files are different, not how they
are different, then use the cmp command. Based on the output of the
following commands, the display.sh and show.sh files contain identical
content (this results in no output when cmp is executed) whereas present.sh
contains different content:

t.me/DevTwitter
 [student@localhost ~]$ ls *.sh
display.sh present.sh show.sh
[student@localhost ~]$ cmp display.sh show.sh
[student@localhost ~]$ cmp display.sh present.sh
display.sh present.sh differ: byte 66, line 5

The cmp command is also useful for comparing two non-text files. For
example, you could compare two files that contain compiled code.

The diff Command

If you want two see how two text files differ, use the diff command:
[student@localhost ~]$ diff display.sh present.sh
5c5
< if [-d /etc]
---
> if [ -d /etc ]
13a14,15
>
> echo "The end of the report"

The output of the diff command is essentially saying, “If you make these
changes, then the files will look the same.” Each section starts with a code
that includes the line of the first file, what action to take and the line of the
second file. For example, 5c5 means “Change line 5 of the first file to look
like line 5 of the second file.”
Additional lines after the “code” line indicate what the changes would look
like:
< if [-d /etc]
---
> if [ -d /etc ]

The line that begins with < shows the current fifth line of the first file. The --
- is just used to separate the lines, and the line that begins with the > shows
the current fifth line of the second file.

Shell Features
The bash shell includes a large number of features designed to make it an
easier and more powerful command-line environment. Some of these
features, such as wildcards and redirection, have already been covered in
Chapter 3.

t.me/DevTwitter
 In this section you learn about more bash shell features, including shell
variables, aliases, and history. Knowing how to use these features will make
it much easier to work in the bash shell and make you a more powerful
software developer.

Shell Variables
Just like programming languages use variables to store values, the bash shell
also stores critical shell information in variables. To create a variable, use the
following syntax: VAR=value. To display a variable, use the echo command
and place a $ character in front of the variable name:
[student@localhost ~]$ EDITOR=vi
[student@localhost ~]$ echo $EDITOR
vi

To display all variables, use the set command. There are many predefined
variables, so you might want to pipe the output to the more or head command
to limit the output. See Listing 4.3 for an example.

Listing 4.3 The set command

Click here to view code image
[student@localhost ~]$ set | head
ABRT_DEBUG_LOG=/dev/null
BASH=/bin/bash
BASHOPTS=checkwinsize:cmdhist:expand_aliases:extglob:extquote:force_
fignore:histappend:interactive_comments:login_shell:progcomp:promptv
ars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_COMPLETION_COMPAT_DIR=/etc/bash_completion.d
BASH_LINENO=()
BASH_REMATCH=()

Variables serve three primary purposes:

To store useful information for the user. For example,
DOCS=/usr/share/docs
To store useful information for the shell or a command. For example,
the EDITOR variable is used to tell commands like visudo and crontab
which editor to use by default. For this to work, you must convert the

t.me/DevTwitter
 variable to an environment variable (see the “Environment Variables”
sidebar).
To store script data. When creating bash shell scripts, you will need to
store information. Variables are very useful for that purpose (see Chapter
8, “BASH Shell Scripting,” for more details on BASH shell scripting).

Environment Variables
By default, variables are only available to the shell they are
created in. However, you can tell the shell to pass variables to
other commands by making them environment variables.
For example, if you want to pass the EDITOR variable to any
command that is executed in the shell, execute the following
commands:5
EDITOR=vi
export EDITOR

Aliases
If you find yourself using the find command daily to search the system for
new shell scripts:
find / -name "*.sh" -ls

At some point you ask yourself, “Why do I need to type this long command
every day?” The fact is, you don’t need to. You can make an alias for this
command, which can be much shorter and easier to type. For example:
alias myfind='find / -name "*.sh" -ls'

Now when you execute the myfind alias, it will run that long find command.
However, you must create this alias every time you log in and every time
you open a new shell. To have this happen automatically, place this alias
command in a file named .bashrc in your home directory. You can also use
this file to create variables that you want enabled every time you log in to
the system.

History
Commands that you execute in a shell are saved in memory so you can
execute them again. To see these commands, execute the history command

t.me/DevTwitter
 (the output could be hundreds of commands, so limit the output with the
tail command):
[student@localhost ~]$ history | tail -n 5
258 alias hidden='ls -ld .*'
259 alias c=clear
260 alias
261 date
262 history | tail -n 5

Each command is assigned a number. You can re-execute a command by

using this number with an ! character preceding it:6
[student@localhost ~]$ !261
date
Sun May 1 01:06:21 PDT 2016

You can also bring up the previous command by pressing the up arrow key.
This allows you to modify a command before re-executing it.

Permissions
Understanding file and directory permissions is critical for Linux developers
because Linux is a multi-user environment and permissions are designed to
protect your work from others. To understand permissions, you first need to
know the types of permissions that are available in Linux and how these
permissions differ when they are applied to files versus when they are
applied to directories.
You also need to know how to set permissions. Linux provides two methods:
the symbolic method and the octal (or numeric) method.

Viewing Permissions
To view the permissions of a file or directory, use the ls -l command:
[student@localhost ~]$ ls -l /etc/chrony.keys
-rw-r-----. 1 root chrony 62 May 9 2015 /etc/chrony.keys

The first ten characters of the output denote the file type (recall that - is for
plain files and d is for directories) and the permissions for the file.
Permissions are broken into three sets: the user owner of the file (root in the
previous example), the group owner (chrony), and all other users (referred to
as “others”).

t.me/DevTwitter
 Each set has three possible permissions: read (symbolized by r), write (w),
and execute (x). If the permission is set, the character that symbolizes the
permission displays. Otherwise, a - character displays to indicate that
permission isn’t set. So, r-x means “read and execute are set, but write is not
set.”

Files versus Directories

What read, write, and execute permissions really mean depends on whether
the object is a file or directory. For files, it means the following:
Read—Can view or copy file contents.
Write—Can modify file contents.
Execute—Can run the file like a program; after you create a program,
you must make it executable before you can run it.

For directories, it means the following:

Read—Can list files in directory.
Write—Can add and delete files in directory (requires execute).
Execute—Can cd into directory or use in pathname.

The write permission on directories is potentially the most dangerous. If a

user has write and execute permission on one of your directories, then that
user can delete every file in that directory.

Changing Permissions
The chmod7 command is used to change permissions on files. It can be used
in two ways: symbolic method and octal method. With the octal method, the
permissions are assigned numeric values:
Read = 4
Write = 2
Execute = 1

With these numeric values, one number can be used to describe an entire
permission set:
7 = rwx

t.me/DevTwitter
 6 = rw-
5 = r-x
4 = r--
3 = -wx
2 = -w-
1 = --x
0 = ---

So, to change the permissions of a file to rwxr-xr--, you execute the

following command:
chmod 754 filename

With octal permissions, you should always provide three numbers, which
will change all the permissions. But what if you only want to change a single
permission of the set? For that, use the symbolic method by passing three
values to the chmod command as shown in Table 4.1.

Table 4.1 Symbolic method values

The following demonstrates adding execute permission to all three sets (user
owner, group owner, and others):
[student@localhost ~]$ ls -l display.sh
-rw-rw-r--. 1 student student 291 Apr 30 20:09 display.sh
[student@localhost ~]$ chmod a+x display.sh
[student@localhost ~]$ ls -l display.sh
-rwxrwxr-x. 1 student student 291 Apr 30 20:09 display.sh

Developer Tools
Knowing how to view files, modify file permissions, and use shell features
are important for all Linux users. Developers also should know how to
compress files and how to use the powerful filtering tool, the grep command.

t.me/DevTwitter
 File Compression Commands
As a developer you will be in a position to transfer files from one system to
another. You might be downloading software from the Internet, uploading
your programs to a server, or sending your programs to someone via email.
In all of these cases, knowing how to merge files into a single file and
compressing this merged file will be useful. This process makes transporting
large amounts of data easy and quick as well as provides something that will
take up less disk space.
Many commands in Linux enable you to create compressed files, including
the gzip, bzip2, and tar commands.

The gzip Command

The purpose of the gzip command is to create a compressed version of a file.
By default, it replaces the original file with the compressed version:
[student@localhost ~]$ cp /usr/share/dict/linux.words .
[student@localhost ~]$ ls -l linux.words
-rw-r--r--. 1 student student 4953680 May 1 09:19 linux.words
[student@localhost ~]$ gzip linux.words
[student@localhost ~]$ ls -l linux.words.gz
-rw-r--r--. 1 student student 1476083 May 1 09:19 linux.words.gz

If you want both the compressed and original file, you have to use the -c
option to send the output to standard output and keep the original file. Of
course, you don’t really want the output to be sent to the screen, so redirect
the compressed output to a file:
[student@localhost ~]$ ls -l linux.words
-rw-r--r--. 1 student student 4953680 May 1 09:19 linux.words
[student@localhost ~]$ gzip -c linux.words > linux.words.gz
[student@localhost ~]$ ls -l linux.words linux.words.gz
-rw-r--r--. 1 student student 4953680 May 1 09:19 linux.words
-rw-rw-r--. 1 student student 1476083 May 1 09:23 linux.words.gz

t.me/DevTwitter
 Note
Typically file extensions, such as .txt and .cvs, are unnecessary
in Linux. However, they are important for files that you create
with the gzip command. This utility expects the extension of
.gz when it uncompresses a file. If you name the file
linux.words.zipped, for example, the gzip command will
attempt to use the file named linux.words.zipped.gz when
uncompressing the file (and this will fail).

To uncompress a gzipped file, use the -d option:8

[student@localhost ~]$ ls -l linux.words.gz
-rw-rw-r--. 1 student student 1476083 May 1 09:23 linux.words.gz
[student@localhost ~]$ gzip -d linux.words.gz
[student@localhost ~]$ ls -l linux.words
-rw-rw-r--. 1 student student 4953680 May 1 09:23 linux.words

The bzip2 Command

The difference between gzip and bzip2 is how they perform the compression
operation. In some cases, gzip results in better compression, whereas in
others the bzip2 command does. The gzip utility is the older of the two and
considered more established, but the bzip2 utility is used fairly often on
modern Linux distributions.
Fortunately, the developers of bzip2 decided to use the same options that the
gzip utility uses:

[student@localhost ~]$ ls -l linux.words

-rw-rw-r--. 1 student student 4953680 May 1 09:23 linux.words
[student@localhost ~]$ bzip2 linux.words
[student@localhost ~]$ ls -l linux.words.bz2
-rw-rw-r--. 1 student student 1711811 May 1 09:23 linux.words.bz2
[student@localhost ~]$ bzip2 -d linux.words.bz2
[student@localhost ~]$ ls -l linux.words
-rw-rw-r--. 1 student student 4953680 May 1 09:23 linux.words

t.me/DevTwitter
 Which One Should You Use?
Keep in mind that gzip and bzip2 are just two of the
compression commands available on Linux. Others include the
zip utility and the xz command. With so many to choose from,
which should you use?
If you are only concerned about the compressed file being used
on Linux, then it really comes down to compression ratio. Try
them all, and find out which compresses the best (or the fastest
because higher compression is often slower).
If you are considering compressing a file that will be used on
other platforms, such as Microsoft Windows, I suggest using
zip or gzip because they use a more standard compression
algorithm.

The tar Command

The gzip and bzip commands are great for compressing a single file, but
what if you want to merge a bunch of files together? Typically, this means
using the tar9 command.
To create a tar file (also called a tar ball), use the following syntax:
[student@localhost ~]$ tar -cvf zip.tar /usr/share/doc/zip-3.0
tar: Removing leading '/' from member names
/usr/share/doc/zip-3.0/
/usr/share/doc/zip-3.0/CHANGES
/usr/share/doc/zip-3.0/LICENSE
/usr/share/doc/zip-3.0/README
/usr/share/doc/zip-3.0/README.CR
/usr/share/doc/zip-3.0/TODO
/usr/share/doc/zip-3.0/WHATSNEW
/usr/share/doc/zip-3.0/WHERE
/usr/share/doc/zip-3.0/algorith.txt

You use the -c option to create the tar file. The -v option stands for verbose
and results in a list of the files that are being merged into the tar file. You use
the -f option to specify the name of the resulting tar file.10

t.me/DevTwitter
 Note
What happens when you try to create a tar file and forget to
provide a name for it? You get the following message:11
tar: Cowardly refusing to create an empty archive

To list the contents of an existing tar file, use the -t option (t for table of
contents:
[student@localhost ~]$ tar -tf zip.tar
usr/share/doc/zip-3.0/
usr/share/doc/zip-3.0/CHANGES
usr/share/doc/zip-3.0/LICENSE
usr/share/doc/zip-3.0/README
usr/share/doc/zip-3.0/README.CR
usr/share/doc/zip-3.0/TODO
usr/share/doc/zip-3.0/WHATSNEW
usr/share/doc/zip-3.0/WHERE
usr/share/doc/zip-3.0/algorith.txt

To extract the files from the tar ball, use the -x option:
[student@localhost ~]$ tar -xf zip.tar

In the current directory there should now be a usr directory with a directory
structure of usr/share/doc/zip-3.0. All the extracted files are in the zip-3.0
directory.
By default, the tar command does not compress. However, you can have the
tar command use either gzip or bzip2 to compress by using the -z (gzip) or
-j (bzip2) options.

The grep Command

Many tools available on Linux are designed to perform operations on text
data, but the most powerful and useful for a software developer is the grep
command. This command is designed to act as a filter, only displaying the
lines of data that match a pattern.

t.me/DevTwitter
 Origin of the Word grep
Clearly grep is not a real word. So, where did the name come
from?
It comes from a feature of the ed editor (an editor that predates
the vi editor that you will learn about in Chapter 5). In the ed
editor, you could display only the lines that contain a pattern by
using the following syntax:
:g/pattern/p
Because the pattern could be a regular expression (regular
expressions are covered in the next section), ed documentation
typically displayed this command as
:g/re/p
The individual who created the grep command (Ken Thompson)
also created the ed feature, so he naturally named the command
after the feature.12

For example, to view all the comment lines in a shell script, execute the
following command:
[student@localhost ~]$ grep "#" /etc/rc.local
#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev
rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during
boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to
ensure
# that this script will be executed during boot.

By default, the grep command matches the pattern regardless of whether it is

part of another word. You can see the result of this by looking at line 2 of
Listing 4.4 where the the is matched within the word then.

Listing 4.4 Matching with grep

t.me/DevTwitter
 Click here to view code image
[student@localhost ~]$ grep the /etc/bashrc | cat -n
1 # will prevent the need for merging in future updates.
2 if [ "$PS1" ]; then
3 if [ -z "$PROMPT_COMMAND" ]; then
4 if [ -e /etc/sysconfig/bash-prompt-xterm ]; then
5 elif [ "${VTE_VERSION:-0}" -ge 3405 ]; then
6 if [ -e /etc/sysconfig/bash-prompt-screen ]; then
7 # if [ "$PS1" ]; then
8 if ! shopt -q login_shell ; then # We're not a login shell
9 # Need to redefine pathmunge, it gets undefined at the
end ➥of /etc/profile
10 if [ "$2" = "after" ] ; then
11 if [ $UID -gt 199 ] && [ "'id -gn'" = "'id -un'" ]; then
12 # and interactive - otherwise just process them to set
envvars
13 if [ -r "$i" ]; then
14 if [ "$PS1" ]; then

If you only want to match a pattern as a separate word, use the -w option:
[student@localhost ~]$ grep -w the /etc/bashrc | cat -n
1 # will prevent the need for merging in future updates.
2 # Need to redefine pathmunge, it gets undefined at the
end ➥of /etc/profile

Regular Expressions
Chapter 3 discussed wildcards, special characters that the bash shell uses to
match filenames in a directory. Wildcards are fairly simple to use, because
filenames are typically small and not very complex. However, text within a
file can be much more rich and complex. To perform flexible matching with
the grep command, use regular expressions (think “wildcards on steroids”).
Regular expressions are a huge topic (seriously, enough to fill up a book
larger than the size of this one). As a developer, you don’t need to know
everything about regular expressions, so to get you started, I just cover the
basics.
As you can see from the following example, the grep command returns
results regardless of where on the line the pattern is found:
[student@localhost ~]$ grep "growths" /usr/share/dict/linux.words
growths
ingrowths
outgrowths
regrowths

t.me/DevTwitter
 undergrowths
upgrowths

If you only want to see the lines that begin with the pattern, use the regular
expression character ^ at the beginning of the pattern:
[student@localhost ~]$ grep "^growths" /usr/share/dict/linux.words
growths

If you only want to see the lines that end with the pattern, use the regular
expression character $ at the end of the pattern.

Remember Listing 4.1?

Listing 4.1 is a bash shell script (with an error) that includes
lines like the following:
ls -l /etc | grep "^d" | wc -l
ls -l /etc | grep "^-" | wc -l
ls -l /etc | grep "^l" | wc -l
These lines took the output of the ls -l command and sent it to
the grep command to display file times. Lines from the ls -l
command that begin with the letter d are directories. By sending
this output to the wc command, you get a count of how many
directories are in the /etc directory.
In Chapter 8, “BASH Shell Scripting,” you will learn about the
rest of what this bash shell script is doing.

Another useful regular expression character is the . character, which

represents exactly one character. In the following example, "r..t" matches
“root” on lines 1 and 2. For line 3 "r..t" matches “r/ft”:
[student@localhost ~]$ grep "r..t" /etc/passwd | cat -n
1 root:x:0:0:root:/root:/bin/bash
2 operator:x:11:0:operator:/root:/sbin/nologin
3 ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

Additional useful grep regular expressions (some of which require using the
-E option for extended regular expressions) include
* Matches zero or more of the previous character.
+ Matches one or more of the previous character (requires using the -E
option).

t.me/DevTwitter
 . Matches any single character.
[ ] Matches a single character from a subset of characters; [abc]
matches either an a, b, or c.
? Matches an optional character; a? means “match either the character
‘a’ or nothing” (requires using the -E option).
| Match one or another; abc|xyz matches either abc or xyz (requires
using the -e option).
\ Escapes the special meaning of a regular expression character; \*
matches simply a * character.

Note
Regular expressions are extremely useful, not just for the grep
command, but many other Linux tools as well as many
programming languages. This is a very good topic to expand
your knowledge on to become a more powerful developer.

Searching for Files with grep

The find and locate commands are useful for finding files by name, but
they can’t find files based on the contents of a file. The grep command can
search all files within a directory structure recursively if you use the -r
option.
When you use the grep command in this manner, you probably want to use
the -l option, which will list matching filenames (rather than listing every
line in every file that matches). You probably also want to redirect STDERR
to suppress error messages for files and directories that you don’t have
permission to. See Listing 4.5 for an example that searches for all bash shell
scripts in the /etc directory structure.

Listing 4.5 Searching with grep

Click here to view code image
[student@localhost ~]$ grep -rl '^#!/bin/bash' /etc/* 2> /dev/null
/etc/auto.net
/etc/auto.smb
/etc/cron.daily/0yum-daily.cron
/etc/cron.daily/man-db.cron
/etc/cron.hourly/0yum-hourly.cron

t.me/DevTwitter
 /etc/init.d/netconsole
/etc/kernel/postinst.d/51-dracut-rescue-postinst.sh
/etc/NetworkManager/dispatcher.d/11-dhclient
/etc/NetworkManager/dispatcher.d/13-named
/etc/pki/tls/certs/renew-dummy-cert
/etc/ppp/ip-down
/etc/ppp/ip-up
/etc/ppp/ipv6-up
/etc/qemu-ga/fsfreeze-hook
/etc/rc.d/init.d/netconsole
/etc/rc.d/rc.local
/etc/rc.local
/etc/sysconfig/network-scripts/ifdown-eth
/etc/sysconfig/network-scripts/ifdown-tunnel
/etc/sysconfig/network-scripts/ifup-aliases
/etc/sysconfig/network-scripts/ifup-eth
/etc/sysconfig/network-scripts/ifup-sit
/etc/sysconfig/network-scripts/ifup-tunnel
/etc/sysconfig/network-scripts/ifup-wireless
/etc/sysconfig/network-scripts/ifdown-ib
/etc/sysconfig/network-scripts/ifup-ib
/etc/sysconfig/raid-check
/etc/vsftpd/vsftpd_conf_migrate.sh
/etc/X11/xinit/xinitrc.d/50-xinput.sh
/etc/X11/xinit/xinitrc.d/zz-liveinst.sh
/etc/X11/xinit/Xclients
/etc/X11/xinit/Xsession

Linux Humor
Who needs to go to the movies? Big screen Hollywood action is
available right on your Linux computer. Type the following
command, grab some popcorn, and enjoy:
telnet towel.blinkenlights.nl
P.S. To stop the “movie,” hold down the Ctrl button and press
the ] key. Then type quit at the telnet> prompt and press the
Enter key.

Summary
At this point you should have a solid foundation that will enable you to work
in the Linux command-line environment. You learned the essentials, such as
how to view a file and make use of bash shell features. You should now
know how to secure your files using permissions. In the next chapter you

t.me/DevTwitter
 will build on these tools to learn some important system administrative tasks
of which developers should be aware.

1 It also means that you can wake up a Unix developer who was
cryogenically frozen in the 1970s and that developer would already
understand the basics of working in Linux. Note that the process of
waking up cryogenically frozen developers is beyond the scope of this
book.
2 If you view a non-text file accidentally and it messes up your terminal
display with “garbage” characters, type the reset command and press the
Enter key. Don’t worry if it looks like garbage when you type the
command, it will execute properly and fix your terminal display.
3 Are you wondering what this script actually does? Keep reading this
chapter to find out!
4 This is about as funny as Linux jokes get. I apologize for all future Linux
jokes made throughout this book.
5 You can also do this in one step: export EDITOR=vi
6 The ! character is often called the bang character in Linux. Other
common character nicknames include splat for * and hash for #.
7 Permissions used to be called modes of access, hence the origin of the
name chmod (change mode of access).
8 You can also use the gunzip command.
9 The command comes from the phrase Tape ARchive. Or it could be TApe
aRchive. You pick.
10 Note that for the tar command, the - character before the options is
optional. So, tar cvf is the came as tar -cvf. A handful of Linux
commands don’t require the - character before options.
11 Okay, some Linux jokes are a little bit funny.
12 With this new Linux trivia, you are bound to be the life of any party.

t.me/DevTwitter
 5. Text Editors
As a developer, you will edit text files on a regular basis. This can
sometimes pose a challenge because Linux offers a variety of editors. In
many cases, you should be able to pick your editor, but sometimes you might
be forced to use one of the standard editors, such as the vi editor.
This chapter focuses on the editors that are available on Linux distributions.
The primary focus is on the vi (or vim) editor because it has the advantage of
always being available on whatever Linux distribution you are working on. I
introduce some additional editors to help you determine which tool will
work best for your situation.

The vi Editor
Consider the early days of Unix, the precursor to Linux: A developer would
sit down at a keyboard, ready to edit a program that he is working on. He
stares at the printer (yes, printer, not monitor) considering what commands
to execute. Monitors were very rare in the early 1970s and even if a
developer had one, it was primarily designed to display the output of
executed code, not to interactively edit files.
Instead, the developer would use a simple command-based editor, such as
the ed editor. With this editor a developer could perform operations such as
list the contents of a file (print the file), modify specific characters of a file,
or save the contents of a file. However, he accomplished all this in a way
that might seem foreign today. The developer wouldn’t see what he was
editing, but rather just assume the commands were successful (or print the
file to verify).1
When monitors became more commonplace, the ed editor seemed like a
clumsy way to edit a text file. In the mid-1970s a replacement editor named
vi (short for visual) was introduced to Unix.2 It was a great improvement
over the ed editor3 because you could actually see and move around in your
document.

Why Learn vi?

t.me/DevTwitter
 You will soon discover that some editors are easier to use than vi. This
discovery will result in your asking the question, “Why the heck should I
bother learning the vi editor?” Several good reasons exist, even if you never
plan on using the vi editor on a daily basis:
vi requires no GUI: Many of the editing tools that this chapter
introduces require a graphical user interface. Normally that is not a
problem, but on many servers, the GUI isn’t even installed because it
tends to be a resource (CPU, memory, and hard drive space) hog. If you
might be called upon to edit code on a server, then you might need to
know a command-line editor like the vi editor.
vi is a very stable standard: You could take a cryogenically frozen
developer from the 1970s who used the vi editor, unfreeze him today
and he could edit files using a modern vi editor. Certainly, features have
been added to the vi editor since the 1970s, but the core functionality of
vi does not change, making it very easy for you to use throughout your
career without having to “relearn” how newer versions work.
vi is always there: In every Linux distribution (and all Unix ones as
well) the vi editor is there. After you know how to edit files with the vi
editor, you can edit files on any Linux machine.4
Speed of use: Although you might not believe this initially, you can
actually edit files quickly using the vi editor. Granted, this will take
many years of practice, but because you never need to use the mouse,
you never need to take your hands off your keyboard when editing a file.
The commands are also very short, so you can edit files very quickly.

What Is vim?
The vim editor5 was released in 1991 as a clone of the vi editor. The vim
editor has the same base functionality as the vi editor, but it has several
additional features. Some of these features can be useful for software
developers.
Your distribution might only have the vi editor. Many distributions have both
the vi and vim editor. On some distributions, the command vi is actually a
link to the vim editor.
An easy way to tell whether you are using the vi or vim editor is to try
executing the vi command. If you are using vim, a message like the

t.me/DevTwitter
 following appears: VIM – Vi IMproved. If you don’t get this message, then
you are using a standard vi editor.6

Note:
Unless stated otherwise, the commands shown in this chapter
work in both the vi and vim editors. Any command that only
works in vim is denoted as such.

Essential vi Commands
To become an expert vi user can take a lot of practice, but to be able to
effectively edit files requires the knowledge of a subset of the large amount
of vi commands.
It helps to have a large file to edit. All Linux distributions should come with
a /etc/services file that is typically thousands of lines long. You can start
by first copying this file to your home directory and then edit the copy with
the vi command:
[student@fedora ~]$ cp /etc/services.
[student@fedora ~]$ vi services

Entering Insert Mode

When you first start the vi editor, you are placed in the command mode. This
mode allows you to perform commands such as moving around the screen,
copying text, and deleting text.
While in the command mode, you can’t insert new text into your document
because all the keyboard keys are assigned to command tasks. To insert new
text, you must use the s command to move from command mode to insert
mode. These commands include the following:
i—New text will appear before the cursor position.
a—New text will appear after the cursor position.
I—New text will appear at the beginning of the line.
A—New text will appear at the end of the line.
o—Opens a new line below the line that contains the cursor; new text
will be placed on this line.

t.me/DevTwitter
 O—Opens a new line above the line that contains the cursor; new text
will be placed on this line.

See Figure 5.1 for a visual description of how these vi commands work.

Figure 5.1 vi commands to enter insert mode

Note that when you are using the vim editor and enter the insert mode, the
bottom part of the screen changes to indicate this. See the -- INSERT -- at
the bottom of Figure 5.2.

Figure 5.2 The insert mode

t.me/DevTwitter
 If you are working in a standard vi editor, then -- INSERT -- does not appear
at the bottom of the screen by default. To enable this feature in a standard vi
editor, type the following command while in command mode:
:set showmode

When you want to return to command mode, just press the escape (Esc) key.
This is normally denoted in documentation as <ESC>. If you return to the
command mode, then the -- INSERT -- should disappear from the bottom of
the screen.

Movement Commands
While in the command mode, you can move the cursor in your document by
using a variety of keys. One of the common methods is to move the cursor
one character to the left or right or one line up or down. You do this by using
either the arrow keys on your keyboard or using the h, j, k, and l keys7 as
shown in Figure 5.3.

Figure 5.3 vi movement commands

Many additional movement commands are available, including the
following:
$ Move to the last column (character) on the current line

t.me/DevTwitter
 0 Move to the first column (character) on the current line
w Move to the beginning of the next word or punctuation mark
W Move to past the next space
b Move to the beginning of the previous word’s punctuation mark
B Move to the beginning of the previous word, ignoring punctuation
e Move to the end of next word or punctuation mark
E Move to the end of next word, ignoring punctuation
) Move forward one sentence
( Move back one sentence
} Move forward one paragraph
{ Move back one paragraph
H Move to the top of the screen
M Move to the middle of the screen
L Move to the bottom of the screen
[[ Move to the beginning of the document
]] Move to the end of the document
G Move to the end of the document (same as ]])
xG Move to line x (you can also use :x)

Note that these are just some of the movement commands. Here’s a
suggestion: spend some time trying out these movement commands and then
create a cheat sheet of the commands that you feel will be most useful to
you. Make this a cheat sheet that you can add more commands to as you
learn additional useful commands.8

Repeater Modifiers
In the previous section, you discovered that you can jump to a specific line
in a document by typing a number followed by a G while you are in the
command-mode. For example, the command 7G will take you to line number
7 in the document.
Placing a number before a command acts as a modifier. You can use
modifiers on many different command-mode commands; for example:
3w Move forward three words

t.me/DevTwitter
 5i Insert something five times9
4( Move back three paragraphs

You can also use repeat modifiers on commands like deleting, copying, and
pasting. Typically, if there is a command-mode command that you would
logically want to execute multiple times, then repeat modifiers should work
with that command.

Are You Trying These Commands?

I had you copy the /etc/services file to your home directory so
you can try the commands out. Remember, if you get stuck in
insert mode, just press the Esc key to return to command mode.
Don’t worry if you end up with a big mess in this file. It is just a
practice file and you are about to learn how to correct mistakes.

Undoing
You can undo whatever change has been made to the document by typing the
u character in command mode. In the standard vi editor, you can only undo a
single action;10 in fact, the u command acts as an undo/redo key.
If you are using the vim editor, you can undo several actions. Just keep
pressing the u character to undo older modifications. You can perform a redo
that undoes the changes performed by the undo command by using the ^r
(Ctrl+r) command.
Suppose you made a large number of changes since you opened the
document and you want to discard them all. In this case, you probably want
to close the document without saving and open it again. To close the
document without saving changes, type the command :q!. You can find
more on this command and other ways to quit the vi editor in the “Saving
and Quitting” section later in this chapter.

Copying, Deleting, and Pasting

The following is a summary of commonly used copying commands. Keep in
mind that you should execute these while in command mode:
yw—Copy word. This actually copies from the current character in a
word until the end of the word (including punctuation) and the white

t.me/DevTwitter
 space after the word. So, if your cursor was on the h in “this is fun,” the
cw command would copy “his ” into memory.
yy—Copy current line.
y$—Copy from current character to end of the line.
yG—Copy current line to the end of the document.

You might be wondering “Why use the y character?” This is because the
process of copying text into the memory buffer used to be called yanking.
The following is a summary of commonly used deleting commands. Keep in
mind, you should execute these while in command mode:11
dw—Delete word; this command actually deletes from the current
character in the word until the end of the word (including punctuation)
and the white space after the word. So, if your cursor was on the h in
“this is fun” the dw command would delete “his ”, resulting in “tis fun.”
dd—Delete current line.
d$—Delete from current character to end of the line.
dG—Delete current line to the end of the document.12

x—Delete the character the cursor is currently on (like a delete key).13

X—Delete the character before the character the cursor is currently on
(like a backspace key).

Where Are the Cut Commands?

When you use a delete command, the text that was deleted is
placed into the copy buffer. As a result, no need exists for a
separate set of cut commands.

Pasting commands can be a bit trickier because how they work depends on
what you are pasting. For example, suppose you had copied a word into the
buffer. In this case, the following describes how the paste commands would
work:
p—Pastes the buffer contents before the cursor
P—Pastes the buffer contents after the cursor

t.me/DevTwitter
 The behavior is a bit different if you copy an entire line (or multiple lines)
into the buffer:
p—Pastes the buffer contents in the line above the cursor
P—Pastes the buffer contents in the line below the cursor

Finding Text
Finding text is a critical function for a software developer who is using the vi
editor because error messages that appear when code is executed often
include bits of code where the error occurs. You can search for text by using
one of the following methods:
/ While in command mode, type the / key; this character appears in the
bottom-left corner of the terminal window. Now type what you want to
search for and then press the Enter key. The vi editor will search forward
in the document for the text you asked it to search for.
? While in command mode, type the ? key; this character appears in the
bottom-left corner of the terminal window. Now type what you want to
search for and then press the Enter key. The vi editor will search
backward in the document for the text you asked it to search for.

Suppose your search didn’t find the specific match you were looking for.
You can use the n command to find the next match. The n command searches
forward when your last search started with the / key and searches backward
when your last search started with the ? key.14
What if you searched for “/one” and realize you will need to press the n key
many times to find what you are looking for? After furiously typing n
repeatedly, you realize you went past the match that you wanted. To reverse
the current search, use the N character (capital N rather than lowercase).
When you’re searching forward, N will reverse and search backward in the
document. When you’re searching backward, N will reverse and search
forward in the document.

t.me/DevTwitter
 Case Sensitive
As with just about everything you use in Linux, the search
function is case sensitive. In other words, a search of /the will
not match the following line:
The end is near

Searching and Replacing

To search for text and replace it with other text, use the following format:
:x,ys/pattern/replace/

The values of x and y represent which lines you want to perform the search
on. For example, to search and replace on only the first ten lines of the
document, use the following syntax:
:1,10s/I/we/

You can use the $ character to represent the last line in the document:
:300,$s/I/we/

So, to perform the substitution on the entire document, use the following:
:1,$s/I/we/

By default, only the first match on each line is replaced. Imagine if the line
you are searching for and replacing looked like the following:
The dog ate the dog food from the dog bowl

If the command :s/dog/cat/ were executed on the previous line, then the
result would be:
The cat ate the dog food from the dog bowl

To replace all occurrences on a line, add a g15 at the end of the search
command:
:s/dog/cat/g

Searching and replacing is case sensitive. Imagine if the line you are
searching and replacing looked like the following:
The Dog ate the dog food from the dog bowl

t.me/DevTwitter
 If the command :s/dog/cat/ were executed on the previous line, then the
result would be:
The Dog ate the cat food from the dog bowl

The result matched the second dog because the first one had a capital D
character. To perform a case-insensitive search and replace, add an i at the
end of the search command:
:s/dog/cat/i

Saving and Quitting

In the previous section, you typed a : character to perform a search and
replace operation. Complex commands are performed in the last line mode,
also called the ex mode in honor of the ex editor. The : character takes you
to the bottom of the screen where the command appears as you type it.
Another operation you can do in this mode is save and quit your document:
:wq

Note
You must save before quitting, so you can’t execute the
command :qw because that will attempt to quit and then save.

You might also want to just save, but continue working:

:w

You can also save to a different document, but there is a little gotcha to this
operation that you will discover. Suppose you want to save changes made to
your services file into a file called myservices. Execute the following:
:s myservices

The changes will be placed into this file. However, any further changes will
be saved by default into the original services file. Most modern editors
“switch” the default save document to whatever the last saved document
was, but vi doesn’t do this. To see your current document, type ^G (Ctrl+g).
So, if you want to edit the new file, you should quit the vi editor and open
the new file.

t.me/DevTwitter
 If you make changes to a file and then try to quit without saving (:q), you
receive an error message like the following:
E37: No write since last change (add ! to override)

To force quit (quit without saving changes), execute the following

command:
:q!

Expanding Your vi Knowledge

Although the discussion has covered many vi commands, it really just
scratches the surface. The vi editor is a very powerful tool with hundreds of
commands. In addition, it provides some very advanced features, such as
syntax highlighting, the capability to create macros, features for editing
multiple files at the same time, and much more.
The vim editor has some very useful built-in documentation, but you must
have a specific software package installed on Red Hat–based distributions16
in order to be able to access this documentation. Chapter 6 covers installing
software in greater detail. For now, just make sure you are logged in as the
root user and execute the following command: yum install vim-enhanced
If this package is installed, you can execute the command :help while in the
vim editor to see a help document. See Figure 5.4 for an example.

t.me/DevTwitter
 Figure 5.4 vim help output
Use your arrow keys (or h, j, k, and l) to scroll through the document. About
20 lines down, you will start to see some subtopics, as shown in Figure 5.5.

t.me/DevTwitter
 Figure 5.5 vim help topics
Each of these topics, such as quickref and usr_01.txt, are separate help
topics. To view them, first exit from the current help document by typing the
:q command. Then type a command like the following, replacing topic with
the full name of the topic you want to view:
:help topic

For example, to see help on “Using syntax highlighting,” type the following
command:
:help usr_06.txt

vimtutor
A great tool to help you learn the vi editor is the vimtutor
command. This command takes you into vi and provides a
useful guide to learning the vim editor.

Additional Editors

t.me/DevTwitter
 A large number of editors are available that you can use in Linux. The focus
of this section is to make you aware of these editors, not to teach you how to
use them.

Note
It is likely that not all of these editors will be installed on your
distribution. You might need to install additional software
packages to have access to them.

Emacs
Like the vi editor, the Emacs editor was developed in the mid-1970s.17
Linux users who like Emacs will praise how easy it is to use and how
customizable it is. If you launch Emacs (just run the emacs command) while
in a GUI-based terminal, it should open a GUI-based version of the program
as shown in Figure 5.6. As you can see, the GUI-based version has menus in
addition to the commands that you can execute via the keyboard.

t.me/DevTwitter
 Figure 5.6 GUI-based Emacs
If you execute the Emacs editor in a command line–only environment, the
editor will look like Figure 5.7.

t.me/DevTwitter
 Figure 5.7 Text-based Emacs

GUI vim?
If you install the vim-X11 software package, you get access to a
GUI-based version of the vim editor. Just execute gvim or vim -
g on Red Hat–based distributions or vim-gtk on Debian-based
distributions.

gedit and kwrite

These editors are fairly standard GUI-based editors (gedit comes with
GNOME and kwrite comes with KDE). If you are used to Notepad on
Microsoft Windows, then you will find these editors fairly easy to use
(although somewhat limited).
The gedit editor typically is installed on distributions that use the GNOME
desktop by default. The kwrite (or KATE) editor typically is installed on
distributions that use the KDE desktop by default. However, you can easily
install gedit on a system that uses KDE desktop or install kwrite on a
GNOME desktop system.

nano and joe

t.me/DevTwitter
 The vi and emacs editors are extremely powerful. In some cases, you might
find that you want to just use a simple edit in a command-line environment.
The gedit and kwrite editors only work in GUI-based environments. The
nano editor is also typically installed by default on most Linux distros.
The nano and joe editors provide a simple interface for editing text files.
They are command line–only editors, so no GUI is required. See Figure 5.8
for an example of the nano editor.

Figure 5.8 The nano editor

lime and bluefish

The lime and bluefish editors take the process of editing a text file to the
next level by providing tools and features that are designed to help a
developer create code. These tools provide features like syntax highlighting,
insertion of code (by clicking a button), and automatic formatting (like
automatically indenting code).
If you are going to make a career of coding on Linux, you should start to
explore these tools (or many of the other similar editing tools). See Figure

t.me/DevTwitter
 5.9 for an example of the bluefish editor.

Figure 5.9 The bluefish editor

Linux Humor
Try this one in your shell:
bo@mintos:~ > cd /
bo@mintos:/ > touch me
touch: cannot touch 'me': Permission denied

Summary
The focus of this chapter was editors, primarily the vi or vim editor. We
covered the basics of editing files in vi/vim, including how to start the editor,
the three modes of operation, how to add and delete text, and some more
advanced features. You were also introduced to some additional editors that
are available in Linux. At this point, using these editors comes down to
practice, which you should get plenty of when you start writing code on
Linux.

1 On some modern Linux distributions, the ed editor still exists. Try typing
the command ed in a terminal window. If the editor does exist on your
system, you will be placed in the ed editor environment (essentially a

t.me/DevTwitter
 blank prompt). In reality, the only ed command you want to know is q
because this quits the ed editor.
2 By the way, vi is pronounced vee-eye, not a single syllable vi.
3 Technically, it was an improvement of the ex editor, which was an
improvement of the em editor, which as an improvement of the ed editor.
Use this Unix trivia to entertain your friends at your next party. On second
thought, maybe just keep this nugget to yourself.
4 Although removing the vi editor is possible, I’ve never heard of any
distribution that has done so.
5 vim = a contraction of Vi IMproved
6 To exit vi/vim, type :q and then press the Enter key.
7 Why use the h, j, k, and l keys when you can use the arrow keys? When
the vi editor was developed, most keyboards didn’t have arrow keys. Even
today, rack-mounted servers are sometimes connected to dump terminals
that have keyboards that lack arrow keys.
8 Or visit the “Vi lovers” page (http://thomer.com/vi/vi.html) and download
one of the reference cards under the “Vi pages/manuals/tutorials” section.
Personally, I prefer to make my own because the commands that someone
else finds useful, I might not find very helpful.
9 Be careful here. I once accidently typed the 8 key twice before pressing
the i key to enter insert mode. After typing pages of text, I pressed the Esc
key and received 88 times the text that I typed. Read on to learn how I
fixed this problem quickly.
10 Actually, if you don’t move to another line, you can undo all changes on
the current line by typing the u character while in command mode.
However, I think you will find that you’ll rarely use this feature because
the moment you move to a new line, these undos are lost.
11 Notice how similar these deleting commands are to the copying
commands. Not only did this allow me to avoid a lot of extra typing in this
book (call me lazy, or just an effective developer who “reuses” good
“code”), but it highlights that most of the copying commands are just like
deleting commands, which should make the process of learning them
easier.
12 In a previous footnote, I mentioned that I once accidently inserted several
pages of text 88 times. To fix this, I went to the first line in the second

t.me/DevTwitter
 “set” and typed the dG command. This deleted all but the first copy of
what I had typed with minimal effort.
13 If you are using the vim editor, you can use the delete key to delete the
character you are currently on. However, the backspace key works like a
back arrow key.
14 Remember the regular expressions that were covered in the section on the
grep command in Chapter 4? If not, go back and read that section again!
Regular expressions show up in many tools in Linux, including the vi
editor. For example, if you search for ^The, the vi editor will only match
lines that begin with The, rather than The anywhere on the line.
15 You can think of g standing for get them all. It actually stands for global.
16 The help feature comes with vim by default on Debian-based
distributions.
17 Emacs versus vi/vim has been referred to as a “religious war” between
the users of these editors. It is a war that I’ve always avoided; be aware
that there are some in the Linux community who take this war very
seriously.

t.me/DevTwitter
 6. System Administration
System administration is a huge topic that incorporates tasks such as
configuring services, maintaining the health of the operating system, and
keeping the system secure. Whole volumes have been devoted to teaching
individuals how to administer a Linux distribution. As a developer, you
should consider leaving the heavy lifting aspects of system administration
to full-time system administrators.
However, that doesn’t mean that you should never take on some of the
responsibilities of system administration. Some tasks you will want to be
able to accomplish without having to bother a system administrator. These
tasks include installing software and maintaining user accounts. This
chapter focuses on the essential system administration tasks that all
software developers should have in their skill set.

Essential Tasks
In almost all cases, you should log in to the system using a regular user
account and avoid logging in as the root user (the system administrator
account). Routinely executing commands as the root user is just asking for
trouble.
The root user has full control over the system, including the capability to
delete all files and directories. The problem with working as the root user
on a regular basis is that you can potentially damage your operating system,
making it unusable. For example, consider the following command (but
don’t run this command!):
[root@fedora ~]$ rm -rf /

If you ran the preceding command while you were logged in as the root
user, every file and directory on the system would be removed. As a regular
user, this could result in loss of files in your home directory, but even that
could be avoided by pressing Ctrl+C after you noticed all the error
messages that would appear as you try to delete files that you don’t have
permission to delete.
So to summarize, the best practice is this: Log in as a regular user and only
assume the identity of the root user if you need to perform a specific task as

t.me/DevTwitter
 the root user.

Gaining Access to the Root Account

You can use three techniques to assume the identity of the root user:
Log in directly as the root user: As previously mentioned, this is not
the ideal method. Even system administrators avoid logging in directly
as the root user.
Use the su command: With the su command you can switch user to
the root account if you know the root password. This command opens a
new shell, and in that new shell you can commands as the root user. To
return to your regular user account, you close the shell by executing the
exit command.
Use the sudo command: With the sudo command, you can execute
commands as the root user without having to even know the root
password. However, this feature does need to be set up by the system
administrator to work properly.

Let’s look at the su and sudo commands in a little more depth.

Using the su Command

To use the su command, execute the command as shown:
[student@localhost ~]$ id
uid=1000(student) gid=1000(student) groups=1000(student),10(wheel)
context=unconfined_
u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[student@localhost ~]$ su - root
Password:
[root@localhost ~]# id
uid=0(root) gid=0(root) groups=0(root)
context=unconfined_u:unconfined_r:unconfined_
t:s0-s0:c0.c1023

Note that the id command displays your current user account. In this case
the id command wasn’t necessary because you can see the current user
name in the prompt.
You will often see the argument root omitted when using the su command.
If you don’t specify a user account name, the root user is assumed by
default:

t.me/DevTwitter
 [student@localhost ~]$ su -
Password:
[root@localhost ~]# id
uid=0(root) gid=0(root) groups=0(root)
context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

The - option is not only strange because it lacks a character after the -, but
it is also very important.1 Without the - character, you will not fully switch
to the root user account because the login scripts for the root user do not
execute. The best way to demonstrate the difference between using the -
and not using it is by looking at the code in Listing 6.1.

Listing 6.1 The - option of the su command

Click here to view code image
[student@localhost ~]$ su root
Password:
[root@localhost student]# pwd
/home/student
[root@localhost student]# echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/student/.lo
cal/bin:/home/student/bin
[root@localhost student]# exit
exit
[student@localhost ~]$ su - root
Password:
[root@localhost ~]# pwd
/root
[root@localhost ~]# echo $PATH
/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/
bin

Note that in Listing 6.1 when the - was not used, the current directory did
not change and the value of the $PATH variable didn’t change to the value for
the root user. Using the - character fully switches you to the root account,
the current directory switches to the root user’s home directory, and the
$PATH variable is set to the proper value for the root user (look at the end:
/root/bin).
In many cases, it won’t matter if you use the - character or not. However,
sometimes not fully switching to the root user account can cause problems.
The best practice is to use the - character when switching to the root
account.

t.me/DevTwitter
 Important Note
Always remember that when you finish executing commands
that require root privileges, switch back to your regular user
account by executing the exit command.

Using the sudo Command

The sudo command allows you to execute commands as the root user
without even knowing the root password but only if this feature has already
been configured. On some distributions, notably Ubuntu and Mint, the sudo
command is set up for the first user account by default:
bo@mintos:~ > sudo id
[sudo] password for bo:
uid=0(root) gid=0(root) groups=0(root)

Note that the password that was requested was not the root password, but
rather the password of the current user (the bo user in this case). The sudo
command takes another command as its argument and will execute that
other command as the root user provided the correct password is provided
and the sudo command has been set up correctly.
To set up the sudo command, add a line like the following in the
/etc/sudoers file2:
!!
bo ALL=(ALL:ALL) ALL

The previous line would allow the user bo to use the sudo command to
execute commands as the root user. Note that you can also apply this
feature to entire groups3 as shown in the following:
bo@mintos:~ > sudo grep %sudo /etc/sudoers
%sudo ALL=(ALL:ALL) ALL
bo@mintos:~ > id
uid=1000(bo) gid=1000(bo)
groups=1000(bo),4(adm),24(cdrom),27(sudo),30(dip),
46(plugdev),108(lpadmin),111(sambashare)

So, the reason why the bo user in the previous example can execute
commands as the root user using the sudo command is because the bo user
is a member of the sudo group.

t.me/DevTwitter
 If you need to give sudo access to a user or group, first switch to the root
account and then execute the visudo command. This command
automatically edits the /etc/sudoers file using the vi or vim editor. One
advantage of using the visudo command rather than the regular vi or vim
editor is that the visudo command performs some basic error checking
when you save your changes.

Displaying Disk Usage

As a developer, displaying disk usage can be an important task. The amount
of free space available will have an impact regarding what software you can
install on the system. In addition, the programs you create might be very
large or create large files, so displaying disk usage can be critical to making
sure enough room exists for your program data.
On a Linux system, the space on the hard drive is broken into chunks called
partitions or volumes. This is also true on other operating systems, such as
Microsoft Windows; however, typically the result is a bit different. Making
a partition out of an entire hard disk is common practice on Windows OS,
whereas in Linux creating several partitions (or volumes4) on one hard disk
is common.
To display these partitions, including how much space is available, execute
the df command as shown in the following:5
[student@localhost ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 6.7G 4.1G 2.6G 61% /
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 88K 1.9G 1% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda1 497M 196M 302M 40% /boot
tmpfs 389M 8.0K 389M 1% /run/user/0

The Filesystem column is used to indicate the partition (/dev/sda1) or

volume (/dev/mapper/centos-root). Lines that don’t represent a path to a
file, such as devtmpfs or tmpfs, are memory-based filesystems and are not
important for this topic.
The Mounted on column indicates to which directory structure the partition
or volume is attached. Recall that, unlike Microsoft Windows, devices are

t.me/DevTwitter
 not assigned drive letters but rather are placed under directory structures,
like the /boot directory.
Based on the output of the df command, you can see how much space is
available. For example, in the previous output, the /boot directory structure
could support up to 302MB more data. The following directory structures
are normally the most critical for developers to be aware of the available
space:
/usr—Location where new software will be installed
/home—Home directories for regular users, including your own account
/tmp—A location to store temporary files. As a developer you might
need to create a file to hold data while your program is executing.
Placing this file in the home directory of the user who is running the
program is not ideal (they might delete it accidently). The /tmp
directory is the best place to store such a file.

Note
If you don’t see /usr, /home, or /tmp in the Mounted on column
of the output of the df command, then these directories are not
separate partitions or volumes, but rather are part of the /
directory structure.

Determining how much space the files in a specific directory use on the
hard disk is also useful. This can be important when you want to see how
much space the removal of some large files within a directory can free up.
To see how much space the files in a directory (and all subdirectories) are
using, use the du command:
[student@localhost ~]$ du -sh /usr/sbin
54M /usr/sbin

The -s option displays a summary of the entire base directory, rather than
each separate subdirectory. The -h option shows human-readable sizes.

Managing Software
Most of what appears in this book works the same (or at least similarly) on
different distributions. Software management is different, because three
different sets of tools are available to enable you to add and remove

t.me/DevTwitter
 software. Which set of tools you use depends on the distribution on which
you are working:
yum and rpm—These tools enable you to manage software on Red Hat
Enterprise Linux, CentOS, Fedora, and other Red Hat–based
distributions.
apt-get and dpkg—These tools enable you to manage software on
Debian, Ubuntu, Mint, and other Debian-based distributions.
zypper and rpm—These tools enable you to manage software on SUSE
and SUSE-based distributions.

The rpm and dpkg commands perform very similar tasks. Historically, they
were designed to install software packages that had been downloaded to the
local system. This function is now normally handled by the yum, apt-get,
and zypper commands, which are used to both download the package and
install it. These commands download the package from a server called a
repository.
The advantage of the yum, apt-get and zypper commands over the rpm and
dpkg commands is that package dependencies are automatically taken care
of. So, if a package needs three other packages to work successfully, the
yum, apt-get, and zypper commands would also download and install these
packages.
You can also use all of these commands to remove software packages.
Again, the yum, apt-get, and zypper commands have an advantage over the
rpm and dpkg commands because they check dependency issues before
removing the package. So, if you try to remove a package that is required
by another package, an error message appears.
So why would you ever use the rpm or dpkg commands? The yum, apt-get,
and zypper commands are really front-end programs that eventually run rpm
and dpkg commands. The rpm and dpkg commands have some more
powerful options that can’t be accessed by the yum, apt-get, or zypper
commands, particularly options regarding querying information regarding
packages. This is much more critical for system administrators than it is for
developers, so you will likely run the yum, apt-get, and zypper commands
much more often than rpm or dpkg.
Developers often install new software packages to enhance what features
(or programming languages) they can use on the system. Keep in mind that

t.me/DevTwitter
 installing and removing software requires root privileges.

Listing and Finding Software

Sometimes one of the challenges to installing software is trying to find the
correct name for the software package. On Red Hat–based systems, you can
execute the yum search command to query the repository for packages that
match a word or pattern:
[root@localhost ~]# yum search editor | head
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: centos.mia.host-engine.com
* epel: linux.mirrors.es.net
* extras: mirrors.sonic.net
* updates: mirror.steadfast.net
======================== N/S matched: editor
=========================
ckeditor.noarch : WYSIWYG text editor to be used inside web pages
ckeditor-samples.noarch : Sample files for ckeditor
dconf-editor.x86_64 : Configuration editor for dconf

The yum search command can produce a lot of output, so consider using the
grep command to perform a secondary filter:
[root@localhost ~]# yum search editor | grep GUI
nedit.x86_64 : A GUI text editor for systems with X
root-guibuilder.x86_64 : GUI editor library for ROOT
torrent-file-editor.x86_64 : Qt based GUI tool designed to create
and edit
To search for a package on a Debian-based system, use the apt-get
search term command (replace term with your search term). To
install a package on a SUSE-based system, use the zypper search -t
term command.

To list currently installed packages on Red Hat–based systems, use the yum
list installed command:
[root@localhost ~]# yum list installed | tail
yelp-libs.x86_64 1:3.14.2-
1.el7 @base
yelp-xsl.noarch 3.14.0-
1.el7 @base
yum.noarch 3.4.3-
132.el7.centos.0.1 @base
yum-langpacks.noarch 0.4.2-
4.el7 @base
yum-metadata-parser.x86_64 1.1.4-

t.me/DevTwitter
 10.el7 @anaconda
yum-plugin-fastestmirror.noarch 1.1.31-
34.el7 @base
yum-utils.noarch 1.1.31-
34.el7 @base
zenity.x86_64 3.8.0-
5.el7 @base
zip.x86_64 3.0-
10.el7 @anaconda
zlib.x86_64 1.2.7-
15.el7 @base

The yum list installed command also produces a lot of output. Consider
piping the output to the more or grep command. Note that the first column
of the output of this command displays the package name, the second
column displays the version of the package, and the third column displays
the repository name where the package was installed from.
To list all installed packages on a Debian-based system, use the dpkg -l
command. To list all installed packages on a Red Hat–based system, use the
rpm -qa command.

Installing Software
On Red Hat–based systems, install a package using the yum install
command as shown in Listing 6.2.

Listing 6.2 The yum install command

Click here to view code image

[student@localhost Desktop]$ su -
Password:
[root@localhost ~]# yum install kernel-doc
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: centos.mia.host-engine.com
* epel: linux.mirrors.es.net
* extras: mirrors.sonic.net
* updates: mirror.steadfast.net
Resolving Dependencies
--> Running transaction check
---> Package kernel-doc.noarch 0:3.10.0-327.28.2.el7 will be
installed
--> Finished Dependency Resolution

Dependencies Resolved

t.me/DevTwitter
 ===================================================================
=======
Package Arch Version Repository
Size
===================================================================
=======
Installing:
kernel-doc noarch 3.10.0-
327.28.2.el7 updates 13 M
Transaction Summary
===================================================================
=======
Install 1 Package
Total download size: 13 M
Installed size: 48 M
Is this ok [y/d/N]: y
Downloading packages:
kernel-doc-3.10.0-327.28.2.el7.noarch.rpm | 13
MB 00:03
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : kernel-doc-3.10.0-
327.28.2.el7.noarch 1/1
Verifying : kernel-doc-3.10.0-
327.28.2.el7.noarch 1/1
Installed:
kernel-doc.noarch 0:3.10.0-327.28.2.el7
Complete!

To install a package on a Debian-based system, use the apt-get install

command. To install a package on a SUSE-based system, use the zypper
install command.

t.me/DevTwitter
 Removing Packages
Although developers often want to install packages on their
own systems, wanting to remove or perform more advanced
package manipulation commands is not as common. The rpm
and dpkg commands were mentioned in the event you want to
learn more about package management, but this is normally
something that interests system administrators more.
In the event you do want to remove a software package, switch
to the root account and run the proper command for your
distribution:6
yum remove package_name
apt-get remove package_name or apt-get purge
package_name
zypper remove package_name

User Accounts
Typically, maintaining user accounts is the responsibility of the system
administrator. However, this can also be an important task for a software
developer because you might want to be able to test your software using
different user accounts. For example, you might want to have different
accounts to test access for unprivileged users to a database.
This section focuses on the basics of creating, modifying, and deleting user
accounts, and also explores the topic of group accounts.

Adding User Accounts

To add a user account, you need root privileges. GUI-based tools are
available that you can use to create user accounts. However, they differ
between distributions. The command-line tools are easy enough and you
can use them to quickly create user accounts.
To create a user account, execute the useradd command as shown in the
following:
[root@localhost ~]# useradd julia
[root@localhost ~]# tail -1 /etc/passwd
julia:x:1001:1001::/home/julia:/bin/bash

t.me/DevTwitter
 [root@localhost ~]# ls /home
julia student

Notice the new entry in the /etc/password file, one of the files that contains
user account information. To see details about the format of this file,
execute the man 5 passwd command.
The new user was provided with a home directory automatically
(/home/julia). This does not happen on all distributions; on some distros
you must specify the name of the home directory with the -d option and tell
the useradd command to create this home directory with the -m option:
[root@localhost ~]# useradd -d /home/julia -m julia

Typically the default settings for the user account are fine for accounts that
you are creating for testing purposes. A few settings that you might want to
modify include the following:
-s Specify the login shell. For example: -s /bin/tcsh
-g Specify the primary group for the account. For example: -g sudo
-G Specify the primary group(s) for the account. For example: -G
sudo,payroll

After creating the user account, assign the new account with a password by
executing the passwd command as shown in the following:
[root@localhost ~]# passwd julia
Changing password for user julia.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

t.me/DevTwitter
 Bad Passwords
You might get a warning message like the following when
assigning a password to a user account:
BAD PASSWORD: The password fails the dictionary check -
it is based on a dictionary word
If the system you are working on has access to the Internet, you
should heed this warning and use a more complex password.
However, if this is an internal-only system, making a more
complex password can be more trouble than it is worth.
Ask yourself, “Will I ever connect this system to the Internet?”
If the answer is “yes,” make a more complex password.

Modifying User Accounts

To change a user account, use the usermod command. The usermod
command accepts the same options as the useradd command. So, to change
the login shell for a user, use the -s option as shown in the following:
[root@localhost ~]# grep julia /etc/passwd
julia:x:1001:1001::/home/julia:/bin/bash
[root@localhost ~]# usermod -s /bin/tcsh julia
[root@localhost ~]# grep julia /etc/passwd
julia:x:1001:1001::/home/julia:/bin/tcsh

If you look at the last field of data in the “julia” line of the /etc/passwd file,
you can see that the login shell has changed from /bin/bash to /bin/tcsh.

Deleting User Accounts

To delete a user account, use the userdel command. If you want to delete
both the account and the user’s home directory, use the -r option. Not using
the -r option removes the account from the /etc/passwd file (and other files
that contain user account information), but does not remove the user’s home
directory and its contents.
[root@localhost ~]# userdel -r julia

Understanding Groups

t.me/DevTwitter
 Chapter 4, “Essential Commands” mentioned group accounts during the
discussion on permissions. To understand how important group membership
is, consider the output of the following commands:
[root@localhost ~]# id sarah
uid=1002(sarah) gid=1002(sarah) groups=1002(sarah)
[root@localhost ~]# ls -l /tmp/sample.txt
-rw-r-----. 1 root wheel 158 Aug 16 21:11 /tmp/sample.txt

Based on the output of the previous id command, you can see that the user
sarah is a member of one group (the group named sarah). If you look at the
output of the previous ls -l command, you can see that the
/tmp/sample.txt file is owned by the user root and the wheel group. So, in
this situation, the permissions for the user sarah are ---, the “others” section
of permissions.
What if the root user wanted the user sarah to be able to view this file? By
adding the user sarah to the wheel group, she would have the permissions
r--, allowing her to view the contents of the file:
[root@localhost ~]# usermod -aG wheel sarah
[root@localhost ~]# id sarah
uid=1002(sarah) gid=1002(sarah) groups=1002(sarah),10(wheel)
[root@localhost ~]# ls -l /tmp/sample.txt
-rw-r-----. 1 root wheel 158 Aug 16 21:11 /tmp/sample.txt

Managing Groups
To create a new group, use the groupadd command:
[root@localhost ~]# groupadd staff

To add a user to a group, use the -G option to the usermod command. Very
important: Make sure you use the -a option with the -G option. Using -G
alone removes the user from all of their secondary groups. See the
following for the wrong way to do this:
[root@localhost ~]# id sarah
uid=1002(sarah) gid=1002(sarah) groups=1002(sarah),10(wheel)
[root@localhost ~]# usermod -G staff sarah
[root@localhost ~]# id sarah
uid=1002(sarah) gid=1002(sarah) groups=1002(sarah),1003(staff)

Notice the output of the previous id commands. You can see that the
usermod command removed the user sarah from the wheel group. The

t.me/DevTwitter
 following demonstrates the right way to add a user to a group:
[root@localhost ~]# id sarah
uid=1002(sarah) gid=1002(sarah) groups=1002(sarah),10(wheel)
[root@localhost ~]# usermod -a -G staff sarah
[root@localhost ~]# id sarah
uid=1002(sarah) gid=1002(sarah)
groups=1002(sarah),10(wheel),1003(staff)

To remove a group, you might want to first use the find command to search
the filesystem for all files owned by that group:
[root@localhost ~]# find / -group staff -ls 2> /dev/null
27304379 4 -rw-r----- 1 root staff 158 Aug 16 21:11 /tmp/sample.txt

This is an important step because you should change the group ownership
of these files to another group before removing the group. After you change
the group ownership, you can use the groupdel command to delete the
group:
[root@localhost ~]# chgrp wheel /tmp/sample.txt
[root@localhost ~]# ls -l /tmp/sample.txt
-rw-r-----. 1 root wheel 158 Aug 16 21:11 /tmp/sample.txt
[root@localhost ~]# groupdel staff

Linux Humor
If you are like me (or 99% of folks who work in Linux), you
will eventually end up typing the command sl instead of the ls
command. Why not make the result a bit more interesting than
bash: sl: command not found...?

First, install the package named sl (choose the right command

for your distribution):
yum install sl
apt-get install sl
zypper install sl
Now, type sl and press the Enter key!

Summary
A system administrator performs many additional tasks that were not
covered in this chapter. However, the chapter did cover the administrative
tasks that you, as a software developer, might routinely perform. You

t.me/DevTwitter
 should now know to switch to the root account to perform system
administration tasks. You also learned how to display disk usage, add and
remove software, and manage group and user accounts.

1 The - option is the same as the -l or -login options.

2 Note that this is a very simple example and perfectly fine for a
standalone system. However, for a system in which security is a concern,
you should learn more about the sudo command or have a system
administrator set up this feature.
3 A group in Linux is a collection of user accounts. Managing groups is
covered in detail later in this chapter.
4 The difference between a partition and a volume is not critical for
developers to understand. If you choose to become a system
administrator, the difference becomes very important because they are
managed differently. Because this book is for developers, I have chosen
not to describe these differences. Consider them both to be a container
where files and directories can be stored.
5 Use the -h option to show the output in human-readable sizes rather than
one kilobyte block sizes.
6 The purge argument to apt-get removes the package entirely. The remove
argument removes everything except the configuration files (left behind
in case you reinstall the software at a later date).

t.me/DevTwitter
 III. Linux Programming Languages
One of the more important benefits to Linux is the vast amount of software
that is freely available for the operating system. This poses both advantages
and disadvantages. For example, having dozens of text editors to choose
from is an advantage because that means you aren’t “locked into” using an
editor that you really don’t like. However, it can also be a disadvantage
because exploring the different editors to find one that suits you best might
take some time.
The same can be said for programming languages. Many languages are
available, and finding the one that best fits your programming style often
takes a lot of trial and error. The focus of this part of the book is to provide
a review of some of the more popular languages that are available for
Linux. Chapter 7 provides general information about these languages and
Chapters 8 through 11 provide additional details about select languages.

t.me/DevTwitter
 7. Overview of Linux Programming Languages
Most Linux programming languages1 can be placed into two general
categories: scripting languages (sometimes called interpreted languages)
and compiled languages (sometimes called structured languages). There
isn’t a strict definition that separates these categories, but the following
provides the essential differences:
Complied languages cannot be executed directly from source code. The
source code must be converted into compiled code first.
Scripts traditionally are not compiled.
Scripting languages are typically easier to learn.
Scripts typically take less coding to perform a task.

As an example of how these categories are not strictly defined, consider

this: Perl is a popular scripting language that is executed directly from
source code, but before executing it is compiled into memory and the
compiled form of the code is executed.

Scripting Languages
Many scripting languages are available for Linux, making it difficult to
create a complete list in this book. However, you should find that the
languages described in this section are the most popular and widely used on
Linux distributions.

BASH Shell Scripting

In Chapters 2–6 you learned the basics of working in Linux and the BASH
shell.2 You can also use those commands you learned in shell scripting
programs. For example, suppose you routinely execute the following
commands:
cd /home
ls -l /home > /root/homedirs
du -s /home/* >> /root/homedirs
date >> /root/homedirs

t.me/DevTwitter
 Instead of executing each of these commands manually, day after day, you
can place all the commands into a file, make the file executable, and then
run the file as a program:
[root@fedora ~]$ more /root/checkhome.sh
#!/bin/bash

cd /home
ls -l /home > /root/homedirs
du -s /home/* >> /root/homedirs
date >> /root/homedirs
[root@fedora ~]$ chmod a+x /root/checkhome.sh
[root@fedora ~]$ /root/checkhome.sh

Because you can use Linux commands natively in BASH shell scripts, this
scripting language can be extremely powerful. Another advantage of using
this language is that you can be confident that just about every Linux (and
UNIX) distribution will have the BASH shell, making it easy to port a
script from one system to another.
In addition to being able to use Linux commands in BASH shell scripts,
you should be aware that this language has other programming features,
such as:
Variables
Loop controls (if, while, and so on)
Exit status values
The ability to source code from other files

With all of its advantages, BASH shell scripting has some disadvantages,
including:
It lacks some advanced programming features, such as object-oriented
programming.
It is often much slower than executing other languages as each
command is normally executed as a separate process.3

Even with these disadvantages, BASH shell scripting is very popular in

Linux. In fact, a search for BASH scripts (files that end in .sh) on a typical
Linux distribution normally yields hundreds of results:
[root@fedora ~]$ find / −name "*.sh" | wc −l
578

t.me/DevTwitter
 Because of this, Chapter 8, “BASH Shell Scripting,” is devoted to
additional details regarding BASH shell scripting.

Perl Scripting
In the mid-1980s, a developer named Larry Wall began work on a new
scripting language that would eventually be named Perl. At the time he was
working on UNIX-based systems, which had tools such as the C
programming language, the Bourne Shell scripting language (precursor to
BASH), and sed and awk (more about these tools later). However, none of
these tools worked as he wanted them to, so he created his own language.
Of course, Larry didn’t want to lose the features that he did like about these
tools, so he combined the features that he liked into his new language. This
resulted in a language that looks a bit like C, a bit like shell scripting, and a
bit like a hodgepodge collection of UNIX utilities.

Which Scripting Language Is the Best?

I believe trying to list the pros and cons of each scripting
language is a mistake. To begin with, this is often a matter of
opinion.4 For example, Perl is a very flexible language whereas
Python (see the next section) is more structured. If I want to
write a quick script and I am not worried about maintaining the
code long term, then flexibility might be a pro and structure a
con. However, if I was working with multiple developers on a
larger product, then structure might be a pro and flexibility a
con.
Rather than trying to compare and contrast the pros and cons of
each scripting language, I try to focus on what developers
typically like about each language and what each language is
typically used for. I would rather you decide what aspect of a
language is a feature versus a liability.

Several aspects of Perl that developers like include the following:

You can very quickly write Perl code because much of what you need
for basic scripting is already built into the core language.

t.me/DevTwitter
 Perl code is very flexible; you are not limited by the structure as much
as some other languages.
Perl’s syntax is fairly simple, derived primarily from the C language.
It normally doesn’t take very long to learn Perl.
Perl has very powerful features, such as robust regular expressions.

Although Perl can be used for many different applications, it is often used
for the following:
Data parsing—Perl has powerful regular expression features that
make it ideal for data munging (pulling chunks from data and
generating reports).
Web development—Perl is often a component of LAMP-based5
technology because of its web development features, including
Common Gateway Interface (CGI).
Code testing—Because Perl is easy and quick to code, developers
often use it to create tools to test their applications.
GUI programs—Additional Perl modules (libraries), such as WxPerl
and Tk, provide Perl programmers with the option of easily creating a
GUI-interface for users to interact with the Perl code.
Administrative tools—System administrators create Perl scripts to
help them automate administrative tasks.

Chapter 9, “Perl Scripting,” provides additional details regarding the

creation of Perl programs.

Python Scripting
The beginnings of Python are best described by Guido van Rossum, its
creator, who wrote the following as a forward for a book on Python
published in 1996:
Over six years ago, in December 1989, I was looking for a “hobby”
programming project that would keep me occupied during the week
around Christmas. My office … would be closed, but I had a home
computer, and not much else on my hands. I decided to write an
interpreter for the new scripting language I had been thinking about
lately: a descendant of ABC that would appeal to Unix/C hackers. I

t.me/DevTwitter
 chose Python as a working title for the project, being in a slightly
irreverent mood (and a big fan of Monty Python’s Flying Circus).
Little did he know that Python would one day become one of the world’s
most popular scripting languages. Since that fateful Christmas break in the
late 1980s, Python has developed over time into a robust programming
language that is the core of many Linux tools and open source projects.
One of the driving philosophies of Python is well-structured code. Python
enforces this with rules such as a very rigid indentation scheme. You can
see how seriously Python developers take the concept of well-structured
code by reading some of the rules defined by the document “Zen of
Python”:
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.

In addition to being a well-structured language, the following components

make Python a popular language:
It has object-oriented features.
It has a large standard library.
It is extendable or embedded.
The data structures provided by Python are more diverse than those of
many languages.

Although Python can be used for many different applications, it is often

used for the following:
Network-based applications—By using Twisted, a Python-based
network framework, network-based applications can be developed.
Web development— The Apache Web Server provides the option of
using Python scripts for dynamic websites.

t.me/DevTwitter
 Scientific applications— Several libraries are available for Python that
make it a good choice to create scientific applications.
System tools— Linux developers often use Python to create system
tools for the operating system.

Additional details regarding the creation of Python programs are provided

in Chapter 10, “Python Scripting.”

Additional Scripting Languages

Although this book focuses on BASH shell, Perl, and Python scripting,
other scripting languages exist that you might want to explore when
considering a Linux scripting language.

Ruby
Developed in the mid-1990s by Yukihiro Matsumoto, Ruby’s origin is best
described by its creator:
I was talking with my colleague about the possibility of an object-
oriented scripting language. I knew Perl (Perl4, not Perl5), but I
didn’t like it really, because it had the smell of a toy language (it still
has). The object-oriented language seemed very promising. I knew
Python then. But I didn’t like it, because I didn’t think it was a true
object-oriented language—OO features appeared to be add-on to the
language. As a language maniac and OO fan for 15 years, I really
wanted a genuine object-oriented, easy-to-use scripting language. I
looked for but couldn’t find one. So I decided to make it.6
Initially Ruby wasn’t as popular as Perl and Python, although it was praised
for being a good object-oriented scripting language. Ruby became more
popular when Ruby on Rails, a web application framework, was introduced.

t.me/DevTwitter
 Ruby Essentials
Is it installed?
root@centos:~# which ruby
/usr/bin/ruby
Install (if needed):
Red Hat/Fedora/CentOS: yum install ruby
Debian/Mint/Ubuntu: apt–get install ruby
Sample script:
root@centos:~# more hello.rb
#!/usr/bin/ruby
puts "Hello World"
root@centos:~# chmod a+x hello.rb
root@centos:~# ./hello.rb
Hello World
To learn more about Ruby, visit http://www.ruby-lang.org.

PHP
PHP is a recursive acronym that stands for PHP: Hypertext Preprocessor,
although when it was originally released PHP stood for Personal Home
Page. Like Ruby, PHP was developed in the mid-1990s. Created by Rasmus
Lerdorf to dynamically create web pages, it has become a popular scripting
language for website designers.
PHP is an important part of the LAMP architecture. Seeing embedded PHP
code within an HTML document is common. PHP can also be used as a
standalone scripting language.

t.me/DevTwitter
 PHP Essentials
Is it installed?
root@centos:~# which php
/usr/bin/php
Install (if needed):
Red Hat/Fedora/CentOS: yum install php5
Debian/Mint/Ubuntu: apt–get install php5
Sample script— standalone example:
root@kali:~# more hello.php
#!/usr/bin/php
<?php
echo "Hello, world\n";
?>
root@kali:~# chmod a+x hello.php
root@kali:~# ./hello.php
Hello World
Sample script— embedded HTML example:
<html>
<head>
<title>PHP Test</title>
</head>
<body>
<?php echo '<p>Hello World</p>'; ?>
</body>
</html>
To learn more about PHP, visit http://www.php.net.

JavaScript
In the mid-1990s the World Wide Web was just starting to become popular.
An organization named Mosaic Communications (later becoming Netscape
Communications) had just developed a graphical web browser and was
exploring the possibility of using a programming language to enhance
HTML. Initially they chose to incorporate Java. However, they soon
decided that a scripting language would be a better solution.
JavaScript was designed to be similar to Java in terms of syntax. However,
it is not a “spin-off” or extension of Java, but rather a scripting language

t.me/DevTwitter
 that has similar syntax.
In addition to being heavily used, JavaScript is also used as an embedded
scripting language in many products including Adobe Acrobat, MongoDB,
and extensions for the Chrome and Opera web browsers.

JavaScript Essentials
Is it installed?
JavaScript is executed as part of the web browser, not typically
installed. However, you can download and install an executable
version from http://www. http://javascript-exe.com.
Sample script—embedded HTML example:
<html>
<head>
<title></title>
</head>
<body>

<script>
console.log("hello world");
</script>
</body>
</html>

To learn more about JavaScript, visit https://developer.mozilla.org.

Tcl
Tcl7 was created in the late 1980s by John Ousterhout. The tool was
originally created for his students to use while he was a professor at the
University of California, Berkeley. As additional features were added to the
language and these students graduated, the popularity of the language grew.
In the early 1990s, the scripting language market was growing crowded and
it was hard for a newer language to build traction. One of the main reasons
Tcl became popular was the introduction of Tk, the Tool Kit extension to
the Tcl language. Essentially a separate language itself, Tk made use of Tcl
as a base language and added features that enabled developers to quickly
and easily create graphical-based, platform-independent programs. The
combination of Tcl and Tk is referred to as Tcl/Tk.

t.me/DevTwitter
 Tcl/Tk Essentials
Is it installed (Tcl=tclsh, Tk=wish)?
root@centos:~# which tclsh
/usr/bin/tclsh
root@centos:~# which wish
/usr/bin/wish
Install (if needed):
Red Hat/Fedora/CentOS: yum install tcl
Red Hat/Fedora/CentOS: yum install tk
Debian/Mint/Ubuntu: apt-get install tcl
Debian/Mint/Ubuntu: apt-get install tk
Sample Tcl script:
root@centos:~# more hello.tcl
#!/usr/bin/tclsh
puts "Hello World"
root@centos:~# chmod a+x hello.tcl
root@centos:~# ./hello.tcl
Hello World
Sample Tk script:
root@centos:~# more hello.tk
#!/usr/bin/wish
button .hello -text "Hello World" -command { exit }
pack .hello
root@kali:~# chmod a+x hello.tk
root@kali:~# ./hello.tk
#Note: output is a graphical program
To learn more about Ruby, visit http://www.tcl.tk.

sed and awk

The sed and awk executables are command-line utilities that also have
programming features. Typically, you use the sed command to parse a data
stream (like a file) one line at a time and preform some sort of change to a
document. For example, the following sed command replaces all numbers
with an X character:8

t.me/DevTwitter
 root@centos:~# more /etc/hosts
127.0.0.1 localhost
127.0.1.1 centos

# The following lines are desirable for IPv6 capable hosts

::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

root@centos:~# sed 's/[0-9]/X/g' /etc/hosts

XXX.X.X.X localhost
XXX.X.X.X centos

# The following lines are desirable for IPvX capable hosts

::X localhost ipX-localhost ipX-loopback
ffXX::X ipX-allnodes
ffXX::X ipX-allrouters

The awk utility is designed to work on database-based information, such as a

system file. For example, consider the following data from the /etc/passwd
file:
root@centos:~# head /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin

With the awk utility, we either display or modify fields of data. For example,
to print just the user name (first field) and login shell (seventh field), use the
following awk command:
root@centos:~# head /etc/passwd | awk -F : '{print $1, $7}'
root /bin/bash
daemon /usr/sbin/nologin
bin /usr/sbin/nologin
sys /usr/sbin/nologin
sync /bin/sync
games /usr/sbin/nologin
man /usr/sbin/nologin
lp /usr/sbin/nologin
mail /usr/sbin/nologin
news /usr/sbin/nologin

t.me/DevTwitter
 Both sed and awk are not only powerful command-line utilities, but they
also have simple programming features, such as variable usage and flow
control. For more details on creating sed scripts, view the man page for sed
or see https://www.gnu.org/software/sed/manual/sed.html. The awk utility
also has a good man page in addition to the following website:
https://www.gnu.org/software/gawk/manual/gawk.html.

Compiled Languages
The primary focus of this book is to introduce developers to Linux and
languages that are specifically popular in Linux. Complied languages, such
as C, C++, and Java9 are not covered in great detail in this book for several
reasons:
These languages are huge topics and more difficult to learn than
scripting languages.
Although these languages certainly exist and are popular in Linux, they
are even more widely used on non-Linux platforms, such as Microsoft
Windows (particularly C++ and Microsoft's C#). In other words, they
aren't languages that are primarily popular in Linux but standard
languages commonly found on many platforms.
The assumption is that you are already a developer and likely already
know one or more of these complied languages.

Although this book won't include specifics on how to create C, C++, or

Java programs, Chapter 11, “C, C++, and Java,” does cover topics
specifically related to writing code in these languages on Linux platforms.
This includes the following:
Handling system libraries
Building packages
Java installation

C Programming Basics
In the event that you are not familiar with the C language, here are some
basics that you should be aware of:
It’s an older and well-established language.
It lacks object-oriented features.

t.me/DevTwitter
 It is often used for low-level tasks, such as the Linux kernel.
It typically requires much more coding because simple tasks require
loading libraries.
Code must be compiled for a specific operating system: Write Once,
Compile Anywhere (WOCA).

C++ Programming Basics

In the event that you are not familiar with the C++ language, here are some
basics that you should be aware of:
It adds features to C.
Its additional features include object-oriented programming.
It’s typically used for more complicated, high-level programming tasks.
Code must be compiled for a specific operating system: Write Once,
Compile Anywhere (WOCA).

Java Programming Basics

In the event that you are not familiar with the Java language, here are some
basics that you should be aware of; it is
An object-oriented language
Designed to be more flexible than C++
Runs via a “virtual machine,” with the result being more portable code:
Write Once, Run Anywhere/Everywhere (WORA/WORE)

IDEs
As a developer you might already be familiar with Integrated Development
Environments (IDEs). For example, if you have developed C or C++ code
on Microsoft Windows platforms, you are likely familiar with Microsoft
Visual Studio.
An IDE provides you with tools to make the process of developing code
easier. These tools could include a debugger program, a special editor that
provides syntax highlighting and other features or features that enable you
to insert new code quickly (and without error).
Many IDEs are available for Linux. Some are very specific to a language,
whereas others are a bit more generic. It is also important to note that some

t.me/DevTwitter
 of these IDEs are free, but others might require a fee of some sort to use
them.
The goal of mentioning IDEs is to encourage you to explore what is
available. Suppose, for example, you explore the different Linux
programming languages and decide that Python is the best language for
you. Before diving into the language further, explore the available IDEs
(more than a dozen exist just for Python) and make a point to learn how to
use the one that best meets your needs.10

Programming Humor
Algorithm (noun): Word used by software developers when
they do not want to explain what their code does.

Summary
This chapter provided you with a foundation in programming languages
that are commonly used on Linux distributions. You learned the difference
between a scripting language and a structured language. You were also
introduced to several different scripting languages that are popular on
Linux. The topics described in this chapter lay the groundwork for the next
four chapters in which you will learn more about BASH, Perl, and Python
scripting as well as essentials regarding writing C, C++, and Java code on
Linux distributions.

1 Although I refer to these languages as “Linux programming languages,”

be aware that most of these languages could also be available on other
platforms, including Microsoft Windows.
2 Given that BASH stands for Bourne-Again SHell, it is actually
redundant to refer to the scripting component of BASH as “BASH shell
scripting.” However, just as the term ATM machine (ATM=Automated
Teller Machine) has become standard, the term BASH shell scripting has
become standard.
3 This isn’t always the case as some commands are built-in shell
commands that don’t require a separate process.
4 Warning: Discussions on this matter often escalate to the level of a
serious “religious war.”

t.me/DevTwitter
 5 LAMP = Linux, Apache HTTP Server, MySQL, and Perl (or PHP).
LAMP is a collection of technologies that make up a solution stack to
provide web services.
6 Posted on the ruby-talk mailing list in 1999.
7 Originally Tool Command Language, Tcl is commonly referred to as
tickle.
8 Does the syntax 's/[0-9]/X/g' look familiar? It should remind you of
the search and replace feature of the vi editor (see Chapter 5, “Text
Editors”). This is because, like vi, sed was derived from older UNIX
editors.
9 Technically Java isn’t a complied language, but it belongs more in this
category than the scripting language category.
10 If you follow my advice on IDEs, you will not be sorry. I can’t tell you
how many developers I have spoken to who are using simple text editors
to create their code and not taking advantage of great debugging tools
that are available. They waste hours of time by using inadequate tools.
Save yourself future headaches and find a good IDE now, not sometime
in the future!

t.me/DevTwitter
 8. BASH Shell Scripting
The primary benefit of BASH scripting is that you can use everything that
is available in the BASH shell within a BASH script. Because your Linux
distribution has hundreds (maybe even thousands) of commands, each of
which can be used in a BASH script, BASH shell scripting can be a very
powerful tool.
This chapter focuses on providing you a firm understanding of how to write
basic BASH scripts as well as introducing you to some of the more
advanced features.

Basics of BASH Scripting

To an extent, you already know many of the basics of BASH scripting
because you have already learned many features of the BASH shell in this
book. For example, you learned about shell variables in Chapter 4,
“Essential Commands.” Shell variables are used in BASH scripting to store
values.
To start a BASH script, enter the following as the first line of the script file:
#!/bin/bash

This special sequence is called the sha-bang, and it tells the system to
execute this code as a BASH script.

The Story Behind sha-bang

To understand the origin of the term sha-bang, you first need to
know two other slang terms. In Linux, a # character is called a
hash and a ! character is called a bang. Put them together and
you get hash-bang or sha-bang for short.
Some documentation referred to this as a she-bang, but
because of the origin of the term, I think sha-bang makes more
sense. In any case, you now know why #! is called sha-bang!

Comments in a BASH script start with a # character and extend to the end
of the line. For example:

t.me/DevTwitter
 echo "hello world" #prints "hello" to the screen

As shown in the preceding example, you can use the echo command to
display information to the user who is running a program. The arguments to
the echo command can contain any text data and can also include the value
of a variable:
echo "The answer is $result"

After creating your BASH script and saving it, you then make it executable:
[student@OCS ~]$ more hello.sh
#!/bin/bash
#hello.sh

echo "hello world!"

[student@OCS ~]$ chmod a+x hello.sh

Now it can be run as a program by using the following syntax:

[student@OCS ~]$ ./hello.sh
hello world!

Note the need to place ./ before the name of the command. This is because
the command might not be in one of the directories specified by the $PATH
variable:
[student@OCS ~]$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/g
ames:/usr/local/games

To avoid the need to include ./ whenever you want to run your script, you
can modify the $PATH variable to include the directory in which your script
is stored. Typically, regular users create a “bin” directory in their home
directory and place scripts in this location:
[student@OCS ~]$ mkdir bin
[student@OCS ~]$ cp hello.sh bin
[student@OCS ~]$ PATH="$PATH:/home/student/bin"
[student@OCS ~]$ hello.sh
hello world!

In addition to the built-in variables that were discussed in Chapter 4,

variables are available in BASH scripts that represent the arguments being
passed into the script. For example, consider the following execution of a
script called test.sh:

t.me/DevTwitter
 [student@OCS ~]$ test.sh Bob Sue Ted

The values “Bob,” “Sue,” and “Ted” are assigned to variables within the
script. The first argument (“Bob”) is assigned to the $1 variable, the second
argument is assigned to the $2 variable, and so on. Additionally, all
arguments collectively are assigned to the $@ variable.
For additional details regarding these positional parameters variables, or
anything related the BASH scripting, consult the man page for bash:
[student@OCS ~]$ man bash

Conditional Expressions
Several conditional statements are available for the BASH shell, including
the if statement:
!!
if [ cond ]
then
statements
elif [ cond ]
then
statement
else
statements
fi

Note the following:

An else if is spelled elif and is optional.
After the if and elif, you need a then statement. However, after an
else, do not include a then statement.
End the if statement with the word if spelled backwards: fi

See Listing 8.1 for an example of an if statement.

Listing 8.1 Sample if statement

Click here to view code image
!!
#!/bin/bash
#if.sh

t.me/DevTwitter
 color=$1

if [ "$color" = "blue" ]
then
echo "it is blue"
elif [ "$color" = "red" ]
then
echo "it is red"
else
echo "no idea what this color is"
fi

Listing 8.1 used the following conditional statement:

!!
[ "$color" = "blue" ]

Quoting variables
Get in the habit of putting double quotes around your variables
in BASH scripts. This is important in the event the variable
hasn’t been assigned a value. For example, suppose the script
in Listing 8.1 was execute with no arguments. The result would
be that the color variable is unassigned and the resulting
conditional statement would be if [ "" = "blue" ]
The result would be false, but without the quotes around
$color, the result would be an error message and the script
would exit immediately. This is because the resulting
conditional statement would be missing one of its key
components after the value of $color has been returned: if [ =
"blue" ]

This syntax performs an implicit call of a BASH command named test that
you can use to perform several comparison tests. This can include integer
(numeric) comparisons, string comparisons, and file testing operations.1 For
example, use the following syntax to test whether the string value that is
stored in the $name1 variable does not equal the string stored in the $name2
variable:
!!
[ "$name1" != "$name2" ]

t.me/DevTwitter
 Important Note
The spacing around the square brackets is very important.
There should be a space before and after each square bracket.
Without these, an error message occurs.

In addition to determining whether two strings are equal or not equal to

each other, you might also find the -n option useful.2 This option
determines whether a string is not empty, which is useful when testing user
input. For example, the code in Listing 8.2 reads data from user input (the
keyboard), assigns the input to the $name variable, and tests to make sure the
user typed something for the name.

Listing 8.2 Testing user input

Click here to view code image
[student@OCS ~]$ more name.sh
#!/bin/bash
#name.sh

echo "Enter your name"

read name

if [ -n "$name" ]
then
echo "Thank you!"
else
echo "hey, you didn't give a name!"
fi
[student@OCS ~]$./name.sh
Enter your name
Bo
Thank you!
[student@OCS ~]$./name.sh
Enter your name
hey, you didn't give a name!

Integer Comparisons
If you want to perform integer (numeric) comparison operations, use the
following:
-eq True if values are equal to each other.

t.me/DevTwitter
 -ne True if values are not equal to each other.
-gt True if first value is greater than second value.
-lt True if first value is less than second value.
-ge True if first value is greater than or equal to second value.
-le True if first value is less than or equal to second value.

File Test Comparisons

You can also perform test operations on files and directories to determine
information about their status. These operations include:
-d True if “file” is a directory.
-f True if “file” is a regular file.
-r True if “file” exists and is readable by the user running the script.
-w True if “file” exists and is writable by the user running the script.
-x True if “file” exists and is executable by the user running the script.
-L True if “first” value is less than or equal to second value.

Flow Control Statements

In addition to if statements, the BASH scripting language has several other
flow control statements:
The while loop—Executes a block of code repeatedly as long as the
conditional statement is true.
The until loop—Executes a block of code repeatedly as long as the
conditional statement is false. Essentially the opposite of a while loop.
The case statement—Similar to an if statement but provides an easier
branching method for multiple situations.
The for loop—Executes a block of code for each item of a list of
values.

The while loop

The following code segment prompts the user for a five-digit number. If the
user complies, the program will continue because the condition of the while
loop will be false. However, if the user provides incorrect data, the

t.me/DevTwitter
 condition of the while loop will be true and the user will be prompted for
the correct data again:
!!
echo "Enter a five-digit ZIP code: "
read ZIP

while echo $ZIP | egrep -v "^[0-9]{5}$" > /dev/null 2>&1

do
echo "You must enter a valid ZIP code – five digits only!"
echo "Enter a five-digit ZIP code: "
read ZIP
done

echo "Thank you"

The egrep command from the previous example may be a bit tricky to
understand. To begin with, the regular expression pattern is matching a
value that is exactly five digits. The –v option is used to return a value if the
pattern is not found. So, if $ZIP contains a valid five-digit number, then
egrep returns a false result because it is trying to find lines that don’t
contain a five-digit number. The egrep command returns a true result if the
$ZIP contains something besides a five-digit number.
Why the > /dev/null 2>&1? Because we don’t want to display anything
from the egrep command, just make use of its true/false return value. All
OS commands return true or false3 when executed, and that is what is
needed here. Any STDOUT or STDERR from the command is unnecessary and
only serves to confuse matters if it is displayed to the user.

The for Loop

A for loop enables you to perform an operation on a set of items. For
example, the following command, when run as the root user, creates five
user accounts:
!!
for person in bob ted sue nick fred
do
useradd $person
done

t.me/DevTwitter
 Loop Control
Like most languages, BASH scripting provides a way to
prematurely exit a loop or to stop the current iteration of a loop
and start a new iteration of a loop. Use the break command to
immediately exit a while, until, or for loop. Use the continue
command to stop the current iteration of a while, until, or for
loop and start the next iteration.

The case Statement

A case statement is designed for when you want to perform multiple
conditional checks. Although you could use an if statement with multiple
elif statements, the syntax of if/elif/else is often more cumbersome than
a case statement.
The syntax for a case statement is:
!!
case var in
cond1) cmd
cmd;;
cond2) cmd
cmd;;
esac

For the preceding syntax example, var represents a variable’s value that you
want to conditionally check. For example, consider the following code:
!!
name="bob"

case $name in
ted) echo "it is ted";;
bob) echo "it is bob";;
*) echo "I have no idea who you are"
esac

The “condition” is a pattern that uses the same matching rules as file
wildcards. An * matches zero or more of any character, a ? matches a single
character, and you can use square brackets to match a single character of a
specific range. You can also use a |character to represent “or.” For example,
consider Listing 8.3, which is used to check a user’s answer to a question:

t.me/DevTwitter
 Listing 8.3 Example of the case statement
Click here to view code image

!!
answer=yes

case $answer in
y|ye[sp]) echo "you said yes";;
n|no|nope) echo "you said no";;
*) echo "bad response";;
esac

User Interaction
The example in Listing 8.3 is a bit of a puzzle because it is intended to
check user input. However, the variable is hard coded. Using actual user
input, which the read statement can gather, would make more sense:
!!
read answer

The read statement prompts the user to provide information and reads that
data (technically from STDIN) into a variable that is specified as the
argument to the read statement. See Listing 8.4 for an example.

Listing 8.4 Example of the read statement

Click here to view code image
!!
read answer

case $answer in
y|ye[sp]) echo "you said yes";;
n|no|nope) echo "you said no";;
*) echo "bad response";;
esac

Additional Information
Do you want to learn more about creating BASH scripts? The following are
good resources for additional information:

t.me/DevTwitter
 man bash—The man page for the BASH shell has a great deal of
information about writing BASH scripts.
http://tldp.org—A website that is (sadly) mostly out of date.4
However, it has one gem of a document called the “Advanced Bash-
Scripting Guide.” Click on the Guides link under the Documents
section and scroll down until you see this guide. The author of this
guide normally updates it on a regular basis. Because the guides are
listed by publication date, this guide is almost always at the top of the
list.

BASH Scripting Humor

This isn’t technically a script humor item, but because you will
likely be editing files using the vim editor…
Open the vim editor by executing the command vim at the
command line. Then type :help 42.

Summary
This book does not cover some additional features of BASH scripting.
However, the goal of this chapter was to provide you with enough
information to determine whether BASH scripting is a good language for
you. If you liked the features and syntax of BASH scripting, consider
exploring the documentation provided to learn more about this flexible
language.

1 See the man page for test to learn more about its comparison operations:
man test
2 A similar option, -z, returns true if the string contains zero characters.
3 Technically they return 0 for “true” and a positive number for “false.”
4 It has a few up-to-date documents, but most are very much outdated.
Look at the publication date of the document and realize that anything
older than a couple of years is probably no longer accurate (but still
might provide you with some useful information).

t.me/DevTwitter
 9. Perl Scripting
Although Perl might have started as a simple scripting language, it has
grown into a robust language designed to tackle many different coding
situations. There is a great deal to learn about Perl scripting (as well as the
other programming languages discussed in this book), so don’t expect to
become an expert overnight.
This chapter’s focus is to give you a firm understanding of how to write
basic Perl scripts as well as provide an understanding of some of Perl’s more
advanced features.

Basics of Perl Scripting

Perl is an unstructured language, which means white space within the
program is typically ignored. For example, the following code prints “hi” to
the screen:
print "hi\n";

This could also be written as follows:1

print
"hi\n"
;

Note that a semicolon (;) is used to end a statement in Perl. Also note that
the \n string represents a newline character. The print statement doesn’t
naturally display a newline character, which makes for awkward output
when multiple print statements are executed.
Comments in Perl begin with a pound sign character (#) and extend to the
end of the line. For example:
# This is my first Perl script
print "hello\n"; #displays "hello"

Executing Perl Code

In most cases, you place Perl code within a file and execute the code as
shown in the following:

t.me/DevTwitter
 [student@OCS ~]$ more hello.pl
print "hello\n";
[student@OCS ~]$ perl hello.pl
hello

Typing the perl command before each execution can become annoying. To
avoid that, make use of the #! line:
[student@OCS ~]$ more hello.pl
#!/bin/perl

print "hello\n";
[student@OCS ~]$ chmod a+x hello.pl
[student@OCS ~]$ ./hello.pl
hello

You can also make use of a feature called the Perl debugger to execute Perl
code interactively. Start by executing the following command: perl -d -e
"1;"
The -d option enters the Perl debugger, which requires valid Perl code. The -
e option means “execute the code provided on the command line.” The "1;"
is valid Perl code; it doesn’t do anything but return the value of true. The
result of this command should be a prompt like the one shown in Listing 9.1.

Listing 9.1 The interactive Perl debugger

Click here to view code image
[student@OCS ~]$ perl -d -e "1;"

Loading DB routines from perl5db.pl version 1.37

Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(-e:1): 1;
DB<1>

At the DB<1 prompt you can type a Perl statement. Press the Enter key to
execute the command. This enables you to test Perl code interactively.2 A
couple of things to note:
Normally you need to place a ; character after each Perl statement. In
the Perl debugger, the Enter key acts like a ; character. This means you
don’t need a ; character at the end of a Perl statement within the
debugger.

t.me/DevTwitter
 A few things don’t work in the Perl debugger (for example, a regular
expression feature called backreferencing). However, almost
everything that works in a Perl script also works in the interactive Perl
debugger.
To exit the Perl debugger, type q and then press the ENTER key.

Additional Perl Documentation

Perl is a language that offers a large number of features, well beyond what is
covered in this book (or any book for that matter). Fortunately, you can use a
number of resources to get additional information about Perl:
perldoc.perl.org—Primary documentation website.
man perl—On UNIX and Linux systems, this command provides
details about Perl.
perldoc—On all platforms, this command provides information about
Perl.

The perldoc command is especially useful. Try running the following

command: perldoc perl
Man page-like output appears that provides information about Perl. Included
in this is a list of additional categories as shown in Figure 9.1.

t.me/DevTwitter
 Figure 9.1 Perl help topics
So, if you want to learn about Perl regular expressions, you could execute
the perldoc perlrequick or perldoc perlretut commands. You can view
dozens of different categories to learn more about Perl. As you learn more
details about Perl while reading this book, consider diving into this
documentation as well.

Variables and Values

Perl has three data structure types:
Scalar—A single data type that can be used as either a string or a
number.
Array—An ordered list of scalar values, separated by commas.
Hash—A collection of unordered values that are referenced by using a
scalar key. Also called an associate array.

t.me/DevTwitter
 Scalar variables are assigned and referenced using a $ character:
[student@localhost Desktop]$ perl -d -e "1;"

Loading DB routines from perl5db.pl version 1.37

Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(-e:1): 1;
DB<1> $name="Bob"
DB<2> print $name
Bob

Note
To read data from the user (the keyboard), use the syntax $name=
<STDIN>;

Several useful built-in Perl statements are used on scalars, including the
statements shown in Listing 9.2.

Listing 9.2 Useful scalar statements

Click here to view code image

[student@localhost Desktop]$ perl -d -e "1;"

Loading DB routines from perl5db.pl version 1.37

Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(-e:1): 1;
DB<1> $name=<STDIN> #grabs data from keyboard and assigns to
$name
Bob Smith
DB<2> print $name #prints "Bob Smith\n"; \n is a newline
character
Bob Smith
DB<3> chomp $name #removes newline character at the end of
string
DB<4> print $name #prints "Bob Smith" without newline
character
Bob Smith
DB<5> $name=lc ($name) #returns lower case "bob smith"
DB<6> print $name
bob smith

t.me/DevTwitter
 Arrays are defined by creating variables that begin with an @ character:
DB<1> @colors=("red", "blue", "green")
DB<2> print "@colors"
red blue green

One confusing element of Perl is how you refer to individual elements in an

array. Because those individual elements are scalar values, you use a $
character to display a single value. For example, to display the first element
in an array you do something like the following:
DB<1> @colors=("red", "blue", "green")
DB<2> print $colors[0]
red

Important statements that manipulate arrays include the following:

push—Adds a new element to end of the array
unshift—Adds a new element to beginning of the array
pop—Removes (and returns) the last element of the array
shift—Removes (and returns) the first element of the array
splice—Add or remove one or more items in any part of the array
sort—Sorts elements in an array

You can perform an operation on each element in an array by using the

foreach statement:

DB<1> @names=("Smith", "Jones", "Rothwell")

DB<2> foreach $person (@names) {print "Hello, Mr. $person\n";}
Hello, Mr. Smith
Hello, Mr. Jones
Hello, Mr. Rothwell

Creating an array can be a pain because of all the quotes3 and commas. To
make life easier, use the qw or qq statements:
DB<1> @colors=qq(red blue green) #same as @colors=("red", "blue",
"green")
DB<2> @colors=qw(red blue green) #same as @colors=('red', 'blue',
'green')

t.me/DevTwitter
 Single versus Double Quotes
A double-quoted string allows for special characters whereas a
single-quoted string does not. For example, the string "hello\n"
means hello followed by a newline character (\n = newline
character). However, the string 'hello\n' means hello\n.

A hash (called a dictionary in some other languages) provides a way to

associate a key with a value. For example, to keep track of the favorite color
of some people, you could use the following code:
DB<1> %favorite=("Sue" => "blue", "Ted" => "green", "Nick" =>
"black")
DB<2> print $favorite{"Ted"}
green

To see all the keys in a hash, use the keys command:

DB<1> %favorite=("Sue" => "blue", "Ted" => "green", "Nick" =>
"black")
DB<2> @people=keys(%favorite)
DB<3> print "@people"
Ted Nick Sue

Special Variables
Several variables in Perl have a special meaning to the
language. Typically these variables have cryptic names, such as
$| or $_. For example, the $$ variable contains the process ID of
the Perl process itself.
Some of these variables hold important information whereas
others can be modified to change the behavior of how Perl
functions. To see a description of all of these special variables,
visit this URL: http://perldoc.perl.org/perlvar.html.

Flow Control
Perl supports many traditional flow control statements, including the if
statement:4
$name=<STDIN>;
chomp $name;

t.me/DevTwitter
 if ($name eq "Tim")
{
print "Welcome, Tim!";
}
elsif ($name eq "Bob")
{
print "Welcome, Bob!";
}
else
{
print "Welcome, stranger!";
}

elsif?
Be careful when writing Perl if statements because the “else if”
statement is oddly spelled. It is a single word, with the second
“e” missing!

Another common conditional statement is the while loop. With the while
loop, a conditional check is performed and, if the condition is true, a block
of code is executed. After the block of code is executed, the conditional
check is performed again. See Listing 9.3 for an example.

Listing 9.3 The while loop

Click here to view code image
print "Enter your age: ";
$age=<STDIN>;
chomp $age;

#Make sure the user entered a proper age:

while ($age < 0)
{
print "You can't be that young!\n";
print "Enter your age: ";
$age=<STDIN>;
chomp $age;
}
print "Thank you!\n";

Additional flow control statements include the following:

until—Opposite of the while statement; executes a block of code as
long as the conditional statement returns false.

t.me/DevTwitter
 unless—Opposite of the if statement; executes a block of code if the
conditional statement is false.
for—Designed to perform a specific number of operations. Example:
for ($i=1; $i <=10; $i++) {#code}
foreach—Performs a block of code for each item of a list (array).

What about Switch or Case Statements?

Natively, Perl doesn’t have a switch or case statement. You
might see older Perl code make use of multiple if/elsif
statements or clever use of other conditional statements to
simulate a switch statement. However, modern Perl provides a
means to access a switch-like statement called given:
use feature "switch";
given ($setting) {
when (/^Code/) { $code = 1 }
when (/^Test/) { $test = 1 }
default { $neither = 1 }
}
The ^Code and ^Test are regular expressions and are covered
later in this chapter.

Many languages support loop control statements such as break and continue.
Perl also supports loop control statements, but they are called last and next
instead of break and continue. You can use these loop controls statements in
while, until for, and foreach loops.

Conditions
Perl supports a large variety of conditional expressions, including numeric
comparison, string comparison, file testing operations, and regular
expressions. You can also use the outcome of a Perl statement, but be aware
that only a handful of built-in Perl statements return a natural true or false
value.
Numeric comparisons include the following:
== Determine whether two numbers are equal to each other.
Example:
if ($age == 35) {}

t.me/DevTwitter
 !=Determine whether two numbers are not equal to each other.
Example:
if ($age != 35) {}
<Determine whether one number is less than another number.
Example:
if ($age < 35) {}
<=Determine whether one number is less than or equal to another
number.
Example:
if ($age <= 35) {}
>Determine whether one number is greater than another number.
Example:
if ($age > 35) {}
>=Determine whether one number is greater than or equal to another
number.
Example:
if ($age >= 35) {}
String comparisons include:
eq Determine whether two scalars are equal to each other.
Example:
if ($name eq “Bob”) {}
neDetermine whether two scalars are not equal to each other.
Example:
if ($name ne “Bob”) {}
ltDetermine whether one scalar is less than another scalar.
Example:
if ($name lt “Bob”) {}
leDetermine whether one scalar is less than or equal to another scalar.
Example:
if ($name le “Bob”) {}
gtDetermine whether one scalar is greater than another scalar.
Example:
if ($name gt “Bob”) {}

t.me/DevTwitter
 ge Determine whether one scalar is greater than or equal to another
scalar.
Example:
if ($name ge “Bob”) {}

File test operators include:5

-r Determine whether a file is readable. Example: if (-r “file”) {}
-w Determine whether a file is writable
-x Determine whether a file is executable
-T Determine whether a file contains text data
-e Determine whether a file exists
-f Determine whether a file exists and is a plain file
-d Determine whether a file exists and is a directory6

Regular expressions are a powerful feature in the Perl programming

language. For example, you can see whether a pattern exists inside of a
scalar variable by using the following code:
$name=<STDIN>;
if ($name =~ m/Bob/)
{
print "yes"
}

When reading Perl code, you should be aware of a few regular expression
features:
You can also perform substitution by using the following syntax:
$name =~ s/Bob/Ted
This replaced “Bob” with “Ted” in the $name variable.
Because matching is more common than substitution, you can drop the
“m” when performing a match:
if ($name =~ /Bob/) {}
The $_variable is called the default variable and is often used by default.
For example:
if ($_ =~ /Bob/) {} is the same as if (/Bob/) {}

Additional Features

t.me/DevTwitter
 In addition to reading from the keyboard (STDIN), you can open files and
read directly from the files. This is referred to as opening a filehandle:7
open (DATA, "<file.txt");
$line=<DATA>;
close DATA; #when finished, close the filehandle

You can also open a file and write to the file:

open (DATA, ">file.txt");
print DATA "This is output";
close DATA; #when finished, close the filehandle

Another important feature in Perl is functions. To create a function, use the

following syntax:
sub welcome
{
print "This is my function";
}

You can call a function by using the following syntax:

&welcome;

By default, any variable created in the main part of the program is available
in a function. Additionally, any variable in a function is available in the main
part of the program:
sub total
{
$z=$x + $y; # $x and $y from main program
}

$x=10;
$y=5;
&total;
print $z; # $z from total subroutine

To make variables private, use the my statement:8

sub total
{
my $z=$x + $y; # $x and $y are not set here
}

my $x=10;
my $y=5;

t.me/DevTwitter
 &total;
print $z; # $z is not set here

To reuse code in other programs, Perl has a feature called modules. Modules
are like libraries in other languages. By calling a module, you will have
access to functions that are shared by that module to your program.
For example, the following module call provides a function called cwd,
which displays the current directory:
[student@OCS ~]$ perl -d -e "1;"

Loading DB routines from perl5db.pl version 1.37

Editor support available.
Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(-e:1): 1;
DB<1> use Cwd;
DB<2> print cwd;
/home/student

A few things to note about modules:

By convention, module names begin with a capital letter. If you see a
use statement that calls something that is in all lowercase characters, it
isn’t a module, but rather another Perl feature called a pragma. Pragmas
are used to change the default behavior of Perl and are well documented
on perldoc.perl.org in the “Pragmas” section.
Perl comes with literally hundreds of built-in modules that enhance the
functionality of the language greatly. These are all well documented on
perldoc.perl.org in the “Modules” section.
Which functions are provided by the module depend on the developer
who created the module. View the module documentation to determine
which functions are provided.
You can create your own modules, but that topic is beyond the scope of
this book. See perldoc.perl.org → language → perlmodlib for details.

t.me/DevTwitter
 Perl Humor
Larry Wall continues to oversee further development of Perl and
serves as the benevolent dictator for life of the Perl project. His
role in Perl is best conveyed by the so-called 2 Rules, taken
from the official Perl documentation:
1. Larry is always by definition right about how Perl should
behave. This means he has final veto power on the core
functionality.
2. Larry is allowed to change his mind about any matter at a later
date, regardless of whether he previously invoked Rule 1.
Got that? Larry is always right, even when he was wrong.

Summary
As a robust language, Perl has many additional features that were not
covered in this chapter. However, the goal of this chapter was to provide you
with enough information to determine whether Perl is a good language for
you. If you liked the features and syntax of Perl, considering exploring the
documentation sources provided to learn more about this flexible language.

1 Just because it could be written this way doesn’t mean you should write
your code like this. As with all languages, you should try to write Perl
code that is easy to read.
2 This method also provides a way for me to demonstrate Perl features
without having to create a full Perl script.
3 Yes, you should use quotes around strings, even though it seems like you
don’t need them in Perl. By default, Perl tries to use an unquoted value as
a function and replaces the function call with the return value of the
function. Placing quotes around the string prevents function calls.
4 Note the formatting of this code. Perl really doesn’t care about formatting,
but someone reading your code will. There is no standard convention as to
how you format your Perl code, but I suggest that you at least be
consistent within the program that you are writing to make it easier to
read.

t.me/DevTwitter
 5 Note that when you look at Perl documentation, file test operators are not
listed in the section regarding operators, but rather under the “Functions”
section. Look for –X under functions to learn more about file test
operators.
6 I know the wording of this seems odd, but a directory is technically a file
in Linux. It is a file that contains specific data, a list of files that are in the
directory.
7 Several techniques actually exist for reading from a file. This is just one
of them.
8 This is a very simplistic view of variable scope in Perl. In reality, Perl
provides you with the capability to have a much more robust (and
complex) scoping process.

t.me/DevTwitter
 10. Python Scripting
Python is a robust programming language that provides many different
features that you would expect from a modern language. One of its many
benefits is the fact that it is object-oriented by nature, making it a good
language for large task programs.
This chapter’s focus is to give you a firm understanding of how to write
basic Python scripts as well as provide you with an understanding of some of
Python’s more advanced features.

Basics of Python Scripting

Unlike Perl (discussed in Chapter 9, “Perl Scripting”), Python is very much a
structured language. It is very sensitive to white space, to the point that
improper white space usage causes a program to crash with compile errors.
When you create a block of code, you must indent the entire block with the
same number of white space characters. For example, consider the following
code fragment, focusing on how the code is indented:
x = 25
if x > 15:

print x
a = 1
else:
print x
a = 2

The preceding code fragment results in a compile time error message

because the second print statement isn’t properly indented. The positive
thing about enforced structure is that it makes your code easier to read (for
both others and yourself when you look over code that you wrote months or
years ago). The negative thing is that it can be a pain when you miss a space
or accidently use a tab instead of four spaces.1

t.me/DevTwitter
 Use an Editor to Avoid Indentation Issues
To avoid issues with indentation, use an editor designed to
perform auto indentation. Many such editors are designed
specifically for Python, but you can also use a generic editor
such as vim.
To enable auto indentation in vim, start the editor and then
execute the command :set autoindent. Bonus: If you edit a file
that ends with .py, the vim editor will color code key Python
statements.

Executing Python Code

Currently, the two primary versions of Python are 2.x and 3.x. At the time of
the writing of this book, version 2.x was the more popular of the two, so the
book covers 2.x syntax. To determine which version of Python is installed on
your Linux distribution, execute the python command with the -V option or
enter the Python interactive shell by executing the python command with no
arguments:
[student@OCS ~]$ python
Python 2.7.5 (default, Oct 11 2015, 17:47:16)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> quit()
[student@OCS ~]$

Note that the python command not only displays the version of Python, but it
also places you into an interactive Python shell where you can test Python
code on the fly. To exit this Python shell, enter the quit() statement as
shown in the previous example.
To execute a Python script that is stored in a file, use the following syntax:
[student@OCS ~]$ python script.py

Typing the python command before each execution can become annoying.
To avoid that, make use of the #! line:
[student@OCS ~]$ more hello.py
#!/bin/python

t.me/DevTwitter
 print "hello"
[student@OCS ~]$ chmod a+x hello.py
[student@OCS ~]$ ./hello.py
hello

As demonstrated from the previous example, the print statement in Python

is used to produce output. By default, this output goes to STDOUT.

A Note about .pyc and .pyo Files

Your Python script names should end with .py. You will
sometimes also see files that end in .pyc, and this can lead to
some confusion. Not touching these files is best because they
are compiled versions of Python code and not something that
you can edit directly.
These files are created when a Python library is called. The idea
is that the compile process takes time and each time a library is
called, its code would have to be compiled. To make the process
more efficient, Python automatically saves this code into files
that end in .pyc. So, if you call a library called input.py, you
should expect to see a file called input.pyc after the program
that calls the library is executed.
When Python is invoked with the -O option, a .pyo file is
generated. Like .pyc files, this is compiled code, but it is
optimized complied code.

Additional Documentation
Initially you probably want to look at the man page for python: man python.
This provides you with some of the basics of the python executable, but
there isn’t much focus on how to write code in Python.
However, at the bottom of the python man page are some very useful links to
additional documentation, as shown in Figure 10.1.

t.me/DevTwitter
 Figure 10.1 Python documentation
The URL http://docs.python.org/ will likely be the most useful resource
when you start learning Python, but the other links will also prove valuable
over time.

Variables and Values

Python has several data structure types, including:
Numeric variables—A single data type that is used to store numeric
values.
String variables—A single data type that is used to store string values.
Lists—An ordered list of numeric or string values.
Dictionaries—A collection of unordered values that are referenced by
using a key.

Numbers and strings are different in that you can perform certain operations
on numbers (addition, subtraction, and so on) but not on strings as
demonstrated in Listing 10.1.

Listing 10.1 Numbers versus strings

Click here to view code image
[student@localhost Desktop]$ python
Python 2.7.5 (default, Oct 11 2015, 17:47:16)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> a=100
>>> print a
100
>>> b=200
>>> print a + b
300
>>> c="hello"
>>> print a + c

t.me/DevTwitter
 Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Note the error that occurred in the last statement of Listing 10.1 when the
string variable was used in a numeric operation.
Python has a rich set of operations that can be performed on strings. For
example, you can use the following to capitalize a string:
>>> name="ted"
>>> name=name.capitalize()
>>> print name
Ted

Python is an object-oriented language. Variables store typed objects

(numeric type, string type, and so on), and to call a method on an object, you
use the notation var.method(). So, name.capitalize() calls the capitalize
method on the object in the name variable.
Note that Python has a few traditional functions as well. For example,
consider the following code:
>>> name="ted"
>>> print len(name)
3

The len function takes an object as an argument and returns the length of the
object. Granted, using both method calls and functions can sometimes be
confusing, but keep in mind that most of Python consists of method calls,
and functions are fairly rare.
To create a list, use the following syntax:
>>> colors=["red", "blue", "yellow", "green"]
>>> print colors
['red', 'blue', 'yellow', 'green']

Use the following notation to access elements in a list:

>>> colors=["red", "blue", "yellow", "green"]
>>> print colors[1]
blue
>>> print colors[1:3]
['blue', 'yellow']

Important methods that manipulate lists include the following:

t.me/DevTwitter
 append—Adds a new element to the end of the list.
insert—Adds a new element to a specific position in the list.
extend—Adds the elements of a list to another list.
del—Removes an element from a list based on an index position.
pop—Removes the last element from a list.
remove—Removes an element from a list based the value of the element.

The Tuple
You are likely to come across another data structure that works
much like a list called a tuple. Consider a tuple to be a list that,
once created, can’t be modified. Technically, tuples are
immutable, a fancy way of saying unchangeable. They are a
clever way of having constant-like data structures.

To create a dictionary in Python, use the following syntax:

>>> age={'Sarah': 25, "Julia": 18, "Nick": 107}
>>> print age
{'Sarah': 25, 'Nick': 107, 'Julia': 18}
>>> print age['Sarah']
25

To add a key-value pair to a dictionary, use the following syntax:

>>> age['Bob']=42

Times occur when you will want to get a list of all the keys of a dictionary.
To accomplish this, use the keys() method:2
>>> age={'Sarah': 25, "Julia": 18, "Nick": 107}
>>> print age.keys()
['Sarah', 'Nick', 'Julia']

Other Data Structures

Python has other data structures that you should consider
exploring. For example, you can create sets of data within
Python. This is useful because set objects have methods that
enable you to find items that appear in two sets or items that
only appear in a specific set.

t.me/DevTwitter
 Flow Control
Python supports many traditional flow control statements, including the if
statement:
age=15
if age >= 16:
print "you are old enough to drive"
elif age == 15:
print "you are old enough for a permit"
else:
print "sorry, you can't drive yet"

Another common conditional statement is the while loop. With the while
loop, a conditional check is performed and, if the condition is true, a block
of code is executed. After the block of code is executed, the conditional
check is performed again. See Listing 10.2 for an example.

Listing 10.2 The while loop

Click here to view code image

#!/bin/python

age = int(raw_input('Please enter your age: '))

while (age < 0):

print "You can't be that young!";

age = int(raw_input('Please enter your age: '))

print "Thank you!";

Note that in Listing 10.2, the raw_input() function gets data from the user
(STDIN, likely the keyboard) and the int() function converts this data into an
integer object.
To perform an operation on each item of a list, use the for loop:
>>> colors=["red", "blue", "yellow", "green"]
>>> for hue in colors:
... print hue
...
red
blue
yellow
green

t.me/DevTwitter
 Many languages support loop control statements such as break and continue.
Python has these statements, which you can use in while loops or for loops.
The break statement exits the loop prematurely and the continue statement
stops the current iteration of the loop and starts the next iteration. Python
also supports an else statement that you can use to execute additional code if
the loop is terminated without a break statement.

Conditions
Python supports a large variety of conditional expressions, which enable you
to perform comparison operations on like objects. For example, you can
compare strings to strings, dictionaries to dictionaries, and so on.
Comparison operators include the following:
== Determine whether two objects are equal to each other.
!= Determine whether two objects are not equal to each other.
< Determine whether one object is less than another object.
<= Determine whether one object is less than or equal to another object.
> Determine whether one object is greater than another object.
>= Determine whether one object is greater than or equal to another
object.

Additional Features
In addition to reading from the keyboard, you can open files and read
directly from the files. For example:
>>> data=open('test.py', 'r')
>>> print data
<open file 'test.py', mode 'r' at 0x7f4db20ba1e0>

The first argument to the open statement is the filename to open. The second
argument is how to open it. The value 'r' means to open the file for reading.
You can also open a file and write to the file by using the value of 'w'.
You can use several methods for reading from and writing to a file after
opening it:
read()—Reads the entire file. Example: total = data.read().
readline—Reads one line from the file.
write—Writes data to a file. Example: data.write("hello").

t.me/DevTwitter
 Another important feature in Python is functions. To create a function, use
the following syntax:
def welcome():
print "This is my function"

You can call a function by using the following syntax:

welcome()

To reuse code in other programs, Perl has a feature called modules. By

calling a module, you have access to functions that are shared by that
module in your program.
For example, the following module call provides a function called path that
displays a list of directories where Python libraries are held:
>>> import sys
>>> print sys.path
['', '/usr/lib64/python27.zip', '/usr/lib64/python2.7',
'/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk',
'/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload',
'/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-
packages/gtk-2.0', '/usr/lib/python2.7/site-packages'] /home/student

Programming Humor
Definition of recursion:
1. Stop when you understand the definition.
2. See definition of recursion.

Summary
This chapter serves as an introduction to the Python language. As with most
languages discussed in this book, Python offers many other features. The
purpose of this chapter was to introduce you to the language to determine
whether it might meet your software development needs.

1 You might be thinking, “Why did he say four spaces and not five or
eight?” Python Enhancement Proposals (PEPs) are a major component of
Python development. PEP8 is the “Style Guide for Python Code,” and it
states that the standard for indentation is four spaces.

t.me/DevTwitter
 2 Note that the return value of the keys are not in the order they were
originally created in. Remember that a dictionary is an unordered
collection of key-value pairs.

t.me/DevTwitter
 11. C, C++, and Java
The purpose of this chapter is different from the last three chapters.
Although the goals of Chapters 8–10 were to introduce you to new
languages, the assumption in this chapter is that you already have a
background coding C, C++, or Java. These languages are popular on all
operating systems, including Microsoft Windows.
Instead of covering the basics of C, C++, and Java, the focus on this chapter
is on topics that are related to these languages. Specifically, the goal is to
provide you with information regarding the Linux operating system’s impact
on how you create programs in these languages.

Understanding System Libraries

A library is a file that contains compiled code (typically C or C++) that
developers use to add more functionality to their programs. Normally a
library includes functions and declarations that are shared with the calling
program. Rarely does a library do anything on its own because the goal is to
define things that will be used by the calling program.
Another program calls libraries via the #include statement in the source
code. For example:
#include <stdio.h>

The two types of library calls are

Static:
Code that is included in the program at compile time
Filename typically ends in .a
Makes binary larger, but doesn’t require additional run-time files
Shared:
Code is included at run time
More flexible (can change library at any time)
Binary code is smaller; however, missing library causes program not
to execute

t.me/DevTwitter
 Managing Shared Library Files
On production machines, the task of managing shared library files is often
the responsibility of the system administrator. However, on your own system
you may take on this responsibility. Even if you don’t, understanding the
basics of how these files are managed can make you a better developer.
Usually, shared library files are stored in one of the following locations on
Linux distributions:
/lib or /lib64
/usr/lib or /usr/lib64
/usr/local/lib or /usr/local/lib64

If your operating system is a 32-bit distribution, expect to see the libraries

under /lib, /usr/lib, and /usr/local/lib. On 64-bit platforms, the /lib64,
/usr/lib/64, and /usr/local/lib64 directories are where you can expect to
file libraries. You may also see file libraries under the 32-bit directories
because some applications may run as 32-bit even under a 64-bit operating
system. See Figure 11.1 for an example of the /lib64 directory.

Figure 11.1 Shared libraries.

These shared libraries follow the naming convention of libname.so.ver. In
this case, the name is a unique name for the library and ver is used to indicate
the version number of this library—for example, libkpathsea.so.6.1.1.
The idea of managing shared libraries on the system is to add or remove
libraries as needed. This requires access to the system as the root user
because only the root user can modify the configuration files.

t.me/DevTwitter
 The primary configuration file for shared libraries is the /etc/ld.so.conf
file. However, typically only a single line is in this file:
[root@localhost ~]# more /etc/ld.so.conf
include ld.so.conf.d/*.conf

The include line in this file tells the system to also use all the configuration
files in the specified directory. In the case of the preceding example, that
would be all the files that end in .conf in the /etc/ld.so.conf.d directory:
[root@localhost ~]# ls /etc/ld.so.conf.d
dyninst-x86_64.conf libiscsi-x86_64.conf
kernel-3.10.0-327.el7.x86_64.conf mariadb-x86_64.conf

A big advantage exists for using this include method. Suppose you create
some software that requires some shared libraries. To install that software on
a system, you need to tell the system where these new libraries are. Instead
of having to create an installation program that modifies the primary
configuration file (/etc/ld.so.conf), your installation program can simply
copy a configuration file into the /etc/ld.so.conf.d directory. Conversely,
the uninstall program that you create to remove the software could simply
delete this file from the /etc/ld.so.conf.d directory.
The configuration file itself is simple. It just contains a directory in which
the shared libraries are stored:
[root@localhost ~]# more /etc/ld.so.conf.d/libiscsi-x86_64.conf
/usr/lib64/iscsi
[root@localhost ~]# ls /usr/lib64/iscsi/
libiscsi.so.2 libiscsi.so.2.0.10900

To add new shared libraries to the system, you first download the libraries to
the system and place them into a directory. After adding new libraries, you
create a configuration file in the /etc/ld.so.conf.d directory and then
execute the ldconfig command.1 You should perform all of these tasks as
the root user:
[root@localhost ~]# ls /usr/lib64/test
mylib.so.1
[root@localhost ~]# cat /etc/ld.so.conf.d/libtest.conf
/usr/lib64/test
[root@localhost ~]# ldconfig

Regular users can’t successfully execute the ldconfig command. However,

if a regular user wants to use a custom shared library, then the user can

t.me/DevTwitter
 download this file into her home directory and make use of the
LD_LIBRARY_PATH to indicate the location of custom library files:

[student@localhost ~]$ ls lib

mylib.so.1
[student@localhost ~]$ export LD_LIBRARY_PATH=/home/student/lib

Viewing Shared Library Files

You can see what shared libraries a specific command uses by using the ldd
command as demonstrated in Listing 11.1.

Listing 11.1 The ldd command

Click here to view code image
[root@localhost ~]# ldd /bin/cp
linux-vdso.so.1 => (0x00007ffc35df9000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f93faa09000)
libacl.so.1 => /lib64/libacl.so.1 (0x00007f93fa800000)
libattr.so.1 => /lib64/libattr.so.1 (0x00007f93fa5fa000)
libc.so.6 => /lib64/libc.so.6 (0x00007f93fa239000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f93f9fd8000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f93f9db2000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f93f9bae000)
/lib64/ld-linux-x86-64.so.2 (0x00007f93fac42000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f93f9992000)

The purpose of using the ldd command is to troubleshoot problems with

code that you are writing. This command tells you not only what libraries are
being called, but specifically which directory each library is being called
from. This can be extremely useful when a library is not behaving as you
would expect it to behave.

Building Packages
You have successfully completed creating your software and are ready to
package the software so it can be installed. The two common techniques for
packaging software are by using RPM and Debian.2 Typically, you use RPM
to build packages on Red Hat–based distributions (RHEL, Fedora, CentOS,
and so on) and Debian on Debian-based systems (Debian, Ubuntu, Mint, and
so on).

t.me/DevTwitter
 Note
Creating a software package with RPM or Debian can be very
complex. The examples provided in this chapter are very
general and designed to provide an overview of how to build
packages.

Building RPM Packages

To build an RPM, first install the rpm-build software package as
demonstrated in Listing 11.2. (Note that some output of the yum command
was removed.)

Listing 11.2 Install rpm-build

Click here to view code image
[root@localhost ~]# yum -y install rpm-build
Resolving Dependencies
--> Running transaction check
---> Package rpm-build.x86_64 0:4.11.3-17.el7 will be installed
--> Processing Dependency: patch >= 2.5 for package: rpm-build-
4.11.3-17.el7.x86_64
--> Processing Dependency: system-rpm-config for package: rpm-build-
4.11.3-17.el7.x86_64
--> Processing Dependency: perl(Thread::Queue) for package: rpm-
build-4.11.3-17.el7.x86_64
--> Running transaction check
---> Package patch.x86_64 0:2.7.1-8.el7 will be installed
---> Package perl-Thread-Queue.noarch 0:3.02-2.el7 will be installed
---> Package redhat-rpm-config.noarch 0:9.1.0-68.el7.centos will be
installed
--> Processing Dependency: dwz >= 0.4 for package: redhat-rpm-
config-9.1.0-68.el7.centos.noarch
--> Processing Dependency: perl-srpm-macros for package: redhat-rpm-
config-9.1.0-68.el7.centos.noarch
--> Running transaction check
---> Package dwz.x86_64 0:0.11-3.el7 will be installed
---> Package perl-srpm-macros.noarch 0:1-8.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

Transaction Summary
====================================================================
========================================================
Install 1 Package (+5 Dependent packages)

t.me/DevTwitter
 Total download size: 451 k
Installed size: 944 k
Downloading packages:
Running transaction
Installing : patch-2.7.1-8.el7.x86_64
1/6
Installing : dwz-0.11-3.el7.x86_64
2/6
Installing : perl-Thread-Queue-3.02-2.el7.noarch
3/6
Installing : perl-srpm-macros-1-8.el7.noarch
4/6
Installing : redhat-rpm-config-9.1.0-68.el7.centos.noarch
5/6
Installing : rpm-build-4.11.3-17.el7.x86_64
6/6
Verifying : redhat-rpm-config-9.1.0-68.el7.centos.noarch
1/6
Verifying : perl-srpm-macros-1-8.el7.noarch
2/6
Verifying : perl-Thread-Queue-3.02-2.el7.noarch
3/6
Verifying : rpm-build-4.11.3-17.el7.x86_64
4/6
Verifying : dwz-0.11-3.el7.x86_64
5/6
Verifying : patch-2.7.1-8.el7.x86_64
6/6

Installed:
rpm-build.x86_64 0:4.11.3-17.el7

Dependency Installed:
dwz.x86_64 0:0.11-3.el7 patch.x86_64 0:2.7.1-8.el7
perl-Thread-Queue.noarch 0:3.02-2.el7
perl-srpm-macros.noarch 0:1-8.el7 redhat-rpm-config.noarch 0:9.1.0-
68.el7.centos

Complete!

The next step is a bit tricky because you need to create a directory structure
that contains various files, including your software and instructions on how
to install the software. I recommend downloading the source code of a
sample project and using that as a template. For example, after downloading
the source code for the dovecot package, you can expand the source code by
using the rpm command as demonstrated in Listing 11.3.3

t.me/DevTwitter
 Listing 11.3 Install source RPM
Click here to view code image
root@localhost ~]# rpm -ivh /tmp/dovecot-2.2.10-5.el7.src.rpm
Updating / Installing...
1:dovecot-1:2.2.10-
5.el7 ################################# [100%]
[root@localhost ~]# ls rpmbuild/
SOURCES SPECS
[root@localhost ~]# ls rpmbuild/SOURCES
dovecot-1.0.beta2-mkcert-permissions.patch dovecot-2.2.9-
nodevrand.patch
dovecot-1.0.rc7-mkcert-paths.patch dovecot-2.2-pigeonhole-
0.4.2.tar.gz
dovecot-2.0-defaultconfig.patch dovecot.conf.5
dovecot-2.1.10-reload.patch dovecot.init
dovecot-2.1.10-waitonline.patch dovecot.pam
dovecot-2.1-privatetmp.patch dovecot.sysconfig
dovecot-2.2.10-CVE_2014_3430.patch dovecot.tmpfilesd
dovecot-2.2.10.tar.gz prestartscript
[root@localhost ~]# ls rpmbuild/SPECS
dovecot.spec

Your source code is stored in the SOURCES directory. The spec file is used to
define how to install the software. As you can tell from the following output,
this file can be quite large:
[root@localhost ~]# wc -l rpmbuild/SPECS/dovecot.spec
1849 rpmbuild/SPECS/dovecot.spec

Yes, the spec file for the dovecot package is 1,849 lines long. Not all spec
files will be this large (although some will be larger). Spec files are a large
topic by themselves. However, they are much like BASH shell scripts, so if
you read a few existing spec files, you should be able to figure out how to
create your own (or modify an existing spec file).
After your source files are in the correct directory and you have created a
spec file, you can build your package using the following command:
rpmbuild -ba ~/rpmbuild/SPECS/name.spec

This command creates two new subdirectories: RPMS and SRPMS. In those
directories will be your package’s RPM files.

Building Debian Packages

t.me/DevTwitter
 The process of building a Debian package is very similar to building a RPM
package. The following are the general steps:
1. Download source code file (typically a tar file).
2. Edit configuration files:
debian/changelog
debian/rules
debian/control
3. Use dpkg-buildpackage to build the package.

Exploring Java Installation and Basics

In older Linux distributions, it was rare that Java came installed by default.
However, almost all distributions have Java installed as part of the typical
installation process. You can see whether it is installed by using the which
command and, if it is installed, use the java command to determine which
version is installed:
[root@localhost ~]# which java
/bin/java
[root@localhost ~]# java -version
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

If Java isn’t installed, you can use the apt-get command on Debian-based
systems to install it. The package name for Java on Debian-based systems is
openjdk-X.X.X.jdk (X.X.X represents the version number).
If Java isn’t installed on a Red Hat–based system, you can use the yum
command to install it. The package name for Java on Debian-based systems
is java-X.X.X-openjdk.

Programming Humor
Real programmers count from 0!

Summary
This chapter introduced some key concepts and features of Linux that are
related to programming in C, C++, and Java. You learned how library files
are managed and how to package software for distribution. You should also

t.me/DevTwitter
 now know how to determine whether Java is installed and, if not, how to
install a version of Java.

1 No news is good news for the ldconfig command. No output means the
command worked successfully.
2 Other methods you might want to consider include APK, TGZ, and PET.
For example, TGZ is when you use the tar command to merge all files
into a single file and then the gzip command to compress the tar file.
3 For this example, I actually break a best practice policy: don’t build
packages while logged in as root. However, I am just reviewing the basics
of package building. Consider exploring the use of mock, a system that
can be used to build RPMs using a regular user account rather than the
root account.

t.me/DevTwitter
 IV. Using Git
One of the biggest headaches that developers must deal with is different
versions of source code. Sometimes you just need to “go back” to a
previous version of code. Maintaining these versions manually can be
cumbersome and time consuming.
Compounding the problem is when multiple programmers work together on
a single piece of source code. A large program can be tens of thousands of
lines of code with different programmers responsible for different portions
of the code.
Version control software like Git can handle the complicated task of
maintaining different versions of source code.

t.me/DevTwitter
 12. Git Essentials
This chapter introduces you to Git, including how to install the necessary
software to access Git servers where your software project will be stored.

Version Control Concepts

To understand Git and the concept of version control, looking at version
control from an historical perspective is helpful. There have been three
generations of version control software.

The First Generation

The first generation was very simple. Developers worked on the same
physical system and “checked out” one file at a time.
This generation of version control software made use of a technique called
file locking. When a developer checked out a file, it was locked and no other
developer could edit the file. Figure 12.1 illustrates the concept of this type
of version control.
Examples of first-generation version control software include Revision
Control System (RCS) and Source Code Control System (SCCS).

The Second Generation

The problems with the first generation included the following:
Only one developer can work on a file at a time. This results in a
bottleneck in the development process.
Developers have to log in directly to the system that contains the
version control software.

These problems were solved in the second generation of version control

software. In the second generation, files are stored on a centralized server in
a repository. Developers can check out separate copies of a file. When the
developer completes work on a file, the file is checked in to the repository.
Figure 12.2 illustrates the concept of this type of version control.

t.me/DevTwitter
 Figure 12.1 First-generation version control software

Figure 12.2 Second-generation version control software

If two developers check out the same version of a file, then the potential for
issues exists. This is handled by a process called a merge.

t.me/DevTwitter
 What Is a Merge?
Suppose two developers, Bob and Sue, check out version 5 of a
file named abc.txt. After Bob completes his work, he checks
the file back in. Typically, this results in a new version of the
file, version 6.
Sometime later, Sue checks in her file. This new file must
incorporate her changes and Bob’s changes. This is
accomplished through the process of a merge.
Depending on the version control software that you use, there
could be different ways to handle this merge. In some cases,
such as when Bob and Sue have worked on completely different
parts of the file, the merge process is very simple. However, in
cases in which Sue and Bob worked on the same lines of code
in the file, the merge process can be more complex. In those
cases, Sue will have to make decisions, such as whether Bob’s
code or her code will be in the new version of the file.
After the merge process completes, the process of committing
the file to the repository takes place. To commit a file
essentially means to create a new version in the repository; in
this case, version 7 of the file.

Examples of second-generation version control software include Concurrent

Versions System (CVS) and Subversion.

The Third Generation

The third generation is referred to as Distributed Version Control Systems
(DVCSs). As with the second generation, a central repository server contains
all the files for the project. However, developers don’t check out individual
files from the repository. Instead, the entire project is checked out, allowing
the developer to work on the complete set of files rather than just individual
files. Figure 12.3 illustrates the concept of this type of version control.

t.me/DevTwitter
 Figure 12.3 Third-generation version control software
Another (very big) difference between the second and third generation of
version control software has to do with how the merge and commit process
works. As previously mentioned, the steps in the second generation are to
perform a merge and then commit the new version to the repository.
With third-generation version control software, files are checked in and then
they are merged. To understand the difference between these two techniques,
first look at Figure 12.4.

t.me/DevTwitter
 Figure 12.4 Second-generation merge and commit
In phase 1 of Figure 12.4, two developers check out a file that is based on
the third version. In phase 2, one developer checks that file in, resulting in a
version 4 of the file.
In phase 3 the second developer must first merge the changes from his
checked-out copy with the changes of version 4 (and, potentially, other
versions). After the merge is complete, the new version can be committed to
the repository as version 5.
If you focus on what is in the repository (the center part of each phase), you
will see that there is a very straight line of development (ver1, ver2, ver3,
ver4, ver5, and so on). This simple approach to software development poses
some potential problems:
Requiring a developer to merge before committing often results in
developers’ not wanting to commit their changes on a regular basis. The
merge process can be a pain and developers might decide to just wait
until later and do one merge rather than a bunch of regular merges. This

t.me/DevTwitter
 has a negative impact on software development as suddenly huge chunks
of code are added to a file. Additionally, you want to encourage
developers to commit changes to the repository, just like you want to
encourage someone who is writing a document to save on a regular
basis.
Very important: Version 5 in this example is not necessarily the work
that the developer originally completed. During the merging process, the
developer might discard some of his work to complete the merge
process. This isn’t ideal because it results in the loss of potentially good
code.

A better, although arguably more complex, technique can be employed. It is

called Directed Acyclic Graph (DAG), and you can see an example of how
it works in Figure 12.5.

Figure 12.5 Third-generation commit and merge

t.me/DevTwitter
 Phases 1 and 2 are the same as shown in Figure 12.4. However, note that in
phase 3 the second “check in” process results in a version 5 file that is not
based on version 4, but rather independent of version 4. In phase 4 of the
process, versions 4 and 5 of the file have been merged to create a version 6.
Although this process is more complex (and, potentially, much more
complex if you have a large number of developers), it does provide some
advantages over a “single line” of development:
Developers can commit their changes on a regular basis and not have to
worry about merging until a later time.
The merging process could be delegated to a specific developer who has
a better idea of the entire project or code than the other developers have.
At any time, the project manager can go back and see exactly what work
each individual developer created.

Certainly an argument exists for both methods. However, keep in mind that
this book focuses on Git, which uses the Directed Acyclic Graph method of
third-generation version control systems.

Installing Git
You might already have Git1 on your system because it is sometimes
installed by default (or another administrator might have installed it). If you
have access to the system as a regular user, you can execute the following
command to determine whether you have Git installed:
ocs@ubuntu:~$ which git
/usr/bin/git

If Git is installed, then the path to the git command is provided, as shown in
the preceding command. If it isn’t installed, then you either get no output or
an error like the following:
[ocs@centos ~]# which git
/usr/bin/which: no git in (/usr/lib64/qt-
3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
:/root/bin)

As an administrator on a Debian-based system, you could use the dpkg

command to determine whether the git package has been installed:
root@ubuntu:~# dpkg -l git
Desired=Unknown/Install/Remove/Purge/Hold

t.me/DevTwitter
 | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-
aWait/
➥Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-========-=============-=============-
========================================
ii git 1:1.9.1-1ubun amd64 fast, scalable, distributed
➥revision con

As an administrator on a Red Hat–based system, you could use the rpm

command to determine whether the git package has been installed:
[root@centos ~]# rpm -q git
git-1.8.3.1-6.el7_2.1.x86_64

If Git isn’t installed on your system, you must either log in as the root user
or use sudo or su to install the software. If you are logged in as the root user
on a Debian-based system, you can use the following command to install
Git:
apt-get install git

If you are logged in as the root user on a Red Hat–based system, you can use
the following command to install Git:
yum install git

Get More Than Git!2

Consider installing the software package named git-all. This
package includes some additional dependency packages that
add more power to Git. Although you might not make use of
these features in this introductory book, having them available
when you are ready to perform more advanced Git functions
will be good.

Git Concepts and Features

One of the challenges to using Git is just understanding the concepts behind
it. If you don’t understand the concepts, then all the commands just seem
like some sort of black magic. This section focuses on the critical Git
concepts as well as introduces you to some of the basic commands.

t.me/DevTwitter
 Git Stages
It is very important to remember that you “check out”3 an entire project and
that most of the work you do will be local to the system that you are working
on. The files that you check out will be placed in a directory under your
home directory.
To get a copy of a project from a Git repository, you use a process called
cloning. Cloning doesn’t just create a copy of all the files from the
repository; it actually performs three primary functions:
Creates a local repository of the project under the project_name/.git
directory in your home directory. The files of the project in this location
are considered to be “checked out” from the central repository.
Creates a directory where you can directly see the files. This is called
the working area. Changes made in the working area are not
immediately version controlled.
Creates a staging area. The staging area is designed to store changes to
files before you commit them to the local repository.

This means that if you were to clone a project called Jacumba, the entire
project would be stored in the Jacumba/.git directory under your home
directory. You should not attempt to modify these directly. Instead, look
directly in the ~/Jacumba directory and you will see the files from the project.
These are the files that you should change.
Suppose you made a change to a file, but you have to work on some other
files before you were ready to commit changes to the local repository. In that
case, you would stage the file that you have finished working on. This would
prepare it to be committed to the local repository.
After you make all changes and stage all files, then you commit them to the
local repository. See Figure 12.6 for a visual demonstration of this process.

t.me/DevTwitter
 Figure 12.6 Git stages
Realize that committing the staged files only sends them to the local
repository. This means that only you have access to the changes that have
been made. The process of “checking in” the new versions to the central
repository is called a push.
I explain each of these steps in greater detail later. Consider this to be an
introduction to the concepts that will help you understand the process better
when I introduce the Git commands.

Choosing Your Git Repository Host

First, the good news: Many organizations provide Git hosting—at the time
of this writing, more than two dozen choices. This means you have many
options to choose from. That’s the good news…and the bad news.
It is only bad news because it means you really need to spend some time
researching the pros and cons of the different hosting organizations. For
example, most don’t charge for basic hosting but do charge for large-scale
projects. Some only provide public repositories (anyone can see your
repository) whereas others allow you to create private repositories. There are
many other features to consider.
One of the features that might be high on your list is a web interface.
Although you can do just about all repository operations locally on your
system, being able to perform some operations via a web interface can be
very useful. Explore the interface that is provided before making your
choice.

t.me/DevTwitter
 Consider This
I am not going to make a suggestion regarding which Git host
you should use because it really does vary based on what your
needs are. Many websites provide up-to-date comparisons
between the various Git hosts, and I strongly recommend you
do your homework before deciding on one.
At the very least, I recommend considering the following:
https://bitbucket.org
http://www.cloudforge.com
http://www.codebasehq.com
https://github.com
http://gitlab.com
Note that I chose gitlab.com for the examples in the book. Any
of the hosts in the preceding list would have been just fine for
the book; I just chose gitlab.com because it happened to be the
one I used on my last Git project.

Configuring Git
Now that you have gotten through all the theory,4 it is time to actually do
something with Git. This next section assumes the following:
You have installed the git or git-all software package on your system.
You have created an account on a Git hosting service.

The first thing you want to do is perform some basic setup. Whenever you
perform a commit operation, your name and email address will be included
in the metadata. To set this information, execute the following commands:
ocs@ubuntu:~$ git config --global user.name "Bo Rothwell"
ocs@ubuntu:~$ git config --global user.email
"[email protected]"

Obviously you will replace "Bo Rothwell" with your name and
"[email protected]" with your email address. The next step is to clone
your project from the Git hosting service. Note that before cloning, only one
file is in the user's home directory:

t.me/DevTwitter
 ocs@ubuntu:~$ ls
first.sh

The following cloned a project named ocs:

ocs@ubuntu:~$ git clone https://gitlab.com/borothwell/ocs.git
Cloning into 'ocs'...
Username for 'https://gitlab.com': borothwell
Password for 'https://[email protected]':
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
Checking connectivity... done.

After successful execution, notice a new directory in the user’s home

directory:
ocs@ubuntu:~$ ls
first.sh ocs

If you switch to the new directory, you can see what was cloned from the
repository (only one file so far exists in the repository):
ocs@ubuntu:~$ cd ocs
ocs@ubuntu:~/ocs$ ls
README.md

Next, create a new file in the repository directory. You can either create one
from scratch or copy a file from another location:
ocs@ubuntu:~/ocs$ cp ../first.sh

Remember, anything placed in this directory is not version controlled

because this is the working directory. To put it in the local repository, you
first have to add it to the staging area and then you need to commit it to the
repository:
ocs@ubuntu:~/ocs$ git add first.sh
ocs@ubuntu:~/ocs$ git commit -m "added first.sh"
[master 3b36054] added first.sh
1 file changed, 5 insertions(+)
create mode 100644 first.sh

The git add command places the file in the staging area. The git commit
command takes all the new files in the staging area and commits them to the
local repository. You use the -m option to add a message; in this case the
reason for the commit was given.

t.me/DevTwitter
 It is important to highlight that no changes have been made to the repository
on the server. The git commit command only updates the local repository.
You can see that the server repository has not been modified by looking at
Figure 12.7, which shows a screenshot of the web-based interface of the
current project. Notice that the original file, README.md, was pushed to the
server several days ago, but the new file, first.sh, does not have an entry.

Figure 12.7 Server repository is unchanged after executing the git commit
command
Most likely you would make additional changes to your local project and
then "check in" (push) the changes to the server’s repository:
ocs@ubuntu:~/ocs$ git push -u origin master
Username for 'https://gitlab.com': borothwell
Password for 'https://[email protected]':
Counting objects: 4, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 370 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://gitlab.com/borothwell/ocs.git
12424f5..3b36054 master -> master
Branch master set up to track remote branch master from origin.

See Figure 12.8 to verify the push was successful.

t.me/DevTwitter
 Figure 12.8 Server repository is changed after executing the git push
command
At this point all changes of the files from the staging area have been updated
to the local repository and the central server repository.

Git Humor
Wiser words may have never been spoken: “Commit early,
commit often. A tip for version controlling—not for
relationships.”
—Anonymous

Summary
This chapter focused on the key concepts of Git. You should now understand
the concepts of version control in addition to specifics about how to perform
version control with Git.

1 When I refer to Git with a capital I, I am referring to the software project.

When I refer to git with a lowercase g and in code font, I am referring to
either the command or the software package (most often the command).
2 It appears that all those Linux “jokes” have rubbed off on me.
3 I use parentheses around “check out” because in Git the process is really
called cloning. However, some developers who are used to second-
generation version control software are more comfortable with the term

t.me/DevTwitter
 “check out.” I tend to use the terms interchangeably when initially
explaining the Git process.
4 Trust me, the theory will help a lot!

t.me/DevTwitter
 13. Manage Files with Git
Git provides a rich collection of features, but the core set of features are the
ones that you are going to use the most often. This includes commands that
enable you to stage files and commit files to the local repository, as well as
create branches. This chapter focuses on these topics as well as a few
related ones.

Basic Configuration
You might want to perform several configuration operations. For example,
you can set your default editor by executing the following command:
ocs@ubuntu:~/ocs$ git config --global core.editor vi

You can view current configuration settings by executing the git config --
list command:

ocs@ubuntu:~/ocs$ git config --list

user.name=Bo Rothwell
[email protected]
core.editor=editor_name
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://gitlab.com/borothwell/ocs.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master

You can store configuration information in multiple locations. One location

is in your own home directory:
ocs@ubuntu:~/ocs$ more ~/.gitconfig
[user]
name = Bo Rothwell
email = [email protected]
[core]
editor = vi

An administrator can also store configuration information for all users in

the /etc/gitconfig file. The additional settings that you see when you

t.me/DevTwitter
 execute the git config --list command are either default settings or are
derived from the information in the local repository.

Getting Help
Besides looking at man pages for information about git
commands, you might consider the git help command. When
run with no additional arguments, the git help command
provides a synopsis of how the command should be executed
followed by a summary of git commands (config, add, clone,
commit, push, and so on).

To see information about a specific command, execute git

help command. For example: git help config. This
automatically puts you in the man page for that command.

git status
In Chapter 12, “Git Essentials,” you learned how to clone an existing
repository from a central repository server. You also learned the lifecycle of
how local files are committed to the local repository and then pushed to the
central repository server:
1. Create the file in the repository directory.
2. Add the file to the staging area with the git add command.
3. Commit the file to the repository with the git commit command.
4. Push the file to the central server with the git push command.

Note
You can combine the git add and git commit commands into a
single operation by using the -a option to the git commit
command.

Imagine you are working on some files one day and it’s getting late. It is
Friday afternoon and you just can’t wait for the weekend to start. On the
following Monday you arrive at work and realize you have no idea in what
area you left your file. Were they added to the staging area? All of them or
just some? Did you commit any of them to the local repository?

t.me/DevTwitter
 This is when you want to run the git status command:
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean

Note
It makes a difference which directory you are in when you
execute the git status command. For example, if you are in
your home directory, all the files in this directory will be
checked to see whether they are in the repository:
ocs@ubuntu:~$ git status
On branch master
Initial commit
Untracked files:
(use "git add <file>..." to include in what will be
committed)

.bash_history
.bash_logout
.bashrc
.gitconfig
.lesshst
.profile
.viminfo
ocs/

nothing added to commit but untracked files present

(use "git add"
➥to track)
Make sure you are in the repository directory before running
this command!

If you make changes to a file and don’t add it to the staging area, then the
output of the git status command will look like the following:

t.me/DevTwitter
 ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:

(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working
directory)

modified: first.sh

no changes added to commit (use "git add" and/or "git commit -a")

Notice in the previous output the Changes not staged for commit section.
The command output also is helpful in showing you how you can stage the
command with the git add command or stage and commit with the git
commit -a command.
If a file has been added to the staging area, but not committed to the local
repository, then the output of the git status command will look like the
following:
ocs@ubuntu:~/ocs$ git add first.sh
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes to be committed:
(use "git reset HEAD <file>..." to unstage)

modified: first.sh

Changes to be committed means the modified file is in the staging area but
not in the local repository. If a file has been committed to local repository,
but not committed to the central repository server, then the output of the git
status command will look like the following:

ocs@ubuntu:~/ocs$ git commit -m "demostrating status"

[master 9eb721e] demostrating status
1 file changed, 2 insertions(+), 1 deletion(-)
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)

nothing to commit, working directory clean

t.me/DevTwitter
 Note that nothing to commit, working directory clean means the staging
area no longer contains any files and the working directory reflects the
current contents of the local repository. Also note the message that states
Your branch is ahead of 'origin/master' by 1 commit. This makes it
clear that you need to execute git push to push the contents of the local
repository to the central repository server.
Executing the git status command after successfully executing the git
push command results in the output demonstrated in the original example of
this chapter as shown in Listing 13.1.

Listing 13.1 All files up to date

Click here to view code image
ocs@ubuntu:~/ocs$ git push -u origin master
Username for 'https://gitlab.com': borothwell
Password for 'https://[email protected]':
Counting objects: 5, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 381 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://gitlab.com/borothwell/ocs.git
3b36054..9eb721e master -> master
Branch master set up to track remote branch master from origin.
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean

Handling a Multiple Location Situation

Because of the staging area, the possibility exists to have one version of the
file in the local repository, a second version in the staging area, and a third
in the working directory. This can happen after you edit a file, add it to the
staging area, and then edit the file again. When this happens, the output of
the git status command will look like that shown in Listing 13.2.

Listing 13.2 Three versions of a file

Click here to view code image
ocs@ubuntu:~/ocs$ git status
On branch master

t.me/DevTwitter
 Your branch is up-to-date with 'origin/master'.

Changes to be committed:
(use "git reset HEAD <file>..." to unstage)

modified: first.sh

Changes not staged for commit:

(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working
➥directory)

modified: first.sh

In the output of Listing 13.2, you can see that the first.sh file is listed both
as Changes to be committed and Changes not staged for commit. Now
you must make a choice:
Commit both versions of the file—First execute the git commit
command, then execute git add, and then execute git commit again.
Commit the last version of the file—First execute git add and then
execute git commit.1

A useful option to the git status command is the -s option, which shows a
much more condensed output:
ocs@ubuntu:~/ocs$ git status -s
M first.sh
A showmine.sh
?? hidden.sh

Each file is preceded by up to two characters. The first character indicates

the status in the staging area and the second character indicates the status in
the working directory. The first.sh file has been modified in the working
directory, but you can tell that it has not been staged yet because of the
space character in front of the "M" character. Note the difference if the file
is staged:
ocs@ubuntu:~/ocs$ git add first.sh
ocs@ubuntu:~/ocs$ git status -s
M first.sh
A showmine.sh
?? hidden.sh

t.me/DevTwitter
 The A for the showmine.sh file means this is a new file that has been staged
(because the A is in the first column), but it hasn’t been committed to the
repository yet. The ?? means that hidden.sh is new and hasn’t been staged
or committed yet. Any file that is up to date in the local repository is not
listed when you execute the git status -s command.

Telling Git to Ignore a File

In some cases, you want to have a file in the working directory but not have
it ever staged or placed in the repository. For example, maybe you want to
keep track of some notes about the project, but only for your own purposes.
Of course, you could just never add this file to the staging area, but this
would mean that git status will never return nothing to commit, working
directory clean.
To have git commands ignore a file, create a file named .gitignore in the
working directory and place the filename to ignore inside of this file:
ocs@ubuntu:~/ocs$ touch notes
ocs@ubuntu:~/ocs$ git status -s
?? notes
ocs@ubuntu:~/ocs$ vi .gitignore #added notes as shown below:
ocs@ubuntu:~/ocs$ cat .gitignore
notes
ocs@ubuntu:~/ocs$ git status -s
?? .gitignore

Notice that you must also place the .gitignore file itself in the .gitignore
file:
ocs@ubuntu:~/ocs$ git status -s
?? .gitignore
ocs@ubuntu:~/ocs$ vi .gitignore #added .gitignore as shown
below:
ocs@ubuntu:~/ocs$ cat .gitignore
notes
.gitignore
ocs@ubuntu:~/ocs$ git status -s

t.me/DevTwitter
 Note
You can also use wildcard characters (*, ?, and [range]) in the
.gitignore file to match a collection of files. For example, I
like to name my own files with a .me extension. In the
.gitignore file I include the pattern *.me to have the git
commands ignore all of my files.
You can use a pattern that ends with a / to indicate an entire
directory.

If You Are Following Along…

I’ve tried to make this process as transparent as possible, but
there are some cases in which I am going to be executing some
commands that don’t show up in the body of the text. For
example, at this point I have added several versions of the
first.sh file (as well as two new files: the showmine.sh and
hidden.sh files) to the local repository. I want to use these in
later examples, so I decided to push them to the central
repository server:
ocs@ubuntu:~/ocs$ git push -u origin master
Username for 'https://gitlab.com': borothwell
Password for 'https://[email protected]':
Counting objects: 10, done.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (8/8), 865 bytes | 0 bytes/s,
done.
Total 8 (delta 1), reused 0 (delta 0)
To https://gitlab.com/borothwell/ocs.git
9eb721e..07bb91c master -> master
Branch master set up to track remote branch master from
origin.
However, this isn't critical to the topic at hand, so I opted not to
show this in the main text. In the future, if I do some "behind
the scenes" commands, I will mention the commands that I
executed in a footnote. That way if you are following along,
you should end up with the same results that I do.

t.me/DevTwitter
 Removing Files
Suppose one day you create a file purely for testing purposes and then,
without really thinking about it, you commit it to the local repository:2
ocs@ubuntu:~/ocs$ git add test.sh
ocs@ubuntu:~/ocs$ git commit "update 27"

Later, you realize your mistake and want to remove the file. Simply
removing it from the working directory isn’t enough, as demonstrated in
Listing 13.3.3

Listing 13.3 Deleting file from working directory

Click here to view code image
ocs@ubuntu:~/ocs$ rm test.sh
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)

Changes not staged for commit:

(use "git add/rm <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working
directory)

deleted: test.sh

no changes added to commit (use "git add" and/or "git commit -a")

Notice the suggestion to execute the git rm command, which will stage the
file for removal from the repository. See Listing 13.4 for an example of the
git rm command.

Listing 13.4 Staging a file to be deleted

Click here to view code image
ocs@ubuntu:~/ocs$ git rm test.sh
rm 'test.sh'
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)

Changes to be committed:

t.me/DevTwitter
 (use "git reset HEAD <file>..." to unstage)

deleted: test.sh

Lastly, perform a git commit command to remove the file from the
repository:
ocs@ubuntu:~/ocs$ git commit -m "deleting test.sh file"
[master 2b44792] deleting test.sh file
1 file changed, 1 deletions(-)
delete mode 100644 test.sh

Handling Branches
You decide that you want to test some new features of the project that you
are working on, but you don’t want this to impact the current development
process. This is an ideal time to create a branch.
When you first create a project, the code is associated with a branch called
master. If you want to create a new branch, execute the git branch
command:
ocs@ubuntu:~/ocs$ git branch test

This doesn’t mean you are suddenly working in the new branch. As you can
see from the output of the git status command, the git branch command
doesn’t change your current branch:
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is ahead of 'origin/master' by 2 commits.
(use "git push" to publish your local commits)

nothing to commit, working directory clean

The first line of the output of the previous command, On branch master
denotes that you are still working in the master branch. To switch to the
new branch, execute the git checkout command:4
ocs@ubuntu:~/ocs$ git checkout test
Switched to branch 'test'

Switching actually does two things:

Makes it so any future commits occur on the test branch
Makes it so the working directory reflects the test branch

t.me/DevTwitter
 The second item makes more sense with a demonstration. First, observe the
following commands, which will end up with a new version of the
hidden.sh file being placed in the test branch repository:
ocs@ubuntu:~/ocs$ git add hidden.sh
ocs@ubuntu:~/ocs$ git commit -m "changed hidden.sh"
[test ef2d7d5] changed hidden.sh
1 file changed, 1 insertion(+)

Note what the file looks like in the current working directory:
ocs@ubuntu:~/ocs$ more hidden.sh
#!/bin/bash
#hidden.sh

echo "Listing only hidden files:"

ls -ld .* $1

If we switch the project back to the master branch, you can see how the
hidden.sh file in the working directory is different (note the missing echo
line, which was added for the test branch only):
ocs@ubuntu:~/ocs$ git checkout master
Switched to branch 'master'
Your branch is ahead of 'origin/master' by 2 commits.
(use "git push" to publish your local commits)
ocs@ubuntu:~/ocs$ more hidden.sh
#!/bin/bash
#hidden.sh
ls -ld .* $1

You can see the changes that are made on different branches, along with the
comments you provided for each change, by using the git log command:
ocs@ubuntu:~/ocs$ git log --oneline --decorate --all
ef2d7d5 (test) changed hidden.sh
2b44792 (HEAD, master) deleting test.sh file
19198d7 update 27
07bb91c (origin/master, origin/HEAD) adding showmine.sh and
hidden.sh
75d717b added first.sh
9eb721e demostrating status5
3b36054 added first.sh
12424f5 add README

The--online option has the git log command provide a one-line summary
of each change. The--decorate option requests additional information such

t.me/DevTwitter
 as the branch name. The--all option asks to see the log for all branches,
not just the current branch.

Pushing Branches
Recall the command to push changes to the central repository server:
git push -u origin master

Did you wonder at the time what the word master meant? You can probably
guess now that it was the branch that you were pushing to the central
repository. If you wanted to push the test branch, then you would have to
execute the following command as well:6
git push -u origin test

There is more to the story of branching besides just creating a branch and
switching back and forth between branches. For example, at some point you
might want to merge files from branches together. You might also want to
see how versions of files are different between different branches. Chapter
14, “Manage Differences in Files” covers these topics.

Git Humor
“Be careful not to remove the branch you're standing on.”
—Anonymous

Summary
After reading this chapter, you should have a firm understanding of how the
working directory, staging area, and local repository work. You should also
know how to determine which of these locations a file is currently "in" as
well as how to specify files that the git commands should ignore. You
should understand the basics of branching, including how to create
branches, switch between branches, add/commit in different branches, and
push different branches to the central repository server.

1 The resulting output of this git commit command tends to cause

confusion: 1 file changed, 2 insertions(+), 1 deletion(-). Because

t.me/DevTwitter
 two versions of the file were staged (two insertions), one had to be
deleted before the other was committed to the repository.
2 This potential mistake might be the reason why you must execute git
add and then git commit for new files (you can't just execute git commit
-a to add a new file).
3 However, nothing is wrong with executing a regular rm command
because you probably want to delete this file from the working directory.
This just isn't going to end up removing it from the repository.
4 You can create a branch and switch to it by using the -b option to the git
checkout command: git checkout -b test
5 Just a side note for those of you paying close attention: the spelling error
(demostrating, when it should be demonstrating) is actually how the
output of the command was displayed.
6 If are following along, you should know that I did execute both git push
commands to update the master and test branch. I use this data in
Chapter 14 examples.

t.me/DevTwitter
 14. Manage Differences in Files
Typically, developers love writing code. Unfortunately, writing code is not
the only responsibility of software developers. Searching for differences
between two versions of source code files and merging them into a new
version is a major part of writing code.
The Git software tries to make this process easier by providing you with
tools that display the difference between files and help you merge files
together. These tools are the focus of this chapter.

Executing Diffs
You arrive at work Monday morning, ready to start on your project.
Executing the git status command when you haven't worked on the project
for a while is always a good habit, so you execute the command and
discover that you have a file in the working area that hasn't been staged:
ocs@ubuntu:~/ocs$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:

(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working
directory)

modified: hidden.sh

no changes added to commit (use "git add" and/or "git commit -a")

You could just stage and commit the file, but you wonder what changes were
made to this file. Maybe you didn't finish making the changes last Friday.
This might pose problems, so comparing the version of the file in the
working directory with the version that has most recently been committed to
the local repository is best. You can do this by executing the git diff
command:
ocs@ubuntu:~/ocs$ git diff hidden.sh | cat -n
1 diff --git a/hidden.sh b/hidden.sh
2 index 05151ce..714482b 100644
3 --- a/hidden.sh
4 +++ b/hidden.sh

t.me/DevTwitter
 5 @@ -1,4 +1,5 @@
6 #!/bin/bash
7 #hidden.sh
8
9 +echo "Hidden files:"
10 ls -ld .* $1

The output of this command requires a bit of explaining.1 Ignore the first
two lines of output because they aren't important at this point.
Lines 3 and 4 refer to the two versions of the file. Each version is assigned a
letter (a or b) to distinguish between the two. Line 3 refers to the version that
has been committed, whereas line 4 refers to the version in the working
directory.
Line 5 gives “directions” on how to make the two files look the same. In this
case, it is simply saying “at line 4 of the 'a' file add line 5 of the 'b' file.”
Following these directions would make the files look the same by making
the committed version look like the version in the working directory.
Lines 6–10 visually show what changes would have to take place in the
committed version to make it look like the version in the working directory.
A + before a line means "add this" and a – before a line means "remove
this."

Note
The output of the git diff command is referred to as patch
output. You can use this output to patch a file (essentially,
upgrade the file to the most current version). Chapter 15,
“Advanced Git Features” covers this process.

Understanding that this comparison is performed on a line-by-line basis is

important. A single change on one line would mean the lines are completely
different to the git diff command. For example, look at Listing 14.1 and
notice that the only difference between lines 9 and 11 is a single character.

Listing 14.1 A single difference

Click here to view code image
ocs@ubuntu:~/ocs$ git diff hidden.sh | cat -n
1 diff --git a/hidden.sh b/hidden.sh
2 index 05151ce..6de92ae 100644

t.me/DevTwitter
 3 --- a/hidden.sh
4 +++ b/hidden.sh
5 @@ -1,4 +1,5 @@
6 #!/bin/bash
7 #hidden.sh
8
9 -ls -ld .* $1
10 +echo "Hidden files:"
11 +ls -ldh .* $1

By default, the git diff command compares versions in the following two
situations:
If the working directory version is different than the committed version,
but only if the working version hasn't been staged
If the working directory version is different than the staged version

That means if you stage a file, the git diff command won't compare the
staged version to the committed version, at least not by default. As you can
see, there is no output for the following git diff command:
ocs@ubuntu:~/ocs$ git add hidden.sh
ocs@ubuntu:~/ocs$ git diff | cat -n

To compare a staged version to a committed version, use the --staged option

as shown in Listing 14.2.

Listing 14.2 Staged versus committed difference

Click here to view code image
ocs@ubuntu:~/ocs$ git diff --staged hidden.sh | cat -n
1 diff --git a/hidden.sh b/hidden.sh
2 index 05151ce..6de92ae 100644
3 --- a/hidden.sh
4 +++ b/hidden.sh
5 @@ -1,4 +1,5 @@
6 #!/bin/bash
7 #hidden.sh
8
9 -ls -ld .* $1
10 +echo "Hidden files:"
11 +ls -ldh .* $1

Dealing with White Space

t.me/DevTwitter
 A useful option to the git diff command is the --check option, which looks
for white spaces. To understand the importance of this, first look at the
output of Listing 14.3.

Listing 14.3 White space mystery

Click here to view code image

ocs@ubuntu:~/ocs$ git diff --staged hidden.sh | cat -n

1 diff --git a/hidden.sh b/hidden.sh
2 index 6de92ae..519eb3c 100644
3 --- a/hidden.sh
4 +++ b/hidden.sh
5 @@ -1,5 +1,5 @@
6 #!/bin/bash
7 -#hidden.sh
8 +#hidden.sh
9
10 echo "Hidden files:"
11 ls -ldh .* $1

Based on the output of Listing 14.3, lines 7 and 8 are different. However,
they look exactly the same. To see why they are different use the --check
option:
ocs@ubuntu:~/ocs$ git diff --staged --check | cat -n
1 hidden.sh:2: trailing whitespace.
2 +#hidden.sh

The message trailing whitespace means some sort of white space

characters (spaces, tabs, and so on) are at the end of the line.2

Comparing Branches
You can also use the git diff command to compare files in different
branches. For example, to see a list of files that are different between two
branches, use the following command:
ocs@ubuntu:~/ocs$ git diff --name-status master..test
M hidden.sh

In the previous git diff command, the --name-status option provides a

summary of the files that are different in the two branches. The two
branches, master and test, are listed, separated by .. characters.

t.me/DevTwitter
 To see the differences between the versions in the two branches, use the
syntax shown in Listing 14.4.

Listing 14.4 git diff between branches

Click here to view code image

ocs@ubuntu:~/ocs$ git diff master:hidden.sh test:hidden.sh

diff --git a/master:hidden.sh b/test:hidden.sh
index 519eb3c..804fcf7 100644
--- a/master:hidden.sh
+++ b/test:hidden.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-#hidden.sh
+#hidden.sh

-echo "Hidden files:"

-ls -ldh .* $1
+echo "Listing only hidden files:"
+ls -ld .* $1

If you find the output of the git diff command to be confusing, consider
using the git difftool command:3
ocs@ubuntu:~/ocs$ git difftool hidden.sh

This message is displayed because 'diff.tool' is not configured.

See 'git difftool --tool-help' or 'git help config' for more
details.
'git difftool' will now attempt to use one of the following tools:
opendiff kdiff3 tkdiff xxdiff meld kompare gvimdiff diffuse
diffmerge ecmerge p4merge araxis bc3 codecompare emerge vimdiff

Viewing (1/1): 'hidden.sh'

Launch 'vimdiff' [Y/n]: y
2 files to edit

Note how it prompts you for the tool to use to display the differences. It also
lists the tools that could be available.4 If you want to use a different tool than
the one it chooses, then execute the command as shown in the following:
git difftool --tool=<tool> file

The output of git difftool appears in a more readable format. For example,
see Figure 14.1 for the output of git difftool when you use the vimdiff
command as the display tool.5

t.me/DevTwitter
 Figure 14.1 git difftool using vimdiff

Merging Files
Suppose you create a new branch to add a new feature to a file as
demonstrated in Listing 14.5.

Listing 14.5 Features branch

Click here to view code image
ocs@ubuntu:~/ocs$ more showmine.sh
#!/bin/bash
#showmine.sh

echo "Your processes:"

ps -fe | grep $USER | more
ocs@ubuntu:~/ocs$ git checkout -b feature127
Switched to a new branch 'feature127'
ocs@ubuntu:~/ocs$ vi showmine.sh
ocs@ubuntu:~/ocs$ more showmine.sh
#!/bin/bash
#showmine.sh

echo -n "Enter name username or press enter: "

read person

t.me/DevTwitter
 echo "${person:-$USER} processes:"
ps -fe | grep "^${person:-$USER}" | more

After testing out this new feature (look at the last two lines of the output of
Listing 14.5 to see the new feature), you are ready to implement it in the
master branch. To do this, you will need to merge the content from the
feature127 branch into the master branch. Start by committing all changes
in the feature127 branch and then switch back to the master branch:
ocs@ubuntu:~/ocs$ git commit -a -m "feature added to showmine.sh"
[feature127 2e5defa] feature added to showmine.sh
1 file changed, 5 insertions(+), 2 deletions(-)
ocs@ubuntu:~/ocs$ git checkout master
Switched to branch 'master'
Your branch is ahead of 'origin/master' by 3 commits.
(use "git push" to publish your local commits)

You must be in the branch that you want to merge into in order to correctly
run the next command. The following git merge command merges the
changes from the feature127 branch into the master branch:
ocs@ubuntu:~/ocs$ git merge feature127
Updating 4810ca8..2e5defa
Fast-forward
showmine.sh | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)

After the feature has been implemented, you may decide that the feature127
branch is no longer necessary. To remove this branch, execute the following
command:
ocs@ubuntu:~/ocs$ git branch -d feature127
Deleted branch feature127 (was 2e5defa).

This merge process can be more complex. For example, there was a separate
branch named test that was branched off an earlier version of the master
branch. In the test branch, the most recent showmine.sh script looks like the
following:
ocs@ubuntu:~/ocs$ git checkout test
ocs@ubuntu:~/ocs$ more showmine.sh
#!/bin/bash
#showmine.sh

echo "Your programs:"

t.me/DevTwitter
 ps -fe | grep $USER | more

echo -n "Enter a PID to stop: "

read proc
kill $proc

The current version of showmine.sh that has been committed to the master
branch looks like the following:
ocs@ubuntu:~/ocs$ git checkout master
Switched to branch 'master'
Your branch is ahead of 'origin/master' by 4 commits.
(use "git push" to publish your local commits)
ocs@ubuntu:~/ocs$ more showmine.sh
#!/bin/bash
#showmine.sh

echo -n "Enter name username or press enter: "

read person

echo "${person:-$USER} processes:"

ps -fe | grep "^${person:-$USER}" | more

Should you merge the changes from the master into the test branch? Or
should you merge the changes from the test branch into the master branch?
Typically, if you have more work to do in the test branch, merge the
changes from the master into the test branch. Otherwise, merge the changes
from the test branch into the master branch.
The following example merges the changes from the master branch into the
test branch:

ocs@ubuntu:~/ocs$ git checkout test

Switched to branch 'test'
Your branch is ahead of 'origin/test' by 1 commit.
(use "git push" to publish your local commits)
ocs@ubuntu:~/ocs$ git merge master
Auto-merging showmine.sh
CONFLICT (content): Merge conflict in showmine.sh
Auto-merging hidden.sh
CONFLICT (content): Merge conflict in hidden.sh
Automatic merge failed; fix conflicts and then commit the result.

You can see that the merge was not completed because the automated merge
process ran into some conflicts. You can also see these conflicts by
executing the git status command:
ocs@ubuntu:~/ocs$ git status
On branch test

t.me/DevTwitter
 Your branch is ahead of 'origin/test' by 1 commit.
(use "git push" to publish your local commits)

You have unmerged paths.

(fix conflicts and run "git commit")

Unmerged paths:
(use "git add <file>..." to mark resolution)

both modified: hidden.sh

both modified: showmine.sh

no changes added to commit (use "git add" and/or "git commit -a")

This makes it clearer that two files have conflicts, not just one. If you look at
the new showmine.sh file in the working directory, it will look something like
Listing 14.6.

Listing 14.6 Merged file

Click here to view code image
ocs@ubuntu:~/ocs$ cat -n showmine.sh
1 #!/bin/bash
2 #showmine.sh
3
4 <<<<<<< HEAD
5 echo "Your programs:"
6 ps -fe | grep $USER | more
7
8 echo -n "Enter a PID to stop: "
9 read proc
10 kill $proc
11
12 =======
13 echo -n "Enter name username or press enter: "
14 read person
15
16 echo "${person:-$USER} processes:"
17 ps -fe | grep "^${person:-$USER}" | more
18 >>>>>>> master

Essentially, the file contains the contents of each file. Rather than try to edit
this file directly, one way to handle these conflicts is to use the git
mergetool command:6

ocs@ubuntu:~/ocs$ git mergetool showmine.sh

t.me/DevTwitter
 This message is displayed because 'merge.tool' is not configured.
See 'git mergetool --tool-help' or 'git help config' for more
details.
'git mergetool' will now attempt to use one of the following tools:
opendiff kdiff3 tkdiff xxdiff meld tortoisemerge gvimdiff diffuse
diffmerge ecmerge p4merge araxis bc3 codecompare emerge vimdiff
Merging:
showmine.sh

Normal merge conflict for 'showmine.sh':

{local}: modified file
{remote}: modified file
Hit return to start merge resolution tool (vimdiff):

The git mergetool command displays the files using one of the diff tools.
For example, using the vimdiff tool displays as shown in Figure 14.2.7

Figure 14.2 git mergetool using vimdiff

The first time you see this output, you might be a bit overwhelmed. It isn't as
bad as you might think. To see how this works, look at Figure 14.3.

t.me/DevTwitter
 Figure 14.3 Understanding vimdiff
BASE is what the file looked like when the two branches were last in sync.
LOCAL represents how the file looks in the current branch (the test branch
in this example). REMOTE represents how the file looks like in the branch
that is being merged into this branch (the master branch in this example).
The file at the bottom is the file that you are creating/merging.
As you can see from Figure 14.2, the first three lines of all versions of the
file are identical. If you go to the first line that differs between the LOCAL
and REMOTE, you can execute the :diffget RE command to grab the code
from the REMOTE file. This would grab the echo, read, echo, and ps
commands in this example.
Suppose you want to input specific lines from one of the three files? For
example, you want to copy the last four lines of the LOCAL version into the
merged area. In this case you essentially do a copy and paste operation.
To switch to the LOCAL window, hold down the Ctrl button and then press
the W key twice (Ctrl+W+W). Then copy the four lines (go to the first line
that you want to copy then type 4yy). Use Ctrl+W+W to move forward until
the cursor returns to the merged copy. Then move to where you want to
make the change and press the P key.
After making all of your changes, you need to save and quit all four
versions. The easiest way to do this is the :wqa vim command.

Merge all files and then stage/commit them with the git commit -a
command.8

t.me/DevTwitter
 Git Humor
"Git knows what you did last summer!"
—Anonymous

Summary
At this point you should now know how to create branches, see the
difference in versions of files, and merge files from one branch to another.

1 If you run this on your own system and don't pipe the output to the cat
command, you should see some color highlight that helps you understand
the data.
2 Note that I executed git commit at this point.
3 Note that you need to install the git-all package if you want to use the
git difftool command.
4 Most likely not all of these tools are installed on your system.
5 Note that I executed git commit at this point.
6 Note that you need to install the git-all package if you want to use the
git mergetool command.
7 The vimdiff utility highlights differences using colors that are sure to
give you a headache if you look at them too long. I highly suggest
executing the :diffoff! command immediately to avoid visual problems
and dizziness. However, when you want to make changes to the file, you
need to turn this feature back on by executing the :window diffthis
command.
8 If you are following along, be aware that at the end of this chapter I
completed the merge of all files (including the hidden.sh file), committed
all changes to the test branch, and pushed all changes for both the test
and master branch to the central repository server.

t.me/DevTwitter
 15. Advanced Git Features
At the heart of Git are the repositories. Chapters 13 and 14 introduced you to
tools that enabled you to work with the local repository. In this chapter you
learn how to interact with the central repository server.

Managing Repositories
From your perspective, the central repository server is also considered the
remote repository as opposed to the repository that is on your system (the
local repository).

To see the location of your remote repository, execute the git remote -v
command:
ocs@ubuntu:~/ocs$ git remote -v
origin https://gitlab.com/borothwell/ocs.git (fetch)
origin https://gitlab.com/borothwell/ocs.git (push)

Both lines point to the same location. The first line refers to the process of
downloading content from the remote repository and the second line refers
to the process of uploading content to the remote repository.
Very likely you will be working on separate projects, each with its own
remote repository. To access another project, execute the git remote add
command:1
ocs@ubuntu:~/ocs$ git remote add docs
➥ https://gitlab.com/borothwell/docs.git
ocs@ubuntu:~/ocs$ git remote -v
docs https://gitlab.com/borothwell/docs.git (fetch)
docs https://gitlab.com/borothwell/docs.git (push)
origin https://gitlab.com/borothwell/ocs.git (fetch)
origin https://gitlab.com/borothwell/ocs.git (push)

The docs argument is how you want to refer to this remote repository locally.
The last argument is the URL path to the remote repository.
Of course, after you have added the remote repository, you should also clone
it using the git clone command:
ocs@ubuntu:~/ocs$ cd ..
ocs@ubuntu:~$ git clone https://gitlab.com/borothwell/docs.git

t.me/DevTwitter
 Cloning into 'docs'...
Username for 'https://gitlab.com': borothwell
Password for 'https://[email protected]':
warning: You appear to have cloned an empty repository.
Checking connectivity... done.
ocs@ubuntu:~$ ls
docs ocs
ocs@ubuntu:~$ ls -a docs
. .. .git

Even though this is an empty repository, you can see that it has been cloned
by the fact it created a ~/docs directory and a docs/.git directory. To
perform work in this project, simply create files in the ~/docs directory and
execute the git commands while in this directory.

Getting Content from the Remote Server

The process of getting the content from the remote server to the local
repository and working directory is fairly simple. In this situation, you
execute the git clone command to duplicate everything in the project from
the remote server to the local system.
However, this process can become much more complex later in the
development cycle when you want to include changes from the remote
server (likely from other developers) to your local repository and working
directory. You can use several different methods, each with a different
intended result.
Before getting into the process of getting content from the remote server,
visualizing how content is sent from the working directory to other locations
might be helpful. Look at Figure 15.1 for a demonstration.

Figure 15.1 Git commands to send changes from the working directory
In Figure 15.1 you can see the flow of version changes. The git add
command sends the working version to the staged area. The git commit

t.me/DevTwitter
 command sends the staged version to the local repository. The git commit -
a command can send a version from the working directory to the staged area
and then to the local repository. Lastly, the git push command sends the
changed versions of the files to the remote repository.2

When to Commit versus When to Push

I am often asked when to use git commit versus git push.
Some organizations provide rules or guidelines for when a
developer should use commit versus push. If your organization
doesn’t have any guidelines, I suggest using the following
guidelines
Commit often! It is the means by which you can undo a mistake
or go back to an older version. Think of it like a “save as” of a
document.
Push when you have something to share to other developers.
Pushing multiple times a day for subtle file changes makes the
process of merging and applying changes much more difficult.

File versions cannot only be sent from the working directory up to the
remote repository, but they can also be propagated from the remote
repository to the local repository and the working directory. See Figure 15.2
for a visual description of the commands that can perform these actions.3

Figure 15.2 Git commands to retrieve changes from the remote repository
Details regarding these commands:
The git fetch command downloads the latest version history of files
from the remote repository to the local repository. This command does
not change your working directory, but you can use the git checkout

t.me/DevTwitter
 command to make the working directory contain the latest versions of
the files of the branch.

Important
The git fetch command only downloads the file versions. It
does not perform any merging.
The git pull command downloads the latest version history of
files from the remote repository to the local repository, but this
command also performs a merge operation and updates the
content of your working directory. The merging process is the
same as that described in Chapter 14.
The git rebase command takes all the changes from a branch
and applies them to the current branch. This is a process called
patching, which is described in more detail later in this chapter.

Connecting via SSH

The default (and recommended) method of communication with the remote
repository is HTTPS. Another method that can be used is SSH. Both
methods should provide a secure (and encrypted) communication method.
Typically, HTTPS is recommended for a few reasons:
No additional setup is required when you use HTTPS. If you choose to
use SSH, then you must generate a SSH key and upload it to the remote
repository.
When using HTTPS, Git makes use of a feature called credential
helper automatically to cache your password. This means when you
connect to the remote repository, you won’t have to provide your
password each time. This could also be configured for SSH, but it isn’t
done so automatically (you need to configure this using a SSH feature
called ssh-agent).
The third possible reason is related to firewalls. In networks that are
highly secured with strict firewalls, the HTTPS network port is more
likely to be “open” than the SSH network port.

This doesn’t mean you shouldn’t use SSH, just that the developers of Git
recommend HTTPS. If you do want to use SSH, then you first need to

t.me/DevTwitter
 execute the following command:
ocs@ubuntu:~/ocs$ ssh-keygen -t rsa

This creates an SSH key in the ~/.ssh/id_rsa.pub file. You must upload this
file to the SSH server, which you normally do via the web interface. See
Figure 15.3 for an example of this interface on gitlab.com.

Figure 15.3 Uploading SSH keys on gitlab.com

After uploading the SSH keys to the remote repository server, you can have
the git command use SSH instead of HTTP by using the following syntax:
git clone ssh://user@server/project.git

Another method that is popular is to use the following syntax:

git clone user@server:project.git

Patching
The idea of patching is that you will discover situations in which it won’t be
easy to perform a simple merge between two different branches. Several
reasons exist as to why this might be, including the following:
When you want to implement changes from a branch that hasn’t been
made available on the central repository
When you want to implement changes from a specific version of a file

Actually several different techniques can be utilized to perform patching.

The most basic (and common) method is to use the git diff command that
was discussed in Chapter 14 to generate a diff file. You do this by executing
the git diff command and redirecting the output to a file. For example:

t.me/DevTwitter
 git diff A B > file.patch

This patch file will be used to modify an existing checked-out file to include
the differences. This is performed by using the git apply command. For
example:
git apply file.patch

Often the patch file is generated on one system and then copied to another
system before applying. It is also typically a good idea to use the --check
option to test the patch before actually applying it:
git apply --check file.patch
git apply file.patch

Git Humor
“Nurture your git-twigs and they will grow into a full branch.”
—Anonymous

Summary
In this chapter you learned how to manage repositories. You also learned
how to change the method of connecting to a remote repository from HTTPS
to SSH. Last, you were introduced to the concept of patching in Git.

1 I created this new project by using the web interface provided by

gitlab.com.
2 None of these commands are new because they were covered in previous
chapters. However, visualizing this process can help you understand the
next topic better.
3 Note that the git clone command is not included in Figure 15.2. This is
because typically you only run this command once when you first
download the remote repository.

t.me/DevTwitter
 Index
Symbols
“ (double quotes), 124
‘ (single quotes), 124
- option (su command), 84–85
/ (forward slash)
command, 75
in paths, 29
/bin directory, 30
/home directory, 30
/media directory, 30
/root directory, 30
/sbin directory, 30
/tmp directory, 30
/usr/bin directory, 30
/usr/sbin directory, 30
/usr/share/doc directory, 27
: (colon) command, 76
; (semicolon) in Perl statements, 119
| (pipe) in redirection, 44–45
#! (sha-bang)
in bash shell scripting, 111
in Perl, 120
in Python, 132
$ (dollar sign) command, 71
( (left parenthesis) command, 72
) (right parenthesis) command, 71
* (asterisk) as wildcard, 38
? (question mark)
command, 75
as wildcard, 39

t.me/DevTwitter
 [[ (double left square brackets) command, 72
[] (square brackets) as wildcard, 39
]] (double right square brackets) command, 72
{ (left curly brace) command, 72
} (right curly brace) command, 72
< (left arrow) in redirection, 43
> (right arrow) in redirection, 41–42
>> (double right arrow) in redirection, 41
0 (zero) command, 71
2> (two right arrow) in stderr redirection, 42–43

A
A command, 69
a command, 69
absolute paths, 31–33
adding user accounts, 91–92
aliases, 56
append method, 135
apt-get command, 88
apt-get install command, 91
arguments (in command line), 22
arrays, 122–124
asterisk (*) as wildcard, 38
auto indentation in vim, 132
awk utility, 105–107

B
B command, 71
b command, 71
bash shell. See shell
/bin directory, 30
bluefish editor, 82
branches (Git), 169–171
comparing files, 176–177

t.me/DevTwitter
 creating, 169
merging files, 178–182
pushing, 171
removing, 179
switching, 169–170
break command, 116, 137
bzip2 command, 60–61

C
C programming language, 107–108
packages, 142–145
system libraries, 139–142
C++ programming language, 108
packages, 142–145
system libraries, 139–142
capitalize method, 134
case statements
in bash shell scripting, 115, 117
in Perl, 126
cat command, 49–50
cd command, 32–33
change directory process, 31
changing permissions, 58–59
chmod command, 58–59
choosing
Linux distribution, 14–15
repository host, 156–157
cloning, 155
closed source, 4–5
cmp command, 54
colon (:) command, 76
command line
aliases, 56

t.me/DevTwitter
 benefits of, 47–48
comparing files, 54–55
cmp command, 54
diff command, 54–55
components of, 22
compressing files, 59–62
bzip2 command, 60–61
gzip command, 60
tar command, 61–62
documentation, 22–27
/usr/share/doc directory, 27
help command, 22–23
info command, 26–27
man pages, 23–26
finding files, 52–54
find command, 52–54
locate command, 52
grep command, 62–66
finding files, 65–66
origin of name, 62
regular expressions, 63–65
history, 56–57
logging in via, 16–17
number of commands, 48
permissions, 57–59
changing, 58–59
files versus directories, 58
viewing, 57
redirection, 41–45
shell, 21
terminal windows, 21
variables, 55–56
vi editor. See vi editor

t.me/DevTwitter
 viewing files, 48–52
cat command, 49–50
file command, 48–49
head command, 51
less command, 50–51
more command, 50–51
tail command, 51
wc command, 51–52
wildcards, 38–40
comments
in bash shell scripting, 111
in Perl, 119
commercial software, 4–5
comparing files, 54–55
branches, 176–177
cmp command, 54
diff command, 54–55
git diff command, 173–177
staged versus committed files, 175
white space, 175–176
comparison operators in Python, 137
compiled languages, 107
C programming language, 107–108
C++ programming language, 108
Java, 108, 145–146
packages, 142–145
Debian, 145
RPM, 142–145
scripting languages versus, 97
system libraries, 139–142
managing, 140–141
viewing, 142
compressing files, 59–62

t.me/DevTwitter
 bzip2 command, 60–61
gzip command, 60
tar command, 61–62
conditional expressions
in bash shell scripting, 113–115
in Perl, 126–128
in Python, 137
configuring Git, 157–162
connecting to repositories, 186–187
continue command, 116, 137
copying
files, 37–38
in vi editor, 73
copyleft, 8
cp command, 37–38
credential helper, 186

D
d$ command, 74
DAG (Directed Acyclic Graph), 153
data structures. See variables
dd command, 74
Debian, 14, 145
debugger (Perl), 120–121
del method, 135
deleting
directories, 36–37
user accounts, 93
in vi editor, 74
desktops, 18–21
df command, 87
dG command, 74
dictionaries, 133, 135. See also hashes

t.me/DevTwitter
 diff command, 54–55
Directed Acyclic Graph (DAG), 153
directories
creating, 36
-d option (ls command), 40
deleting, 36–37
listing files in, 33–35
managing, 36–37
most used, 30
naming conventions, 30–31
permissions, 58
root, 29
wildcards, 38–40
working, 31
disk usage, displaying, 86–88
display manager, 15–16
Distributed Version Control Systems (DVCSs), 151
distribution. See Linux distribution
distro. See Linux distribution
documentation, 22–27. See also resources for information
/usr/share/doc directory, 27
help command, 22–23
info command, 26–27
man pages, 23–26
vi editor, 77–79
dollar sign ($) command, 71
double left square brackets ([[) command, 72
double quotes (“), 124
double right arrow (>>) in redirection, 41
double right square brackets (]]) command, 72
dpkg command, 88
dpkg -l command, 90
DVCSs (Distributed Version Control Systems), 151

t.me/DevTwitter
 dw command, 74

E
E command, 71
e command, 71
echo command, 55, 112
ed editor, 67–68
editors
bluefish, 82
ed, 67–68
Emacs, 79–80
gedit, 81
joe, 81
kwrite, 81
lime, 82
nano, 81
vi. See vi editor
vim, 68–69
egrep command, 116
else statement, 137
elsif statement, 125
Emacs editor, 79–80
environment variables, 56
executables, 23
executing
bash shell scripts, 112
Perl code, 120–121
Python code, 132–133
exit command, 17, 85
extend method, 135

F
file command, 48–49
file locking, 149

t.me/DevTwitter
 file test comparisons
in bash shell scripting, 115
in Perl, 127–128
files
comparing, 54–55
branches, 176–177
cmp command, 54
diff command, 54–55
git diff command, 173–177
staged versus committed files, 175
white space, 175–176
compressing, 59–62
bzip2 command, 60–61
gzip command, 60
tar command, 61–62
copying, 37–38
finding, 52–54
find command, 52–54
grep command, 65–66
locate command, 52
hidden, 34
ignoring in Git, 166–167
listing, 33–35
managing, 37–45
merging, 178–182
multiple locations in Git, 165–166
naming conventions, 30–31
permissions, 58
reading
in Perl, 128–129
in Python, 137
redirection, 41–45
removing in Git, 168–169

t.me/DevTwitter
 retrieving from remote repository, 184–186
viewing, 48–52
cat command, 49–50
file command, 48–49
head command, 51
less command, 50–51
more command, 50–51
tail command, 51
wc command, 51–52
wildcards, 38–40
filesystem. See also directories; files
defined, 29
listing files in, 33–35
most used directories, 30
naming conventions, 30–31
navigating, 31–33
find command, 52–54, 94
finding
files, 52–54
find command, 52–54
grep command, 65–66
locate command, 52
software, 89–90
text in vi editor, 75
first generation version control software, 149
flags (in command line), 22
flow control statements
in bash shell scripting, 115–117
in Perl, 124–126
in Python, 136–137
foreach statement, 123, 125
for loops
in bash shell scripting, 115, 116

t.me/DevTwitter
 in Perl, 125
in Python, 136
forward slash (/)
command, 75
in paths, 29
FOSS (Free and Open Source Software), 6
Four Freedoms, 6
free software, 5–7
Free Software Foundation (FSF), 6
functions
in Perl, 129
in Python, 138

G
G command, 72
gedit editor, 81
Git
branches, 169–171
choosing repository host, 156–157
comparing files, 173–177
branches, 176–177
staged versus committed files, 175
white space, 175–176
configuring, 157–160, 161–162
connecting to repositories, 186–187
git status command, 162–165
ignoring files, 166–167
installing, 154–155
managing repositories, 183–184
merging files, 178–182
multiple file locations, 165–166
patching, 187
removing files, 168–169

t.me/DevTwitter
 resources for information, 162
retrieving content from remote repository, 184–186
stages, 155–156
git add command, 158, 162
git apply command, 187
git branch command, 169
git checkout command, 169
git clone command, 184
git commit command, 158, 162, 185
git config command, 161–162
git diff command, 173–177, 187
branches, 176–177
staged versus committed files, 175
white space, 175–176
git difftool command, 177
git help command, 162
git log command, 170
git merge command, 178
git mergetool command, 181
git push command, 159, 185
git remote add command, 183
git remote command, 183
git rm command, 168–169
git status command, 162–166
git-all package, 155
given statement, 126
grep command, 62–66, 89
finding files, 65–66
origin of name, 62
regular expressions, 63–65
groupadd command, 93
groupdel command, 94
groups

t.me/DevTwitter
 explained, 93
managing, 93–94
GUI
desktops, 18–21
logging in via, 15–16
man page viewer, 26
vim editors in, 81
gzip command, 60

H
H command, 72
h command, 71
hashes, 122, 124. See also dictionaries
head command, 51
help command, 22–23
help documentation. See documentation
hidden files, listing, 34
history command, 56–57
/home directory, 30
HTTPS, connecting to repositories, 186–187

I
I command, 69
i command, 69
id command, 84
IDEs (Integrated Development Environments), 108–109
if statements
in bash shell scripting, 113
in Perl, 124–125
in Python, 136
ignoring files in Git, 166–167
immutable, 135
info command, 26–27
insert method, 135

t.me/DevTwitter
 insert mode (vi editor), 69–70
installing
Git, 154–155
Java, 145–146
Linux distribution, 13–14
software, 90–91
integer comparisons in bash shell scripting, 115
Integrated Development Environments (IDEs), 108–109
international licenses, 8
interpreted languages. See scripting languages

J
j command, 71
Java, 108
installing, 145–146
packages, 142–145
system libraries, 139–142
JavaScript, 103–104
joe editor, 81

K
k command, 71
kernel, 13
keys command, 124
kwrite editor, 81

L
L command, 72
l command, 71
last line mode, 76
last statement, 126
ldd command, 142
left arrow (<) in redirection, 43
left curly brace ({) command, 72

t.me/DevTwitter
 left parenthesis (() command, 72
len function, 134
less command, 50–51
libraries. See system libraries
licensing, open source
benefits of, 7
categories of, 8
examples, 8–9
resources for information, 9
terminology, 8
lime editor, 82
links. See resources for information
Linux distribution
choosing, 14–15
defined, 13
installing, 13–14
Linux systems, logging in, 15–18
via command line, 16–17
via GUI, 15–16
via network, 17–18
listing
files, 33–35
software, 89–90
listings
all files up to date, 164–165
case statement example, 117
cat -n command, 49–50
deleting file from working directory, 168
features branch, 178
git diff between branches, 176
head command, 51
if statement example, 113
install rpm-build, 143–144

t.me/DevTwitter
 install source RPM, 144–145
ldd command, 142
matching with grep, 63
merged file, 180
numbers versus strings, 134
the - option of su command, 85
read statement example, 118
scalar statements, 122–123
searching with grep, 65–66
set command, 55
single difference, 174–175
staged versus committed difference, 175
staging file to be deleted, 168–168
testing user input, 114–115
three versions of file, 165
while loop, 125
white space mystery, 175–176
yum install command, 90
lists, 133, 134–135
locate command, 52
logging into Linux systems, 15–18
via command line, 16–17
via GUI, 15–16
via network, 17–18
loops
in bash shell scripting, 115–117
in Perl, 124–126
in Python, 136–137
ls command, 33–35
-d option, 40
-l option, 57

t.me/DevTwitter
 M command, 72
man 5 passwd command, 91
man command, 23–26
man pages, 23–26
managing
directories, 36–37
files, 37–45
filesystem, 33–35
groups, 93–94
software, 88
/media directory, 30
merge process, 150–151
merging files, 178–182
mkdir command, 36
modifiers (vi editor), 72
modifying user accounts, 92
modules
in Perl, 129–130
in Python, 138
more command, 50–51
movement commands (vi editor), 71–72
multiple file locations in Git, 165–166
mv command, 38
my statement, 129

N
N command, 75
n command, 75
naming conventions for files and directories, 30–31
nano editor, 81
navigating filesystem, 31–33
network, logging in via, 17–18
newline characters, 119

t.me/DevTwitter
 next statement, 126
nonreusable licenses, 8
numeric comparisons in Perl, 126–127
numeric variables, 133–134

O
O command, 69
o command, 69
octal method (changing permissions), 58–59
open source
defined, 5
free software, 5–7
licensing
benefits of, 7
categories of, 8
examples, 8–9
resources for information, 9
terminology, 8
open statement, 137
options (in command line), 22

P
P command, 74
p command, 74
packages, 142–145
Debian, 145
RPM, 142–145
paging, 44
passwd command, 92
passwords, complexity of, 92
pasting in vi editor, 74
patch output, 174
patching, 186, 187
paths, 29

t.me/DevTwitter
 for bash shell scripts, 112
types of, 31–33
Perl, 99–100, 119–130
comments, 119
conditional expressions, 126–128
executing code, 120–121
flow control statements, 124–126
functions, 129
modules, 129–130
print statements, 119
reading files, 128–129
resources for information, 121–122
as unstructured, 119
variables, 122–124
perl command, 120
perldoc command, 121–122
permissions, 57–59
changing, 58–59
files versus directories, 58
viewing, 57
permissive permissiveness, 8
permissiveness, 8
PHP, 102–103
pipe (|) in redirection, 44–45
piping, 44–45
pop method, 135
pop statement, 123
positional parameter variables, 112
pragmas in Perl, 130
print statements, 119, 132
private variables, 129
programming languages
compiled languages, 107

t.me/DevTwitter
 C programming language, 107–108
C++ programming language, 108
Java, 108, 145–146
packages, 142–145
system libraries, 139–142
IDEs, 108–109
scripting languages
awk utility, 105–107
bash shell, 98–99, 111–118
comparing, 99
compiled languages versus, 97
JavaScript, 103–104
Perl, 99–100, 119–130
PHP, 102–103
Python, 100–101, 131–138
Ruby, 102
sed utility, 105–107
Tcl/Tk, 104–105
proprietary software, 4–5
push (Git), 156, 159
push statement, 123
pushing branches (Git), 171
pwd command, 32
.pyc files, 132–133
.pyo files, 132–133
Python, 100–101, 131–138
conditional expressions, 137
executing code, 132–133
flow control statements, 136–137
functions, 138
modules, 138
reading files, 137
resources for information, 133

t.me/DevTwitter
 as structured language, 131–132
variables, 133–136
python command, 132

Q
qq statement, 124
question mark (?)
command, 75
as wildcard, 39
quitting vi editor
and saving, 76–77
without saving, 73, 77
quoting variables, 114
qw statement, 124

R
read() method, 137
read statements, 117–118
reading files
in Perl, 128–129
in Python, 137
readline method, 137
Red Hat, 14
redirection, 41–45
regular expressions
in bash shell scripting, 63–65
in Perl, 128
relative paths, 31–33
remote repository. See repositories
remove method, 135
removing. See also deleting
branches in Git, 179
files in Git, 168–169
software, 91

t.me/DevTwitter
 repeater modifiers (vi editor), 72
replacing text in vi editor, 75–76
repositories
connecting via SSH, 186–187
managing, 183–184
retrieving content from, 184–186
repository host, choosing, 156–157
resources for information. See also documentation
bash shell scripts, 118
Git, 162
open source licensing, 9
Perl, 121–122
Python, 133
right arrow (>) in redirection, 41–42
right curly brace (}) command, 72
right parenthesis ()) command, 71
rm command, 36–37, 38
rmdir command, 36
/root directory, 30
root directory, 29
root user, accessing, 84–86
logging in directly, 83
with su command, 84–85
with sudo command, 85–86
rpm command, 88
RPM packages, 142–145
rpm -qa command, 90
Ruby, 102
running. See executing

S
s command, 69
saving in vi editor, 76–77

t.me/DevTwitter
 /sbin directory, 30
scalars, 122–123
scripting languages
awk utility, 105–107
bash shell, 98–99, 111–118
comments, 111
conditional expressions, 113–115
echo command, 112
executing code, 112
file test comparisons, 115
flow control statements, 115–117
integer comparisons, 115
positional parameter variables, 112
resources for information, 118
sha-bang (#!), 111
user interaction, 117–118
comparing, 99
compiled languages versus, 97
JavaScript, 103–104
Perl, 99–100, 119–130
comments, 119
conditional expressions, 126–128
executing code, 120–121
flow control statements, 124–126
functions, 129
modules, 129–130
print statements, 119
reading files, 128–129
resources for information, 121–122
as unstructured, 119
variables, 122–124
PHP, 102–103
Python, 100–101, 131–138

t.me/DevTwitter
 conditional expressions, 137
executing code, 132–133
flow control statements, 136–137
functions, 138
modules, 138
reading files, 137
resources for information, 133
as structured language, 131–132
variables, 133–136
Ruby, 102
sed utility, 105–107
Tcl/Tk, 104–105
searching. See finding
second generation version control software, 149–151
sed utility, 105–107
selecting. See choosing
semicolon (;) in Perl statements, 119
set command, 55
sets, 136
sha-bang (#!)
in bash shell scripting, 111
in Perl, 120
in Python, 132
shared library calls, 139
shell
defined, 21
scripting, 98–99, 111–118
comments, 111
conditional expressions, 113–115
echo command, 112
executing code, 112
file test comparisons, 115
flow control statements, 115–117

t.me/DevTwitter
 integer comparisons, 115
positional parameter variables, 112
resources for information, 118
sha-bang (#!), 111
user interaction, 117–118
variables, 55–56
shift statement, 123
single quotes (‘), 124
sl package, 94
Slackware/SUSE, 14
software
installing, 90–91
listing and finding, 89–90
managing, 88
removing, 91
sort statement, 123
source code
closed source, 4–5
defined, 3–4
open source
benefits of, 7
defined, 5
free software, 5–7
licensing categories, 8
licensing examples, 8–9
resources for information, 9
terminology, 8
version control, 147
choosing Git repository host, 156–157
comparing files, 173–177
configuring Git, 157–160, 161–162
connecting to repositories, 186–187
first generation software, 149

t.me/DevTwitter
 Git branches, 169–171
Git stages, 155–156
git status command, 162–165
ignoring files, 166–167
installing Git, 154–155
managing repositories, 183–184
merging files, 178–182
multiple file locations, 165–166
patching, 187
removing files, 168–169
resources for information, 162
retrieving content from remote repository, 184–186
second generation software, 149–151
third generation software, 151–154
special purpose licenses, 8
special variables in Perl, 124
splice statement, 123
square brackets ([]) as wildcard, 39
SSH, connecting to repositories, 186–187
ssh-agent, 186
stages (Git), 155–156
staging area (Git), 156
Stallman, Richard, 6
standard licenses, 8
static library calls, 139
stderr (standard error), 41
redirecting, 42–43
stdin (standard input), 41
redirecting, 43–44
stdout (standard output), 41
redirecting, 41–42
strict permissiveness, 8
string comparisons in Perl, 127

t.me/DevTwitter
 string variables, 133–134
strong copyleft, 8
structured languages, Python as, 131–132. See also compiled languages
su command, 84–85
sudo command, 84, 85–86
switch statements in Perl, 126
switching branches (Git), 169–170
symbolic method (changing permissions), 58–59
system administration
disk usage, displaying, 86–88
groups
explained, 93
managing, 93–94
root user
accessing, 84–86
logging in directly, 83
software
installing, 90–91
listing and finding, 89–90
managing, 88
removing, 91
user accounts, 91
adding, 91–92
deleting, 93
modifying, 92
system libraries, 139–142
managing, 140–141
viewing, 142

T
tail command, 51
tar command, 61–62
Tcl/Tk, 104–105

t.me/DevTwitter
 terminal windows, 21
test command, 113–115
text
finding in vi editor, 75
replacing in vi editor, 75–76
text editors. See editors
third generation version control software, 151–154
/tmp directory, 30
touch command, 38
tuples, 135
two right arrow (2>) in stderr redirection, 42–43

U
u command, 73
undoing actions (vi editor), 73
unless statement in Perl, 125
unshift statement, 123
unstructured languages, Perl as, 119
until loops
in bash shell scripting, 115
in Perl, 125
user accounts, 91
adding, 91–92
deleting, 93
modifying, 92
user interaction in bash shell scripting, 117–118
useradd command, 91
userdel command, 93
usermod command, 92, 93
/usr/bin directory, 30
/usr/sbin directory, 30
/usr/share/doc directory, 27

t.me/DevTwitter
 variables
environment, 56
in Perl, 122–124
positional parameters, 112
private, 129
in Python, 133–136
quoting, 114
shell, 55–56
version control, 147
first generation software, 149
Git
branches, 169–171
choosing repository host, 156–157
comparing files, 173–177
configuring, 157–160, 161–162
connecting to repositories, 186–187
git status command, 162–165
ignoring files, 166–167
installing, 154–155
managing repositories, 183–184
merging files, 178–182
multiple file locations, 165–166
patching, 187
removing files, 168–169
resources for information, 162
retrieving content from remote repository, 184–186
stages, 155–156
second generation software, 149–151
third generation software, 151–154
vi editor
benefits of, 68
copying in, 73
deleting in, 74

t.me/DevTwitter
 documentation, 77–79
finding text, 75
history of, 67–68
insert mode, 69–70
movement commands, 71–72
pasting in, 74
quitting
and saving, 76–77
without saving, 73, 77
repeater modifiers, 72
replacing text, 75–76
undoing actions, 73
viewing
files, 48–52
cat command, 49–50
file command, 48–49
head command, 51
less command, 50–51
more command, 50–51
tail command, 51
wc command, 51–52
permissions, 57
system libraries, 142
vim editor, 68–69
auto indentation, 132
vimdiff tool, 181–182
vimtutor command, 79
visudo command, 86

W
W command, 71
w command, 71
Wall, Larry, 130

t.me/DevTwitter
 wc command, 51–52
weak copyleft, 8
websites. See resources for information
while loops
in bash shell scripting, 115, 116
in Perl, 125
in Python, 136
white space, comparing files, 175–176
wildcards, 38–40
working area (Git), 156
working directory, 31
write method, 137

X
X command, 74
x command, 74
xG command, 72
xman command, 26

Y
y$ command, 73
yanking, 73
yG command, 73
yum command, 88
yum install command, 90
yum list installed command, 89
yum search command, 89
yw command, 73
yy command, 73

Z
zero (0) command, 71
zypper command, 88
zypper install command, 91

t.me/DevTwitter
t.me/DevTwitter
 Code Snippets
Many titles include programming code or configuration examples. To
optimize the presentation of these elements, view the eBook in single-
column, landscape mode and adjust the font size to the smallest setting. In
addition to presenting code and configurations in the reflowable text format,
we have included images of the code that mimic the presentation found in
the print book; therefore, where the reflowable format may compromise the
presentation of the code listing, you will see a “Click here to view code
image” link. Click the link to view the print-fidelity code image. To return
to the previous page viewed, click the Back button on your device or app.

t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter
t.me/DevTwitter