Process groups, jobs and sessions

A new process group is created every time we execute a command or a pipeline of commands
in a shell. Inside a shell, a process group is usually called a job. In its turn, each process
group belongs to a session. Linux kernel provides a two-level hierarchy for all running
processes (look at Image 3 below). As such, a process group is a set of processes, and a
session is a set of related process groups. Another important limitation is that a process group
and its members can be members of a single session.

$ sleep 100 # a process group with 1 process

$ cat /var/log/nginx.log | grep string | head # a process group with 3 processes

Process groups

A process group has its process group identificator PGID and a leader who created this group.
The PID of the group leader is equal to the corresponding PGID. As so, the type
of PID and PGID are the same, and is (pid_t)[https://ftp.gnu.org/old-gnu/Manuals/glibc-
2.2.3/html_node/libc_554.html]. All new processes created by the group members inherit
the PGID and become the process group members. In order to create a group, we
have setpgid() and setpgrp() syscalls (man 2 getpgrp()).

A process group lives as long as it has at least one member. It means that even if the group
leader terminates, the process group is valid and continues carrying out its duties. A process
can leave its process group by:

 joining another group;

 creating its own new group;

 terminating.

Linux kernel can reuse PIDs for new processes if only the process group with
that PGID doesn’t have members. It secures a valid hierarchy of processes.

A leader of a process group can’t join another process group because the constraints
between PID of the process and PGID of the members of the group will be violated.

Two interesting features of process groups are:

1. a parent process can wait() for its children using the process group id;

2. a signal can be sent to all members of a process group by using killpg() or kill() with a
negative PGID parameter.
The below command sends a SIGTERM(15) to all members of the process group 123:

$ kill -15 -123

Sessions

For its part, a session is a collection of process groups. All members of a session identify
themselves by the identical SID. It’s also the pid_t type, and as a process group, also
inherited from the session leader, which created the session. All processes in the session share
a single controlling terminal (we’ll talk about this later).

A new process inherits its parent’s session ID. In order to start a new session a process should
call setsid() (man 2 setsid). The process running this syscall begins a new session, becomes
its leader, starts a new process group, and becomes its leader too. SID and PGID are set to the
process’ PID. That’s why the process group leader can’t start a new session: the process
group could have members, and all these members must be in the same session.

Basically, a new session is created in two cases:

1. When we need to log in a user with an interactive shell. A shell process becomes a
session leader with a controlling terminal (about this later).

2. A daemon starts and wants to run in its own session in order to secure itself (we will
touch daemons in more detail later).

The following image shows a relationship between a session, its process groups and

processes.
Image 3. – 2 level hierarchy of processes

 ❶ – Session id (SID) is the same as the session leader process (bash) PID.

 ❷ – The session leader process (bash) has its own process group, where it’s a leader,
so PGID is the same as its PID.

 ❸, ❹ – The session has 2 more process groups with PGIDs 200 and 300.

 ❺, ❻ – Only one group can be a foreground for a terminal. All other process groups
are background. We will touch on these terms in a minute.

 ❼, ❽, ❾ – All members of a session share a pseudoterminal /dev/pts/0.

In order to get all the above information for a running process, we can read
the /proc/$PID/stat file. For example, for my running bash shell $$ porcess:

$ cat /proc/$$/stat | cut -d " " -f 1,4,5,6,7,8 | tr ' ' '\n' | paste <(echo -ne
"pid\nppid\npgid\nsid\ntty\ntgid\n") -

pid 8415 # PID

ppid 8414 # parent PID

pgid 8415 # process group ID

sid 8415 # sessions ID

tty 34816 # tty number

tgid 9348 # foreground process group ID

 pid – the process id.

 ppid – the PID of the parent of this process.

 pgrp – the process group id of the process.

 sid – the session id of the process.

 tty – the controlling terminal of the process. (The minor device number is contained in
the combination of bits 31 to 20 and 7 to 0; the major device number is in bits 15 to
8.)

 tgid – the id of the foreground process group of the controlling terminal of the
process.
Streams and Pipes
Like plumbing pipes where you can connect and disconnect sections to redirect water to
different places, so too can you connect and disconnect streams of data.

The Standard Streams

There are three standard streams, stdin (said standard input or standard in,) stdout (said
standard output or standard out,) and stderr (said standard error or standard err.)

stdin
stdin is an input stream to a program and the other two are output streams. stdout for all non-
error text, the normal output. stderr is just for error information.

stdout

stdout is where all normal output goes. If it's not caught or redirected, stdout by default goes
to your terminal screen (as does stderr too.) A good example of that is our friend cat. cat take
a file and concatenates it to the stdout. If you do cat file1.txt and don't redirect that stream
somewhere else, it ends up at the terminal screen and you get the ouptut of the file. What if
we wanted to redirect the output? We can!

echo "this will get output to the file and not to stdout" 1> new-file.txt

The 1> redirect stdout from heading to the terminal and into a file, new-file.txt. We don't see
the output of echo; it's been redirected. So what if we wanted to do that to cat?

cat new-file.txt 1> yet-new-file.txt

We concatenated new-file.txt to stdout and then redirected stdout to a file. So basically we

did cp with more steps and concepts. But hey! Now you can see the power of piping stdout to
different places.

Replacing vs appending

So far we've been replacing files with 1>. What if we want to append? That is to say, instead
of replacing the file's contents, we just want to add new stuff to the end? That's
where 1>> comes in (this will work with all streams, not just 1 / stdout.)

ls -lsah 1> ls.txt

ls -lsah 1>> ls.txt

It's two outputs of ls -lsah.

stderr
While 1> redirects stdout, 2> redirect stderr. So if I say ls -lsah 2> error-log.txt what would
you expect to happen? You'll see the normal output of stdout to the terminal since we didn't
redirect it and it will create a new file called error-log.txt with nothing in it.

do something that causes an error then. Run cat file-that-doesnt-exist.txt 2> error-
log.txtNothing gets output since nothing is concatenated to stdout (since that file doesn't
exist) and cat: file-that-doesnt-exist.txt: No such file or directory gets output to error-log.txt.

All the same things apply too with 2> for replacing a file and 2>> for appending.

Note, so far everything we've done has had either error information or normal info and not
both. Normally it'll have both.

Redirecting both stderr and stdout

ls -lsah 1> stdout.txt 2> stderr.txt

This will redirect each of those streams to different files.

We can have them go to the same file too!

ls -lsah 1> ls.txt 2> ls.txt

Or, even more easily,

ls -lsah > ls.txt

Yes, if you put no number there, it will output both streams to the same file. /dev/null

Sometimes you want to run a program and you don't really care what the output is; you just
want to run the program. Say hello to /dev/null which is the programming equivalent of the
infinite abyss. Anything that gets output to /dev/null is thrown away. Let's say you're running
a program that's very noisy and you really only care if there's an error.

ls -lsah 1> /dev/null # assume this is a very noisy program

This will run the command and only print the errors. Everything else gets chucked into the
infinite abyss. Useful sometimes

stdin

stderr and stdout direct the text from a program to a file. With stdin, we can direct the
contents of a file into a program via the stdin. Try this

cat < ls.txt

But let's say it's a very long file and we want to find one very specific line. We could do this:

grep "error-log.txt" < ls.txt

Using stdin and stdout

What if we want to have both stdin and stdout and then throw away the errors?

grep "error-log.txt" < ls.txt 1> ls2.txt 2> /dev/null

A good example is if I'm running a web server from a computer: I want the output from the
server (like who logged in, anayltics, metrics, security stuff, etc.) to live in one place so I can
keep track of the logs and I want the error logs to live in another place so I can debug
successfully what's going on with my server. In this case, stderr and stdout are very useful.
What if I need to to input some secrets like passwords and cache keys to a server in order to
start it up? stdin is definitely one way to do that.

Pipes
if we want to use the output of one program into another? Enter the pipe (sometimes called
vertical bar,) |. This takes what one program outputs and puts it into the next one. This opens
up a lot of possibilites. Let's redo that one we had above with grep using cat and |.

cat ls.txt | grep "error-log.txt"

cat will concatentate ls.txt to stdout and then | will take the output of that and run that as stdin
to grep.

It's usually a very long list since Linux has a lot running all the time. Try running ps aux and
see how long it is. It can be much longer too if you're running a server. Notice the last thing it
outputs is the ps aux command itself that you used to find it.

Let's use grep to find just that line and nothing else. Try this:

ps aux | grep "ps aux"

This should output two lines, the ps aux call and the grep we're running to find that ps aux. A
little self referential but the point here is that we're able to find just what we need and leave
the rest behind. And we're doing that with the power of pipes. ps aux find all processes and
outputs that to stdout. We then take that stdout and run that as the stdin to grep. grep then
finds just the lines it needs and outputs just those to its stdout. At this point we don't have
anything else redirecting output streams so it gets output to the terminal window. We
absolutely could redirect that out to a file using 1>.

If you do rm -i *.txt, it'll try to remove all files with .txt extensions. It'll all confirm with you
on each one to say either y for yes or n for no. Try it and say "n" and hit enter for each one.
Notice afterwards you won't have deleted anything.

yes n | rm -i *.txt

The first command, yes n outputs infinite ns to stdout. rm -i *.txt uses those from stdin to
answer n to every question it asks.
tee command in Linux with examples

tee command reads the standard input and writes it to both the standard output and one or
more files. The command is named after the T-splitter used in plumbing. It basically breaks
the output of a program so that it can be both displayed and saved in a file. It does both the
tasks simultaneously, copies the result into the specified files or variables and also display the
result.

SYNTAX:

tee [OPTION]... [FILE]...

Options :
1.-a Option : It basically do not overwrite the file but append to the given file.
Suppose we have

file1.txt

Input: geek

for

geeks

and file2.txt

Input:geeks

for

geeks

SYNTAX :

geek@HP:~$ wc -l file1.txt|tee -a file2.txt

OUTPUT :

3 file1.txt

geek@HP:~$cat file2.txt

OUTPUT:

geeks

for

geeks

3 file1.txt

2.–help Option : It gives the help message and exit.

SYNTAX :

geek@HP:~$ tee --help

3.–version Option : It gives the version information and exit.
SYNTAX :

geek@HP:~$ tee --version

Application

Suppose we want to count number of characters in our file and also want to save the output to
new text file so to do both activities at same time, we use tee command.

geek@HP:~$ wc -l file1.txt| tee file2.txt

OUTPUT:

geek@HP:~$15 file1.txt

Here we have file1 with 15 characters, so the output will be 15 and the output will be stored
to file2. In order to check the output we use :

geek@HP:~$ cat file2.txt

OUTPUT:

geek@HP:~$15 file1.txt
Command-line text editors in Linux

There are two command-line text editors in Linux®: vim and nano.

You can use one of these two available options should you ever need to write a script, edit a
configuration file, create a virtual host, or jot down a quick note for yourself. These are but a
few examples of what you can do with these tools.

While these tools might seem daunting at first, any Linux user should become accustomed to
using at least one. Ask any Linux Administrator or regular user, and you soon find that
everyone has their favorite.

This article covers the basics of each editing tool and how they differ.

Vim

Vim comes from Vi Improved because it is the successor of an older editor called vi. Because
this editor (through its predecessor) has been around a long time, Linux Administrators or
coders usually favor it. Vim is typically used by people who are familiar with Linux because
it can have a bit of an uphill learning curve.

You can use vim to edit, write, and save a document from the command line. It does this
through the use of two different modes:

 Command

 Insert

By default, the vim editor opens in command mode. To open the vim editor, use the
following syntax at the command line:

$ vim (name of the file)

or

$ vim (full path of the file)

To start writing or editing, you must enter insert mode by pressing the letter i on your
keyboard (“I” for insert). You should see ---INSERT--- at the bottom of your terminal page if
you did it correctly.

When you are finished typing, and you want to save your work, you need to exit insert mode.
Press the escape (esc) key, which places you back in command mode. Then you can save
your work.

After you press escape, press shift + ;. The bottom of your terminal screen changes to reflect
that you did it correctly. You now see a : where the ---INSERT--- was.

After you see the : in the lower left-hand corner of your vim editor, type w and then
press enter to save your work. Then, you can either type i again to go back into insert mode
if you want to continue writing, or you can quit the file. To quit, press shift + ; again,
type q and then press enter. This saves your file and closes vim. You should see your usual
terminal screen again.

You can also enter both the save and quit functions at the same time. To save and quit vim in
one command, type wq after the : and then press enter. The file saves and closes.

If you start working on a file, but you change your mind, you can exit without saving. To do
this, enter command mode by pressing esc followed by shift + ;. After you see the : at the
lower left, enter q!. This force-quits vim without saving. ! is the force function.

Those commands are the ones that you are going use most of the time, but you can use the
following cheat sheet if you want to do more complex actions with vim.

Vim editor cheat sheet

Use the following commands in command mode:

 h - Moves the cursor to the left by one character; you can also press the left arrow.

 j - Moves the cursor one line down; you can also press the down arrow.

 k - Moves the cursor one line up; you can also press the up arrow.

 l - Moves the cursor to the right by one character; you can also press the right arrow.

 w - Moves the cursor one full word to the right.

 b - Moves the cursor one full word to the left.

 0 - Moves the cursor to the beginning of the current line.

 $ - Moves the cursor to the end of the current line.

 ~ - Changes the case of the current character.

 dd - Deletes the current line.

 D - Deletes everything on the line to the right of the cursor’s current position.

 x - Deletes the current character.

 u - Undo the last command.

 . - Repeats the last command.

 :w - Saves current file, but does not exit.

 :wq - Saves current file, and quits.

The following commands place you into insert mode:

 i - Inserts to the left of the current cursor position.

 a - Appends to the right of the current cursor position.

 dw - Deletes the current word.

 cw - Changes the current word.

Nano

Nano is a newer text editor in Linux systems. It’s simpler and easier to use than vim.

To open a file with nano, use the following syntax at the command line:

$ nano (name of the file)

or

$ nano (full path of the file)

After the nano editor opens, you can begin typing. When you’re ready to save your work,
press ctrl + o, which is called a write out. It saves your current work while allowing you to
continue your work. If you’re done, you can save and quit by pressing ctrl + x. When you
save a file in nano, your current work is color-coded based on what you’re writing.

Another major difference with nano is that you can access a list of commands within the
editor, but you can use the following cheat sheet as well.

Nano editor cheat sheet

Note: The commands in the following list use ^ to indicate that you should press the ctrl key
along with the other key. For example ^G means that you should press ctrl + G.

 ^G - Get Help.

 ^X - Exit. Nano then asks if you want to save with a Y or N option.

 ^O - Write Out; also known as save.

 ^R - Read File. Enter the name of a file you want to paste into the current document at
your cursor’s position.

 ^W - Where Is; Search function.

 ^\ - Replace.

 ^K - Cut text.

 ^U - Uncut text.
 ^J - Justify.

 ^T - To spell.

 ^C - Current Position; Cancel save.

 ^_ - Go to line.

 Command Substitution:
 Command substitution is a mechanism that is followed by programmers in a shell
script. In this mechanism, the output of a command replaces the command itself. Shell
operates the expansion by executing a command and then replacing the command
substitution with the standard output of the command. In simple words, the output of a
UNIX command is bundled and then used as a command.
 To understand it in a better way, let us consider an example. The seq command in
Linux is used to print numbers from START to END in steps of INCREMENT.
 Syntax:
 seq START INCREMENT END
 Return type:
 Prints numbers from START to END each in the new line by the difference of
INCREMENT.
 Example:
 In the below script we are printing numbers from 2 to 40 with a difference of 2. In
other words, we are printing even numbers up to 40.
 #!/bin/sh

 # your code goes here

 seq 2 2 40
 Output:


 We can use the output of the above command as a new command. Consider the below
script,
 Example:
 #!/bin/sh
 echo $(seq 2 2 40)
 Output:

 Variables and command expansion:

 During the process of command substitution, the output of the command can be
assigned to a variable, just like any other value.
 Example:
 In the below script we have assigned the result of the echo command to both the
strings in variables, “variable1” and “variable2” respectively. Then we have used
these variables in the echo command.
 #!/bin/sh
 variable1=$(echo 'Full form of gfg is' )
 variable2=$(echo 'GeekforGeeks')
 echo "$variable1 : $variable2"

 Output:

 Loss of newlines in command substitution:

 In the command substitution mechanism, If the output of a command that is being
substituted contains any trailing newlines, then in that case the trailing newlines are
deleted after the substitution. Note that embedded newlines are not deleted, but they
might be removed during word splitting.
 Example:
 In this script, we are using the seq command. Now seq command prints numbers to
the console and appends a newline character after each number is printed to the
console.
 #!/bin/sh
seq 1 2 19
 Output:
 Example:

 In this script, we have used the result of the above command and substituted it in the
command. As you can see in the output, numbers from 1 to 39 with the difference of 2
are printed to console. But this time they all are printed on the same line that is
without any newline character.
#!/bin/sh
echo $(seq 1 2 39)
Output: