Introduction to

Linux
Shak Hossain
Topics for Today

● Linux Overview
● Linux Interaction - Shell and Commands
● I/O redirection (pipes, etc.)
● Navigating the file system
● Processes and job control
● Editors
● Creating and Running Code
About Me

● Shak Hossain
● Adjunct Professor
How about you?

● Who has experience programming?

● Using Linux?
● Using compute clusters?
● Using the Shared Computing Cluster (SCC)?
Linux
What, Who, When, Where & Why
What is Linux?

● Unix-like computer operating system assembled under the

model of free and open-source software development and
distribution.
● These operating systems share the Linux kernel.
○ Typically have the GNU utilities
● Comes in several “distributions” to serve different purposes.
What is Linux? gcc

wc bash emacs
● Bird’s eye view
multitasking

tcsh
cat device file grep
access Hardware system
sh

Kernel awk
sort

Shell

Utilities
Who is Linux?

Linus Torvalds Richard Stallman

Linux is an O/S core originally written A set of programs written by Richard
by Linus Torvalds. Now almost 10,000 Stallman and others. They are the GNU
developers including major technology utilities.
companies like Intel and IBM.
When is Linux?

~1991
Where is Linux?

● World Wide Web

○ 67% of the world’s web-servers run Linux (2016)
● Research/High-Performance Compute
○ Google, Amazon, NSA, 100% of TOP 500 Supercomputers.
● Modern Smartphones and devices
○ The Android phone
○ Amazon Kindle
○ Smart TVs/Devices
Why Linux?

● Free and open-source.

● Powerful for research data centers
● Personal for desktops and phones
● Universal
● Community (and business) driven.

The most common OS used by Engineers and Researchers

when working on a server or computer cluster.
Connecting
Let’s use Linux
Local System

Remote Server
Connecting from
Different Platforms
SSH X-Win SFTP
Microsoft MobaXterm
Windows https://mobaxterm.mobatek.net

Apple Terminal (Built XQuartz Cyberduck

MacOS in) https://www.xquartz.org https://cyberduck.io

Terminal (Built Various (Built

Linux X11(Built in)
in) in)
Microsoft Windows
You need software that emulates an “X” terminal and
that connects using the “SSH” Secure Shell protocol.

● Recommended:
○ MobaXterm
http://mobaxterm.mobatek.net/

● Alternatives:
○ SSH/X-Windows: X-Win32
https://www.bu.edu/tech/services/support/desktop/distribution/xwindows/

○ SFTP: Filezilla
https://filezilla-project.org/
Apple MacOS

● SSH: Terminal
○ Built in to macOS
Applications > Utilities > Terminal

● X-Windows: XQuartz
○ Download: https://www.xquartz.org/ (Note: This install
requires a logout.)

● SFTP: Your choice

○ Filezilla: https://filezilla-project.org/
○ Cyberduck: https://cyberduck.io
○ Many others
Apple macOS Built in!
Apple macOS is built on
Darwin -- a derivative of
● SSH: Terminal 4.4BSD-Lite2 and FreeBSD
○ Built in to macOS
Applications > Utilities > Terminal

● X-Windows: XQuartz
○ Download: https://www.xquartz.org/
○ Note: This install requires a logout.

● SFTP: Your choice

○ Filezilla: (Cross-platform, open-source)
https://filezilla-project.org/ (macOS native, drag-and-
○ Cyberduck: https://cyberduck.io drop)
○ Many others
Linux
Interaction
Shell, Prompt, Commands and System Use
Linux: The Shell
● Program that interprets commands and sends them to the OS

● Provides:
○ Built-in commands
○ Programming control structures
○ Environment variables

● Linux supports multiple shells.

○ The default on SCC is Bash.
Bash” = “Bourne-again Shell”
(GNU version of ~1977 shell written by
Stephen Bourne)
Linux: The “Prompt”
Your Current
Username Directory

username@hostname ~]$

The System Inp

Name ut
( In Linux “ ~ ” is a shorthand for your home
directory. )
Linux: The “prompt”

Your Username Current Directory

username@hostname ~]$

The System Name Input

( In Linux “ ~ ” is a shorthand for your home

Linux: Command Basics
[username@scc1 ~]$ command --option argument

● Command: Command/program that does one thing

● Options: Change the way a command does that one thing
Short form: Single-dash and
one letter e.g. ls -a
Long form: Double-dash and
a word e.g. ls --all
For more information
● Argument: about
Provides the any command,
input/output use
that the man or info
command
(e.g. “man ls”)
interacts with.
Commands: Hands-On

● After you connect, type

○ whoami # my login
○ hostname # name of this computer
○ echo “Hello, world” # print characters to screen
○ echo $HOME # print environment variable
○ echo my login is $(whoami) # replace $(xx) with program output
○ date # print current time/date
○ cal # print this month’s calendar
○ shazam # bad command
Commands: Hands-On

● After you connect, type

○ whoami # my login
○ hostname # name of this computer
○ echo “Hello, world” # print characters to screen
○ echo $HOME # print environment variable
○ echo my login is $(whoami) # replace $(xx) with program output
○ date # print current time/date
○ cal # print this month’s calendar
○ shazam # bad command
Commands: Hands-On Options

● After you connect, type

○ whoami # my login
○ hostname # name of this computer
○ echo “Hello, world” # print characters to screen
○ echo $HOME # print environment variable
○ echo my login is $(whoami) # replace $(xx) with program output
○ date # print current time/date
○ cal # print this month’s calendar
○ shazam # bad command
Commands: Hands-On Options
● Commands have three parts; command, options and arguments/parameters.

Example: cal –j 3 1999. “cal” is the command, “-j” is an option (or switch), “3”
and “1999” are arguments/parameters.

[username@scc1 ~]$ cal -j 3 1999

● What is the nature of the prompt?

● What was the system’s response to the command?
Commands

“Small programs that do one thing well”

● The Unix Programming Environment, Kernighan and Pike

… at its heart is the idea that the power of a system comes more from the
relationships among programs than from the programs themselves. Many
UNIX programs do quite trivial things in isolation, but, combined with other
programs, become general and useful tools.
Commands: Selected text processing utilities
● awk Pattern scanning and processing
● cat language Display file(s)
● cut Extract selected fields of each line of a
● diff file Compare two files Just a few of the
● grep Search text for a pattern commands for
● head Display the first part of text processing
● less files
● sed Display files on a page-by-page basis
● sort Stream editor (esp. search and
● split replace) Sort text files
● tail Split files
● tr Display the last part of a file
● uniq Translate/delete characters
● wc Filter out repeated lines in a
file Line, word and character
count
Variables and Environment Variables
● Variables are named storage locations.
○ USER=augustin
○ foo=“this is foo’s value”

● “Environment variables” are variables used and shared by the shell

○ For example, $PATH tells the system where to find commands.

● Environment variables are shared with programs that the shell

runs.
Bash variables
● To create a new variable, use the assignment operator ‘=‘

[username@scc1 ~]$ foo=“this is foo’s value”

● The foo variable can be printed with

echo
[username@scc1 ~]$ echo
$foo this is foo’s value

● To make $foo visible to programs run by the shell (i.e., make it an

“environment variable”), use export:
[username@scc1 ~]$ export foo
Environment Variables
● To see all currently defined environment variable, use printenv:
[username@scc1 ~]$ printenv
HOSTNAME=scc1
TERM=xterm-256color
SHELL=/bin/bash
HISTSIZE=1000
TMPDIR=/scratch
SSH_CLIENT=168.122.9.131 37606 22
SSH_TTY=/dev/pts/191
USER=cjahnke
MAIL=/var/spool/mail/cjahnke
PATH=/usr3/bustaff/cjahnke/apps/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
PWD=/usr3/bustaff/cjahnke/linux-materials
LANG=C
MODULEPATH=/share/module/bioinformatics:/share/module/chemistry
SGE_ROOT=/usr/local/ogs-ge2011.11.p1/sge_root
Command History and Command Line Editing
● Try the history command
● Choose from the command history using the up ↑ and down ↓ arrows
● To redo your last command, try !!
● To go further back in the command history try !, then the number as shown
by history (e.g., !132). Or, !ls, for example, to match the most recent ‘ls’
command.
● What do the left ← and right → arrow do on the command line?
● Try the <Del> and <Backspace> keys
Help with Commands
● Type
○ date –-help
○ man date
○ info date

● BASH built-ins
○ A little different from other commands
○ Just type the command ‘help’
○ Or ‘man bash’

Yes, you can always Google it.

On using ‘man’ with ‘less’
● The man command outputs to a pager called less, which
supports many ways of scrolling through text:
○ Space, f # page forward
○ b # page backward
○ < # go to first line of file
○ > # go to last line of file
○ / # search forward (n to repeat)
○ ? # search backward (N to repeat)
○ h # display help
○ q # quit help
I/O Redirection
I/O redirection with pipes
● Many Linux commands print to “standard output”, which defaults to
the terminal screen. The ‘|’ (pipe) character can be used to
divert or “redirect” output to another program or filter.
○ w # show who’s logged on
○ w | less # pipe into the ‘less’ pager
○ w | grep ‘tuta’ # pipe into grep, print lines containing ‘tuta’
○ w | grep –v ‘tuta’ # print only lines not containing ‘tuta’
○ w | grep ‘tuta’ | sed s/tuta/scholar/g
# replace all ‘tuta’ with ‘scholar’
More examples of I/O redirection
● Try the following (use up arrow to avoid retyping each line):
○ w | wc # count lines
○ w | cut –d ‘ ’ –f1 | sort # sort users
○ w | cut –d ‘ ’ –f1 | sort | uniq # eliminate duplicates
● We can also redirect output into a file:
○ w | cut –d ‘ ’ –f1 | sort | uniq > users
● Note that ‘awk’ can be used instead of ‘cut’:
○ w | awk ‘{print $1;}’ | sort | uniq > users
● Quiz:
○ How might we count the number of distinct users currently logged in?
For extra credit, how can we avoid over-counting by 2? (Hint: use ‘tail’.)
The Filesystem
The Linux File System
● The structure resembles an upside-down tree
● Directories (a.k.a. folders) are collections of files and other
directories.
● Every directory has a parent except for the root directory.
● Many directories have subdirectories.
Navigating the File System

● Essential navigation commands:

○ pw print current directory
d list files
○ ls change directory
○ cd
Navigating the File System

We use pathnames to refer to files and directories in the Linux file

system.

● There are two types of pathnames:

○ Absolute – The full path to a directory or file; begins with /
○ Relative – A partial path that is relative to the current working directory;
does not begin with /
Navigating the File System

● Special characters interpreted by the shell for filename

expansion:
○ ~ your home directory (e.g., /usr1/tutorial/tuta1)
○ . current directory
○ .. parent directory
○ * wildcard matching any filename
○ ? wildcard matching any character
○ TAB try to complete (partially typed) filename
Navigating the File System

● Examples:
○ cd /usr/local Change directory to /usr/local/lib
○ cd ~ Change to home directory (could just type ‘cd’)
○ pw Print working (current) directory
d
○ cd .. Change directory to the “parent” directory
○ cd / Change directory to the “root”
○ ls –d pro* Listing of only the directories starting with “pro”
The ls Command
● Useful options for the “ls” command:
○ ls -a List all files, including hidden files beginning with a “.”
○ ls -ld * List details about a directory and not its contents
○ ls -F Put an indicator character at the end of each name
○ ls –l Simple long listing
○ ls –lR Recursive long listing
○ ls –lh Give human readable file sizes
○ ls –lS Sort files by file size
○ ls –lt Sort files by modification time (very useful!)
Some Useful File Commands
● cp [file1] [file2] copy file
● mkdir [name] make directory
● rmdir [name] remove (empty)
● mv [file] [destination] directory move/rename
● rm [file] file remove (-r for
● file [file] recursive) identify file
● less [file] type
● head -n N [file] page through file
● tail -n N [file] display first N lines
● ln –s [file] [new] display last N lines
● cat [file] [file2…] create symbolic
● tac [file] [file2…] link display file(s)
● touch [file] display file in reverse order
● od [file] update modification time
display file contents, esp.
binary
Manipulating files and directories
● Examples:
○ cd # The same as cd ~
○ mkdir test
○ cd test
○ echo ‘Hello everyone’ > myfile.txt
○ echo ‘Goodbye all’ >> myfile.txt
○ less myfile.txt
○ mkdir subdir1/subdir2 # Fails. Why?
○ mkdir -p subdir1/subdir2 # Succeeds
○ mv myfile.txt subdir1/subdir2
○ cd ..
○ rmdir test # Fails. Why?
○ rm –rf test # Succeeds
Symbolic links
● Sometimes it is helpful to be able to access a file from multiple locations
within the hierarchy. On a Windows system, we might create a “shortcut.” On
a Linux system, we can create a symbolic link:

○ mkdir foo # make foo

○ touch foo/bar directory # create
○ ln –s foo/bar . empty file
# create link in current dir.
Finding a needle in a haystack
● The find command has a rather unfriendly syntax, but can be
exceedingly helpful for locating files in heavily nested directories.

● Examples:
○ find ~ -name bu –type d # search for “bu” directories in ~
○ find . –name my-file.txt # search for my-file.txt in .
○ find ~ -name ‘*.txt’ # search for “*.txt” in ~

● Quiz:
○ Can you use find to locate a file called “needle” in your haystack
directory?
○ Extra credit: what are the contents of the “needle” file?
Processes & Job Control
Processes and Job Control
● As we interact with Linux, we create numbered instances of running programs
called “processes.” You can use the ‘ps’ command to see a listing of your
processes (and others!). To see a long listing, for example, of all processes
on the system try:

[username@scc1 ~]$ ps -ef

● To see all the processes owned by you and other members of the class, try:

[username@scc1 ~]$ ps -ef | grep tuta

Processes and job control
● Use “top” to see active processes.

Tasks: 408 total, 1 running, 407 sleeping, 0 stopped, 0

zombie
Cpu(s): 0.3%us, 0.1%sy, 0.0%ni, 99.6%id, 0.0%wa, 0.0%st
0.0%hi,
Mem: 0.0%si,
99022756k total, 69709936k used, 29312820k free,
525544k buffers Swap: 8388604k8388604k
total, free, 0k used, cached
65896792k

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

7019 root 20 0 329m 137m 4852 S 4.0 0.1 217:01.56 sge_qmaster
38246 isw 20 0 88724 2764 1656 S 0.7 0.0 0:01.28 sshd
41113 cjahnke 20 0 13672 1512 948 R 0.7 0.0 0:00.03 top
2324 root 20 0 0 0 0 S 0.3 0.0 0:21.82 kondemand/2
7107 nobody 20 0 89572 10m 2400 S 0.3 0.0 2:18.05 gmond
27409 theavey 20 0 26652 1380 880 S 0.3 0.0 0:34.84 tmux
1 root 20 0 25680 1604 1280 S 0.0 0.0 0:05.74 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.07 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.89 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:01.72 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/0
(refreshes every 2
Foreground/background
● Thus far, we have run commands at the prompt and waited for them
to complete. We call this running in the “foreground.”
● Use the “&” operator, to run programs in the “background”,
○ Prompt returns immediately without waiting for the command to complete:

[username@scc1 ~]$
mycommand & ← process id
[1] 54356
[username@scc1 ~]$
Process Control Practice
● Let’s look at the “countdown” script, in your scripts folder for practice

[username@scc1 ~]$ cd ~/scripts

[username@scc1 ~]$ cat countdown

● Make the script executable with chmod:

[username@scc1 ~]$ chmod +x countdown

● First, run it for a few seconds, then kill with Control-C.

[username@scc1 ~]$ ./countdown 100
100
99
98
^ ← Ctrl-C = (^C)
C
Process control
● Now, let’s try running it in the background with &:
[username@scc1 ~]$ ./countdown 60 &
[1] 54355
[username@scc1 ~]$
60
59

● The program’s output is distracting, so redirect it to a file:

[username@scc1 ~]$ countdown 60 > c.txt &
[1] 54356
[username@scc1 ~]$
Process control
● Type ‘ps’ to see your countdown process.

● Also, try running ‘jobs’ to see any jobs running in the background from this
bash shell.

● To kill the job, use the ‘kill’ command, either with the five-digit process id:
○ kill 54356

● Or, you can use the job number (use ‘jobs’ to see list) with ‘%’:
○ kill %1
Backgrounding a running job with C-z and
‘bg’
● Sometimes you start a program, then decide to run it in the background.

[username@scc scripts]$ ./countdown 200 > c.out

1 ← Ctrl-Z = (^Z)
^Z
[1]+ ./countdown 200 > c.out
Stopped
[username@scc1 scripts]$ bg
[1]+ ./countdown 200 > c.out &

[username@scc scripts]$ jobs

1
[1]+ ./countdown 200 > c.out &
Running
[username@scc1 scripts]$
Editors
Regular expressions
● Many Linux tools, such as grep and sed, use strings that describe sequences
of characters. These strings are called regular expressions.
Here are some examples:

○ ^foo # line begins with

○ bar$ “foo” # line ends with
○ [0-9]\{3\} “bar” # 3-digit number
○ .*a.*e.*i.*o.*u.* # words with vowels in order*
File Editors
● gedit
○ Notepad-like editor with some programming features (e.g., syntax highlighting). Requires
X-Windows.
● nano
○ Lightweight editor. Non-Xwindows.
● emacs
○ Swiss-army knife, has modes for all major languages, and can be customized. Formerly
steep learning curve has been reduced with introduction of menu and tool bars. Can be used
under Xwindows or not.
● vim
○ A better version of ‘vi’ (an early full-screen editor). Very fast, efficient. Steep learning
curve. Popular among systems programmers. Terminal or X-Windows.
Creating and Running Code
“Hello, world” in C
● cd to “~/c”, and read hello.c into your editor of choice.
● Modify the text on the printf line between “[“ and “]” and save the file.
● Produce an executable file called “hello” by compiling the program with gcc:

[username@scc1 ~]$ gcc –o hello hello.c

● Run the program at the command line:

[username@scc1 ~]$ ./hello

● Optional: modify countdown script to run hello program

Obtaining the Supplementary Course Material

● In browser, navigate to http://rcs.bu.edu/tutorials

○ Scroll to Introduction to Linux and select “Cheat
Sheets”.

● See also other Linux tutorials:

○ http://www.tutorialspoint.com/unix/
○ Edx Linux intro [Google “edx linux”]
○ http://www.cse.sc.edu/~okeefe/tutorials/unixtut/
Questions?
Course Agenda
Part 1: Introduction to basic topics in
Linux
❖ Getting Started
❖ About Linux
❖ Working with Files and Directories
❖ Linux Essential Commands (Part 1)
Getting Started
Logging into HPCC Resources (HPCC Users):
▪ Linux/Mac Users:
• SSH (Secure Shell): Freely available on Linux/Unix/MacOS and used via the Terminal.

▪ Windows Users:
• MobaXterm (Recommended):
– https://mobaxterm.mobatek.net
• Putty:
– https://www.putty.org

▪ User Guides, Other Alternatives, and Details:

• See the “Connecting to HPCC Systems” link at http://www.hpcc.ttu.edu/userguides/
About
Linux…
A brief history of Linux
GNU Project:
▪ Richard Stallman started the GNU project in 1983
• After AT&T imposed commercial licensing on Unix Operating System

• Reprogrammed some of the Unix Tools. running on a small kernel

(TRIX)

• Needed more advanced and reliable OS Kernel for his programs

Linux:
▪ Linus Torvalds created the Linux kernel in the early 1990s
• Linus released the first kernel under the GPL (General Public License)

– Requires that the source code remains freely available

Linux Distributions
Linux has various distributions
(Distros):
▪ Why many
• Server distributions?
vs. Desktop
• Commercial user support
• Special hardware support

▪ Debian/RedHat/SUSE/…
• The major difference is the software package management on these distros.
 Structure of Linux OS
Linux
Shell
• Here is where you • User software
start interacting with and applications
the OS • High-level programs
Supplement
• Multiuser environments
al
Software

Core
Linux
Tools
• OS core commands
• System-level programs
• GNU tools • Process/Thread
• Compilers scheduling
Linux • Memory Management
Kernel • File and I/O management
• Device drivers
• System calls
Linux Shell
What is the shell?
▪ The shell is a program that takes commands from user’s keyboard and passes
them to the operating system to execute.
• Many shell programs are available for Linux:
• bash, sh, csh, tcsh, ksh, zsh, …
• For this tutorial, we will use bash, but other shells are conceptually similar

quanah:$ command

What is Terminal?
▪ Terminal is a program that opens in a window and lets users interact with the
shell.
 Working with
Files and
Directories
Understanding Linux Directory Structure
/
/
boot
/boot
etc
/etc
tmp
/tmp
hom
/home
e /home/mahmadia
mydi
mahmadi /home/mahmadia/mydir
ra /home/eraider
vareRaider /var
lustr /lustre
e /lustre/work
work /lustre/scratch
.. scratc
Basic Directory Operations
Where am I?
▪ pwd command to print working
directory
quanah:$ pwd
/home/username

▪ ls command to list contents of the current

directory
quanah:$
ls
test1.txt
Mydir1
mydir2
Basic Directory Operations
Make/Remove/Go into a directory?
▪ mkdir command to make a new
directory
quanah:$ mkdir my_new_dir
quanah:$
ls
▪my_new_dir
cd command to change into a
directory
quanah:$ cd my_new_dir

▪ rmdir command to remove an empty

directory
quanah:$ rmdir my_new_dir
Basic Directory Operations
More about ls. (Commands with options)
▪ ls -a (--all) list all files including hidden
files
quanah:$ ls -a

▪ ls -l show file
details
quanah:$ ls -l

▪ Combine multiple options for a

command
quanah:$ ls -a -
l quanah:$ ls -
al
Special Directory Names (Characters)
▪ Return to home directory
(~) cd
quanah:$
quanah:$ cd ~
quanah:$ cd
~/
▪ Current directory
(.)
quanah:$ ls .
quanah:$
ls ./
▪ Parent directory
(..)
quanah:$ cd ..
quanah:$ cd ../
quanah:$
ls ../../
quanah:$ cd
Absolute VS. Relative path

Absolute Path of a file/directory:

▪ Always starts with / and goes all the way to the file or directory
• For instance:
quanah:$ cat /home/mahmadia/mydir/file1.txt

Relative Path of a file/directory:

▪ It defines from the current or parent
directory
• For instance:

quanah:$ cat ../mydir/file1.txt

File and Directory Names
Naming: What to use: •
•
A-Z
A-z
▪ Letters and numbers [A-Z a-z 0-9]: • 0-9
• dot (.)
• File and directory names are case sensitive (e.g. ‘M’ vs ‘m’) • dash (-)
▪ (.) can be used anywhere in the file/directory • underscore (_)
name
• Using (.) at the beginning of the file/directory name makes it hidden
• Using (.) alone will be considered as current working directory
• () parenthesis
What to Avoid: • "
double quotes
▪ Avoid spaces in the file • '
single quote
names
• “my file name.txt” 🡪 “my_file_name.txt” • ?
▪ Avoid these characters: Question mark
• $
• () " ' ? $ *\ / :
dollar sign
• *
A few Basic File Operations
Let’s learn a few file operations before Exercise
#1
▪quanah:$ creates an empty file (Not the primary job!)
touch touch
my_file quanah:$ ls
my_file

▪ echo prints its arguments to the standard output

▪ > redirects the standard output to a file (Creates/Rewrite
the file)
▪ >> redirects the standard output to a file (Append to the file)
• Moreecho
quanah:$ details"hello
on output redirects >
world!" in next few slides
hello.txt quanah:$ echo "second line"
A few Basic File Operations (Cont.)
Let’s learn some file operations before Exercise
#1
▪quanah:$
cat shows
cat the content of a file(s)
hello.txt Hello
world!
second line
▪ wc counts number of lines, words, and characters in a text
file
quanah:$ wc hello.txt
2 4 25 hello.txt
quanah:$ wc -l
hello.txt
2 hello.txt
quanah:$ wc -w hello.txt
Exercise #1
1. Return to your home directory
2. Create a new directory and name it “exercise1”
3. Go into the “exercise1” directory
4. Create a new text file and name it “test1.txt”. Then add the following lines into
the file:
Exercise #1:
This is a test file
here is the last line
5. Now create a new directory under the “exercise1” directory and name it
“subdir1”.
6. Go into the “subdir1” and try to print the content of “test1.txt” on your terminal.
7. Return to the “exercise1” directory and count the number of lines in “test1.txt”.
File and Directory Operations
Copy Files and Directories:
▪ cp command to make a copy of a file(s) or directory(s)
• -r: Recursively copy the directories and subdirectories as
well
File and Directory Operations
Move/Rename Files and Directories:
▪ mv command to make a copy of the source file(s) and
directory(s) into the destination and remove the source at the end.
• Can be used for renaming a file/directory as well
File and Directory Operations
Deleting Files and Directories:
▪ rm command to remove file(s) and directory(s) permanently
• Removing files and directories by this command may not be recovered
easily (This is an exception for /home directory on HPCC clusters)
• -r: Recursively copy the directories and subdirectories as well
Output Redirection
▪ So far, all the commands that we used sent their output to
the screen. We can control this in Linux:
• < redirects to the standard input
• > redirects the standard output
• >> appends the standard output
• 1> redirects the standard output (equals to >)
• 2> redirects the standard error
• &> redirects the standard output and standard error
• | (pipe) pass the output of one command to another
Exercise #2
1. Return to your home directory
2. Create a new directory and name it “exercise2”
3. Go to the “exercise2” directory
4. Copy the “test1.txt” file from the “exercise1” directory to
the “exercise2” directory
5. Make a new directory under “exercise2” and name it “subdir2”
6. Rename the file “test1.txt” to “test2.txt”
7. List the contents of “exercise2” and redirect the output to “ls.out”
file
8. Remove everything under the “exercise2” directory
Linux Essential Commands 2
File/Directory Ownership and Permissions:
▪ Every file/directory belongs to a specific user or a group of users

▪ Every user/group many have permissions to read, write,

and/or
User Group Other
execute
rwx rwx rwx

▪ If you set write permission for a directory, you can (create/delete) new
entries

▪ If you set read permission for a directory, you can list (ls) the contents

▪ If you set execute permission for a directory, you can (cd) into the directory
Linux Essential Commands 2
File/Directory Ownership and Permission
Examples:
▪ chmod command changes the rwx mode bits of a file or directory

• +/-: adds or removes the mode bits

• u: Sets the permissions for the owner of the file/directory
• g: Sets the permissions for the group that the owner belongs to
• o: Sets the permissions for the other users
• a: Sets the same permissions for user/group/other
Linux Essential Commands 2
File/Directory Ownership and Permission:
▪ With stat or ls –l commands you can check the ownership
and permissions of a file or directory

▪ whoami command Displays the username of the current user

▪ groups command prints the groups a user belongs to

Linux Essential
Commands
(Part 1)
Linux Essential Commands 1
Check the manual page of a command:
▪ man command to search and open the manual page of a
Linux command
man cp

• /word search the man page for the text

‘word’
• n search for the next found match
• N search for the previous found match
• g Go to the beginning of the manual
page
• G Go to the end of the manual page
Linux Essential Commands 1
Other useful commands to find out more:
▪ Several built-in references are available within the operating
system.
• info more detailed tutorials and examples.
• apropos
quanah:$ info cpSearch the manual database for a “string”.
quanah:$ apropos
▪mkdir
Many reference guides are available.
• Search the web for “reference” – for example “bash reference”.
• Specifically for bash, try finding the “bash advanced scripting guide” (but only
when you are ready for the advanced stuff!
• Guides are available for other shells and commands also.
Linux Essential Commands 1

Paging through long text files/outputs:

▪ more command for paging through text one screenful at a time

• More only allows to move the text forward
quanah:$ more ./README
quanah:$ cat ./README |
more
▪ Less command is an improved version of more command
which allows backward movement in the file as well as
forward movement
quanah:$ less ./README
quanah:$ cat ./README |
less
Linux Essential Commands 1
Display a partial content of a file/output:
▪ head command to see first lines of a file (10 lined by
default)
• -n head
quanah:$ <NUM>: First NUM
-n 15 ./README number of a file/output
quanah:$ cat ./README | head
-3
▪ tail command to see last lines of a file (10 lined by
default)
• -n <NUM>: Last NUM number of a file/output
quanah:$ tail -n 20 ./README
quanah:$ cat ./README | tail
-5
Linux Essential Commands 1
Searching for pattern(s) in files/outputs:
▪ grep command to print lines in a file matching a pattern
• -i: Ignore case distinctions in both the PATTERN and the input
files
• -v: Invert the sense of matching, to select non-matching lines
quanah:$ grep "test"
../exercise1/test1.txt This is a test
file
quanah:$ grep "is"
../exercise1/test1.txt Exercise:
This is a test file
here is the last
Linux Essential Commands 1
History of your commands:
▪ history command to see the commands you have typed so
far.
• Your command are saved to the .bash_history file under your
home directory.
quanah:$ history | less
quanah:$ history | tail -15
quanah:$ history | grep
“cp”
Exercise #3
1. Search in your history and try to list your last 10 “ls”
commands that you have used

2. Check the manual of the “man” command and see what type
of manuals are available on Linux
Coming in Part 2 of this Training …
Part 2: Introduction to more advanced topics in
Linux
❖ Linux Essential Commands (Part 2)
❖ Text Editors in Linux
❖ Linux Environment Variables
❖ Basic Bash Scripting in Linux
Course Agenda
Part 2: Introduction to more advanced topics in
Linux
❖ Linux Essential Commands (Part 2)
❖ Text Editors in Linux
❖ Linux Environment Variables
❖ Basic Bash Scripting in Linux
Review
Command Description
pwd Prints Current Working Directory
ls Lists the contents of a directory
cd Change the current path to the destination directory
mkdir Makes a new directory
rmdir Removes an empty directory
cp Copy file or directory
mv Move/Rename file or directory
rm Delete file or directory
cat Concatenates and prints the contents a file
Review
Command Description
echo Write arguments to the standard output
wc word, line, character, and byte count
man Search and open the manual page of a Linux command
more Paging through text one screenful at a time
less Improved version of more allows backward/forward movement
head Display first lines of a file
tail Display last lines of a file
grep Print lines in a file matching a pattern
history See the commands you have typed so far
Linux Essential
Commands
(Part 2)
Linux Essential Commands 2
Define the file type:
▪ Unlike Windows, files extensions rarely define the type of a file in Linux.
• For example: “file.txt” many not be a TEXT file.
▪ file command displays the file type along with a brief summary of the
file contents.
quanah:$ file myfile
myfile: ASCII text
quanah:$ file /home/mahmadia/program.py
/home/mahmadia/program.py: Python script, ASCII text
executable quanah:$ file test.tar.gz
test.tar.gz : gzip compressed data, from Unix, last modified:
Wed Sep 4 14:04:10 2019
Linux Essential Commands 2
CRLF Line Terminator:
▪ Windows text editors such as notepad add a Carriage Return Line Feed
(CRLF) character at the end of each line of the text which causes
problems with many Linux applications.
▪ dos2unix command fixes the CRLF issue in text files from Windows.

quanah:$ file windows.txt

windows.txt: ASCII text, with CRLF line
terminators quanah:$ dos2unix windows.txt
dos2unix: converting file windows.txt to Unix
format ... quanah:$ file windows.txt
windows.txt : ASCII text
Linux Essential Commands 2
Compression and File Packaging:
▪ zip command packages and compresses files and directories
• zip [OPTIONS] zip_file
file_dir1 file_dir2 …

• -r : Add the directories and subdirectories contents into the zip file
quanah:$ ls
mydir
test1.txt
quanah:$ zip -r archive.zip ./*
adding: ./test1.txt (deflated
62%) adding: ./mydir/ (deflated
0%) quanah:$ ls
archive.zip mydir test1.txt
Linux Essential Commands 2
Compression and File Packaging:
▪ unzip command lists and extracts the contents of a zipped
file
• -l : Lists the contents of a zipped file
quanah:$
ls
archive.zi
p
quanah:$ unzip
archive.zip quanah:$ ls
archive.zip mydir test1.txt
Linux Essential Commands 2
Compression and File Packaging:
▪ Other compression commands available in
Linux
Command Description Decompression File Ext.
zip Packages and compresses files unzip .zip
and directories
gzip A GNU tool for compressing or gunzip .gz
expanding files/directories gzip -d
bzip2 Compresses files using the bunzip2 .bz,
Burrows- Wheeler block sorting text bzip2 -d .bz2
compression algorithm.
xz Similar to gzip and bzip2 unxz .xz
Linux Essential Commands 2
Archiving:
▪ tar command saves many files and directories into a single “archive”
file
• tar OPTIONS dest_file src1 src2 …

• -f define the archive file path/name

• -c Create a new archive
• -a Append to the existing archive file
• -x Extract the contents of an archive file
• -z Compress archive file with gzip
• -j Compress archive file with bzip2
• -v verbosely list files processed
Linux Essential Commands 2
Archiving:
▪ tar command
examples:
quanah:$ ls
mydir
test1.txt
quanah:$ tar -cf
myarchive.tar ./* quanah:$ ls
myarchive.tar mydir test1.txt
quanah:$ tar -xvf
myarchive.tar test1.txt
mydir/
quanah:$ tar -czf docs.tar.gz
/home/mahmadia/docs quanah:$ ls
Docs.tar.gz myarchive.tar mydir
Linux Essential Commands 2
Download files from internet:
▪ wget command downloads files from internet
• -O : (capital O) defined the name of the destination file on your
system
quanah:$ wget
“https://repo.anaconda.com/miniconda/Miniconda3- latest-
Linux-x86_64.sh” -O miniconda3.sh
Exercise #4
1. Go to your home directory
2. Create a new directory and name it “exercise4”
3. Go to the “exercise4” directory
4. Choose a small directory from your home directory
5. Try to archive and compress the directory by tar and save it under
the “exercise4” directory.
6. Now try to decompress the tar file that you just created
7. Check the type of the file
8. Now try to untar the file
Linux Essential Commands 2
Access Control List (ACL):
▪ Second level of discretionary permissions that override the standard
ugo/rwx
▪ better granularity in setting access to a file or a directory
▪ Recommended on HPCC cluster environments
▪ getfacl shows the file/directory Access Control List
• getfacl <File | Directory>

▪ setfacl modify/remove the ACL permissions

• setfacl -m [u|g]:<username>:rwx <File|Directory>

• setfacl -x [u|g]:<username> <File|Directory>

Text Editors in
Linux
Text Editors in Linux
How to edit text files in Linux?
▪ There are many text editors available on Linux
• nano is a small, simple and friendly editor
• vi/vim is a powerful text editor which can be used to edit all kinds of
text
• emacs is part of the GNU project written by Richard Stallman

▪ In this training course we will cover nano and vi/vim

▪ Let’s look into nano (Demo)
Text Editors in Linux
How does vi/vim work?
▪ vi/vim is a very popular text editor among programmers and
system administrators
▪ It supports many programming and scripting languages
▪ Suitable for more advanced file editing
▪ vi/vim has two modes:
1. Text mode: which can be enabled by typing i (insert) or a (append)

2. Command mode: which will be enabled by pressing the Esc key on

keyboard.
Text Editors in Linux
Some useful vi/vim
commands:description
command command description
! Forces the action i insert
:q quit a append
:q! Force quit x Delete a character
:w write y[count]y Yank (copy) [count] lines
:wq Write and quit d[count]d Cut (Delete) [count] lines
:x Write and quit p Paste after the current line

▪ Let’s look into vim

(Demo)
Exercise #5
1. Go to your home directory
2. Create a new directory and name it “exercise5”
3. Go to the “exercise5” directory
4. Open a new text file with the editor of your choice and type the lines
below:

Exercise #5
This is a Linux text editor exercise
We finally made it!
5. Save the file and exit
6. Try to display the contents of the file on your screen
7. Change the permission of the file as below:
• Set read/write permission for the owner
• Set read-only permission to your group
Environment
Variables in
Linux
Linux Environment Variables
What is the environment variable?
▪ Environment Variables stores any user-defined or system-defined information
that can be accessed within the shell.
▪ Environment Variables are useful for passing data to programs or being used in
shell scripts.
▪ Defining a variable is very simple (do not put spaces around = symbol)
quanah:$ VAR_NAME=“This is a variable”

▪ When referencing a variable place a ($) before the variable

name
quanah:$ echo
$VAR_NAME This is a
variable
Linux Environment Variables
Common Linux Environment
Variables
HOME Pathname of the current user’s home directory
PATH Colon separated list of directories where commands can be found
SHELL Name of the current Shell program
PWD Print current working directory
USER Print current username
TERM The type of the terminal
HOSTNAME Displays computer’s hostname
Linux Environment Variables
PATH Environment Variable
▪ Shell uses the PATH environment variable to locate commands
▪ The PATH variable is colon (:) separated, and can be displayed with
echo
quanah:$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

▪ You can add a directory into the PATH variable of your own
environment
quanah:$ export PATH="/home/username/bin:$PATH"
/home/username/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

▪ export command promotes a shell variable to an environment

variable
Linux Environment Variables
Set Environment Variables at login time
▪ ~/.bashrc : Commands for environment variables that you want to set
up at login time should be included in your ~/.bashrc file.
• For HPCC users we highly recommend using modules instead of
modifying the ~/.bashrc file. (Modules are covered in HPCC New
User Training)
quanah:$ vim ~/.bashrc

export WORK=/lustre/work/mahmadia
export
SCRATCH=/lustre/scratch/mahmadia
export PATH=“$PATH:$WORK/bin”
Basic Bash
Scripting In
Linux
Basic Bash Scripting in Linux
What is Bash Script?
▪ Bash script is an executable file contains Bash shell commands which
could be used to automate and simplify things.
• Shell script is a text file starts with (#!) followed by the path to the
shell interpreter (i.e. /bin/bash)
quanah:$ vim
myscript.sh #!/bin/bash
echo “Hello World!”

quanah:$ chmod +x
myscript.sh quanah:$
./myscript.sh
Hello World!
Basic Bash Scripting in Linux
Control flows
▪ The syntax of the if-then-else clause is as
following:
if [ $var -eq 1 ]; then
…
elif [ $var -ne 1 ]; then
…
elif [ $var -gt 1 ]; then
…
elif [ $var -lt 1 ]; then
…
fi
Basic Bash Scripting in Linux
Loops
▪ The syntax of the for…in loop is as
following:
for VARIABLE in 1 2 3 … N; do
…
done

for VARIABLE in file1 file2 file3; do

…
done

For VARIABLE in `Linux command`; do

…
Exercise #6
1. Go to your home directory
2. Create a new directory and name it “exercise6”
3. Go to the “exercise6” directory
4. Create a script file and name it “show_dirs.sh”
5. Program the “show_dirs.sh” to go over all the directories under your
home directory and print the following message for each directory:
/home/username contains --> the_directory_name

then print the following line for “exercise6” directory

This is the last directory --> excercise6
6. Make your script file executable and run it.