1 Using Linux Tools
1 Using Linux Tools
1/83
Outline
1 2
3 4
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 2/83
Outline
1 2
3 4
Introduction
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 3/83
Introduction
What is the Linux OS? Free Open Source Operating System Free Free as in Free Speech, not Free Beer Open-Source Permit modifications and redistribution of source code Unix-inspired Linux Kernel + Application software Runs on a variety of hardware Also called GNU/Linux
4/83
Why Linux?
Introduction
Free as in Free Beer Secure & versatile Why Linux for Scientific Computing? Free as in Free Speech Can run for ever Libraries Parallel Computing
5/83
Outline
1 2
3 4
Getting Started
Introduction
Getting Started
Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 6/83
Logging in
Getting Started
GNU/Linux does have a GUI Command Line for this module Hit Ctrl + Alt + F1 Login
7/83
Where am I?
Getting Started
8/83
What is in there?
Getting Started
9/83
New folders
Getting Started
10 / 83
Moving around
Getting Started
$ cd ..
We could use absolute path instead of relative
$ cd /home/user/sees/
Dept. of CS(SSV College) Using Linux Tools 11 / 83
New files
Getting Started
touch command creates a blank file $ pwd /home/user $ cd sees $ touch first $ ls first
12 / 83
Outline
1 2
3 4
Getting Help
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 13 / 83
Getting Help
$ man touch
Shows all tasks that the command can perform Hit q to quit the man page For more, see the man page of man
$ man man
Dept. of CS(SSV College) Using Linux Tools 14 / 83
Getting Help
$ ls -R
Create a new directory along with parents, if required
Getting Help
16 / 83
Outline
1 2
3 4
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 17 / 83
Removing files
rm is used to delete files $ rm foo rm works only for files; not directories
Additional arguments required to remove a directory -r stands for recursive. Removes directory and all of its content
Copying Files
$ cp linux-tools/scripts/foo linux-tools/
New file-name can be used at target location foo copied to new location with the name bar
Copying Directories
20 / 83
Moving Files
cp and rm would be one way mv command does the job Also takes -i option to prompt before overwriting
$ ls course sees became a sub-directory of course mv command doesnt over-write directories -i option is useful when moving files around mv to rename move to same location with new name
Dept. of CS(SSV College) Using Linux Tools 21 / 83
Outline
1 2
3 4
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership
5
6 7
Looking at files
The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 22 / 83
/ is called the root directory It is the topmost level of the hierarchy For details man hier
23 / 83
drwxr-xr-x
5 root users
The first column shows the permission information First character specifies type of the file Files have -; Directories have d 3 sets of 3 characters for user, group and others r, w, x for read, write, execute Either the corresponding character or - is present
24 / 83
chmod command is used -R option to recursively change for all content of a directory
Change permissions of foo.sh from -rw-r--r-- to
-rwxr-xr--
25 / 83
Symbolic modes
Reference Class Description u user the owner of the file g group users who are members of the files group o others users who are not hte owner of the file or members of a all all three of the above; is the same as ugo Operator Description + adds the specified modes to the specified classes removes the specified modes from the specified classes = the modes specified are to be made the exact modes for the spe Mode Name Description r read read a file or list a directorys contents w write write to a file or directory x execute execute a file or recurse a directory tree
Dept. of CS(SSV College) Using Linux Tools 26 / 83
27 / 83
Outline
1 2
3 4
Looking at files
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership
5
6 7
Looking at files
The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 28 / 83
cat
Looking at files
$ cat foo.txt
Concatenates the text of multiple files
29 / 83
less
Looking at files
$ less wonderland.txt
q: Quit Arrows/Page Up/Page Down/Home/End: Navigation ng: Jump to line number n
30 / 83
wc
Looking at files
Statistical information about the file the number of lines in the file the number of words the number of characters
$ wc wonderland.txt
31 / 83
Looking at files
let you see parts of files, instead of the whole file head - start of a file; tail - end of a file show 10 lines by default
$ head wonderland.txt -n option to change the number of lines $ head -n 1 wonderland.txt tail is commonly used to monitor files -f option to monitor the file Ctrl-C to interrupt $ tail -f /var/log/dmesg
Dept. of CS(SSV College) Using Linux Tools 32 / 83
cut
Looking at files
Allows you to view only certain sections of lines Lets take /etc/passwd as our example
root:x:0:0:root:/root:/bin/bash
View only user names of all the users (first column)
cut
Looking at files
Allows choosing on the basis of characters or bytes Example below gets first 4 characters of /etc/passwd
34 / 83
paste
Looking at files
Outline
1 2
3 4
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 36 / 83
$ cut -d " " -f 2- marks1.txt \ > /tmp/m_tmp.txt $ paste -d " " students.txt m_tmp.txt
or
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt The first solution used Redirection The second solution uses Piping
37 / 83
Redirection
The standard output (stdout) stream goes to the display Not always, what we need First solution, redirects output to a file > states that output is redirected; It is followed by location to redirect
$ command > file1 > creates a new file at specified location appends to a file at specified location
38 / 83
Redirection . . .
39 / 83
Redirection . . .
Following example shows stderr redirection Error is printed out in the first case Error message is redirected, in the second case
$ cut -d " " -c 2- marks1.txt \ > /tmp/m_tmp.txt $ cut -d " " -f 2- marks1.txt 1> \ /tmp/m_tmp.txt 2> /tmp/m_err.txt
1> redirects stdout; 2> redirects stderr
Piping
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - instead of FILE asks paste to read from stdin cut command is a normal command the | seems to be joining the two commands Redirects output of first command to stdin, which becomes input to the second command This is called piping; | is called a pipe
41 / 83
Piping
42 / 83
Tab-completion
Hit tab to complete an incompletely typed word Tab twice to list all possibilities when ambiguous completion Bash provides tab completion for the following.
1 2 3 4 5 6
File Names Directory Names Executable Names User Names (when they are prefixed with a ) Host Names (when they are prefixed with a @) Variable Names (when they are prefixed with a $)
43 / 83
History
Bash saves history of commands typed Up and down arrow keys allow to navigate history Ctrl-r searches for commands used
44 / 83
/<>!$%^&*|{}[]"~; $ ls file.*
Lists file.ext files, where ext can be anything
$ ls file.?
Lists file.ext files, where ext is only one character
45 / 83
Outline
1 2
3 4
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 46 / 83
sort
sort can be used to get sorted content Command below prints student marks, sorted by name
47 / 83
sort . . .
The command below sorts based on marks in first subject
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt -\ | sort -t " " -k 2 -rn -t specifies the delimiter between fields -k specifies the field to use for sorting -n to choose numerical sorting -r for sorting in the reverse order
48 / 83
grep
grep is a command line text search utility Command below searches & shows the marks of Anne alone $ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt | grep Anne grep is case-sensitive by default
49 / 83
grep . . .
-i for case-insensitive searches $ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt | grep -i Anne -v inverts the search To see everyones marks except Annes $ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt | grep -iv Anne
50 / 83
tr
tr . . .
-d deletes all specified characters Only a single character set argument is required The following command removes carriage return characters (converting file in DOS/Windows format to the Unix format) $ cat foo.txt | tr -d \r > bar.txt -c complements the first set of characters The following command removes all non-alphanumeric characters $ tr -cd [:alnum:]
Dept. of CS(SSV College) Using Linux Tools 52 / 83
uniq
uniq -u gives lines which do not have any duplicates uniq -d outputs only those lines which have duplicates -c displays the number of times each line occurs
$ sort items.txt | uniq -u $ sort items.txt | uniq -dc
53 / 83
Outline
1 2
3 4
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators
5
6 7
8 9 10 11
Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 54 / 83
Shell scripts
#!/bin/bash mkdir ~/marks cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - \ | sort > ~/marks/results.txt
55 / 83
Shell scripts . . .
Save the script as results.sh Make file executable and then run
56 / 83
57 / 83
echo
echo command prints out messages #!/bin/bash mkdir ~/marks cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - \ | sort > ~/marks/results.txt echo "Results generated."
58 / 83
#!/bin/bash mkdir ~/marks cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - \ | sort > ~/marks/$1 echo "Results generated." $1 corresponds to first command line argument $n corresponds to nth command line argument It can be run as shown below $ ./results.sh grades.txt
Dept. of CS(SSV College) Using Linux Tools 59 / 83
PATH
60 / 83
Outline
1 2
3 4
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 61 / 83
5
6 7
8 9 10 11
Control Structures
if-else for loops while loops
62 / 83
if
63 / 83
if-else
Checks whether argument is negative or not
#!/bin/bash if test $1 -lt 0 then echo "number is negative" else echo "number is non-negative" fi
$ ./sign.sh -11
64 / 83
[ ]-aliasfortest
#!/bin/bash if [ $1 -lt 0 ] then echo "number is negative" else echo "number is non-negative" fi
spacing is important, when using the square brackets
65 / 83
if-else
An example script to greet the user, based on the time
#!/bin/sh # Script to greet the user # according to time of day hour=date | cut -c12-13 now=date +"%A, %d of %B, %Y (%r)" if [ $hour -lt 12 ] then mess="Good Morning \ $LOGNAME, Have a nice day!" fi
66 / 83
if-else . . .
if [ $hour -gt 12 -a $hour -le 16 ] then mess="Good Afternoon $LOGNAME" fi if [ $hour -gt 16 -a $hour -le 18 ] then mess="Good Evening $LOGNAME" fi echo -e "$mess\nIt is $now" $LOGNAME has login name (env. variable)
backquotes store commands outputs into variables
67 / 83
for
Problem Given a set of .mp3 files, that have names beginning with numbers followed by their names 08 - Society.mp3 rename the files to have just the names. Also replace any spaces in the name with hyphens. Loop over the list of files Process the names, to get new names Rename the files
68 / 83
for
for
tr & cut
Replace all spaces with hyphens using tr -s Use cut & keep only the text after the first hyphen
while
Continuously execute a block of commands until condition becomes false program that takes user input and prints it back, until the input is quit
while [ "$variable" != "quit" ] do read variable echo "Input - $variable" done exit 0
72 / 83
Environment Variables
Pass information from shell to programs running in it
Behavior of programs can change based on values of variables Environment variables vs. Shell variables Shell variables - only current instance of the shell
Environment variables - valid for the whole session Convention - environment variables are UPPER CASE
73 / 83
Environment Variables . . .
The following commands show values of all the environment variables
$ export PATH=$PATH:$HOME/bin
74 / 83
Outline
1 2
3 4
Miscellaneous Tools
Introduction
Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at files The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
Dept. of CS(SSV College) Using Linux Tools 75 / 83
5
6 7
8 9 10 11
find
Miscellaneous Tools
Find files in a directory hierarchy Offers a very complex feature set Look at the man page! Find all .pdf files, in current dir and sub-dirs
76 / 83
cmp
Miscellaneous Tools
77 / 83
diff
Miscellaneous Tools
78 / 83
tar
Miscellaneous Tools
tarball - essentially a collection of files May or may not be compressed Eases the job of storing, backing-up & transporting files
79 / 83
Extracting an archive
Miscellaneous Tools
$ mkdir extract $ cp allfiles.tar extract/ $ cd extract $ tar -xvf allfiles.tar -x Extract files within the archive -f Specify the archive file -v Be verbose
80 / 83
Creating an archive
Miscellaneous Tools
81 / 83
Compressed archives
Miscellaneous Tools
82 / 83
Miscellaneous Tools
Bash reads /etc/profile, ~/.bash_profile, ~/.bash_login, and ~/.profile in that order, when starting up as a login shell.
~/.bashrc is read, when not a login shell Put any commands that you want to run when bash starts, in this file.
83 / 83