UNIX Shell-Scripting Basics

Agenda
What is a shell? A shell script? Introduction to bash Running Commands Applied Shell Programming

What is a shell?

What is a shell?

/bin/bash

What is a shell?

#!/bin/bash

What is a shell?
INPUT

shell

OUTPUT

ERROR

What is a shell?
Any Program But there are a few popular shells

Bourne Shells

/bin/sh /bin/bash Bourne-Again Shell

Steve Bourne

Other Common Shells

C Shell (/bin/csh) Turbo C Shell (/bin/tcsh) Korn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

C Shell (/bin/csh) Turbo C Shell (/bin/tcsh) Korn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

/bin, /usr/bin, /usr/local/bin /sbin, /usr/sbin, /usr/local/sbin /tmp /dev /home/borwicjh

What is a Shell Script?

A Text File With Instructions Executable

What is a Shell Script?

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? A Text File

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

An aside: Redirection
cat > /tmp/myfile cat >> /tmp/myfile cat 2> /tmp/myerr cat < /tmp/myinput cat <<INPUT Some input INPUT cat > /tmp/x 2>&1

INPUT 0 env OUTPUT 1 ERROR 2

What is a Shell Script? How To Run

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? What To Do

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? Executable

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? Running it

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

Finding the program: PATH

% ./hello.sh echo vs. /usr/bin/echo % echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin % which echo /usr/bin/echo

Variables and the Environment

% hello.sh bash: hello.sh: Command not found % PATH=$PATH:. % hello.sh Hello, world

An aside: Quoting
% echo $USER $USER % echo $USER borwicjh % echo \ % echo deacnet\\sct deacnet\sct % echo \ \

Variables and the Environment

% env [variables passed to sub-programs] % NEW_VAR=Yes % echo $NEW_VAR Yes % env [PATH but not NEW_VAR] % export NEW_VAR % env [PATH and NEW_VAR]

Welcome to Shell Scripting!

How to Learn
man
man bash man cat man man

man k
man k manual

Learning the Bash Shell, 2nd Ed. Bash Reference Cards http://www.tldp.org/LDP/abs/html/

Introduction to bash

Continuing Lines: \
% echo This \ Is \ A \ Very \ Long \ Command Line This Is A Very Long Command Line %

Exit Status
$? 0 is True % ls /does/not/exist % echo $? 1 % echo $? 0

Exit Status: exit

% cat > test.sh <<_TEST_ exit 3 _TEST_ % chmod +x test.sh % ./test.sh % echo $? 3

Logic: test
% % 0 % % 1 test 1 -lt 10 echo $?

test 1 == 10 echo $?

Logic: test
test [ ]
[ 1 lt 10 ]

[[ ]]
[[ this string =~ this ]]

(( ))
(( 1 < 10 ))

Logic: test
[ [ [ [ -f /etc/passwd ] ! f /etc/passwd ] -f /etc/passwd a f /etc/shadow ] -f /etc/passwd o f /etc/shadow ]

An aside: $(( )) for Math

% echo $(( 1 + 2 )) 3 % echo $(( 2 * 3 )) 6 % echo $(( 1 / 3 )) 0

Logic: if
if something then : # elif a contraction of else if: elif something-else then : else then : fi

Logic: if
if [ $USER eq borwicjh ] then : # elif a contraction of else if: elif ls /etc/oratab then : else then : fi

Logic: if
# see if a file exists if [ -e /etc/passwd ] then echo /etc/passwd exists else echo /etc/passwd not found! fi

Logic: for
for i in 1 2 3 do echo $i done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: C-style for

for (( expr1 expr2 expr3 do list done ; ; ))

Logic: C-style for

LIMIT=10 for (( a=1 ; a<=LIMIT ; a++ )) do echo n $a done

Logic: while
while something do :
done

Logic: while
a=0; LIMIT=10 while [ "$a" -lt "$LIMIT" ] do echo -n "$a a=$(( a + 1 )) done

Counters
COUNTER=0 while [ -e $FILE.COUNTER ] do COUNTER=$(( COUNTER + 1)) done Note: race condition

Reusing Code: Sourcing

% cat > /path/to/my/passwords <<_PW_ FTP_USER=sct _PW_ % echo $FTP_USER % . /path/to/my/passwords % echo $FTP_USER sct %

Variable Manipulation
% FILEPATH=/path/to/my/output.lis % echo $FILEPATH /path/to/my/output.lis % echo ${FILEPATH%.lis} /path/to/my/output % echo ${FILEPATH#*/} path/to/my/output.lis % echo ${FILEPATH##*/} output.lis

It takes a long time to become a bash guru

Running Programs

Reasons for Running Programs

Check Return Code
$?

Get Job Output

OUTPUT=`echo Hello` OUTPUT=$(echo Hello)

Send Output Somewhere

Redirection: <, > Pipes

Pipes
Lots of Little Tools echo Hello | \ wc -c
INPUT 0 echo OUTPUT 1 ERROR 2

A Pipe!
INPUT 0 wc OUTPUT 1 ERROR 2

Email Notification
% echo Message | \ mail s Heres your message \ [email protected]

Dates
% DATESTRING=`date +%Y%m%d` % echo $DATESTRING 20060125 % man date

FTP the Hard Way

ftp n u server.wfu.edu <<_FTP_ user username password put FILE _FTP_

FTP with wget

wget \ ftp://user:[email protected]/file wget r \ ftp://user:[email protected]/dir/

FTP with curl

curl T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file

Searching: grep
% % % % % grep grep grep grep grep rayra /etc/passwd r rayra /etc r RAYRA /etc ri RAYRA /etc rli rayra /etc

Searching: find
% find /home/borwicjh \ -name *.lis

[all files matching *.lis]

% find /home/borwicjh \ -mtime -1 name *.lis

[*.lis, if modified within 24h]

% man find

Searching: locate
% locate .lis

[files with .lis in path]

% locate log

[also finds /var/log/messages]

Applied Shell Programming

Make Your Life Easier

TAB completion Control+R history cd Study a UNIX Editor

pushd/popd
% cd /tmp % pushd /var/log /var/log /tmp % cd .. % pwd /var % popd /tmp

Monitoring processes
ps ps ef ps u oracle ps C sshd man ps

DOS Mode Files

#!/usr/bin/bash^M FTP transfer in ASCII, or dos2unix infile > outfile

sqlplus
JOB=ZZZTEST PARAMS=ZZZTEST_PARAMS PARAMS_USER=BORWICJH sqlplus $BANNER_USER/$BANNER_PW << _EOF_ set serveroutput on set sqlprompt "" EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); _EOF_

sqlplus
sqlplus $ARG1 if [ $? then exit 1 fi if [ -e then $USER/$PASS @$FILE_SQL \ $ARG2 $ARG3 ne 0 ]

/file/sql/should/create ]

[use SQL-created file]

fi
Ask Amy Lamy!

Passing Arguments
% cat > test.sh <<_TEST_ echo Your name is \$1 \$2 _TEST_ % chmod +x test.sh % ./test.sh John Borwick ignorethis Your name is John Borwick

INB Job Submission Template

$1: user ID $2: password $3: one-up number $4: process name $5: printer name

% /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT

Scheduling Jobs
% 0 0 * 0 % % crontab -l 0 * * * daily-midnight-job.sh * * * * hourly-job.sh * * * * every-minute.sh 1 * * 0 1AM-on-sunday.sh EDITOR=vi crontab e man 5 crontab

Other Questions?
Shells and Shell Scripts bash Running Commands bash and Banner in Practice