Statistics Function

Implementation

Traditional Programming Approach

Rolando V. Raqueño Tuesday, April 8, 2025

 minimum.pro
function minimum, x
n = n_elements(x)
answer = x(0)
for i=1L, n-1 do begin
if( answer gt x(i) ) then $
answer = x(i)
endfor
return, answer
end
Rolando V. Raqueño Tuesday, April 8, 2025
 sum.pro
function sum, x
n = n_elements( x )
answer = 0
for i = 0, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end

Rolando V. Raqueño Tuesday, April 8, 2025

 Overflow Problems
• What if the array is an integer array?
• Sums will have a maximum limit based on
the number of bits specified by the integer
type.

Rolando V. Raqueño Tuesday, April 8, 2025

 Corrected sum.pro
function sum, x
n = n_elements( x )
answer = 0.0D
for i = 0L, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end

Rolando V. Raqueño Tuesday, April 8, 2025

 mean.pro
function mean, x
n = n_elements(x)
answer = sum(x) / n
return, answer
end

Rolando V. Raqueño Tuesday, April 8, 2025

 Integer Problems
IDL> b=9/5
IDL> print,b
1

Rolando V. Raqueño Tuesday, April 8, 2025

 Integer Problems
IDL> b=9/double(5)
IDL> print,b
1.8000000

Rolando V. Raqueño Tuesday, April 8, 2025

 Corrected mean.pro
function mean, x
n = n_elements(x)
answer = sum(x) / double(n)
return, answer
end

Rolando V. Raqueño Tuesday, April 8, 2025

 sum_squares.pro
function sum_squares, x
squares = x^2.0
answer= sum( squares )
return, answer
end

Rolando V. Raqueño Tuesday, April 8, 2025

 variance.pro
function variance, x
n = double(n_elements(x))
answer=(sum_squares(x)-(sum(x)^2.0/n))/
(n-1)
return, answer
end

Rolando V. Raqueño Tuesday, April 8, 2025

 sd.pro
function sd, x
answer = sqrt(variance(x))
return, answer
end

Rolando V. Raqueño Tuesday, April 8, 2025

 Shell Programming

Rolando V. Raqueño Tuesday, April 8, 2025

 UNIX Tip of the Day
• Directory maneuvering commands pushd,
popd, and, dirs

% cd /usr/tmp
% pwd
/usr/tmp
% pushd ~rvrpci
% pwd
/cis/staff/rvrpci

Rolando V. Raqueño Tuesday, April 8, 2025

 UNIX Tip of the Day
% dirs
/cis/staff/rvrpci /usr/tmp
% pushd
% pwd
/usr/tmp
% dirs
/usr/tmp /cis/staff/rvrpci
% pushd /usr/local/bin

Rolando V. Raqueño Tuesday, April 8, 2025

 UNIX Tip of the Day
% dirs
/usr/local/bin /usr/tmp
/cis/staff/rvrpci
% pwd
/usr/local/bin
% pushd
% dirs
/usr/tmp /usr/local/bin
/cis/staff/rvrpci

Rolando V. Raqueño Tuesday, April 8, 2025

 UNIX Tip of the Day
% dirs
/usr/tmp /usr/local/bin
/cis/staff/rvrpci
% pwd
/usr/tmp
% pushd +2
% pwd
/cis/staff/rvrpci

Rolando V. Raqueño Tuesday, April 8, 2025

 UNIX Tip of the Day
% dirs
/cis/staff/rvrpci /usr/tmp
/usr/local/bin
% popd
% dirs
/usr/tmp /usr/local/bin

Rolando V. Raqueño Tuesday, April 8, 2025

IMPORTANT UNIX Concepts
• Environment and Shell Variables
– These allow you to customize your UNIX
environment
– They are different in terms of their SCOPE
• SCOPE determines the visibility of a variable

Rolando V. Raqueño Tuesday, April 8, 2025

Other IMPORTANT UNIX Concepts
• Environment Variable
– Examples are TERM and DISPLAY
– Set a particular variable to a value by using the
setenv command
– You can print the value of a particular variable
or all the environment variable using the
printenv command

Rolando V. Raqueño Tuesday, April 8, 2025

 % Environment Variables %
• Examples
– To set environment variables
% setenv TERM vt100
% setenv DOG Goofy
– print out the terminal type
% printenv TERM
vt100
– print out all environment variables
% printenv

Rolando V. Raqueño Tuesday, April 8, 2025

 Shell Variables
• Shell variables are similar to Environment
variables except they have a limited scope,
i.e., they exist only in the shell which they
are defined.
• Environment variables on the other hand,
exist in all its children shells
• To illustrate this concept, let us look at the
following example

Rolando V. Raqueño Tuesday, April 8, 2025

Environment vs. Shell Variables
% set prompt = "Parent Shell > "
Parent Shell > setenv DOG Goofy
Parent Shell > set mouse=Mickey
Parent Shell > printenv DOG
Goofy
Parent Shell > echo $mouse
Mickey
Parent Shell > xterm &

(YOU SHOULD NOW HAVE A NEW xterm WINDOW)

THIS IS KNOWN AS
“SPAWNING A NEW (OR CHILD) PROCESS”

Rolando V. Raqueño Tuesday, April 8, 2025

Environment vs. Shell Variables
(IN THE NEW xterm WINDOW, DO THE FOLLOWING)

% set prompt = "Child Shell > "

Child Shell > printenv DOG
Goofy
Child Shell > echo $mouse
mouse: Undefined variable.

Rolando V. Raqueño Tuesday, April 8, 2025

Environment vs. Shell Variables
Child Shell > setenv DOG Pluto
Child Shell > set mouse=Minnie
Child Shell > printenv DOG
Pluto
Child Shell > echo $mouse
Minnie
Child Shell > exit

(THE xterm WINDOW SHOULD NOW GO AWAY - THIS

PROCESS HAS NOW BEEN KILLED)

Rolando V. Raqueño Tuesday, April 8, 2025

Environment vs. Shell Variables
Parent Shell >
Parent Shell > printenv DOG
Goofy
Parent Shell > echo $mouse
Mickey
Parent Shell >

Rolando V. Raqueño Tuesday, April 8, 2025

Environment & Shell Variables

• Why is this important?

– UNIX uses Environment and Shell Variables
control a number of processes
– Customizes your working environment
– Variables used for UNIX Scripts
• They are typically defined and initialized in
your .login and .cshrc files

Rolando V. Raqueño Tuesday, April 8, 2025

 Useful Shell Variables
filec#Allows file completion
path #List of command
directories
cdpath #List of candidate
directories to cd into
history #Number of commands to
remember
Rolando V. Raqueño Tuesday, April 8, 2025
 What is shell programming?
• Shell programming
– automate a set of UNIX commands.
– Just like any programming language
– “wrappers”
• black box a customized collection of UNIX
commands.
– Example of shell programs
.login
.cshrc

Rolando V. Raqueño Tuesday, April 8, 2025

 .login file
set path=($HOME/bin /usr/local/bin \
/usr/ucb /usr/sbin /bin /usr/bin \
/usr/bin/X11 .)
stty dec new
tset -I -Q
set mail=/usr/spool/mail/$USER
set editmode = emacs
umask 077
biff n
date

Rolando V. Raqueño Tuesday, April 8, 2025

 .cshrc file
if ($?prompt) then
set notify
set history = 100
set savehist = 100
alias pd pushd
alias pop popd
alias vt100 "set term = vt100"
endif

Rolando V. Raqueño Tuesday, April 8, 2025

 When these files are
executed?
.cshrc
– is automatically executed when you start a new
shell
.login
– only gets executed once when you first login

Can be re-executed by giving the source

command
% source .cshrc

Rolando V. Raqueño Tuesday, April 8, 2025

 Other useful .login
and .cshrc entries
set filec
set cdpath=(~ ~rvrpci/pub ~/mythesis)

Other common entries

set path=( $path /usr/local/bin)

set path=(/usr/local/bin $path)

Rolando V. Raqueño Tuesday, April 8, 2025

 User defined shell program
• Determine name of command
• Determine input, output, and option
arguments
• Determine UNIX commands to execute
• Establish error trapping
• Make shell program executable

Rolando V. Raqueño Tuesday, April 8, 2025

 A simple shell program
• dd command to swap bytes

% dd if=input.dat of=output.dat
bs=2 conv=swab

• Very difficult to remember

• Very little utility to non-UNIX geeks
(normal people)

Rolando V. Raqueño Tuesday, April 8, 2025

 We would rather see...

% swap_bytes input.dat output.dat

Rolando V. Raqueño Tuesday, April 8, 2025

 Special Shell Variables Set
% swap_bytes input.dat output.dat

$0 $1 $2
command $argv[1] $argv[2]

Rolando V. Raqueño Tuesday, April 8, 2025

 Another Special
Shell Variables
% swap_bytes input.dat output.dat

$#argv
Indicates how many arguments are present
In this case, 2

Rolando V. Raqueño Tuesday, April 8, 2025

 shell program
swap_bytes

#!/bin/csh -f
dd if=$1 of=$2 bs=2 conv=swab

Rolando V. Raqueño Tuesday, April 8, 2025

 Making swap_bytes
shell script executable

% ls -l swap_bytes
-rw------- ... swap_bytes
% chmod u+x swap_bytes
% ls -l swap_bytes
-rwx------ ... swap_bytes

Rolando V. Raqueño Tuesday, April 8, 2025

 To run swap_bytes
• swap_bytes becomes just another
unix command!

% swap_bytes input.dat output.dat

Rolando V. Raqueño Tuesday, April 8, 2025

 Limitation of swap_bytes
• No error trapping
• Should give usage when typing command

% swap_bytes
usage: swap_bytes input_file output_file

Rolando V. Raqueño Tuesday, April 8, 2025

 Improvement to swap_bytes
#!/bin/csh -f
if ( $#argv != 2 ) then
echo "usage: $0 input_file output_file"
exit 1
endif
dd if=$1 of=$2 bs=2 conv=swab

Rolando V. Raqueño Tuesday, April 8, 2025

 Commad exit status
• By convention

exit 0
Indicates successful command completion

exit 1 (or non-zero)

Indicates some error condition

Rolando V. Raqueño Tuesday, April 8, 2025

 Informational message from
swap_bytes
• UNIX style informational message

% swap_bytes
usage: swap_bytes input_file output_file

Rolando V. Raqueño Tuesday, April 8, 2025

 Interactive swap_bytes
• If you want a “friendlier” shell program
– Have it query the user for the inputs

• Another special shell variable can be

used

$<
Rolando V. Raqueño Tuesday, April 8, 2025
 Interactive swap_bytes
#!/bin/csh -f
if ( $#argv != 2 ) then
echo -n "Please enter the input file> "
set input=$<
echo -n "Please enter the output file> "
set output=$<
endif
dd if=$input of=$output bs=2 conv=swab

Rolando V. Raqueño Tuesday, April 8, 2025

Interactive swap_bytes example

• User simply types the command

% swap_bytes
Please enter the input file> input.dat
Please enter the output file> output.dat

Rolando V. Raqueño Tuesday, April 8, 2025

 UNIX Quotes

Rolando V. Raqueño Tuesday, April 8, 2025

 A note about quotes in UNIX
% set a=ls
% echo a
% echo $a
% set b=“$a”
% echo $b
% set b=‘$a’
% echo $b
% set b=`$a`
% echo $b

Rolando V. Raqueño Tuesday, April 8, 2025

 A note about shell variables
• Shell variables can also double up as
arrays
• Using the previous example,
% echo $b
% echo $b[1]
% echo $#b
% echo $b[$#b]

Rolando V. Raqueño Tuesday, April 8, 2025

A more complex shell program
• In pbmplus utilities,
rawtopgm conversion exists
pgmtoraw conversion does not

• A version of pgmtoraw in a programming

language like C
– Time consuming
– Program will likely be used infrequently

• Solution: shell program

Rolando V. Raqueño Tuesday, April 8, 2025

 pgmtoraw shell script design
• Define input and output files
• Figure out dimensions of input image
• Determine number of bytes for input image
• Determine number of bytes for header
• Need to strip out the header bytes
• Write out headerless image data

Rolando V. Raqueño Tuesday, April 8, 2025

 Define input and output files
pgmtoraw
#!/bin/csh -f
set command_name=$0
set number_args=$#argv
if( $number_args != 1 ) then
echo “usage:$command_name input_file_name”
exit 1
endif
.
.
.

Rolando V. Raqueño Tuesday, April 8, 2025

 Dimensions of input image
( pnmfile)
% more test_data.pgm
P2
3 3
255
1 2 3
4 5 6
7 8 9

% pnmfile test_data.pgm
test_data.pgm: PGM plain, 3 by 3 maxval 255

Rolando V. Raqueño Tuesday, April 8, 2025

 pgmtoraw (continued)
set input_file=$1
set pnm_info = `pnmfile $input_file`
set image_type = $pnm_info[2]
set data_type = $pnm_info[3]
set width = $pnm_info[4]
set height = $pnm_info[6]
set maxval = $pnm_info[8]
set pixel_bytes = 1
@ image_bytes = $width * $height

Rolando V. Raqueño Tuesday, April 8, 2025

 pgmtoraw (continued)
set file_info=`wc -c $input_file`
set bytes_in_file = $file_info[1]
@ header = $bytes_in_file - $image_bytes
dd if=$input_file bs=$pixel_bytes skip=$header

Rolando V. Raqueño Tuesday, April 8, 2025

 Resulting pgmtoraw utility
• Uses existing routines to obtain information
pnmfile
wc
dd
• Functional tool written in 20 command lines

Rolando V. Raqueño Tuesday, April 8, 2025

 Current Limitations of
Resulting pgmtoraw utility
• No check between “ASCII” vs. “RAW” pgm
if( data_type == ‘plain,’) ...
• No provisions for multibyte per pixel case
– Use pnmfile results to check for above cases
– endian case needs to be addressed for multibyte
case
• Above conditions can be addressed by suite
of UNIX commands

Rolando V. Raqueño Tuesday, April 8, 2025

 Shell Scripts Wrappers
and IDL
• Another utility formerly missing in
pbmplus
jpegtopnm or pnmtojpeg

• IDL has jpeg read/write capability

– Create a “wrapper” that makes an idl
program pbmplus-like

Rolando V. Raqueño Tuesday, April 8, 2025

 pnmtojpeg.pro
pro pnmtojpeg, input_file, output_file

read_ppm, input_file, image

write_jpeg, output_file, image

end

Rolando V. Raqueño Tuesday, April 8, 2025

Usage of pnmtojpeg.pro in IDL
IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

Rolando V. Raqueño Tuesday, April 8, 2025

Usage of pnmtojpeg.pro in IDL
IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

• For IDL to automatically find pnmtojpeg.pro

– It must be in the current working directory

– Directory containing pnmtojpeg.pro must be defined

in the ENVIRONMENT VARIABLE
• IDL_PATH

Rolando V. Raqueño Tuesday, April 8, 2025

 IDL_PATH
environment variable
setenv IDL_DIR /cis/common/rsi/idl_5
setenv IDL_PATH
\+$IDL_DIR/lib:\+$IDL_DIR/examples:
\+/dirs/common/rsi/idl_5:\+/dirs/common/lib/
idl:\+~/lib/idl

Rolando V. Raqueño Tuesday, April 8, 2025

 pnmtojpeg.csh
#!/bin/csh -f
echo pnmtojpeg “,” “’”$1”’” “,” “’”$2”’” | idl

Rolando V. Raqueño Tuesday, April 8, 2025

 Usage of pnmtojpeg.csh
% pnmtojpeg.csh image.pnm image.jpg

Rolando V. Raqueño Tuesday, April 8, 2025

Limitation of pnmtojpeg.csh
• Does not conform to pbmplus piping,
i.e.,
% tifftopnm file.tif | pnmscale
2.0 > new_file.pnm

• No error trapping

Rolando V. Raqueño Tuesday, April 8, 2025

 Usage cases of pnmtojpeg
(no command line arguments)

% tifftopnm file.tif | pnmscale

2.0 | pnmtojpeg > new_file.jpg

Rolando V. Raqueño Tuesday, April 8, 2025

 Usage cases of pnmtojpeg
(1 command line argument)

% pnmtojpeg image.pnm > image.jpg

Rolando V. Raqueño Tuesday, April 8, 2025

 Usage cases of pnmtojpeg
(2 command line arguments)

% pnmtojpeg image.pnm image.jpg

Rolando V. Raqueño Tuesday, April 8, 2025

 Yet another wrapper
pnmtojpeg
#!/bin/csh -f

# If user interrupts process, jump to

stop
onintr stop
# $0 is the command name
# $#argv is the number of arguments
# $$ is the process id

Rolando V. Raqueño Tuesday, April 8, 2025

 Code for no argument case
if($#argv == 0) then
set input_file = /usr/tmp/$0_input_$$
set output_file = /usr/tmp/$0_output_$$
cat > $input_file
pnmtojpeg.csh $input_file $output_file
cat $output_file
.
.
.

Rolando V. Raqueño Tuesday, April 8, 2025

 Code for 1 argument case
else if($#argv ==1) then
set input_file = $1
set output_file = /usr/tmp/$0_output_$$
pnmtojpeg.csh $input_file $output_file
cat $output_file
.
.
.

Rolando V. Raqueño Tuesday, April 8, 2025

 Code for 2 argument case
else
set input_file = $1
set output_file = $2
pnmtojpeg.csh $input_file $output_file
endif

#clean up when finished

stop:
rm -f /usr/tmp/$0_input_$$
rm -f /usr/tmp/$0_output_$$

Rolando V. Raqueño Tuesday, April 8, 2025

 pnmtojpeg summary
• Produced a “new” pbmplus utility
• Used UNIX shell scripting
– Argument handling
– Scratch space /usr/tmp
– Process id handling
– Clean up
• Integrated IDL program and commands
• 21 lines of UNIX commands

Rolando V. Raqueño Tuesday, April 8, 2025

 Summary
• The “dot” files
• Basics of Shell Scripting
• Special Shell Variables
• Seamless integration of UNIX to other
utilities (IDL)

Rolando V. Raqueño Tuesday, April 8, 2025

 Other Shell Constructs
to keep in mind
• foreach
• while
• case

Rolando V. Raqueño Tuesday, April 8, 2025