Ans 6

Pico is the easiest editor to learn, emacs is the most powerful editor and has a built-in tutorial, and vi is present on essentially all Unix systems. These editors are all text-mode editors and do not require an X-server to be running on your PC.

vi Editor
General Introduction
The vi editor (short for visual editor) is a screen editor which is available on almost all Unix systems. Once you have learned vi, you will find that it is a fast and powerful editor. vi has no menus but instead uses combinations of keystrokes in order to accomplish commands. If you are just beginning to learn Unix, you might find the Pico editor easier to use (most command options are displayed at the bottom of the screen). If you use the Pine email application and have composed or replied to a message you have probably already used Pico as it is used for text entry. For more information please refer to the Pine/Pico page.

Starting vi
To start using vi, at the Unix prompt type vi followed by a file name. If you wish to edit an existing file, type in its name; if you are creating a new file, type in the name you wish to give to the new file. %vi filename Then hit Return. You will see a screen similar to the one below which shows blank lines with tildes and the name and status of the file. ~ ~ "myfile" [New file]

vi's Modes and Moods

vi has two modes: the command mode and the insert mode. It is essential that you know which mode you are in at any given point in time. When you are in command mode, letters of the keyboard will be interpreted as commands. When you are in insert mode the same letters of the keyboard will type or edit text. vi always starts out in command mode. When you wish to move between the two modes, keep these things in mind. You can type i to enter the insert mode. If

you wish to leave insert mode and return to the command mode, hit the ESC key. If you're not sure where you are, hit ESC a couple of times and that should put you back in command mode. General Command Information As mentioned previously, vi uses letters as commands. It is important to note that in general vi commands:

are case sensitive - lowercase and uppercase command letters do different things are not displayed on the screen when you type them generally do not require a Return after you type the command.

You will see some commands which start with a colon (:). These commands are ex commands which are used by the ex editor. ex is the true editor which lies underneath vi -- in other words, vi is the interface for the ex editor.

Entering Text
To begin entering text in an empty file, you must first change from the command mode to the insert mode. To do this, type the letter i. When you start typing, anything you type will be entered into the file. Type a few short lines and hit Return at the end of each of line. Unlike word processors, vi does not use word wrap. It will break a line at the edge of the screen. If you make a mistake, you can use the Backspace key to remove your errors. If the Backspace key doesn't work properly on your system, try using the Ctrl h key combination.

Cursor Movement
You must be in command mode if you wish to move the cursor to another position in your file. If you've just finished typing text, you're still in insert mode and will need to press ESC to return to the command mode. Moving One Character at a Time Try using your direction keys to move up, down, left and right in your file. Sometimes, you may find that the direction keys don't work. If that is the case, to move the cursor one character at the time, you may use the h, j, k, and l keys. These keys move you in the following directions:
h j left one space down one space l k right one space up one space

If you move the cursor as far as you can in any direction, you may see a screen flash or hear a beep. Moving among Words and Lines While these four keys (or your direction keys) can move you just about anywhere you want to go in your file, there are some shortcut keys that you can use to move a little more quickly through a document. To move more quickly among words, you might use the following:

w b e

moves moves word, moves

the cursor forward one word the cursor backward one word (if in the middle of a b will move you to the beginning of the current word). to the end of a word.

To build on this further, you can precede these commands with a number for greater movement. For example, 5w would move you forward five words; 12b would move you backwards twelve words. [You can also use numbers with the commands mentioned earlier. For example, 5j would move you down 5 characters.] Command Keys and Case You will find when using vi that lower case and upper case command keys are interpreted differently. For example, when using the lower case w, b, and e commands, words will be defined by a space or a punctuation mark. On the other hand, W, B, and E commands may be used to move between words also, but these commands ignore punctuation. Shortcuts Two short cuts for moving quickly on a line include the $ and the 0 (zero) keys. The $ key will move you to the end of a line, while the 0 will move you quickly to the beginning of a line. Screen Movement To move the cursor to a line within your current screen use the following keys:
H M L moves the cursor to the top line of the screen. moves the cursor to the middle line of the screen. moves the cursor to the last line of the screen.

To scroll through the file and see other screens use:

ctrl-f ctrl-b ctrl-u ctrl-d scrolls scrolls scrolls scrolls down one screen up one screen up a half a screen down a half a screen

Two other useful commands for moving quickly from one end to the other of a document are G to move to the end of the file and 1G to move to the beginning of the file. If you precede G with a number, you can move to a specific line in the document (e.g. 15G would move you to line 15). Moving by Searching One method for moving quickly to a particular spot in your file is to search for specific text. When you are in command mode, type a / followed the text you wish to search for. When you press Return, the cursor will move to the first incidence of that string of text. You can repeat the search by typing n or search in a backwards direction by using N.

Basic Editing
To issue editing commands, you must be in command mode. As mentioned before, commands will be interpreted differently depending upon whether they are issued in lower or upper case.

Also, many of the editing commands can be preceded by a number to indicate a repetition of the command. Deleting (or Cutting) Characters, Words, and Lines To delete a character, first place your cursor on that character. Then, you may use any of the following commands:
x X dw dd D deletes deletes deletes deletes deletes the character under the cursor. the character to the left of your cursor. from the character selected to the end of the word. all the current line. from the current character to the end of the line.

Preceding the command with a number will delete multiple characters. For example, 10x will delete the character selected and the next 9 characters; 10X will delete the 10 characters to the left of the currently selected character. The command 5dw will delete 5 words, while 4dd deletes four lines. Pasting Text using Put Often, when you delete or cut text, you may wish to reinsert it in another location of the document. The Put command will paste in the last portion of text that was deleted since deleted text is stored in a buffer. To use this command, place the cursor where you wish the deleted text to appear. Then use p to reinsert the text. If you are inserting a line or paragraph use the lower case p to insert on the line below the cursor or upper case P to place in on the line above the cursor. Copying Text with Yank If you wish to make a duplicate copy of existing text, you may use the yank and put commands to accomplish this function. Yank copies the selected text into a buffer and holds it until another yank or deletion occurs. Yank is usually used in combination with a word or line object such as the ones shown below:
yw yy copies a word into a buffer (7yw copies 7 words) copies a line into a buffer (3yy will copy 3 lines)

Once the desired text is yanked, place the cursor in the spot in which you wish to insert the text and then use the put command (p for line below or P for line above) to insert the contents of the buffer. Replacing or Changing Characters, Words, and Lines When you are using the following commands to replace text, you will be put temporarily into insert mode so that you can change a character, word, line, or paragraph of text.
r R replaces the current character with the next character you enter/type. Once you enter the character you are returned to command mode. puts you in overtype mode until you hit ESC which will then return you to command mode.

cw changes and replaces the current word with text that you type. A dollar sign marks the end of the text you're changing. Pressing ESC when you finish will return you to command mode.

Inserting Text If you wish to insert new text in a line, first position the cursor to the right of where you wish the inserted text to appear. Type i to get into insert mode and then type in the desired text (note that the text is inserted before the cursor). Press ESC to return to command mode. Inserting a Blank Line To insert a blank line below the line your cursor is currently located on, use the o key and then hit ESC to return to the command mode . Use O to insert a line above the line the cursor is located on. Appending Text You can use the append command to add text at any place in your file. Append (a) works very much like Insert (i) except that it insert text after the cursor rather than before it. Append is probably used most often for adding text to the end of a line. Simply place your cursor where you wish to append text and press a. Once you've finished appending, press ESC to go back to command mode. Joining Lines Since vi does not use automatic word wrap, it is not unusual in editing lines to end up with lines that are too short and that might be improved if joined together. To do this, place your cursor on the first line to be joined and type J. As with other commands, you can precede J with a number to join multiple lines (4J joins 4 lines). Undoing Be sure to remember this command. When you make a mistake you can undo it. DO NOT move the cursor from the line where you made the change. Then try using one of the following two commands:
u U undoes the last change you made anywhere in the file. Using u again will "undo the undo". undoes all recent changes to the current line. You can not have moved from the line to recover the original line.

Closing and Saving Files

When you edit a file in vi, you are actually editing a copy of the file rather than the original. The following sections describe methods you might use when closing a file, quitting vi, or both.

Quitting and Saving a File The command ZZ (notice that it is in uppercase) will allow you to quit vi and save the edits made to a file. You will then return to a Unix prompt. Note that you can also use the following commands:
:w :q :wq to save case of to quit to quit your file but not quit vi (this is good to do periodically in machine crash!). if you haven't made any edits. and save edits (basically the same as ZZ).

Quitting without Saving Edits Sometimes, when you create a mess (when you first start using vi this is easy to do!) you may wish to erase all edits made to the file and either start over or quit. To do this, you can choose from the following two commands:
:e! :q! reads the original file back in so that you can start over. wipes out all edits and allows you to exit from vi.

More about Combining Commands, Objects, and Numbers

Now that you've learned some basic vi commands you might wish to expand your skills by trying some fancy combination steps. Some commands are generally used in combination with a text object. We've already seen some examples of this. For example, when you use the command dw to delete a word, that combines the delete (d) command with the word (w) text object. When you wish to delete multiple words, you might add a number to this combination. If you wished to delete 2 words you might use 2dw or d2w. Either of these combinations would work. So, as you can see, the general format for a command can be (number) (command) (text object) or (command) (number) (text object) You might wish to try out some of the following combinations of commands and objects:
Command Text Object d (delete) w (word to the left) y (yank/copy) b (word to the right or backward) c (change) e (end of word) H (top of the screen) L (bottom of the screen) M (middle of the screen) 0 (zero - first character on a line) $ (end of a line) ( (previous sentence) ) (next sentence) [ (previous section) ] (next section)

Repeating a Command
If you are doing repetitive editing, you may wish to use the same command over and over. vi will allow you to use the dot (.) to repeat the last basic command you issued. If for example, you

wished to deleted several lines, you could use dd and then . (dot) in quick succession to delete a few lines.

A Quick Word about Customizing Your vi Environment

There are several options that you can set from within vi that can affect how you use vi. For example, one option allows you to set a right margin that will then force vi to automatically wrap your lines as you type. To do this, you would use a variation of the :set command. The :set command can be used to change various options in vi. In the example just described, you could, while still in vi, type :set wrapmargin=10 to specify that you wish to have a right margin of 10. Another useful option is :set number. This command causes vi to display line numbers in the file you are working on. Other Options To view a listing of other options, you could type :set all. To view only those options which are currently in effect, you can type set: by itself. Options that you set while in a vi session will apply during that session only. To make permanent changes to your vi environment, you could edit your .exrc file. However, you should not edit this file unless you know what you are doing!

Useful vi Commands
Cut/Paste Commands:
x dw dd D d$ :u p,P location J "[a-z]nyy "[a-z]p/P line delete one character (destructive backspace) delete the current word (Note: ndw deletes n numbered words) delete the current line (Note: ndd deletes n numbered lines) delete all content to the right of the cursor same as above undo last command paste line starting one line below/above current cursor combine the contents of two lines yank next n lines into named buffer [a-z] place the contents of selected buffer below/above the current

Extensions to the Above Commands:

:3,18d 16,25m30 23,29co62 delete lines 3 through 18 move lines 16 through 25 to after line 30 copy specified lines and place after line 62

Cursor Relocation commands:

:[n] shift g h/l/j/k ^f/^b goto line [n] place cursor on last line of text move cursor left, right, down and up move forward, backward in text, one page

^u/^d $ 0

move up, down one half page move to end of line move to beginning of line

Extensions to the Above:

b words) e ( ) move backwards one word (Note: nb moves back n number of move to end of current word move to beginning of curent block move to the end of current block

Searching and Substitution commands:

/ [string] ? [string] n N cw c$ c0 :1,$s/s1/s2/g r search forward for string search backwards for string repeat last search repeat search in opposite direction change the contents of the current word, (use ESC to stop replacement mode) Replace all content to the right of cursor (exit replacement mode with ESC) Replace all content to the left of cursor (exit with ESC) (Yow!) global replacement of string1 with string2 replace current character with next character typed

Entering the Insert Mode:

i I a A o/O ESC Begin inserting text at current cursor location Begin inserting text at the beginning of the current line Begin appending text, one character to the right of current cursor location Begin appending text at the end of the current line Begin entering text one line below\above current line Exit insertion mode and return to command mode

Exiting and Entering VI

ZZ :wq :e! :q made) :w save file and exit VI same as above return to last saved version of current file quit without save, (Note :q! is required if changes have been write without exit (:w! to force write)

Fancy Stuff:
:1,10w file :340,$w >> file :sh ^d :![command] write lines 1 through 10 to file newfile write lines 340 through the end of the file and append to file newfile escape temporarily to a shell return from shell to VI execute UNIX command without leaving VI

:r![command] :r[filename] :$r newfile document :r !sort file through :n :^g :set :set :set :set number showinsert all ai

read output of command into VI read filename into VI read in newfile and attach at the end of current read in contents of file after it has been passed the UNIX sort open next file (works with wildcard filenames, ex: vi file*) list current line number show line numbers show flag ("I") at bottom of screen when in insert mode display current values of VI variables set autoindent; after this enter the insert mode and tab, from this point on VI will indent each line to this location. Use ESC to stop the indentations. set the autoindent tab one tab stop to the right set the autoindent tab one stop to the left sets default tab space to number n shift contents of line one tab stop to the right shift contents of line one tab stop to the left

^T ^D :set tabstop=n >> <<

Ans 7a
A filter is a small and (usually) specialized program in Unix-like operating systems that transforms plain text (i.e., human readable) data in some meaningful way and that can be used together with other filters and pipes to form a series of operations that produces highly specific results. As is generally the case with command line (i.e., all-text mode) programs in Unix-like operating systems, filters read data from standard input and write to standard output. Standard input is the source of data for a program, and by default it is text typed in at the keyboard. However, it can be redirected to come from a file or from the output of another program. Standard output is the destination of output from a program, and by default it the display screen. This means that if the output of a command is not redirected to a file or another device (such as a printer) or piped to another filter for further processing, it will be sent to the monitor where it will be displayed. Numerous filters are included on Unix-like systems, a few of which are awk, cat, comm, csplit, cut, diff, expand, fold, grep, head, join, less, more, paste, sed, sort, spell, tail, tr, unexpand, uniq and wc. It is a simple matter to construct a pipeline of commands with a highly specific function by stringing multiple filters together with pipes. As a trivial example, the following would display the last three files in the directory /sbin (which contains basic programs used for system maintenance or administrative tasks) whose names contain the string (i.e., sequence of characters) mk:
ls /sbin | grep mk | sort -r | head -3

The ls command lists the contents of /sbin and pipes its output (using the pipe operator, which is designated by the vertical bar character) to the filter grep, which searches for all files and directories that contain the letter sequence mk in their names. grep then pipes its output to the sort filter, which, with its -r option, sorts it in reverse alphabetical order. sort, in turn, pipes its output to the head filter. The default behavior of head is to read the first ten lines of text or output from another command, but the -3 option here tells it read only the first three. head thus writes the first three results from sort (i.e., the last three filenames or directory names from /sbin that contain the string mk) to the display screen. cat is one of the most frequently used commands on Unix-like systems. It is best known for its ability to display the contents of files rather than for its ability to transform them, and thus it might not initially appear to fall into the category of a filter. However, it has the two additional (and not unrelated) functions of creating files and concatenating (i.e., combining) copies of them, which clearly makes it a filter.

In the next example, cat combines copies of file1, file2 and file3, and this is piped to wc, a filter which by default writes the number of bytes, words and lines to the display monitor:
cat file1 file2 file3 | wc

Although most filters are highly specialized programs with a very limited range of functions, there are a few exceptions. Most notable among them is awk, a pattern matching program that has evolved into a powerful and full-fledged programming language. An important tenet of the Unix philosophy has been to try to develop every program (at least every command line program) so that it is a filter rather than just a stand-alone program.

Ans 7 b

A Unix pipe provides a one-way flow of data. For example, if a Unix users issues the command
who | sort | lpr

then the Unix shell would create three processes with two pipes between them:

A pipe can be explicitly created in Unix using the pipe system call. Two file descriptors are returned--fildes[0] and fildes[1], and they are both open for reading and writing. A read from fildes[0] accesses the data written to fildes[1] on a first-in-first-out (FIFO) basis and a read from fildes[1] accesses the data written to fildes[0] also on a FIFO basis.

When a pipe is used in a Unix command line, the first process is assumed to be writing to stdout and the second is assumed to be reading from stdin. So, it is common practice to assign the pipe write device descriptor to stdout in the first process and assign the pipe read device descriptor to stdin in the second process. This is elaborated below in the discussion of multiple command pipelines.

Multiple Command Pipelines: Architecture

Creating a pipeline between two processes is fairly simple, but building a multiple command pipeline is more complicated. The relationship between all of the processes in question is different than what one would expect when creating a simple pipeline between two processes. Normally a pipeline between two processes results in a fork() where child and parent are able to communicate. In an extension of this model to n pipes, it is natural to assume a chain of processes in which each is the child of the previous one, until the n'th child is forked. But this model does not work because the parent shell must wait for the last command in the pipeline to complete, not the first, as would be the case with a chained pipeline. A multiple process pipeline can be represented graphically as:

In this example we see that the parent shell (163) forks one child process (202) and then waits for it to complete. The child process (202) is the parent of all the pipe command processes. The child creates two pipes and then calls fork() for each of its children (203 and 204). Each new child

process redirects STDIN and STDOUT to a pipe appropriately and calls exec() to execute the proper command. A process that has been exec()ed will never return. When the child (202) of the parent shell (163) reaches the last command it simply redirects STDIN to the second pipe and exec()s the last command. The parent (163) waits for this last command to exit. This is very important. The parent shell must wait on the last command to finish before continuing. If it does not, interactive commands such as "less" will not work properly. The processes in the above figure have the following relationships:
Parent PID ---------163 202 202 Child PID --------202 203 204

One important thing to note here is that each process in the pipeline is a child of the original child of the shell (pid 202). They are not children of each other the further down the pipeline we go. Another thing to note is that only the shell (process 163) executes a wait. All the others simply die after they exec their respective command.
Multiple Command Pipelines: File Descriptor Considerations

In this sample implementation of multiple pipes, the process that is the child of the shell, i.e., 202 above, is responsible for creating all the needed pipes before it forks off any of its children. Thus, each of children has a set of file descriptors for all pipes in the total pipeline. In this example, processes 203 and 204 each have descriptors for the two pipes that are created. The fact that all forked processes have all the file descriptors implies that a programmer must address two problems: o Specifying for each process exactly which pipe file descriptor among the many it has access to is its stdin and its stdout, and o Eventually closing all file descriptors that comprise its pipes so that the pipes don't hang. To take care of the first problem, you must use the Unix command dup2(), to duplicate a pipe file descriptor to stdin or stdout (whichever is appropriate), e.g.,
dup2(pipefd, stdin) or dup2(pipefd, stdout)

To address the second problem, you must be sure that each forked process closes all of its pipe file descriptors before it execs its respective command. In the figure above, for example, when process 202 creates two pipes before it forks process 203 and 204, processes 202, 203 and 204 will all have all four descriptors for the two pipes. In this case, each of these processes, after dup2'ing its respective descriptor to standard in or standard out, must close all four descriptors.

Alternate Architecture for Multiple Pipeline Commands

A second process architecture to implement pipes is shown below. Duplication of files descriptors and closing all inherited descriptors must still be carefully addressed.

This architecture has an advantage over the previous one in that it can be implemented such that each forked shell has knowledge of two pipes and four file descriptors, maximum (shell 202 need create only one pipe itself). In the previous architecture, all the subsequent shells inherit all the pipes and file descriptors from shell 202.
Example Pipe Programs

In /usr/class/cis762/shell/examples there is an example program demonstrating how to use dup2, pipe and exec in the creation of pipes. Please note that in the example the piped commands are hard-coded, i.e., the pipes do not handle any command, or any number of piped commands, as is needed for a general shell piping mechanism. For this reason, the program uses execlp instead of execvp. execlp is easier to use when you know the command name ahead of time. execvp is easier to use if you are generating the command name in a character array and passing an array element to the exec command. You may want to look at a second example of one, hard-coded pipe.

Ans8
A program that normally reads its input from the terminal (standard input) or normally writes its output to the terminal (standard output) may become annoying if you would rather send the input from a file instead of the keyboard or send the output to a file instead of the terminal. This annoyance can be avoided if you happen to be swift with the redirection operators. The redirection operators are "<", ">", and ">>". The first is used to send input to a command. The second is used to create a file and send the output to it. The third is used to append the output to an existing file. An example of the first redirection operator was already given in the Electronic Mail section. Suppose you wanted to put a list of all the people logged on into a file called neatguys with the current time listed at the top of the file.
date > neatguys

would create a file with the date and time in it, and
who >> neatguys

would append the list of users logged on.

cat is one of the most frequently used commands on Unix-like operating systems. It has three related functions with regard to text files: displaying them, combining copies of them and creating new ones. cat's general syntax is
cat [options] [filenames] [-] [filenames]

The square brackets indicate that the enclosed items are optional. Reading Files The most common use of cat is to read the contents of files, and cat is often the most convenient program for this purpose. All that is necessary to open a text file for viewing on the display monitor is to type the word cat followed by a space and the name of the file and then press the ENTER key. For example, the following will display the contents of a file named file1:
cat file1

The standard output (i.e., default destination of the output) for cat, as is generally the case for other command line (i.e., all-text mode) programs, is the monitor screen. However, it can be redirected from the screen, for example, to another file to be written to that file or to another command to use as the input for that command. In the following example, the standard output of cat is redirected using the output redirection operator (which is represented by a rightward pointing angular bracket) to file2:
cat file1 > file2

That is, the output from cat is written to file2 instead of being displayed on the monitor screen. The standard output could instead be redirected using a pipe (represented by a vertical bar) to a filter (i.e., a program that transforms data in some meaningful way) for further processing. For example, if the file is too large for all of the text to fit on the monitor screen simultaneously, as is frequently the case, the text will scroll down the screen at high speed and be very difficult to read. This problem is easily solved by piping the output to the filter less, i.e.,
cat file1 | less

This allows the user to advance the contents of the file one screenful at a time by pressing the space bar and to move backwards by pressing the b key. The user can exit from less by pressing the q key. The standard input (i.e., the default source of input data) for cat, as is generally the case for other commands on Unix-like systems, is the keyboard. That is, if no file is specified for it to open, cat will read whatever is typed in on the keyboard. Typing the command cat followed by the output redirection operator and a file name on the same line, pressing ENTER to move to the next line, then typing some text and finally pressing ENTER again causes the text to be written to that file. Thus, in the following example the text that is typed on the second line will be written to a file named felines:
cat > felines This is not about a feline.

The program is terminated and the normal command prompt is restored by pressing the CONTROL and d keys simultaneously. Repeating the above example without using a redirection operator and specifying a destination file, i.e.,
cat This is not about a feline.

causes the text to be sent to standard output, i.e., to be repeated on the monitor screen.

Concatenation The second role of cat is concatenation (i.e., stringing together) of copies of the contents of files. (This is the source of cat's curious name.) Because the concatenation occurs only to the copies, there is no effect on the original files. For example, the following command will concatenate copies of the contents of the three files file1, file2 and file3:
cat file1 file2 file3

The contents of each file will be displayed on the monitor screen (which, again, is standard output, and thus the destination of the output in the absence of redirection) starting on a new line and in the order that the file names appear in the command. This output could just as easily be redirected using the output redirection operator to another file, such as file4, using the following:
cat file1 file2 file3 > file4

In the next example, the output of cat is piped to the sort filter in order to alphabetize the lines of text after concatenation and prior to writing to file4:
cat file1 file2 file3 | sort > file4

File Creation The third use for cat is file creation. For small files this is often easier than using vi, gedit or other text editors. It is accomplished by typing cat followed by the output redirection operator and the name of the file to be created, then pressing ENTER and finally simultaneously pressing the CONTROL and d keys. For example, a new file named file1 can be created by typing
cat > file1

then pressing the ENTER key and finally simultaneously pressing the CONTROL and d keys. If a file named file1 already exists, it will be overwritten (i.e., all of its contents will be erased) by the new, empty file with the same name. Thus the cautious user might prefer to instead use the append operator (represented by two successive rightward pointing angular brackets) in order to prevent unintended erasure. That is,
cat >> file1

That is, if an attempt is made to create a file by using cat and the append operator, and the new file has the same name as an existing file, the existing file is, in fact, preserved rather than overwritten, and any new text is added to the end of the existing file.

Text can be entered at the time of file creation by typing it in after pressing the ENTER key. Any amount of text can be typed, including text on multiple lines. cat can also be used to simultaneously create a new file and transfer to it the data from an existing file. This is accomplished by typing cat, the name of the file from which the output will come, the output redirection operator and the name of the file to be created. Then pressing ENTER causes the new file to be created and written to. For example, typing the following and then pressing ENTER creates a new file named file2 that contains a copy of the contents of file1:
cat file1 > file2

There is no effect on the contents of file1. (The same thing can, of course, be accomplished just as easily using cp command, which is used to copy files, i.e., cp file1 file2, but the above example does illustrate the great versatility of cat.) A slight modification to the above procedure makes it possible to create a new file and write text into it from both another file and the keyboard. A hyphen surrounded by spaces is added before the input file if the typed-in text is to come before the text from the input file, and it is added after the input file if the typed-in text is to go after the text from the input file. Thus, for example, to create a new file file6 that consists of text typed in from the keyboard followed by the contents of file5, first enter the following:
cat - file5 > file6

Or to create a new file file8 that consists of the contents of file7 followed by text typed in from the keyboard, first enter the following:
cat file7 - > file8

In either case, then press ENTER to move to a new line and type the desired text on any number of lines. Finally press ENTER once more followed by pressing the CONTROL and d keys simultaneously to execute (i.e., run) the command. An example of a practical application for this use of cat is the creation of form letters (or other documents) for which only the top parts (e.g., dates and names) are customized for each recipient.

Redirection in Unix
Every UNIX process is assigned 3 files at process creation time: Standard Input (file 0), Standard Output (file 1) Standard Error (file 2).

In most cases these files should be allowed to retain their default direction, standard input points at the keyboard (or the file if the process is created as part of a script), standard output and standard error point to the display. However, there are times when these destinations are not compatible with the desires of the programmer. In such cases, it is important to know how to redirect these three files.

Standard Input
The < operator is usually used to direct input into a process or command. For example, the command
cat < myfile

would take input from the file designated as myfile rather than from standard input. (There is another way to accomplish this, and that is to use the file name as an option to the cat command
cat myfile

There is a subtle difference here, you cannot use file matching metacharacters with redirection. The command
cat my*

is valid but
cat < my*

is not. You can use file matching metachartacters if the file name is an option, but not if the file name is the source for input redirection. Redirection must happen from an explicitly specified file or device.

Standard Output
There are two output redirection operators in UNIX. 1. date > myfile redirects the output of a process (date) to the file specified after the metacharacter, in this case, myfile. Two significant things happen when this type of redirection is used: First, UNIX looks to see if the file specified already exists. If it does, then the next byte pointer (NBP) in the file is set to byte zero. That means the contents of the file are instantly, absolutely, totally, and irrevocably lost. For example, this code shows the fastest cat possible:
cat myfile > myfile

When the shell starts to execute this command, it first does file handling, and sees the redirection. Since the first thing it does when a file is the target of a redirection is to set the next byte pointer to byte zero, the file "myfile" suddenly becomes very, very empty. Cating an empty file is pretty quick! Next, UNIX begins transferring the data created by the process to the specified file. (Note: on some Unix systems and in some shells, you or a kind system administrator may have set the "noclobber" shell variable. If this is the case, redirection of output to an existing file will be blocked. The noclobber shell variable is available in most shells, the usual exception is the Bourne shell.) 2. date >> myfile redirects the output of a process and appends it to the end of the existing file, myfile. For this redirection to work, the file should already exist. In some operating systems, this command will not create a new file, and may generate an error message. In other systems, like the Sun OS this command will create a new file if myfile does not exist.

Standard Error
The redirection of standard error is similar to the redirection of standard output. The only difference is the way the file must be specified.
Bourne Shell and its descendants:

In the Bourne shell, the redirection command for standard error is

2>

indicating that file number 2 is being redirected. (Remember that standard error is file number 2.) For example,
ls > mydir 2> mydirerr

will redirect the regular output from the command to the file mydir, and any error messages to the file mydirerr. (With standard output, the file number is optional. ls 1> mydir is a valid redirection, and is the same as ls > mydir.)
C Shell and its descendants:

It is slightly more difficult to direct Standard Error separately from standard output. The command ls >& mydir will direct both standard output and standard error to mydir.

To create different files for the standard output and the error messages, you must use a subshell command. What you need to do is direct standard output to one file, then redirect both standard out and standard error to a second file. This will result in standard output messages sent to one file and the standard error messages to the second file. The code to perform this looks like:
(myprog > outfile) >& errorlog

The way this works is that the code inside the parentheses will be executed first, (the term for this is a subshell.) Within the subshell, the standard output data stream is directed to the file outfile. When the subshell finishes its work, the second redirection sends the data from both standard out and standard error to the file errorlog. However, all that is left in that output stream are the error messages, because the data intended for standard out has already been redirected inside the subshell. This is a somewhat cumbersome, albeit elegant solution to meet the need to divide the two output streams.

Alternate input redirection

The here document

The last form of redirection is only useful within shell scripts. It is called a "here document", and I have absolutely NO idea why it is called that. The code below shows how to use a here document in a script. The shell will redirect all of the lines between the beginning marker for the here document and the ending marker into the command specified just as if the text were coming from standard input. In some systems, the "here document" will create a temporary file. In the example below, the here document is used to provide a menu to the user. The second example shows how the same result could be accomplished using "echo" statements instead.
# present a menu cat << alldone What would you like to do today? Play with shell scripts Write wonderful awk scripts Invent powerful sed scripts zippity do and dah alldone

The code above will generate the following output:

What would you like to do today? Play with shell scripts Write wonderful awk scripts Invent powerful sed scripts zippity do and dah

Notice that the indentation is maintained, that is one of the benefits of this type of redirection, you can easily lay out the page. The same output could be realized by a set of "echo" commands but as you can see from the example below, it is a LOT more coding!
echo"What echo" echo" echo" echo" would you like to do today?" Play with shell scripts" Write wonderful awk scripts" Invent powerful sed scripts" zippity do and dah"

Another interesting thing about "here documents" is how they handle shell variables. If you use the form shown above, the shell will expand any shell variables located within the text for the here document, but if you add the protected slash to the ending mark, the shell will NOT expand shell variables. The following two code snippets differ only in the way the the ending mark is coded, however the output is significantly different:
#!/bin/sh var1="super" var2="duper" cat << enuf This is a $var1 $var2 example of a here document enuf

The output of the script above looks like:

This is a super duper example
#!/bin/sh var1="super" var2="duper" cat << \enuf This is a $var1 $var2 example of a here document enuf

Notice the slash before the ending mark. (I know, hard to miss, it is a bit obvious, color-wise.) Because of that slash, the shell will not expand any shell variables in the here document, as we see from the output shown below:
This is a $var1 $var2 example of a here document

Even though here documents are usually only used in scripts, they are very powerful tools. Experiment with them.

Ans 9 Introduction to Environment Variables

Introduction to Environment Variables
As is the case with virtually all programming languages today, the Unix shell provides support for program data storage via the use of variables. In the Unix shell, these variables are referred to as environment or shell variables. As with all variables, shell variables can be assigned values, changed, manipulated and displayed. Shell variables can be used at the command line and/or used within shell programs. There are system defined environment variables, as well as user defined environment variables. The only difference between the two is who created and assigned values to them. While similar to other programming language variables, shell variables do have some different characteristics such as:

shell variables do not need to be declared (or typed) all shell variables are of type string there are no memory allocation considerations, nor memory cleanup (e.g. memory free or garbage collection) issues
variable=value

Environment variable assignment is accomplished using the assignment operator (=) as follows: where the value on the right of the = operator is assigned to the variable on the left of the = operator. Note variable assignment always works from right to left. Also pay close attention to the format of the assignment operator where there can be NO spaces on either side of the equal sign (except within the C shell). Including spaces before or after the assignment operator will result in errors. When naming environment variables, the rules and conventions are as follows; all variables must begin with a letter (or an underscore), followed by zero or more alpha-numeric chars or underscores. While there is a maximum length, it is quite large and should not be a limiting factor. By convention, variable names are typically typed as uppercase (which helps distinguish them from shell commands). Also by good programming convention, varible names should be descriptive as to what their purpose is. For example, if you are going to use a variable to store a functions return value of zero (as a character), you would do the following:
$ RETVAL=0 [Enter]

When accessing the data stored within a shell variable, you must prefix the variable name with a dollar sign ($). As the shell interprets a variable name following the $, it substitutes the value of

the variable at that point. The echo command is typically used to display the value of a shell variable, for example if the user types:
$ echo $RETVAL [Enter] 0

the variables value of 0 is displayed. Note that if the $ prefix is omitted, the shell will gladly display exactly what the user has told it to display (the string "RETVAL"), and will not display an error. For example:
$ echo RETVAL [Enter] RETVAL

Differing from other languages, shell variables do not cause errors if they are accessed when they are undefined, or have no value. For example:
$ echo $RET_VAL [Enter] $

will merely return a blank line. You can set variables to the values of directory locations, e.g.
$ MY_BIN=/home/mthomas/bin [Enter]

You then have the capability to do things like (assuming the directory location is valid):
$ cd $MY_BIN [Enter] $ pwd [Enter] /home/mthomas/bin

You can add to the value of an existing shell variable such as:
$ MY_BIN=$MY_BIN/new_bin [Enter]

Note that if the user wishes to set the variable to a NULL or empty string, this can be done a couple of ways as follows:
RETVAL= RETVAL="" RETVAL=''

One important note here is the user can also remove the existance of an environment variable with the unset command. The unset command is substantially different than setting a variables value to NULL, and can have drastic effects (more on this later). To unset a variable, you use the unset command as follows:
unset VARIABLE_NAME

Note you do not use the $ when unsetting a variable since you wish to unset the variable, not the variables contents (make sure you understand this). Using the $ when unsetting a variable will result in an error. There is also an alternate notation for referring to and working with shell variables. This notation's syntax is ${VAR} rather than $VAR and is generally referred to as parameter expansion. Imagine you have a filename stored in a variable, and you want to rename the filename and append the number 1 to the filename. You might try something like:
$ mv $FNAME $FNAME1 [Enter]

This will not work since the shell will think $FNAME1 is a variable, and since it has no value associated with it, an error will occur. Thus you can make this work using the following command:
$ mv $FNAME ${FNAME}1 [Enter]

Variables and Child Processes

When you define a variable within a shell, or a child process, the variable is essentially a local variable. This implies the value of the variable can only be seen in that processes environment. Examine this behavior in the following diagrams:

In the example above, the variable X is assigned the (local) value of 10 in the parent (ksh) process environment. When a child process (bash) is created, the value of $X is not visible in that environment since it is local and visible only in the parent process environment.

In the example above, a child process (bash) is started, a local variable (Y) in the child process is assigned the value of 20, which is then displayed, and control is returned to the parent (via an exit command). When the variable Y is examined in the parent process, it holds no value since the variable Y in the child was local and its value was only visible in the child environment. If child processes could never see environment variables, variables would have severe limitations and not be all that useful. Thus there is a way to make variables visible to child processes. This is done using the export command as follows:
$ X=10 [Enter] $ export X [Enter]

What this essentially does is makes the variable X a global variable (sort of). This means is the value of the variable X is now visible in all child processes created from the parent where it was exported. Note the we export the variable and not the value stored in the variable ($X). Environment variables can be exported at any time including before their use. However it makes sense to export variables near their assignment for program readability (unless this is not desired

behavior). A variable need only be exported once; once global, it remains global.You can also do things like the following:
$ X=foo ; Y=bar [Enter] $ export X Y [Enter]

The semicolon (;) in the above example allows multiple single commands to be placed on a single line. This is commonly done for readability reasons (more on this later). Also note that some newer shells allow combining of variable assignment and exportation as follows:
$ export X=foo

Examine the behavior of an exported variable in following diagram:

This can be taken a step further; if a variable is exported, it is visible in all child processes (including processes created from child processes, and so on) created from the same parent. One extremely important point needs to be made here, while exporting a variable makes it global in scope, there is no way to change the value of a varible in a parent process from a child process.
$ X=Fred ; Y=Barney [Enter] $ export Y [Enter] $ echo "X: $X Y: $Y" [Enter] X: Fred Y: Barney $ ksh [Enter] $ echo "X: $X Y: $Y" [Enter] X: Y: Barney $ Y=Betty [Enter] $ echo "X: $X Y: $Y" [Enter] X: Y: Betty $ exit [Enter] $ echo "X: $X Y: $Y" [Enter] X: Fred Y: Barney # assign X, Y # make Y global # examine X, Y) # create new child # X local so empty # new value for Y # return to parent # original values

To summarize environment variables and child processes:

Any shell variable not exported is a local variable whose value will not be visible to any child processes.

Any shell variable which is exported will be visible to all child processes, and all processes of child processes, and so on. Global variables can be assigned new values, where the new value will then be visible in child processes from that point forward. Child processes can never change a value of a variable in a parent process context.

Arithmetic Using Shell Variables

As mentioned at the beginning of this section, all shell variables are of type string. This might lead one to conclude that arithmetic using shell variables is not possible because you can't do arithmetic on strings. Not so fast! The standard tried and true way of doing arithmetic is using the expr command. In general, expr is used as follows:
expr operand1 operator operand2

Note the spaces on either side of the operator, these are mandatory and a source of frequent errors. Some of the possible operators include:
addition + subtraction multiplication * here) division modulus / % # adding 10 + 2 # must be written as \*, more

Thus you can do arithmetic in the shell such as:

$ expr 10 + 2 [Enter] 12

$ I=10 $ expr $I + 2 [Enter] 12 # same using a variable

$ expr 10 \* 2 [Enter] 20

# multiplying by 2

There is an alternative way to performing arithmetic calculations available in some of the newer shells (e.g. bash, ksh93). This newer method (sometimes referred to as let) uses the following syntax: $((expression)) . For example:
$ echo $((10 + 2)) [Enter] 12 # with spaces around operator

$ echo $((10+2)) [Enter] 12

# without spaces

$ X=10 [Enter] $ echo $((X + 2)) [Enter] 12 # note no $ on X

$ echo $(($X + 2)) [Enter] 12

# with $ on X

$ echo $((X * 2)) [Enter] 20

# no \ on *

Other Shell Variables

In general, there are 3 types of variables; standard variables, positional variables, and special shell variables. The discussion of variables to this point has been about standard environment variables. Postitional variables (also called positional parameters) are named due to the way they are assigned values, that is based upon their position on the command line. Unlike standard variables, positional variables are typically defined (and accessed) from within a shell script. As mentioned, these variables are named based upon their respective position on the command line. Thus the first argument on the command line (following the program name) is assigned to the variable named 1, the second argument on the command is assigned to the variable named 2, and so on. As with standard variables, access to the values stored within positional variables is achieved using the dollar sign prefix, thus the variables are commonly referred to as $1, $2, etc. In the example below, the name of the shell program is my_script, and the first positional parameter (named 1) has the value within the script of arg1, the second variable the value of arg2, etc.

Thus, if we wish to see the value stored in the first postional parameter, we could do the following from within the my_script program (note that this only works from within the my_script program):
echo $1 arg1

Positional parameters provide the programmer with a powerful way to "pass data into" a shell program while allowing the data to vary. If we had a shell program named hello that contained the following statement:
echo Hello Fred! How are you today?

this would not be very interesting to run, unless perhaps your name was Fred. However if the program was modified like this:
echo "Hello $1! How are you today?"

This would allow us to pass single data values "into" the program via positional parameters as illustrated in the following diagram:

We could then run the program as follows, using varying values to pass into the positional variable $1.
$ hello Fred [Enter] Hello Fred! How are you today? $ hello Barney [Enter] Hello Barney! How are you today?

It should be obvious that this would be a much more useful program. Keep in mind that many behaviors of standard variables are also behaviors of positional variables. For example, if you did not assign a value to the first positional variable, you would not get an error, rather behavior as follows:
$ hello [Enter] Hello ! How are you today?

Similarly, if there are more command line arguments than positional variables, the extra arguments are simply ignored, for example:
$ hello Fred Barney Dino [Enter] Hello Fred! How are you today?

There are also variables referred to as special variables within the various Unix shells. While there are many special variables, I am going to focus on four here. The first three of these are closely related to the positional parameters discussed above, and thus must be referenced from within the program itself. These variables have the names of $*, $# and $0 (note that these special variables are actually named *, #, and 0, but values stored within must be accessed by $*, etc.). The $0 variable is the name of the program/script being executed. The variable $* represents the list of all command line arguments passed into the script. This is effectively a list containing all positional variables. The $# variable is the number of arguments contained in the $* list. Thus the positional variables for any script range from 1 to $#. Referring to the following diagram, we observe:

the variable $0 refers to the name of the script, the positional parameters refer to each individual command line argument, and the variable $* refers to the entire argument list. We could then modify our hello program to contain the following 2 command statements:
#!/bin/ksh # program name: hello echo "I am program $0 and I have $# argument(s)." echo "Hello $*! How are you today?"

We could then run the program as follows:

$ hello Fred [Enter] I am program hello and I have 1 argument(s). Hello Fred! How are you today? $ hello Fred Barney Dino [Enter] I am program hello and I have 3 argument(s). Hello Fred Barney Dino! How are you today?

Should a user wish, one can also pass NULL or empty values into a shell program as follows:
$ hello Fred "" Dino [Enter] I am program hello and I have 3 argument(s). Hello Fred Dino! How are you today?

In this scenario, there are still 3 values being passed into the hello program (as evident with the $# variable), it is simply that the 2nd argument is NULL. To change directions a bit, the special variable $? is a slightly different creature. Up to this point, I have implied that no data values (e.g. environment variables or changes made to enviroment variables) return from child processes to the parent parent. This is mostly true, except in a single

case. The variable $? contains the numeric exit status of the last returning child process. This is the only data value ever returned from a child processto the parent. A value of zero implies success (however that is measured) and a non-zero (not necessarily a one) implies non-success. Note that the status of success and non-success are relative to the commands executed. It is extremely important to understand while this is a binary condition, the values are not zero and one, they are zero and non-zero (frequently it will be one, but not always). Refer to the following diagram:

As mentioned above success and non-success is measured differently for each command. In general, there are three scenarios to illustrate this concept. The first is that of success. An example of this is the attempt to list a file which does exist, and results in that file being listed. The second scenario is that of non-success. An example of this is the attempt to list a file that does not exist. This may not be a catastrophic event, that is if the file does not exist, it may need to be created. However, this is still an non-success by the ls command. The third scenario is that of more severe non-success (i.e. failure), for example mispelling the ls command. In the latter two cases, the $? variable will return a non-success status, but the meaning of each is different. Examine the following examples:
$ ls foo [Enter] foo $ echo $? [Enter] 0 $ ls bar [Enter] ls: bar: No such file or directory $ echo $? [Enter] non-success 1 $ sl foo [Enter] ksh: sl: not found $ echo $? [Enter] success 127 # list file foo # file exists, return success # list file bar # file does not exist, return

# typing error # erroneous command, return non-

Note that the $? variable only returns the status from the last returning child process. If we examine the following:
$ ls bar [Enter] # list file bar

ls: bar: No such file or directory $ echo $? [Enter] non-success 1 $ echo $? [Enter] 0

# file does not exist, return

In the second echo of $?, the status returned was 0 because the previous child process was the previous echo statement, which was indeed successful. Sometimes you will want to store the return status of a process for examination at a later time. In this case, you can simply store the value of the $? variable in a ordinary value as follows:
ERR1_STATUS=$?

Command Summary

echo - display a line of text (including the contents of variables) env - display the current environment (when used without a command) expr - expression evaluation let - alternative method for expression evaluation (see also let built-in and Arithmetic Expansion) parameter expansion - see Parameter Expansion for ${parameter} syntax positional parameters - description of the positional parameters, i.e. $*, $?, $#, etc. unset - unset the specified variable (or function)