Assignment 1
Assignment 1
Assignment 1
INTRODUCTION
A shell, or command line interpreter program, is a mechanism with which each interactive user can send
commands to the OS and by which the OS can respond to the user. Whenever a user has successfully
logged in to the computer, the OS causes the user process assigned to the login port to execute a specific
shell. The OS does not ordinarily have a built-in window interface. Instead, it assumes a simple character-oriented interface in which the user types a string of characters (terminated by pressing the Enter or
Return key) and the OS responds by typing lines of characters back to the screen. If the human-computer
interface is to be a graphical windows interface, then the software that implements the window manager
subsumes the shell tasks that are the focus of this exercise. Thus the character-oriented shell assumes a
screen display with a fixed number of lines (usually 25) and a fixed number of characters (usually 80) per
line.
Once the shell has initialized its data structures and is ready to start work, it clears the 25-line display and
prints a prompt in the first few character positions on the first line. Linux systems are usually configured to
include the machine name as part of the prompt. For example, my Linux machine is named
kiowa.cs.colorado.edu, so the shell prints, as its prompt string:
kiowa>
or
bash>
depending on which shell I am using. The shell then waits for the user to type a command line in response
to the prompt. The command line could be a string such as:
kiowa> ls al
terminated with an <ENTER> or return character (in Linux, this character is represented internally by the
NEWLINE character, '\n'). When the user enters a command line, the shell's job is to cause the OS to
execute the command embedded in the command line.
Every shell has its own language syntax and semantics. In the standard Linux shell, bash, a command line
has the form:
command [arg1] [arg2] ... [argN]
in which the first word is the command to be executed and the remaining words are arguments expected
by that command. The number of arguments depends on which command is being executed. For example,
the directory listing command may have no arguments-simply by the user's typing ls or it may have
arguments prefaced by the negative - character, as in ls al, where a and l are arguments. The
command determines the syntax for the arguments, such as which of the arguments may be grouped (as
for the a and l in the ls command), which arguments must be preceded by a "-" character, and
whether the position of the argument is important.
Other commands use a different argument-passing syntax. For example, a g++ compiler command might
look like:
kiowa> g++ -g -o deviation -S main.cpp inout.cpp lmath
in which the arguments g, o deviation, S, main.cpp, inout.cpp, and lmath are all passed to
the C++ compiler, g++.
The shell relies on an important convention to accomplish its task: The command for the command line is
usually the name of a file that contains an executable program, for example, ls and g++ (files stored in
/bin on most UNIX-style machines). In a few cases, the command is not a filename but rather a
command that is implemented within the shell. For example, cd (change directory) is usually
implemented within the shell itself rather than in a file in /bin. Because the vast majority of the commands
are implemented in files, you can think of the command as actually being a filename in some directory on
the machine. This means that the shell's job is to:
1 - find the file
2 - prepare the list of parameters for the command,
3 - cause the command to be executed using the parameters.
Many shell programs are used with UNIX variants, including the original Bourne shell (sh), the C shell
(csh) with its additional features over sh, the Korn shell, and the standard Linux shell (bash). All have
followed a similar set of rules for command line syntax, though each has a superset of features.
For the first command line, the shell will find the g++ command (the C++ compiler) in the /bin directory and
then, when the g++ command is executed, pass it the string main.cpp. The C++ compiler will translate
the C++ program that is stored in main.cpp and write the resulting object file named main.o in the current
directory. The next command links the object file into an executable. The third command is simply the
name of the file to be executed, main, without any parameters. The shell finds the main file in the current
directory and then executes it.
Consider the following steps that a shell must take to accomplish its job.
1. Print a prompt.
A default prompt string is available, sometimes hardcoded into the shell, for example the single
character string %, #, or >. When the shell is started, it can look up the name of the machine on
which it is running and prepend this string to the standard prompt character, for example a prompt
string such as kiowa>. The shell also can be designed to print the current directory as part of the
prompt, meaning that each time that the user types cd to change to a different directory, the prompt
string is redefined. Once the prompt string is determined, the shell prints it to stdout whenever it is
ready to accept a command line.
2. Get the command line.
To get a command line, the shell performs a blocking keyboard input operation so that the process that
executes the shell will be asleep until the user types a command line in response to the prompt. Once
the user types the command line (and terminates it with a NEWLINE ('\n') character), the command
line string is returned to the shell.
3. Parse the command.
The syntax for the command line is trivial. The parser begins at the left side of the command line and
scans until it sees a whitespace character (such as space, tab, or NEWLINE). The first word is the
command name, and subsequent words are the parameters.
4. Find the file.
The shell provides a set of environment variables for each user. These variables are first defined in the
user's Iogin file (for the bash shell this is /home/<username>/.bashrc) but they can be modified at
any time by using the set command. The PATH environment variable (whose value can be viewed by
typing echo $PATH at the bash shell) is an ordered list of absolute pathnames specifying where the
shell should search for command files. If the Iogin file has a line such as:
set path=(.:/bin:/usr/bin)
then the shell will first look in the current directory (since the first full pathname is ":."), then in /bin,
and finally in /usr/bin. If no file with the same name as the command can be found in any of the
specified directories, then the shell notifies the user that it is unable to find the command.
5. Prepare the parameters. The shell simply passes the parameters to the command as the argv
array of pointers to strings.
load the file into the address space currently being used by the calling process (overwriting and
discarding the previous program),
set the argv array and environment variables for the new program execution, and start the process
executing at the new program's entry point.
Various versions of execvp() are available at the system call interface, differing in the way that
parameters are specified (for example, some use a full pathname for the executable file and others do
not).
wait()
The wait() system call is used by a process to block itself until the kernel signals the process to execute
again, for example because one of its child processes has terminated. When the wait() call returns as a
result of a child process's terminating, the status of the terminated child is returned as a parameter to the
calling process.
When these three system calls are used, here is the code skeleton that a shell might use to execute a
command:
// Child
if (fork() == 0)
{
execvp(fullpathname, argv);
}
// Parent
else
{
int status=0;
wait(&status);
cout << "Child exited with status of " << status << endl;
}
I/O Redirection
A process, when created, has three default file identifiers: stdin, stdout, and stderr. These three file
identifiers correspond to the C++ objects cin, cout, and cerr. If the process reads from stdin (using
cin) then the data that it receives will be directed from the keyboard to the stdin file descriptor. Similarly,
data received from stdout (using cout) and stderr (using cerr) are mapped to the terminal display.
The user can redefine stdin or stdout whenever a command is entered. If the user provides a filename
argument to the command and precedes the filename with a less than character "< then the shell will
substitute the designated file for stdin; this is called redirecting the input from the designated file.
The user can redirect the output (for the execution of a single command) by preceding a filename with the
right angular brace character, ">" character. For example, a command such as
kiowa> we < main.cpp > program.stats
will create a child process to execute the we command. Before it launches the command, however, it will
redirect stdin so that it reads the input stream from the file main.cpp and redirect stdout so that it
writes the output stream to the file program.stats.
The shell can redirect I/O by manipulating the child process's file descriptors. A newly created child
process inherits the open file descriptors of its parent, specifically the same keyboard for stdin and the
terminal display for stdout and stderr. (This expands on why concurrent processes read and write the
same keyboard and display.) The shell can change the child's file descriptors so that it reads and writes to
files rather than to the keyboard and display.
Each process has its own file descriptor table in the kernel. When the process is created, the first entry in
this table, by convention, refers to the keyboard (stdin) and the second two refer to the terminal display.
Next, the C++ runtime environment and the kernel manage stdin, stdout, and stderr so that:
C++ Object Name
cin
cout
cerr
Alternative Linux
Name
stdin
stdout
stderr
File
Descriptor
Table Index
0
1
2
Device Referred
To
Keyboard
Terminal Display
Terminal Display
If you want to perform I/O redirection, you need to connect stdin to a file which can be read for input
instead of the keyboard, or stdout to a file which can accept output instead of the terminal display.
Lets consider stdin specifically. The key to getting I/O redirection to work for stdin is to replace the
contents of the file descriptor entry for stdin, currently the keyboard, with the file descriptor entry for the
file you want to use for input.
To accomplish this, you need to access the file descriptors for stdin and the file you want to read from.
You know how to get the file descriptor for stdin, thats just the number 0. But how do you get the file
descriptor for the file you want to use for input? Unfortunately, you cant get the file descriptor directly from
a C++ ifstream object. Instead you have to issue the following Linux system call:
int newstdin = open(main.cpp,O_RDONLY);
The second argument to the open() call is an open this file for reading only flag. Now newstdin has a file
descriptor for the file main.cpp.
Next you have to replace the contents of the stdin file descriptor with the main.cpp file descriptor. Use
the following sequence of Linux system calls:
close(0);
dup(newstdin);
close(newstdin);
The close(0) call wipes out the contents of file descriptor table entry 0, which is the table entry for stdin.
The dup(newstdin) call copies the contents of the newstdin file descriptor table entry to the first empty table
entry in the file descriptor table. In this case, that will be entry 0. Now stdin is will use the file main.cpp
instead of the keyboard for input. There are now two file descriptors which are linked to main.cpp. The
final close(newstdin) cleans things up so that only the stdin file descriptor is linked to main.cpp.
A similar set of calls is used if you want to redirect stdout to an output file:
int newstdout = open(program.stats,O_WRONLY|O_CREAT,S_IRWXU|S_IRWXG|S_IRWXO);
close(1);
dup(newstdout);
close(newstdout);
Make sure that when you use this open call you include ALL of the flags Ive listed here. Otherwise the
redirection wont work.
Shell Pipes
The pipe is a common IPC mechanism in Linux and other versions of UNIX. By default, a pipe employs
asynchronous send and blocking receive operations. Optionally, the blocking receive operation may be
changed to be a nonblocking receive. Pipes are FIFO (first-in/first out) buffers designed with an API that
resembles as closely as possible a low level file I/O interface. A pipe may contain a system-defined
maximum number of bytes at any given time, usually 4KB. As indicated in Figure 2.2, a process can send
data by writing it into one end of the pipe and another can receive the data by reading the other end of the
pipe.
int pid;
int thePipe[2];
pipe(thePipe);
pid = fork();
// Parent
if(pid > 0)
{
char message="Hello";
char messageLen=6;
write(thePipe[1], message, messageLen);
cout << "Parent sent: " << message << endl;
}
// Child
else if (pid == 0)
{
char message[6];
int messageLen=6;
read(thePipe[0], message, messageLen);
cout << "Child received: " << message << endl;
}
Pipes enable processes to copy information from one address space to another by using the UNIX file
model. The pipe read and write ends can be used in most system calls in the same way as a file
descriptor. Further, the information written to and read from the pipe is a byte stream. UNIX pipes do not
explicitly support messages, though two processes can establish their own protocol to provide structured
messages. Also, library routines are available that can be used with a pipe to communicate via messages.
A process that does not intend to use a pipe end should close so that end-of-file (EOF) conditions can be
detected.
A named pipe can be used to allow unrelated processes to communicate with each other. Typically in
pipes, the children inherit the pipe ends as open file descriptors. In named pipes, a process obtains a pipe
end by using a string that is analogous to a filename but that is associated with a pipe. This allows any set
of processes to exchange information by using a public pipe whose end names are filenames. When a
process uses a named pipe, the pipe is a system-wide resource, potentially accessible by any process.
Just as files must be managed so that they are not inadvertently shared among many processes at one
time, named pipes must be managed, by using low level file system commands.
Problem A
Write a C++ program that will act as a shell command line interpreter for the Linux kernel. Your shell
program should use the same style as the bash shell for running programs. In particular, when the user
types a line such as:
$ command [parameter1] .... [parameterN]
your shell should parse the command line to build argv. It should search the directory system (in the order
specified by the PATH environment variable) for a file with the same name as the first identifier (which may
be a relative filename or a full pathname). If the file is found, then it should be executed with the optional
parameter list, as is done with bash. If the file is not found, then an error should be printed.
Use the execvp() system call to execute the file that contains the command. You will also need to
become familiar with the Linux fork() and wait() functions.
When the command has completed executing, your shell should prompt the user for another command.
Heres an example of the shell running a simple hello world program followed by the ls command:
BobShell> ./main
Hello World!
BobShell> ls
main.cpp main.o main makefile
BobShell>
Attacking Problem A
The exercise introduction generally describes how a shell behaves. It also implicitly provides a plan of
attack, summarized here. This plan describes several debugging versions that you can use for Part A, then
apply to the other parts as required.
1. Organize the shell to initialize variables and then to perform an endless loop until the shell detects an
EOF condition. When you are reading from stdin, an EOF condition can be:
-
Develop a very simple version that prints the prompt character and then waits for the user to type a
command. After it reads the command, it should print the command to stdout.
2. Refine your simple shell so that it parses the command typed by the user. The parser should do the
following:
-
char* argv[100];
Set the first location in the array after your C-Strings to NULL. You are going to pass the argv
array to the execvp() system call. This system call does NOT take argc as an argument, but the
call still needs to detect the end of the argv array. To detect the end of the argv array, execvp
looks for the first array entry that has the value of NULL.
3. In the next debug version, use argv[0] to find the executable file. In this version, simply print the
filename.
-
Construct a simple version that can find only command files that are in the current directory.
Enhance your program so that it can find command files that are specified with an absolute
pathname.
Enable your program to search directories according to the string that is stored in the shell PATH
environment variable. You can access all of the environment variables by adding the following two
lines before main() in your program:
#include <unistd.h>
extern char *environ[];
The last item in the array is a NULL C-String.
Determine
argv[0]
argv[1]
argv[2]
...
argv[N]
argv[N+1]
Problem B
Add functionality to the shell from Problem A so that a user can use the "&" operator as a command
terminator. A command terminated with "&" should be executed concurrently with the shell (rather than the
shell's waiting for the child to terminate before it prompts the user for another command).
Problem C
Modify your shell program so that the user can redirect the stdin or stdout file descriptors by using the "<"
and ">" characters as filename pre-fixes. For example:
BobShell> ./main > out.txt
BobShell> cat out.txt
Hello World!
BobShell>
will place the output of the Hello World program into the file out.txt rather than sending it to the
screen (which is stdout).
Also, allow your user to use the pipe operator, "I", to execute two processes concurrently, with stdout from
the first process being redirected as the stdin for the second process. For example:
BobShell> cat out.txt
Hello World!
BobShell> cat out.txt | wc l
1
BobShell>
The cat command sends the contents of the out.txt file to stdout. The wc l command counts the
number of characters, words, and lines typed into stdin. The two commands can be strung together
using the pipe operator | so that the output of cat (sent to stdout) is connected to the input of wc
(received from stdin).
You can design your program so that you only have to handle redirection OR pipes in one command line,
but not both.
Attacking Problem C
Dont worry about mixing the &, redirection operators (< and >), and pipe operator | in the same
command.
Concentrate on getting the redirection operators to work first. The key to getting these to work is to
understand the I/O Redirection discussion in this document.
Watch out for file permission problems with the redirection operators. I needed to bone up on the man
pages for file() to get these flags correct. The default permissions for the file open for write meant
subsequent redirects to the same file didn't work of because permission problems.
int newstdin = open(inputFileName.c_str(),O_RDONLY);
To get command line pipes to work, re-read the Shell Pipes section carefully. Since a pipe is just like a
file... theres no reason why you couldnt use I/O Redirection and pipes together!
1 - Have the parent create a pipe.
2 - Spawn a child.
3 - Have the parent redirect stdout to the write portion of the pipe.
4 - Have the parent execute the first command with excecvp().
5 - Have the child redirect stdin to the read portion of the pipe.
6 - Have the child execute the second command with execvp().
7 - The output from the parent command should flow to the child command via the pipe and i/o redirection
you performed on the pipe.
Watch out for (EOF) problems with pipes. I couldn't get the child process to terminate on an eof condition
because the parent didn't close the read portion of the pipe. Make sure that for parent and child that you
call the system call close() for any portion of a pipe you arent using.
General Hints
You should learn to use man pages to get info about important system calls. Sometimes the man page I
wanted didn't come up when I typed man <keyword>. For example:
man wait
gave me lots of information about the bash shell, not the system call wait. But I could
find out about wait by typing:
man 2 wait
I figured this out by looking at the documentation. If you see something like:
See also wait(2)
It means use the syntax above.
You should figure out how to get the error from a system call. If a system call fails, you can find out WHY it
failed in and English friendly way by calling strerror() with the global variable errno. Add this to your code:
#include <errno.h>
#include <string.h>
extern int errno;
. . .
make a system call that fails
cout << strerror(errno) << endl;
DELIVERABLES:
-
All the files necessary to compile, link, and run your program solutions. The final
program that you produce should allow me to test the solutions to A, B, and C.
Use a makefile to build your program so I can do the same thing when I build your
program.
An ELECTRONIC document describing how to run the program you created. Call
this document README.TXT.
These files should be placed in a directory called <username>assignment1.
Use the tar command to place all the files in a single file called
<username>assignment1.tar (e.g. tar cvf bduganassignment1.tar bduganassignment1)