CSCE 515:

Computer Network Programming

------ Web Servers &
Dynamic Web Documents

Wenyuan Xu

http://www.cse.sc.edu/~wyxu/csce515f07.html
Department of Computer Science and Engineering
University of South Carolina
Reference:
http://www.cs.rpi.edu/~hollingd/netprog/notes/dyn_doc/dyn_doc.ppt
Web Server
„ Talks HTTP

„ Looks at METHOD, URI to determine what

the client wants.

„ For GET, URI often is just the path of a file

(relative to some directory on the web
server).

2007 CSCE515 – Computer Network Programming

 GET /foo/blah
/

usr bin www etc

foo fun gif

blah

2007 CSCE515 – Computer Network Programming

In the good old days...
„ Years ago… the WWW was made up of
(mostly) static documents.
… Each URL corresponded to a single file stored
on some hard disk.
„ Today - many of the documents on the
WWW are built at request time.
… URL doesn’t correspond to a single file.

2007 CSCE515 – Computer Network Programming

Dynamic Documents
„ Dynamic Documents can provide:
… automation of web site maintenance
… customized advertising
… database access ea !
o l id
… shopping carts Co
… date and time service

2007 CSCE515 – Computer Network Programming

Web Programming

„ Writing programs that create dynamic

documents has become very important.
„ There are a number of general approaches:
… Create custom server for each service desired.
„ Each is available on different port.
… Have web server run external programs.
… Develop a real smart web server
„ SSI, scripting, server APIs.

2007 CSCE515 – Computer Network Programming

Custom Server
„ Write a TCP server that watches a “well
known” port for requests.
„ Develop a mapping from http requests to
service requests.
„ Send back HTML (or whatever) that is
created/selected by the server process.
„ Have to handle http errors, headers, etc.

2007 CSCE515 – Computer Network Programming

An Example Custom Server
„ We want to provide a time and date
service.
„ Anyone in the world can find out the date
and time (according to our computer)!!!

2007 CSCE515 – Computer Network Programming

Custom Server request
„ We don’t care what is in the http request,
our reply doesn’t depend on it.

„ We assume the request comes from a

browser that wants the content formatted
as an HTML document.

2007 CSCE515 – Computer Network Programming

 WWW based time and date server
Copyright @2003 DaveH Enterprises

… Listenon a well known TCP port.

… Accept a connection.
… Find out the current time and date
… Convert time and date to a string
… Send back some http headers (Content-Type)
… Send the string wrapped in HTML formatting.
… Close the connection.

loop forever

2007 CSCE515 – Computer Network Programming

Accessing our custom server.
„ We can publish the URL to our server, or
embed links to the server in other HTML
documents.
„ We need to make sure the server is
always running (on the published host and
port).
„ Once we are famous we can include
advertisements and make money!

2007 CSCE515 – Computer Network Programming

Another Example
„ Keep track of how many times our server
is hit each day.

„ Report on the number of hits our server

got on any day in the past!

2007 CSCE515 – Computer Network Programming

The Request and Reply
„ The reply now does depend on the
request.

„ We have to remember that the request

comes from a HTTP client, so we need to
accept HTTP requests.

2007 CSCE515 – Computer Network Programming

Time & Date Hit Server
„ Each request comes as a string (URI)
specifying a resource.
nt !
„ Our requests will look like this: plia
Com
Y 2K
/mm/dd/yyyy
„ An example URL for our service:
http://www.timedate.com:4567/02/10/2000

„ We will get a request like:

GET /02/10/2000 HTTP/1.1

2007 CSCE515 – Computer Network Programming

 Fancy means $$$
„ We want to provide a table that lists the
number of hits received each hour of
the day in question

timedate.com hit report for 01/17/1999

hour number of hits

12-1AM 4,320
1-2AM 18,986
2-3AM 246

2007 CSCE515 – Computer Network Programming

 HTML Basics
„ HTML Tables:
… <TABLE> , </TABLE> start/end a table
… <TR> , </TR> start/end a table row
… <TD> , </TD> start/end a table cell
… <TH> , </TH> start/end table header cell

2007 CSCE515 – Computer Network Programming

 timedate.com Hit Table
<TABLE>
<TR>
<TH>hour</TH>
<TH>number of hits</TH>
</TR>
<TR> hour number of hits
<TD>12-1AM</TD> 12-1AM 4,320
<TD>4,320</TD> 1-2AM 18,986
</TR> 2-3AM 246
<TR>
<TD>1-2AM</TD>
<TD>18,986</TD>
</TR>
2007 CSCE515 – Computer Network Programming
 New code
… Record the “hit” in database.
… Read request - parse request to
month,day,year
… Lookup hits for month,day,year in database.
… Send back some http headers (Content-Type)
… Create HTML table and send back to client.
… Close the connection.

2007 CSCE515 – Computer Network Programming

Drawbacks to Custom Server Approach
„ We might have lots of ideas custom
services.
… Each requires dedicated address (port)
… Each needs to include:
„ basic TCP server code
„ parsing HTTP requests

„ error handling

„ headers

„ access control (might want users to pay each time

they check the time and date!)

2007 CSCE515 – Computer Network Programming

Another Approach
„ Take a general purpose Web server (that
can handle static documents) and have it
process requested documents as it sends
them to the client.
„ The documents could contain commands
that the server understands (the server
includes some kind of interpreter).

2007 CSCE515 – Computer Network Programming

Example Smart Server
„ Have the server read each HTML file as it
sends it to the client.
„ The server could look for this:
<SERVERCODE> some command </SERVERCODE>
„ The server doesn’t send this part to the
client, instead it interprets the command
and sends the result to the client.
„ Everything else is sent normally.

2007 CSCE515 – Computer Network Programming

Example Commands

<SERVERCODE> Time </SERVERCODE>

<SERVERCODE> Date </SERVERCODE>
<SERVERCODE> Hitlist </SERVERCODE>

<SERVERCODE> Include file </SERVERCODE>

<SERVERCODE> randomfile directory </SERVERCODE>

2007 CSCE515 – Computer Network Programming

 Example Document
<TITLE>timedate.com Home Page</TITLE>
<H1 ALIGN=CENTER>Welcome to timedate.com</H1>
<SERVERCODE> include fancygraphic
</SERVERCODE>

The current time is

<SERVERCODE> time </SERVERCODE>.<P>

Today is <SERVERCODE> date </SERVERCODE>.

Visit our sponser:

<SERVERCODE> random sponsor </SERVERCODE>

2007 CSCE515 – Computer Network Programming

Real Life - Server Side Includes
„ Many real web servers support this idea
(but not the syntax I’ve shown).

„ Server Side Includes (SSI) provides a set

of commands that a server will interpret.

2007 CSCE515 – Computer Network Programming

SSI Configuration

„ Typically the server is configured to look

for commands only in specially marked
documents (so normal documents aren’t
slowed down).

2007 CSCE515 – Computer Network Programming

SSI Directives
„ SSI commands are called directives
„ Directives are embedded in HTML
comments. A comment looks like this:
<!-- this is an HTML comment -->

„ A directive looks like this:

<!--#command parameter=“arg”-->

2007 CSCE515 – Computer Network Programming

Some SSI Directives
echo: inserts the value of an environment
variable into the page.
SSI servers keep a number of useful things
in environment variables:

DATE_LOCAL

2007 CSCE515 – Computer Network Programming

SSI echo example
The GMT time is:
<!--#echo var ="DATE_GMT"-->

The local time is:

<!--#echo var ="DATE_LOCAL"-->

<!--#config timefmt="%A %B %d, %Y" -->

Today is <!--#echo var="DATE_LOCAL" -->

2007 CSCE515 – Computer Network Programming

 SSI Directives
include: inserts the contents of a text file.
<!--#include file="foo.html" -->

flastmod: inserts the time and date that a

file was last modified.
This file last modified <!--#flastmod
file="ssi.shtml" -->
This file last modified <!--#echo
var="LAST_MODIFIED" -->

2007 CSCE515 – Computer Network Programming

SSI Directives (cont.)
exec: runs an external program and inserts
the output of the program.

Current users:
<!--#exec cmd=“/usr/bin/who”-->

Danger! Danger! Danger!

2007 CSCE515 – Computer Network Programming

 SSI Example
<!--#INCLUDE FILE="header"--><BR>
It is now:
<!--#config timefmt="%I:%M %p (%Z)"-->
<!--#echo var="DATE_LOCAL"-->
<BR>
Today is:
<!--#config timefmt="%A, %B %e, %Y"-->
<!--#echo var="DATE_LOCAL"--><BR>
<BR>
<!--#INCLUDE FILE="footer"--> <BR>

<!--#config timefmt="%D"-->
This file last modified
<!--#echo var="LAST_MODIFIED"-->

2007 CSCE515 – Computer Network Programming

 More Power
„ Some servers support elaborate scripting
languages.
„ Scripts are embedded in HTML documents,
the server interprets the script:
… Microsoft Active Server Pages (ASP)
„ VBScript, .asp
… JaveServer Pages (JSP)
„ java, .jsp
… Hypertext Preprocessor (PHP)
„ .php

2007 CSCE515 – Computer Network Programming

Server Mapping and APIs
„ Some servers include a programming
interface that allows us to extend the
capabilities of the server by writing
modules.
„ Specific URLs are mapped to specific
modules instead of to files.
„ We could write our timedate.com server
as a module and merge it with the web
server.

2007 CSCE515 – Computer Network Programming

External Programs
„ Another approach is to provide a
standard interface between external
programs and web servers.
… We can run the same program from any
web server.
… The web server handles all the http, we
focus on the special service only.
… It doesn’t matter what language we use to
write the external program.

2007 CSCE515 – Computer Network Programming

 Common Gateway Interface
„ CGI is a standard interface to external
programs supported by most (if not all)
web servers.
„ The interface that is defined by CGI
includes:
… Identification of the service (external
program).
… Mechanism for passing the request to the
external program.

2007 CSCE515 – Computer Network Programming

CGI Programming
„ CGI programs are often written in scripting
languages (perl, tcl, etc.),

„ JAVA Servlets
… java technology’s answer to CGI
… Install software that implements Java Servlet
„ Apache Tomcat
„ JavaServer Web Development Kit
„ Sun’s Java Web Server

2007 CSCE515 – Computer Network Programming

 An Servlets example
import java.io.*
import javax.servlet.*;
import javax.servlet.http.*;

public class HelloWWW extends HttpServlet {

public void doGet(HttpServletRequest request,
HttpServletResponse response) throws
ServletException, IOException {
response.setContentType("text/html");
PrintWriter out = response.getWriter();
out.println("<!DOCTYPE HTML PUBLIC \"-
//W3C//DTD HTML 4.0 " + "Transitional//EN\">\n"
+ "<HTML>\n" + "<HEAD><TITLE>Hello
WWW</TITLE></HEAD>\n" + "<BODY>\n" + "<H1>Hello
WWW</H1>\n" + "</BODY></HTML>"); } }

2007 CSCE515 – Computer Network Programming

Result

2007 CSCE515 – Computer Network Programming

SSI vs. CGI
„ SSI:
… StaticHTML + embedded command
… add small amounts of dynamic content to pages,
without doing a lot of extra work

„ CGI
… Generate a complete HTML document in real time
… Powerful

2007 CSCE515 – Computer Network Programming

Concurrent Server
Design Alternatives
Concurrent Server
Design Alternatives
One child process per client

Spawn one thread per client

Preforking multiple processes

Prethreaded Server

2007 CSCE515 – Computer Network Programming

One child per client

• Traditional Unix server:

… TCP: after call to accept(), call fork().
… UDP: after recvfrom(), call fork().
… Each process needs only a few sockets.
… Small requests can be serviced in a small
amount of time.
• Parent process needs to clean up after
children!!!! (call wait() ).

2007 CSCE515 – Computer Network Programming

One thread per client
• Almost like using fork - call pthread_create
instead.
• Using threads makes it easier (less
overhead) to have sibling processes share
information.
• Sharing information must be done
carefully (use pthread_mutex)

2007 CSCE515 – Computer Network Programming

Example
main(int argc, char **argv) main(int argc, char **argv)
{ listenfd=socket(…) { pthread_t tid;
Bind(listenfd…)
Listen(listenfd,LISTENQ); listenfd=socket(…)
Signal(SIGGHLD, sig_chld); Bind(listenfd…)
Signal(SIGINT,sig_int); Listen(listenfd,LISTENQ);
For( ; ;) { For( ; ;) {
connfd = Accept(listenfd, connfd =
…); Accept(listenfd, …);
if ( (pid = Fork())==0) { Pthread_creat(&tid,
Close(listendf); NULL, &doit, (void *)
doit(connfd); connfd);
Close(connfd); }
exit(0); }
} static void doit (void *arg)
} {
Close(connfd); …
} Close( (int) arg);
return NULL;
Process version }
Thread version

2007 CSCE515 – Computer Network Programming

Prefork()’d Server
• Creating a new process for each client is
expensive.

• We can create a bunch of processes, each

of which can take care of a client.

• Each child process is an iterative server.

2007 CSCE515 – Computer Network Programming

Prefork()’d TCP Server
• Initial process creates socket and binds to
well known address.
• Process now calls fork() a bunch of
times.
• All children call accept().
• The next incoming connection will be
handed to one child.

2007 CSCE515 – Computer Network Programming

listen()
Sum of both queues
cannot exceed backlog

Server
accept

Completed connection queue

3-way
handshake
complete
TCP
Incomplete connection queue

arriving
SYN

2007 CSCE515 – Computer Network Programming

Preforking
• As the book shows, having too many
preforked children can be bad.
• Using dynamic process allocation instead
of a hard-coded number of children can
avoid problems.
• The parent process just manages the
children, doesn’t worry about clients.

2007 CSCE515 – Computer Network Programming

Sockets library vs. system call
• A preforked TCP server won’t usually work
the way we want if sockets is not part of
the kernel:
… callingaccept() is a library call, not an atomic
operation.
• We can get around this by making sure
only one child calls accept() at a time
using some locking scheme.

2007 CSCE515 – Computer Network Programming

Prethreaded Server
• Same benefits as preforking.

• Can also have the main thread do all the

calls to accept() and hand off each client
to an existing thread.

2007 CSCE515 – Computer Network Programming

What’s the best server design
for my application?
• Many factors:
… expected number of simultaneous clients.
… Transaction size (time to compute or lookup
the answer)
… Variability in transaction size.
… Available system resources (perhaps what
resources can be required in order to run the
service).

2007 CSCE515 – Computer Network Programming

Server Design
• It is important to understand the issues
and options.
• Knowledge of queuing theory can be a big
help.
• You might need to test a few alternatives
to determine the best design.

2007 CSCE515 – Computer Network Programming