Carnegie Mellon

Network  Programming:  Part  II  

 
15-­‐213:  Introduc;on  to  Computer  Systems  
22nd  Lecture,  Nov.  12,  2015  

Instructors:    
Randal  E.  Bryant  and  David  R.  O’Hallaron  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   1

 Carnegie Mellon

2.  Start  client  
Client  
1.  Start  server  
Server   Sockets  
getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

3.  Exchange  
Client  /   rio_writen rio_readlineb data  
Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

5.  Drop  client  
4.  Disconnect  client  
close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   2
 Carnegie Mellon

Recall:  Socket  Address  Structures  

¢ Generic  socket  address:  
§ For  address  arguments  to  connect,  bind,  and  accept
§ Necessary  only  because  C  did  not  have  generic  (void *)  pointers  when  
the  sockets  interface  was  designed  
§ For  cas;ng  convenience,  we  adopt  the  Stevens  conven;on:    
         typedef struct sockaddr SA;
struct sockaddr {
uint16_t sa_family; /* Protocol family */
char sa_data[14]; /* Address data. */
};

sa_family

Family  Specific  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   3

 Carnegie Mellon

Recall:  Socket  Address  Structures  

¢ Internet-­‐specific  socket  address:  
§ Must  cast  (struct sockaddr_in *)  to  (struct sockaddr *)  
for  func;ons  that  take  socket  address  arguments.    

struct sockaddr_in {
uint16_t sin_family; /* Protocol family (always AF_INET) */
uint16_t sin_port; /* Port num in network byte order */
struct in_addr sin_addr; /* IP addr in network byte order */
unsigned char sin_zero[8]; /* Pad to sizeof(struct sockaddr) */
};

sin_port sin_addr
AF_INET 0 0 0 0 0 0 0 0
sa_family
sin_family
Family  Specific  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   4

 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   5
 Carnegie Mellon

Sockets  Interface:  socket

¢ Clients  and  servers  use  the  socket  func?on  to  create  a  
socket  descriptor:  
int socket(int domain, int type, int protocol)

¢ Example:  
int clientfd = Socket(AF_INET, SOCK_STREAM, 0);

  Indicates  that  we  are  using   Indicates  that  the  socket  

32-­‐bit  IPV4  addresses   will  be  the  end  point  of  a  
connec?on  
 
Protocol  specific!  Best  prac?ce  is  to  use  getaddrinfo  to  
generate  the  parameters  automa?cally,  so  that  code  is  
protocol  independent.  
Bryant  a  nd  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   6
 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   7
 Carnegie Mellon

Sockets  Interface:  bind

¢ A  server  uses    bind  to  ask  the  kernel  to  associate  the  
server’s  socket  address  with  a  socket  descriptor:  
int bind(int sockfd, SA *addr, socklen_t addrlen);

¢ The  process  can  read  bytes  that  arrive  on  the  connec?on  
whose  endpoint  is  addr by  reading  from  descriptor  
sockfd.  
¢ Similarly,  writes  to  sockfd  are  transferred  along  
connec?on  whose  endpoint  is  addr.

Best  prac?ce  is  to  use  getaddrinfo  to  supply  the  

arguments  addr  and  addrlen.    

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   8

 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   9
 Carnegie Mellon

Sockets  Interface:  listen

¢ By  default,  kernel  assumes  that  descriptor  from  socket  
func?on  is  an  ac8ve  socket  that  will  be  on  the  client  end  
of  a  connec?on.  
¢ A  server  calls  the  listen  func?on  to  tell  the  kernel  that  a  
descriptor  will  be  used  by  a  server  rather  than  a  client:  
int listen(int sockfd, int backlog);
 
¢ Converts  sockfd  from  an  ac?ve  socket  to  a  listening  
socket  that  can  accept  connec?on  requests  from  clients.    

¢ backlog is  a  hint  about  the  number  of  outstanding  

connec?on  requests  that  the  kernel  should  queue  up  
before  star?ng  to  refuse  requests.    
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   10
 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   11
 Carnegie Mellon

Sockets  Interface:  accept

¢ Servers  wait  for  connec?on  requests  from  clients  by  
calling  accept:  
  int accept(int listenfd, SA *addr, int *addrlen);

¢ Waits  for  connec?on  request  to  arrive  on  the  connec?on  

bound  to  listenfd,  then  fills  in  client’s  socket  address  
in  addr  and  size  of  the  socket  address  in  addrlen.    
¢ Returns  a  connected  descriptor  that  can  be  used  to  
communicate  with  the  client  via  Unix  I/O  rou?nes.    
 

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   12

 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   13
 Carnegie Mellon

Sockets  Interface:  connect

¢ A  client  establishes  a  connec?on  with  a  server  by  calling  
connect:  
int connect(int clientfd, SA *addr, socklen_t addrlen);

¢ AYempts  to  establish  a  connec?on  with  server  at  socket  

address  addr
§ If  successful,  then  clientfd  is  now  ready  for  reading  and  wri;ng.    
§ Resul;ng  connec;on  is    characterized  by  socket  pair  
 (x:y, addr.sin_addr:addr.sin_port)
§ x  is  client  address  
§ y  is  ephemeral  port  that  uniquely  iden;fies  client  process  on  
client  host  

Best  prac?ce  is  to  use  getaddrinfo  to  supply  the  

arguments  addr  and  addrlen.    
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   14
 Carnegie Mellon

accept  Illustrated  
listenfd(3)
1.  Server  blocks  in  accept,  
Client   Server   wai8ng  for  connec8on  request  
clientfd on  listening  descriptor  
listenfd  

Connec?on  
listenfd(3)
request  
2.  Client  makes  connec8on  request  by  
Client   Server   calling  and  blocking  in  connect
clientfd

listenfd(3)
3.  Server  returns  connfd  from  
Client   Server   accept.  Client  returns  from  connect.  
clientfd connfd(4)
Connec8on  is  now  established  between  
clientfd  and  connfd  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   15

 Carnegie Mellon

Connected  vs.  Listening  Descriptors  

¢ Listening  descriptor  
§ End  point  for  client  connec;on  requests  
§ Created  once  and  exists  for  life;me  of  the  server  

¢ Connected  descriptor  
§ End  point  of  the  connec;on  between  client  and  server  
§ A  new  descriptor  is  created  each  ;me  the  server  accepts  a  
connec;on  request  from  a  client  
§ Exists  only  as  long  as  it  takes  to  service  client  

¢ Why  the  dis?nc?on?  

§ Allows  for  concurrent  servers  that  can  communicate  over  many  
client  connec;ons  simultaneously  
§ E.g.,  Each  ;me  we  receive  a  new  request,  we  fork  a  child  to  
handle  the  request  
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   16
 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   17
 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   18
 Carnegie Mellon

Sockets  Helper:  open_clientfd

¢ Establish  a  connec?on  with  a  server  
int open_clientfd(char *hostname, char *port) {
int clientfd;
struct addrinfo hints, *listp, *p;

/* Get a list of potential server addresses */

memset(&hints, 0, sizeof(struct addrinfo));
hints.ai_socktype = SOCK_STREAM; /* Open a connection */
hints.ai_flags = AI_NUMERICSERV; /* …using numeric port arg. */
hints.ai_flags |= AI_ADDRCONFIG; /* Recommended for connections */
Getaddrinfo(hostname, port, &hints, &listp);
csapp.c  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   19

 Carnegie Mellon

Sockets  Helper:  open_clientfd  (cont)

/* Walk the list for one that we can successfully connect to */
for (p = listp; p; p = p->ai_next) {
/* Create a socket descriptor */
if ((clientfd = socket(p->ai_family, p->ai_socktype,
p->ai_protocol)) < 0)
continue; /* Socket failed, try the next */

/* Connect to the server */

if (connect(clientfd, p->ai_addr, p->ai_addrlen) != -1)
break; /* Success */
Close(clientfd); /* Connect failed, try another */
}

/* Clean up */
Freeaddrinfo(listp);
if (!p) /* All connects failed */
return -1;
else /* The last connect succeeded */
return clientfd;
} csapp.c  
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   20
 Carnegie Mellon

Client   Server   Sockets  

getaddrinfo getaddrinfo Interface  
socket socket
open_listenfd
open_clientfd bind

listen
Connec?on  
request  
connect accept

Client  /   rio_writen rio_readlineb

Server  
Session   Await  connec?on  
rio_readlineb rio_writen request  from  
next  client  
EOF  
close rio_readlineb

close
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   21
 Carnegie Mellon

Sockets  Helper:  open_listenfd

¢ Create  a  listening  descriptor  that  can  be  used  to  accept  
connec?on  requests  from  clients.  
int open_listenfd(char *port)
{
struct addrinfo hints, *listp, *p;
int listenfd, optval=1;

/* Get a list of potential server addresses */

memset(&hints, 0, sizeof(struct addrinfo));
hints.ai_socktype = SOCK_STREAM; /* Accept connect. */
hints.ai_flags = AI_PASSIVE | AI_ADDRCONFIG; /* …on any IP addr */
hints.ai_flags |= AI_NUMERICSERV; /* …using port no. */
Getaddrinfo(NULL, port, &hints, &listp);

csapp.c  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   22

 Carnegie Mellon

Sockets  Helper:  open_listenfd  (cont)  

/* Walk the list for one that we can bind to */
for (p = listp; p; p = p->ai_next) {
/* Create a socket descriptor */
if ((listenfd = socket(p->ai_family, p->ai_socktype,
p->ai_protocol)) < 0)
continue; /* Socket failed, try the next */

/* Eliminates "Address already in use" error from bind */

Setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR,
(const void *)&optval , sizeof(int));

/* Bind the descriptor to the address */

if (bind(listenfd, p->ai_addr, p->ai_addrlen) == 0)
break; /* Success */
Close(listenfd); /* Bind failed, try the next */
} csapp.c  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   23

 Carnegie Mellon

Sockets  Helper:  open_listenfd  (cont)  

/* Clean up */
Freeaddrinfo(listp);
if (!p) /* No address worked */
return -1;

/* Make it a listening socket ready to accept conn. requests */

if (listen(listenfd, LISTENQ) < 0) {
Close(listenfd);
return -1;
}
return listenfd;
} csapp.c  

¢ Key  point:  open_clientfd and  open_listenfd  are  

both  independent  of  any  par?cular  version  of  IP.
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   24
 Carnegie Mellon

Echo  Client:  Main  Rou?ne  

#include "csapp.h"

int main(int argc, char **argv)

{
int clientfd;
char *host, *port, buf[MAXLINE];
rio_t rio;

host = argv[1];
port = argv[2];

clientfd = Open_clientfd(host, port);

Rio_readinitb(&rio, clientfd);

while (Fgets(buf, MAXLINE, stdin) != NULL) {

Rio_writen(clientfd, buf, strlen(buf));
Rio_readlineb(&rio, buf, MAXLINE);
Fputs(buf, stdout);
}
Close(clientfd);
exit(0);
} echoclient.c  
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   25
 Carnegie Mellon

Itera?ve  Echo  Server:  Main  Rou?ne  

#include "csapp.h”
void echo(int connfd);

int main(int argc, char **argv)

{
int listenfd, connfd;
socklen_t clientlen;
struct sockaddr_storage clientaddr; /* Enough room for any addr */
char client_hostname[MAXLINE], client_port[MAXLINE];

listenfd = Open_listenfd(argv[1]);
while (1) {
clientlen = sizeof(struct sockaddr_storage); /* Important! */
connfd = Accept(listenfd, (SA *)&clientaddr, &clientlen);
Getnameinfo((SA *) &clientaddr, clientlen,
client_hostname, MAXLINE, client_port, MAXLINE, 0);
printf("Connected to (%s, %s)\n", client_hostname, client_port);
echo(connfd);
Close(connfd);
}
exit(0);
} echoserveri.c  
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   26
 Carnegie Mellon

Echo  Server:  echo  func?on  

¢ The  server  uses  RIO  to  read  and  echo  text  lines  un?l  EOF  
(end-­‐of-­‐file)  condi?on  is  encountered.  
§ EOF  condi;on  caused  by  client  calling    close(clientfd)

void echo(int connfd)

{
size_t n;
char buf[MAXLINE];
rio_t rio;

Rio_readinitb(&rio, connfd);
while((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) {
printf("server received %d bytes\n", (int)n);
Rio_writen(connfd, buf, n);
}
} echo.c  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   27

 Carnegie Mellon

Tes?ng  Servers  Using  telnet  

¢ The  telnet program  is  invaluable  for  tes?ng  servers  
that  transmit  ASCII  strings  over  Internet  connec?ons  
§ Our  simple  echo  server  
§ Web  servers  
§ Mail  servers  

¢ Usage:    
§ linux> telnet <host> <portnumber>
§ Creates  a  connec;on  with  a  server  running  on  <host>  and    
listening  on  port  <portnumber>

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   28

 Carnegie Mellon

Tes?ng  the  Echo  Server  With  telnet  

whaleshark> ./echoserveri 15213
Connected to (MAKOSHARK.ICS.CS.CMU.EDU, 50280)
server received 11 bytes
server received 8 bytes

makoshark> telnet whaleshark.ics.cs.cmu.edu 15213

Trying 128.2.210.175...
Connected to whaleshark.ics.cs.cmu.edu (128.2.210.175).
Escape character is '^]'.
Hi there!
Hi there!
Howdy!
Howdy!
^]
telnet> quit
Connection closed.
makoshark>

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   29

 Carnegie Mellon

Web  Server  Basics  

¢ Clients  and  servers  communicate   HTTP request  

using    the  HyperText  Transfer   Web  
Web  
Protocol  (HTTP)   client  
server  
(browser)    
§ Client  and  server  establish  TCP  
HTTP response  
connec;on  
(content)  
§ Client  requests  content  
§ Server  responds  with  requested  
content  
HTTP   Web  content  
§ Client  and  server  close  connec;on  
(eventually)  
TCP   Streams  
¢ Current  version  is  HTTP/1.1  
§ RFC  2616,  June,  1999.     IP   Datagrams  

http://www.w3.org/Protocols/rfc2616/rfc2616.html

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   30

 Carnegie Mellon

Web  Content  
¢ Web  servers  return  content  to  clients  
§ content:  a  sequence  of  bytes  with  an  associated  MIME  (Mul;purpose  
Internet  Mail  Extensions)  type  

¢ Example  MIME  types  

§ text/html HTML  document  
§ text/plain Unforma^ed  text  
§ image/gif Binary  image  encoded  in  GIF  format  
§ image/png  Binar  image  encoded  in  PNG  format  
§ image/jpeg  Binary  image  encoded  in  JPEG  format  
 

You  can  find  the  complete  list  of  MIME  types  at:  
http://www.iana.org/assignments/media-types/media-types.xhtml
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   31
 Carnegie Mellon

Sta?c  and  Dynamic  Content  

¢ The  content  returned  in  HTTP  responses  can  be  either  
sta8c  or  dynamic  
§ Sta*c  content:  content  stored  in  files  and  retrieved  in  response  to  
an  HTTP  request  
§ Examples:  HTML  files,  images,  audio  clips  
§ Request  iden;fies  which  content  file  
§ Dynamic  content:  content  produced  on-­‐the-­‐fly  in  response  to  an  
HTTP  request  
§ Example:  content  produced  by  a  program  executed  by  the  
server  on  behalf  of  the  client  
§ Request  iden;fies  file  containing  executable  code  

¢ BoYom  line:  Web  content  is  associated  with  a  file  that  is  
managed  by  the  server  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   32

 Carnegie Mellon

URLs  and  how  clients  and  servers  use  them  

¢ Unique  name  for  a  file:  URL  (Universal  Resource  Locator)  
¢ Example  URL:  http://www.cmu.edu:80/index.html
¢ Clients  use  prefix  (http://www.cmu.edu:80)  to  infer:  
§ What  kind  (protocol)  of  server  to  contact  (HTTP)  
§ Where  the  server  is  (www.cmu.edu)  
§ What  port  it  is  listening  on  (80)  
¢ Servers  use  suffix  (/index.html)  to:  
§ Determine  if  request  is  for  sta;c  or  dynamic  content.  
No  hard  and  fast  rules  for  this  
§
§ One  conven;on:  executables  reside  in  cgi-bin directory  
§ Find  file  on  file  system  
§ Ini;al  “/”  in  suffix  denotes  home  directory  for  requested  content.  
§ Minimal  suffix  is  “/”,  which  server  expands  to  configured  default  
filename  (usually,  index.html)    
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   33
 Carnegie Mellon

HTTP  Requests  
¢ HTTP  request  is  a  request  line,  followed  by  zero  or  more  
request  headers  

¢ Request  line:  <method> <uri> <version>

§ <method> is  one  of    GET, POST, OPTIONS, HEAD, PUT,
DELETE, or TRACE
§ <uri>  is  typically  URL  for  proxies,  URL  suffix  for  servers  
§ A  URL  is  a  type  of  URI  (Uniform  Resource  Iden;fier)  
§ See  h^p://www.ieg.org/rfc/rfc2396.txt  
§ <version>  is  HTTP  version  of  request  (HTTP/1.0  or  HTTP/1.1)  

¢ Request  headers:  <header name>: <header data>  

§ Provide  addi;onal  informa;on  to  the  server  
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   34
 Carnegie Mellon

HTTP  Responses  
¢ HTTP  response  is  a  response  line  followed  by  zero  or  more  
response  headers,  possibly  followed  by  content,  with  blank  line  
(“\r\n”)  separa?ng  headers  from  content.    

¢ Response  line:    
   <version> <status code> <status msg>
§ <version>  is  HTTP  version  of  the  response  
§ <status  code>  is  numeric  status  
§ <status  msg>  is  corresponding  English  text  
§ 200    OK    Request  was  handled  without  error  
§ 301  Moved    Provide  alternate  URL  
§ 404  Not  found  Server  couldn’t  find  the  file  
¢ Response  headers:  <header name>: <header data>
§ Provide  addi;onal  informa;on  about  response  
§ Content-Type: MIME  type  of  content  in  response  body  
§ Content-Length: Length  of  content  in  response  body  
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   35
 Carnegie Mellon

Example  HTTP  Transac?on  

whaleshark> telnet www.cmu.edu 80 Client: open connection to server
Trying 128.2.42.52... Telnet prints 3 lines to terminal
Connected to WWW-CMU-PROD-VIP.ANDREW.cmu.edu.
Escape character is '^]'.
GET / HTTP/1.1 Client: request line
Host: www.cmu.edu Client: required HTTP/1.1 header
Client: empty line terminates headers
HTTP/1.1 301 Moved Permanently Server: response line
Date: Wed, 05 Nov 2014 17:05:11 GMT Server: followed by 5 response headers
Server: Apache/1.3.42 (Unix) Server: this is an Apache server
Location: http://www.cmu.edu/index.shtml Server: page has moved here
Transfer-Encoding: chunked Server: response body will be chunked
Content-Type: text/html; charset=... Server: expect HTML in response body
Server: empty line terminates headers
15c Server: first line in response body
<HTML><HEAD> Server: start of HTML content
…
</BODY></HTML> Server: end of HTML content
0 Server: last line in response body
Connection closed by foreign host. Server: closes connection

¢ HTTP  standard  requires  that  each  text  line  end  with  “\r\n”
¢ Blank  line  (“\r\n”)  terminates  request  and  response  headers   36
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on  
 Carnegie Mellon

Example  HTTP  Transac?on,  Take  2  

whaleshark> telnet www.cmu.edu 80 Client: open connection to server
Trying 128.2.42.52... Telnet prints 3 lines to terminal
Connected to WWW-CMU-PROD-VIP.ANDREW.cmu.edu.
Escape character is '^]'.
GET /index.shtml HTTP/1.1 Client: request line
Host: www.cmu.edu Client: required HTTP/1.1 header
Client: empty line terminates headers
HTTP/1.1 200 OK Server: response line
Date: Wed, 05 Nov 2014 17:37:26 GMT Server: followed by 4 response headers
Server: Apache/1.3.42 (Unix)
Transfer-Encoding: chunked
Content-Type: text/html; charset=...
Server: empty line terminates headers
1000 Server: begin response body
<html ..> Server: first line of HTML content
…
</html>
0 Server: end response body
Connection closed by foreign host. Server: close connection

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   37

 Carnegie Mellon

Tiny  Web  Server  

¢ Tiny  Web  server  described  in  text  
§ Tiny  is  a  sequen;al  Web  server  
§ Serves  sta;c  and  dynamic  content  to  real  browsers  
§ text  files,  HTML  files,  GIF,  PNG,  and  JPEG  images  
§ 239  lines  of  commented  C  code  
§ Not  as  complete  or  robust  as  a  real  Web  server  
§ You  can  break  it  with  poorly-­‐formed  HTTP  requests  (e.g.,  
terminate  lines  with  “\n”  instead  of  “\r\n”)  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   38

 Carnegie Mellon

Tiny  Opera?on  
¢ Accept  connec?on  from  client  
¢ Read  request  from  client  (via  connected  socket)  
¢ Split  into  <method>    <uri>  <version>  
§ If  method  not  GET,  then  return  error  
¢ If  URI  contains  “cgi-bin”  then  serve  dynamic  content  
§ (Would  do  wrong  thing  if  had  file  “abcgi-bingo.html”)  
§ Fork  process  to  execute  program  
¢ Otherwise  serve  sta?c  content  
§ Copy  file  to  output  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   39

 Carnegie Mellon

Tiny  Serving  Sta?c  Content  

void serve_static(int fd, char *filename, int filesize)
{
int srcfd;
char *srcp, filetype[MAXLINE], buf[MAXBUF];

/* Send response headers to client */

get_filetype(filename, filetype);
sprintf(buf, "HTTP/1.0 200 OK\r\n");
sprintf(buf, "%sServer: Tiny Web Server\r\n", buf);
sprintf(buf, "%sConnection: close\r\n", buf);
sprintf(buf, "%sContent-length: %d\r\n", buf, filesize);
sprintf(buf, "%sContent-type: %s\r\n\r\n", buf, filetype);
Rio_writen(fd, buf, strlen(buf));

/* Send response body to client */

srcfd = Open(filename, O_RDONLY, 0);
srcp = Mmap(0, filesize, PROT_READ, MAP_PRIVATE, srcfd, 0);
Close(srcfd);
Rio_writen(fd, srcp, filesize);
Munmap(srcp, filesize);
} ?ny.c  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   40

 Carnegie Mellon

Serving  Dynamic  Content  

¢ Client  sends  request  to  server   GET /cgi-bin/env.pl HTTP/1.1

¢ If  request  URI  contains  the   Client   Server  

string  “/cgi-bin”,  the  Tiny  
server  assumes  that  the  
request  is  for  dynamic  content    

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   41

 Carnegie Mellon

Serving  Dynamic  Content  (cont)  

¢ The  server  creates  a  child  

Client   Server  
process  and  runs  the  
program  iden?fied  by  the  URI  
fork/exec
in  that  process  

env.pl

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   42

 Carnegie Mellon

Serving  Dynamic  Content  (cont)  

¢ The  child  runs  and  generates   Client   Server  

Content  
the  dynamic  content  
Content
¢ The  server  captures  the  
content  of  the  child  and   env.pl
forwards  it  without  
modifica?on  to  the  client  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   43

 Carnegie Mellon

Issues  in  Serving  Dynamic  Content  

¢ How  does  the  client  pass  program   Request  

arguments  to  the  server?  
Client   Content   Server  
¢ How  does  the  server  pass  these  
arguments  to  the  child?  
¢ How  does  the  server  pass  other  info   Content   Create  
relevant  to  the  request  to  the  child?  
¢ How  does  the  server  capture  the   env.pl
content  produced  by  the  child?  
¢ These  issues  are  addressed  by  the  
Common  Gateway  Interface  (CGI)  
specifica?on.  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   44

 Carnegie Mellon

CGI  

¢ Because  the  children  are  wriYen  according  to  the  CGI  

spec,  they  are  ofen  called  CGI  programs.  
 
¢ However,  CGI  really  defines  a  simple  standard  for  
transferring  informa?on  between  the  client  (browser),  
the  server,  and  the  child  process.  

¢ CGI  is  the  original  standard  for  genera?ng  dynamic  

content.  Has  been  largely  replaced  by  other,  faster  
techniques:    
§ E.g.,  fastCGI,  Apache  modules,  Java  servlets,  Rails  controllers  
§ Avoid  having  to  create  process  on  the  fly  (expensive  and  slow).    

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   45

 Carnegie Mellon

The  add.com  Experience  

host   port   CGI  program  
arguments  

Output  page  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   46

 Carnegie Mellon

Serving  Dynamic  Content  With  GET  

¢ Ques?on:  How  does  the  client  pass  arguments  to  the  server?  
¢ Answer:  The  arguments  are  appended  to  the  URI  

¢ Can  be  encoded  directly  in  a  URL  typed  to  a  browser  or  a  URL  
in  an  HTML  link      
§ http://add.com/cgi-bin/adder?15213&18213
§ adder  is  the  CGI  program  on  the  server  that  will  do  the  addi;on.  
§ argument  list  starts  with  “?”  
§ arguments  separated  by  “&”    
§ spaces  represented  by    “+” or “%20”

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   47

 Carnegie Mellon

Serving  Dynamic  Content  With  GET  

¢ URL  suffix:    
§ cgi-bin/adder?15213&18213

¢ Result  displayed  on  browser:    

Welcome to add.com: THE Internet
addition portal.

The answer is: 15213 + 18213 = 33426

Thanks for visiting!

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   48

 Carnegie Mellon

Serving  Dynamic  Content  With  GET  

¢ Ques?on:  How  does  the  server  pass  these  arguments  to  
the  child?  
¢ Answer:  In  environment  variable  QUERY_STRING
§ A  single  string  containing  everything  amer  the  “?”  
§ For  add:  QUERY_STRING  =  “15213&18213”  

/* Extract the two arguments */

if ((buf = getenv("QUERY_STRING")) != NULL) {
p = strchr(buf, '&');
*p = '\0';
strcpy(arg1, buf);
strcpy(arg2, p+1);
n1 = atoi(arg1);
n2 = atoi(arg2);
} adder.c  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   49

 Carnegie Mellon

Serving  Dynamic  Content  with  GET  

¢ Ques?on:  How  does  the  server  capture  the  content  produced  by  the  child?  
¢ Answer:  The  child  generates  its  output  on  stdout.    Server  uses  dup2 to  
redirect  stdout to  its  connected  socket.    
void serve_dynamic(int fd, char *filename, char *cgiargs)
{
char buf[MAXLINE], *emptylist[] = { NULL };

/* Return first part of HTTP response */

sprintf(buf, "HTTP/1.0 200 OK\r\n");
Rio_writen(fd, buf, strlen(buf));
sprintf(buf, "Server: Tiny Web Server\r\n");
Rio_writen(fd, buf, strlen(buf));

if (Fork() == 0) { /* Child */
/* Real server would set all CGI vars here */
setenv("QUERY_STRING", cgiargs, 1);
Dup2(fd, STDOUT_FILENO); /* Redirect stdout to client */
Execve(filename, emptylist, environ); /* Run CGI program */
}
Wait(NULL); /* Parent waits for and reaps child */
} ?ny.c  50
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on  
 Carnegie Mellon

Serving  Dynamic  Content  with  GET  

¢ No?ce  that  only  the  CGI  child  process  knows  the  content  
type  and  length,  so  it  must  generate  those  headers.  

/* Make the response body */

sprintf(content, "Welcome to add.com: ");
sprintf(content, "%sTHE Internet addition portal.\r\n<p>", content);
sprintf(content, "%sThe answer is: %d + %d = %d\r\n<p>",
content, n1, n2, n1 + n2);
sprintf(content, "%sThanks for visiting!\r\n", content);

/* Generate the HTTP response */

printf("Content-length: %d\r\n", (int)strlen(content));
printf("Content-type: text/html\r\n\r\n");
printf("%s", content);
fflush(stdout);

exit(0); adder.c  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   51

 Carnegie Mellon

Serving  Dynamic  Content  With  GET    

bash:makoshark> telnet whaleshark.ics.cs.cmu.edu 15213
Trying 128.2.210.175...
Connected to whaleshark.ics.cs.cmu.edu (128.2.210.175).
Escape character is '^]'.
GET /cgi-bin/adder?15213&18213 HTTP/1.0
HTTP request sent by client
HTTP/1.0 200 OK
Server: Tiny Web Server
HTTP response generated
Connection: close by the server
Content-length: 117
Content-type: text/html
HTTP response generated
Welcome to add.com: THE Internet addition portal. by the CGI program
<p>The answer is: 15213 + 18213 = 33426
<p>Thanks for visiting!
Connection closed by foreign host.
bash:makoshark>

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   52

 Carnegie Mellon

For  More  Informa?on  

¢ W.  Richard  Stevens  et.  al.  “Unix  Network  Programming:  
The  Sockets  Networking  API”,  Volume  1,  Third  Edi?on,  
Pren?ce  Hall,  2003  
§ THE  network  programming  bible.  
¢ Michael  Kerrisk,  “The  Linux  Programming  Interface”,  No  
Starch  Press,  2010  
§ THE  Linux  programming  bible.    
¢ Complete  versions  of  all  code  in  this  lecture  is  available  
from  the  213  schedule  page.    
§ http://www.cs.cmu.edu/~213/schedule.html
§ csapp.{.c,h},  hos;nfo.c,  echoclient.c,  echoserveri.c,  ;ny.c,  adder.c  
§ You  can  use  any  of  this  code  in  your  assignments.    

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   53

 Carnegie Mellon

Addi?onal  slides  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   54

 Carnegie Mellon

Web  History  
¢ 1989:  
§ Tim  Berners-­‐Lee  (CERN)  writes  internal  proposal  to  develop  a  
distributed  hypertext  system  
§ Connects  “a  web  of  notes  with  links”  
§ Intended  to  help  CERN  physicists  in  large  projects  share  and  
manage  informa;on    
¢ 1990:  
§ Tim  BL  writes  a  graphical  browser  for  Next  machines  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   55

 Carnegie Mellon

Web  History  (cont)  

¢ 1992  
§ NCSA  server  released  
§ 26  WWW  servers  worldwide  
¢ 1993  
§ Marc  Andreessen  releases  first  version  of  NCSA  Mosaic  browser  
§ Mosaic  version  released  for  (Windows,  Mac,  Unix)  
§ Web  (port  80)  traffic  at  1%  of  NSFNET  backbone  traffic  
§ Over  200  WWW  servers  worldwide  
¢ 1994  
§ Andreessen  and  colleagues  leave  NCSA  to  form  “Mosaic  
Communica;ons  Corp”  (predecessor  to  Netscape)  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   56

 Carnegie Mellon

HTTP  Versions  
¢ Major  differences  between  HTTP/1.1  and  HTTP/1.0  
§ HTTP/1.0  uses  a  new  connec;on  for  each  transac;on  
§ HTTP/1.1  also  supports  persistent  connec*ons    
mul;ple  transac;ons  over  the  same  connec;on  
§
§ Connection: Keep-Alive
§ HTTP/1.1  requires  HOST  header  
§ Host: www.cmu.edu
§ Makes  it  possible  to  host  mul;ple  websites  at  single  Internet  host  
§ HTTP/1.1  supports  chunked  encoding  
§ Transfer-Encoding: chunked
§ HTTP/1.1  adds  addi;onal  support  for  caching  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   57

 Carnegie Mellon

GET  Request  to  Apache  Server  

From  Firefox  Browser  
URI is just the suffix, not the entire URL
GET /~bryant/test.html HTTP/1.1
Host: www.cs.cmu.edu
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:
1.9.2.11) Gecko/20101012 Firefox/3.6.11
Accept: text/html,application/xhtml+xml,application/
xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
CRLF (\r\n)

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   58

 Carnegie Mellon

GET  Response  From  Apache  Server  

HTTP/1.1 200 OK
Date: Fri, 29 Oct 2010 19:48:32 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.7m
mod_pubcookie/3.3.2b PHP/5.3.1
Accept-Ranges: bytes
Content-Length: 479
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html
<html>
<head><title>Some Tests</title></head>

<body>
<h1>Some Tests</h1>
. . .
</body>
</html>

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   59

 Carnegie Mellon

Data  Transfer  Mechanisms  

¢ Standard  
§ Specify  total  length  with  content-­‐length  
§ Requires  that  program  buffer  en;re  message  
¢ Chunked  
§ Break  into  blocks  
§ Prefix  each  block  with  number  of  bytes  (Hex  coded)  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   60

 Carnegie Mellon

Chunked  Encoding  Example  

HTTP/1.1 200 OK\n
Date: Sun, 31 Oct 2010 20:47:48 GMT\n
Server: Apache/1.3.41 (Unix)\n
Keep-Alive: timeout=15, max=100\n
Connection: Keep-Alive\n
Transfer-Encoding: chunked\n
Content-Type: text/html\n
\r\n
d75\r\n
<html>
First Chunk: 0xd75 = 3445 bytes
<head>
.<link href="http://www.cs.cmu.edu/style/calendar.css" rel="stylesheet"
type="text/css">
</head>
<body id="calendar_body">

<div id='calendar'><table width='100%' border='0' cellpadding='0'

cellspacing='1' id='cal'>

. . .
</body>
</html>
\r\n
0\r\n
\r\n
Second Chunk: 0 bytes (indicates last chunk)
Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   61
 Carnegie Mellon

Proxies  
¢ A  proxy  is  an  intermediary  between  a  client  and  an  origin  server  
§ To  the  client,  the  proxy  acts  like  a  server  
§ To  the  server,  the  proxy  acts  like  a  client  

1.  Client  request   2.  Proxy  request  

Origin  
Client   Proxy  
Server  
4.  Proxy  response   3.  Server  response  

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   62

 Carnegie Mellon

Why  Proxies?  
¢ Can  perform  useful  func?ons  as  requests  and  responses  pass  by  
§ Examples:  Caching,  logging,  anonymiza;on,  filtering,  transcoding  

Client   Request foo.html

A  
foo.html Request foo.html
Proxy   Origin  
foo.html Server  
cache  
Request foo.html
Slower more
foo.html expensive
Client  
B   global network

Fast inexpensive local network

Bryant  and  O’Hallaron,  Computer  Systems:  A  Programmer’s  Perspec;ve,  Third  Edi;on   63