Network Programming - Server & Client A : Basics

Network Programming

Please visit http://www.bogotobogo.com/cplusplus/sockets_server_client.php for general concept

for Network programming, TCP/IP/, socket, etc.

Simple Socket

In the following code, the server sends the current time string to the client:

# server.py
import socket
import time

# create a socket object

serversocket = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)

# get local machine name

host = socket.gethostname()

port = 9999

# bind to the port

serversocket.bind((host, port))

# queue up to 5 requests
serversocket.listen(5)

while True:
# establish a connection
clientsocket,addr = serversocket.accept()
 print("Got a connection from %s" % str(addr))
currentTime = time.ctime(time.time()) + "\r\n"
clientsocket.send(currentTime.encode('ascii'))
clientsocket.close()

Here is the summary of the key functions from socket - Low-level networking interface:

1. socket.socket(): Create a new socket using the given address family, socket type and
protocol number.
2. socket.bind(address): Bind the socket to address.
3. socket.listen(backlog): Listen for connections made to the socket. The backlog
argument specifies the maximum number of queued connections and should be at least 0;
the maximum value is system-dependent (usually 5), the minimum value is forced to 0.
4. socket.accept(): The return value is a pair (conn, address) where conn is a new socket
object usable to send and receive data on the connection, and address is the address
bound to the socket on the other end of the connection.
At accept(), a new socket is created that is distinct from the named socket. This new
socket is used solely for communication with this particular client.
For TCP servers, the socket object used to receive connections is not the same socket
used to perform subsequent communication with the client. In particular, the accept()
system call returns a new socket object that's actually used for the connection. This
allows a server to manage connections from a large number of clients simultaneously.
5. socket.send(bytes[, flags]): Send data to the socket. The socket must be connected to a
remote socket. Returns the number of bytes sent. Applications are responsible for
checking that all data has been sent; if only some of the data was transmitted, the
application needs to attempt delivery of the remaining data.
6. socket.colse(): Mark the socket closed. all future operations on the socket object will fail.
The remote end will receive no more data (after queued data is flushed). Sockets are
automatically closed when they are garbage-collected, but it is recommended to close()
them explicitly.

Note that the server socket doesn't receive any data. It just produces client sockets. Each
clientsocket is created in response to some other client socket doing a connect() to the host and
port we're bound to. As soon as we've created that clientsocket, we go back to listening for more
connections.

# client.py
import socket

# create a socket object

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# get local machine name

host = socket.gethostname()

port = 9999

# connection to hostname on the port.

s.connect((host, port))

# Receive no more than 1024 bytes

tm = s.recv(1024)

s.close()

print("The time got from the server is %s" % tm.decode('ascii'))

The output from the run should look like this:

$ python server.py &

Got a connection from ('127.0.0.1', 54597)

$ python client.py
The time got from the server is Wed Jan 29 19:14:15 2014

Note from http://docs.python.org/2/howto/sockets.html:

"If you need fast IPC between two processes on one machine, you should look into whatever
form of shared memory the platform offers. A simple protocol based around shared memory and
locks or semaphores is by far the fastest technique."

"If you do decide to use sockets, bind the 'server' socket to 'localhost'. On most platforms, this
will take a shortcut around a couple of layers of network code and be quite a bit faster."

In Python 3, all strings are Unicode. For more info, visit Character Encoding.
So, if any kind of text string is to be sent across the network, it needs to be encoded.This is why
the server is using the encode('ascii') method on the data it transmits. Likewise, when a client
receives network data, that data is first received as raw unencoded bytes. If you print it out or try
to process it as text, we're unlikely to get what we expected. Instead, we need to decode it
first.This is why the client code is using decode('ascii') on the result.

Echo Server

This is an echo server: the server that echoes back all data it receives to a client that sent it.

Server:

# echo_server.py
import socket

host = '' # Symbolic name meaning all available interfaces

port = 12345 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(1)
conn, addr = s.accept()
print('Connected by', addr)
while True:
data = conn.recv(1024)
if not data: break
conn.sendall(data)
conn.close()

Client:

# echo_client.py
import socket

host = socket.gethostname()
port = 12345 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.sendall(b'Hello, world')
data = s.recv(1024)
s.close()
print('Received', repr(data))

Note that the server does not sendall()/recv() on the socket it is listening on but on the new
socket returned by accept().

$ python echo_server.py
('Connected by', ('127.0.0.1', 57750))

$ python echo_client.py
('Received', "'Hello, world'")

Network Programming - Server & Client B : File Transfer

Note

In this chapter, we're going to extend Python Network Programming I - Basic Server / Client A,
and try to file transfer from a server to numerous clients. The main purpose is to check the
performance of the server from which clients download files.
Local file transfer

Here is the code to send a file from a local server to a local client.

# server.py

import socket # Import socket module

port = 60000 # Reserve a port for your service.

s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
s.bind((host, port)) # Bind to the port
s.listen(5) # Now wait for client connection.

print 'Server listening....'

while True:
conn, addr = s.accept() # Establish connection with client.
print 'Got connection from', addr
data = conn.recv(1024)
print('Server received', repr(data))

filename='mytext.txt'
f = open(filename,'rb')
l = f.read(1024)
while (l):
conn.send(l)
print('Sent ',repr(l))
l = f.read(1024)
f.close()

print('Done sending')
conn.send('Thank you for connecting')
conn.close()

# client.py

import socket # Import socket module

s = socket.socket() # Create a socket object

host = socket.gethostname() # Get local machine name
port = 60000 # Reserve a port for your service.

s.connect((host, port))
s.send("Hello server!")

with open('received_file', 'wb') as f:

print 'file opened'
while True:
print('receiving data...')
data = s.recv(1024)
print('data=%s', (data))
if not data:
break
# write data to a file
 f.write(data)

f.close()
print('Successfully get the file')
s.close()
print('connection closed')

Output on a local server:

Server listening....
Got connection from ('192.168.56.10', 62854)
('Server received', "'Hello server!'")
('Sent ', "'1 1234567890\\n
...
('Sent ', "'4567890\\n105
...
('Sent ', "'300 1234567890\\n'")
Done sending

Output on a local client:

file opened
receiving data...
data=1 1234567890
2 1234567890
...
103 1234567890
104 123
receiving data...
data=4567890
105 1234567890
106 1234567890
...
299 1234567890

receiving data...
data=300 1234567890
Thank you for connecting
receiving data...
data=
Successfully get the file
connection closed

multithread tcp file transfer on localhost

Our server code above can only interact with one client. If we try to connect with a second client,
however, it simply won't reply to the new client. To let the server interact with multiple clients,
we need to use multi-threading. Here is the new server script to accept multiple client
connections:
# server2.py
import socket
from threading import Thread
from SocketServer import ThreadingMixIn

TCP_IP = 'localhost'
TCP_PORT = 9001
BUFFER_SIZE = 1024

class ClientThread(Thread):

def __init__(self,ip,port,sock):
Thread.__init__(self)
self.ip = ip
self.port = port
self.sock = sock
print " New thread started for "+ip+":"+str(port)

def run(self):
filename='mytext.txt'
f = open(filename,'rb')
while True:
l = f.read(BUFFER_SIZE)
while (l):
self.sock.send(l)
#print('Sent ',repr(l))
l = f.read(BUFFER_SIZE)
if not l:
f.close()
self.sock.close()
break

tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

tcpsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpsock.bind((TCP_IP, TCP_PORT))
threads = []

while True:
tcpsock.listen(5)
print "Waiting for incoming connections..."
(conn, (ip,port)) = tcpsock.accept()
print 'Got connection from ', (ip,port)
newthread = ClientThread(ip,port,conn)
newthread.start()
threads.append(newthread)

for t in threads:
t.join()

# client2.py
#!/usr/bin/env python

import socket

TCP_IP = 'localhost'
TCP_PORT = 9001
BUFFER_SIZE = 1024

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
with open('received_file', 'wb') as f:
print 'file opened'
while True:
#print('receiving data...')
data = s.recv(BUFFER_SIZE)
print('data=%s', (data))
if not data:
f.close()
print 'file close()'
break
# write data to a file
f.write(data)

print('Successfully get the file')

s.close()
print('connection closed')

Below is the output from the server console when we run two clients simultaneously:

$ python server2.py
Waiting for incoming connections...
Got connection from ('127.0.0.1', 55184)
New thread started for 127.0.0.1:55184
Waiting for incoming connections...
Got connection from ('127.0.0.1', 55185)
New thread started for 127.0.0.1:55185
Waiting for incoming connections...

tcp file download from EC2 to local

In the following codes, we made two changes:

1. ip switched to amazon ec2 ip

2. To calculate the time to take download a file, we import time module.

# server3.py on EC2 instance

import socket
from threading import Thread
from SocketServer import ThreadingMixIn

# TCP_IP = 'localhost'
TCP_IP = socket.gethostbyaddr("your-ec2-public_ip")[0]
TCP_PORT = 60001
BUFFER_SIZE = 1024

print 'TCP_IP=',TCP_IP
print 'TCP_PORT=',TCP_PORT

class ClientThread(Thread):

def __init__(self,ip,port,sock):
Thread.__init__(self)
self.ip = ip
self.port = port
self.sock = sock
print " New thread started for "+ip+":"+str(port)

def run(self):
filename='mytext.txt'
f = open(filename,'rb')
while True:
l = f.read(BUFFER_SIZE)
while (l):
self.sock.send(l)
#print('Sent ',repr(l))
l = f.read(BUFFER_SIZE)
if not l:
f.close()
self.sock.close()
break

tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

tcpsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpsock.bind((TCP_IP, TCP_PORT))
threads = []

while True:
tcpsock.listen(5)
print "Waiting for incoming connections..."
(conn, (ip,port)) = tcpsock.accept()
print 'Got connection from ', (ip,port)
newthread = ClientThread(ip,port,conn)
newthread.start()
threads.append(newthread)

for t in threads:
t.join()

# client3.py on local machine

#!/usr/bin/env python

#!/usr/bin/env python

import socket
import time

#TCP_IP = 'localhost'
TCP_IP = 'ip-ec2-instance'
TCP_PORT = 60001
BUFFER_SIZE = 1024

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))

clock_start = time.clock()
time_start = time.time()

with open('received_file', 'wb') as f:

print 'file opened'
while True:
#print('receiving data...')
data = s.recv(1024)
#print('data=%s', (data))
if not data:
f.close()
print 'file close()'
break
# write data to a file
f.write(data)

print('Successfully get the file')

s.close()
print('connection closed')

clock_end = time.clock()
time_end = time.time()

duration_clock = clock_end - clock_start

print 'clock: start = ',clock_start, ' end = ',clock_end
print 'clock: duration_clock = ', duration_clock

duration_time = time_end - time_start

print 'time: start = ',time_start, ' end = ',time_end
print 'time: duration_time = ', duration_time

Server console shows the following output after a connection from my local home machine:

$ python server3.py
TCP_IP= ec2-...
TCP_PORT= 60001
Waiting for incoming connections...
Got connection from ('108.239.135.40', 56742)
New thread started for 108.239.135.40:56742

The ip is isp's:

On my local mac:
$ python client3.py
file opened
file close()
Successfully get the file
connection closed
clock: start = 0.018806 end = 0.038608
clock: duration_clock = 0.019802
time: start = 1434991840.37 end = 1434991840.42
time: duration_time = 0.0457620620728

File downloaded from EC2, received_file is simple, and it looks like this:

From EC2
1
2
3
4
5
6
7
8
9

Download time vs number of clients

Here is the output showing the wall-clock time depending on the number of concurrent
connections:
Our server is located in California, and the following picture compares the download speed
between US and Japan: