2020/21 COMP3234A

 What is Berkeley Unix Socket?

Connectionless and Connection (Connection-oriented) communication
modes (transport layer)
 Client-Server Communication
 Socket functions under Python (and C/C++)
 socket(), bind(), listen(), accept(), connect(), send(), recv(), sendto(), recvfrom(), close()
 Network Byte Order routines
 Concurrent server
 Other utilities

Computer and Communication Networks 2

[ILO4 - Implementation] be able to demonstrate knowledge in using
Socket Interface to design and implement network protocols

Computer and Communication Networks 3

 Computer Networking – A Top-Down Approach Featuring the Internet
7th edition, by J. Kurose et. al
 Sections 2.1, 2.7

 Socket Programming HowTo

 https://docs.python.org/3/howto/sockets.html

Computer and Communication Networks 4

 Socket Programming in Python
 Socket – Low-level networking interface
 https://docs.python.org/3.7/library/socket.html
 Python Module of the Week: Socket – Network Communication
 https://pymotw.com/3/socket/index.html
 TutorialsPoints – Python Network Programming
 https://www.tutorialspoint.com/python3/python_networking.htm

 Beej's Guide to Network Programming (for C interface)

 http://beej.us/guide/bgnet/

 W. Stevens, UNIX Network Programming, Volume 1, Prentice Hall

 Chapter 3 & 4

Computer and Communication Networks 5

 Communication always happens between two parties
 In daily life, we have many modes of communication
 e.g., by phone
 e.g., by post

 Always one party be the “active” one who initiates the communication
Using the telephone Using postal service
• Caller enters callee’s phone number • Sender writes receiver’s address on envelop
• Caller initiates the connection • Sender drops the letter into posting box
• Callee accepts the connection • Postal service picks up the letter
• They start the communication • Postman delivers the letter to receiver’s address
• Either party could end the connection • Sender does not know whether receiver will
receive the letter

Computer and Communication Networks 6

Berkeley sockets allows you to write network applications on top of
TCP or UDP
 Provides a generic service interface for application processes to use the services provide
by Transport Layer for communications

 Was originally designed

 For BSD Unix

 Now
 Industry standard
 Available on almost all operating systems
 Available in different “forms” under different Languages

Computer and Communication Networks 7

 It is an OS abstraction – a bidirectional communication endpoint
 Created dynamically by the program during runtime
 Persists only when application is running
 Used by the application on all communication operations through the socket

 Sockets may be used for communications

 within a process itself
 between processes on the same machine
 Between processes on different machines

 Sockets may be implemented over a number of different channel

types: Unix domain sockets, TCP, UDP, etc.

Computer and Communication Networks 8

 Connection-oriented  Uses by TCP
 Create the local endpoint
 Get a mobile phone with your SIM card
 Identify the remote endpoint
 Find your remote partner's telephone no.
 Initiate a connection
 Call your partner  First, setup logical connection between
 Accept by your partner two peer application processes
 Send and Receive data
 Start the chat  Reliable bidirectional in-sequence
transfer of byte stream
 Multiple send/recv between peer
 Terminate the connection processes
 Finally, connection release

Computer and Communication Networks 9

 Connectionless  Uses by UDP
 Create the local endpoint
 You have a mailbox be available for receiving
mails
 Destination address has to be written in each
 Identify the remote endpoint message
 Write down the destination address of the
recipient on the envelop  Immediate transfer of whole block of
 Send out your mail message
 Using the postal service  No setup overhead & delay

 Deliver to the destined mailbox  Send/recv to/from multiple peer processes by

using the same endpoint (UDP socket)

 Best-effort service only

 Possibly out-of-order
 Possibly loss

Computer and Communication Networks 10

 One application (process)
 Begins execution (be online) first
 Waits passively at a known location
 IP address and port number must be publicly known by the caller/sender
 just like you must know the phone # or postal address of your peer
 IP address tells where the end-system is; port number tells which process in that end-system involves in
the communication
 This Passive program is being called the server

 Another application (process)

 Begins execution later
 Actively contacts (initiate connection or send letter to) the first program
 Active program called a client

 This mode of interaction is used by all network applications (including the

P2P applications)
Computer and Communication Networks 11
 Servers  Clients
 Special-purpose program which  Arbitrary application program which
provides some services actively initiates contact with a
 Waits passively for clients to contact server
 Accepts requests from arbitrary  Request data or service provided by
clients the server
 Can handle multiple remote clients  usually direct interact or control by
simultaneously user

 E.g., Web server (Apache)  E.g., Web browser (Firefox)

Computer and Communication Networks 12

 Each network service must get a unique port number P
 e.g., HTTP - 80, Telnet - 23, SSH - 22, FTP - 21 (20), . . .
 This port # (service) must be known by all clients
 Server
 When a server is up running, it informs OS it is using port P
 No other application within the same machine can use the same port P
 Then, it waits for client requests to arrive
 Client
 A client also needs a port # for communication
 Usually we don't care which port # used by the client
 randomly assigned one port (ephemeral port) but must be unique within the client’s machine
 Constructs request to the server's port P
 Send request to port P of the server computer

Computer and Communication Networks 13

 Valid Range: 0 to 65535 (of size 16 bits)
 Divide into three ranges:
 Well-known ports: 0 to 1023
 Reserved ports
 Can only use by
 system processes
 privileged users
 Registered ports: 1024 to 49151
 Available for ordinary user processes
 Also allow to register with IANA (Internet Assigned Number Authority)
 Dynamic and/or Private ports: 49152 to 65535

Computer and Communication Networks 14

Server does Passive Open

Client does Active Open

Create the socket object

Associate the socket object with

the server’s IP address and port #

Prepare the incoming queue of

the socket
Ready to accept a client
connection

Initiate a connection
to server

recv () send ()

send ()
recv ()

Computer and Communication Networks 15

 Create the socket object

Associate the socket object with

the server’s IP addr and port #

Computer and Communication Networks 16

Computer and Communication Networks 17
To use sockets in your Python programs, you must include the Socket
Module in your programs
 import socket

 Python offers two basic sockets modules

 Socket Module – Low-level networking interface (close to the BSD API)
 SocketServer Module – Provides classes that simplify the development of network
servers

Computer and Communication Networks 18

 socket.socket([family[, type[, proto]]])
 System returns a new socket object for bidirectional communications

 For Internet networking

 family = socket.AF_INET (for IPv4), which is the default
= socket.AF_INET6 (for IPv6)
 type = socket.SOCK_STREAM (default) or socket.SOCK_DGRAM
 protocol - normally is safe to set to 0 (as default) for most cases

TCP
socket.socket() UDP
socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

Computer and Communication Networks 19

 A newly create socket object does not associate with any address
information sockfd = socket.socket()
 socket.bind(address) sockfd.bind( (“147.8.176.23”, 3245) )
 Assign a socket address to the newly created socket
 socket address consists of an IP address and a port number used by the host
 (IP, port) is represented as a tuple – an immutable list

 Bind() mostly used at server end

 Server process must registers its socket address with the system; which must be known to the
clients
 Client process can register a specific address for itself too, but is not required
 Normally, client doesn't need to explicitly call bind() as the system automatically assigns a socket
address to the client

Computer and Communication Networks 20

 Socket addresses are represented as follows:
 A pair (host, port) is used for the IPv4 address family
 Where host is a string representing either a hostname in Internet domain notation like
“i.cs.hku.hk” or an IPv4 address like “100.50.200.5”
 port is an integer

 For AF_INET6 address family, a four-tuple (host, port, flowinfo, scopeid) is used

 If a hostname is used in the host portion of IPv4/v6 socket address, the program may
show a nondeterministic behavior, as Python uses the first address returned from the
DNS resolution as the host address

Computer and Communication Networks 21

 This tells the socket to listen for incoming connections
 This is what differentiates the TCP servers from TCP clients

 socket.listen(backlog)
 Uses by TCP server
 Prepares socket to accept incoming connections
 backlog specifies the max. no. of incoming connection requests that can be queued while
waiting for server to accept them
 Maximum value is system-dependent (usually 5) sockfd = socket.socket()
sockfd.bind( (“147.8.176.23”, 3245) )
sockfd.listen(5)

Computer and Communication Networks 22

  Accept an incoming connection on a listening socket
 socket.accept()
 Used by TCP server
 After calling accept(), the (server) process will be blocked waiting for new incoming
connection;
 Once the connection is established, it returns a pair (conn, address) to the process
 conn is a new socket object, which the server uses it to communicate with the newly
connected client instead of using the original socket (sockfd)
 address is the socket address bound to the socket on the other end of the connection

sockfd = socket.socket()
sockfd.bind( (“147.8.176.23”, 3245) )
sockfd.listen(5)
conn, address = sockfd.accept() Computer and Communication Networks 23
 socket.connect(address)
 Used by TCP client
 Performs a TCP connection setup
 Establish a connection to a server who is waiting at accept()
 Once the socket is connected, you're free to communicate (send and recv)

Csockfd = socket.socket()
Csockfd.connect( (“147.8.178.121”, 80) )

Computer and Communication Networks 24

 Transmit outgoing data  Receive incoming data
 socket.send(bytes[, flags])  socket.recv(bufsize[, flags])
 This socket must be connected to a remote  The socket must be connected to a remote
socket socket
 bytes is the data to be sent which must be a  bufsize – indicate the maximum size of data
bytes object, i.e. a sequence of bytes to retrieve from network
 flags – can safely ignore  flags – can safely ignore
 Returns the number of bytes sent  Returns a bytes object if successful
 it could be less than the length of the bytes  Returns b'' when connection is broken
object
 It is the applications (i.e. the programmer)
responsibility to attempt delivery of the  socket.recv_into(buffer[, nbytes[,
remaining data flags]])
 Receive up to nbytes bytes of data and
 socket.sendall(bytes[, flags]) store the data into a buffer
 Unlike send(), this method continues to  Returns the number of bytes received
send data until either all data has been sent
or an error occurs
 Returns None on success
Computer and Communication Networks 25
 Transmit outgoing data  Receive incoming data
 socket.sendto(bytes, address)  socket.recvfrom(bufsize[, flags])
 bytes is the data to be sent  The counterpart of socket.recv()
 address is the socket address tuple  return a bytes object and the address
of destination socket tuple of the remote peer
 Returns the number of bytes sent
socket.recvfrom_into(buffer[,
nbytes[, flags]])
 The counterpart of
socket.recv_into()

Computer and Communication Networks 26

 socket.close()
 Close the socket
 All future operations on the socket object will fail

Computer and Communication Networks 27

  Problem:  Example: the number
 Different machines use different word orderings 0x12345678
 Little-endian
 Big-endian
 Little endian machine
 These machines may communicate with one another  Lower bytes first
over the network  Stores the four bytes as:
byte location content
184 78
185 56
186 34
0x12345678 187 12

store as  Big endian machine

 Higher bytes first
Big-endian Little-endian  Stores the four bytes as:
transmit byte location content
78 56 34 12 78 56 34 12
184 12
being stored 185 34
186 56
187 78
interpret as 0x78563412
Computer and Communication Networks 28
 Solution:
 For any data objects carry in a message that have data unit size larger than a byte, we
need to perform byte order conversion
 e.g., short integer, long integer, floating point, double, . . .
 How about the text string? e.g., “hello world”
 The basic data unit of a string is the Unicode character, which is a multi-bytes character
 We cannot send the “pure” Unicode string via socket
 We have to encode the string into ASCII or UTF-8 before sending via socket → transform the
string to be a sequence of bytes

 When transmitting message that contains integers, first do the conversion of all data
items with a unit size larger than a byte to Network Byte Order (that is big-endian) when
sending via the network
 Convert it back at the receiving end

Computer and Communication Networks 29

 Before sending out  After receiving
host
 socket.htonl (X) to  socket.ntohl (X)
 X is 32 bits positive number network  socket.ntohs (Y)
 socket.htons (Y) long
 Y is 16 bits positive number

0x12345678
store as
Big-endian machine Little-endian machine
being stored
12 34 56 78 78 56 34 12

ntohl() convert htonl() convert

12 34 56 78 transmit 12 34 56 78

interpret as 0x12345678
Computer and Communication Networks 30
TCPReceiver.py TCPSender.py
#!/usr/bin/python3 #!/usr/bin/python3

import socket import socket

sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# use "" means listening to all adapters in this machine # "localhost" means the loopback device of this machine,
# only valid for IPv4 # which uses the IP addr 127.0.0.1
sockfd.bind( ("", 32341) ) sockfd.connect( ("localhost", 32341) )

print("I_am", socket.gethostname(), "and_I_am_listening_...") print("The_connection_with", sockfd.getpeername(), \

"has_been_established")
sockfd.listen(5)
new, who = sockfd.accept() # Return the TCP connection # Get input for sending
msg = input("Enter your message --> ")
print("A_connection_with", who, "has_been_established")
sockfd.send(msg.encode("ascii"))
message = new.recv(50)
sockfd.close()
print("\'"+message.decode("ascii")+"\'", "is received from", who)

new.close()
sockfd.close()
Computer and Communication Networks 31
 Most errors related to socket or address semantics raise the error
socket.error exception Cltr-C

Example 1 – Only runs TCPSender.py Example 2 – Broken connection

c3234\03-SocketProgramming>python3 TCPSender.py c3234\03-SocketProgramming>python3 TCPSender.py

Traceback (most recent call last): The_connection_with ('127.0.0.1', 32341)
File "TCPSender.py", line 8, in <module> has_been_established
sockfd.connect( ("localhost", 32341) ) Enter your message --> Traceback (most recent call
ConnectionRefusedError: [WinError 10061] No last):
connection could be made because the target File "TCPSender.py", line 14, in <module>
machine actively refused it EOFError

c3234\03-SocketProgramming>python3 TCPReceiver.py
I_am atctam-Home-PC and_I_am_listening_...
A_connection_with ('127.0.0.1', 50624) has_been_established
'' is received from ('127.0.0.1', 50624)

Computer and Communication Networks 32

#!/usr/bin/python3 TCPSender-E.py #!/usr/bin/python3 TCPReceiver-E.py
import socket import socket
import sys
sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sockfd.bind( ("", 32341) )

try: print("I_am", socket.gethostname(), "and_I_am_listening_...")

sockfd.connect( ("localhost", 32341) )
except socket.error as err: sockfd.listen(5)
print("Connection error: ", err) new, who = sockfd.accept() # Return the TCP connection
sys.exit(1)
print("A_connection_with", who, "has_been_established")
print("The_connection_with", sockfd.getpeername(), \
"has_been_established") try:
message = new.recv(50)
try: except socket.error as err:
msg = input("Enter your message --> ") print("Recv error: ", err)
except:
print("Terminated abnormally!!") if message:
sockfd.close() print("\'"+message.decode("ascii")+"\'", "is received from", who)
sys.exit(1) else:
print("Connection is broken")
sockfd.send(msg.encode("ascii"))
new.close()
sockfd.close() 33
sockfd.close()
 Iterative (Sequential) server
 Only accepts (& handles) one client connection at a time
 There can have multiple client requests waiting for the service in the listen queue
 A client at the tail of the queue has to wait for all previous requests to be processed before
being accepted (& served)
 Unacceptable to users if long request blocks short requests

 Concurrent server
 Can handle multiple client requests at a time
 Usually server creates new thread of control (e.g. python thread or pthread) to handle
each request
 Client only waits for its request to be processed

Computer and Communication Networks 34

import socket TCPReceiver-C.py
import threading For brevity, not including error
handling steps
def thd_func(client):
new, who = client
print("A_connection_with", who, "has_been_established")
message = new.recv(50)
print("\'"+message.decode("ascii")+"\'", "is received from", who)
new.close()

sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

sockfd.bind( ("", 32341) )
print("I_am", socket.gethostname(), "and_I_am_listening_...")

sockfd.listen(5)

while (True):
client = sockfd.accept()
newthd = threading.Thread(target=thd_func, args=(client,))
newthd.start()

sockfd.close() Computer and Communication Networks 35

 Who are you ?  Name looking up
 socket.getpeername( ) - get remote socket  socket.gethostbyaddr( ) - given the IP
address (IP, port) of a connected stream address and lookup the host name
socket
 socket.getaddrinfo() – given the hostname,
 Who am I ? the system looks up all IP addresses (IPv4
 socket.getsockname( ) - get socket address and/or IPv6) of that machine
of a connected stream socket itself

 What is my name ?  select.select( )

 socket.gethostname( ) - get the hostname  a powerful tool to blocked-wait for multiple
of current machine active sockets at the same time

 Socket has many parameters and

options
 socket.getsockopt( )
 socket.setsockopt( )
Computer and Communication Networks 36
The socket interface is a set of declaration, definitions, and procedures for
writing client-server programs
 To use TCP, we need to create a stream socket
 To use UDP, we need to create a datagram socket
 Network communication uses big-endian as the network byte order for
transmitting of non-character data units
 To use a TCP server socket, these functions are involved
 socket.socket() socket.bind() socket.listen() socket.accept() [socket.recv()
socket.send()]* socket.close()

 To communicate with a TCP server, clients would invoke

 socket.socket() socket.connect() [socket.send() socket.recv()]* socket.close()

Computer and Communication Networks 37

 To use a UDP server socket, these functions are involved
 socket.socket() socket.bind() [socket.recvfrom() socket.sendto()]* socket.close()

 For the UDP client, these functions are involved

 socket.socket() [socket.sendto() socket.recvfrom()]* socket.close()

 A TCP server usually creates multiple threads to handle many clients

simultaneously

 Once a TCP connection is made, the server switches the dialogue to a

different socket to free up the main socket for additional incoming calls

Computer and Communication Networks 38