MODULE -5

Python provides various options for developing graphical user interfaces (GUIs). Most
important are listed below.
 Tkinter − Tkinter is the Python interface to the Tk GUI toolkit shipped with Python. We
would look this option in this chapter.
 wxPython − This is an open-source Python interface for
wxWindows http://wxpython.org.
 JPython − JPython is a Python port for Java which gives Python scripts seamless access
to Java class libraries on the local machine http://www.jython.org.
There are many other interfaces available, which you can find them on the net.

Tkinter Programming

Tkinter is the standard GUI library for Python. Python when combined with Tkinter provides a
fast and easy way to create GUI applications. Tkinter provides a powerful object-oriented
interface to the Tk GUI toolkit.
Creating a GUI application using Tkinter is an easy task. All you need to do is perform the
following steps −
 Import the Tkinter module.
 Create the GUI application main window.
 Add one or more of the above-mentioned widgets to the GUI application.
 Enter the main event loop to take action against each event triggered by the user.

Example

#!/usr/bin/python

import Tkinter
top = Tkinter.Tk()
# Code to add widgets will go here...
top.mainloop()
This would create a following window −
Tkinter Widgets

Tkinter provides various controls, such as buttons, labels and text boxes used in a GUI
application. These controls are commonly called widgets.
There are currently 15 types of widgets in Tkinter. We present these widgets as well as a brief
description in the following table −

Sr.No. Operator & Description

1 Button

The Button widget is used to display buttons in your application.

2 Canvas
The Canvas widget is used to draw shapes, such as lines, ovals, polygons and rectangles, in your
application.

3 Checkbutton
The Checkbutton widget is used to display a number of options as checkboxes. The user can select
multiple options at a time.

4 Entry
The Entry widget is used to display a single-line text field for accepting values from a user.

5 Frame
The Frame widget is used as a container widget to organize other widgets.
6 Label
The Label widget is used to provide a single-line caption for other widgets. It can also contain
images.

7 Listbox
The Listbox widget is used to provide a list of options to a user.

8 Menubutton
The Menubutton widget is used to display menus in your application.

9 Menu
The Menu widget is used to provide various commands to a user. These commands are contained
inside Menubutton.

10 Message
The Message widget is used to display multiline text fields for accepting values from a user.

11 Radiobutton
The Radiobutton widget is used to display a number of options as radio buttons. The user can
select only one option at a time.

12 Scale
The Scale widget is used to provide a slider widget.

13 Scrollbar
The Scrollbar widget is used to add scrolling capability to various widgets, such as list boxes.

14 Text
The Text widget is used to display text in multiple lines.

15 Toplevel
The Toplevel widget is used to provide a separate window container.

16 Spinbox
The Spinbox widget is a variant of the standard Tkinter Entry widget, which can be used to select
from a fixed number of values.
17 PanedWindow
A PanedWindow is a container widget that may contain any number of panes, arranged
horizontally or vertically.

18 LabelFrame
A labelframe is a simple container widget. Its primary purpose is to act as a spacer or container for
complex window layouts.

19 tkMessageBox
This module is used to display message boxes in your applications.

he Button widget is used to add buttons in a Python application. These buttons can display text
or images that convey the purpose of the buttons. You can attach a function or a method to a
button which is called automatically when you click the button.

Syntax

Here is the simple syntax to create this widget −

w = Button ( master, option=value, ... )

Parameters

 master − This represents the parent window.

 options − Here is the list of most commonly used options for this widget. These options
can be used as key-value pairs separated by commas.

Sr.No. Option & Description

1
activebackground
Background color when the button is under the cursor.

2
activeforeground
Foreground color when the button is under the cursor.

3
bd
Border width in pixels. Default is 2.
4
bg
Normal background color.

5
command
Function or method to be called when the button is clicked.

6
fg
Normal foreground (text) color.

7
font
Text font to be used for the button's label.

8
height
Height of the button in text lines (for textual buttons) or pixels (for images).

9
highlightcolor
The color of the focus highlight when the widget has focus.

10
image
Image to be displayed on the button (instead of text).

11
justify
How to show multiple text lines: LEFT to left-justify each line; CENTER to center them; or
RIGHT to right-justify.

12
padx
Additional padding left and right of the text.

13
pady
Additional padding above and below the text.
14
relief
Relief specifies the type of the border. Some of the values are SUNKEN, RAISED, GROOVE,
and RIDGE.

15
state
Set this option to DISABLED to gray out the button and make it unresponsive. Has the value
ACTIVE when the mouse is over it. Default is NORMAL.

16
underline
Default is -1, meaning that no character of the text on the button will be underlined. If
nonnegative, the corresponding text character will be underlined.

17
width
Width of the button in letters (if displaying text) or pixels (if displaying an image).

18
wraplength
If this value is set to a positive number, the text lines will be wrapped to fit within this length.

Methods

Following are commonly used methods for this widget −

Sr.No. Method & Description

1
flash()
Causes the button to flash several times between active and normal colors. Leaves the button in the
state it was in originally. Ignored if the button is disabled.

2
invoke()
Calls the button's callback, and returns what that function returns. Has no effect if the button is
disabled or there is no callback.

Example
Try the following example yourself −
importTkinter
importtkMessageBox

top =Tkinter.Tk()

defhelloCallBack():
tkMessageBox.showinfo("Hello Python","Hello World")

B =Tkinter.Button(top, text ="Hello", command =helloCallBack)

B.pack()
top.mainloop()
When the above code is executed, it produces the following result −

The Canvas is a rectangular area intended for drawing pictures or other complex layouts. You
can place graphics, text, widgets or frames on a Canvas.

Syntax

Here is the simple syntax to create this widget −

w = Canvas ( master, option=value, ... )

Parameters

 master − This represents the parent window.

 options − Here is the list of most commonly used options for this widget. These options
can be used as key-value pairs separated by commas.

Sr.No. Option & Description

1
bd
Border width in pixels. Default is 2.

2
bg
Normal background color.

3
confine
If true (the default), the canvas cannot be scrolled outside of the scrollregion.

4
cursor
Cursor used in the canvas like arrow, circle, dot etc.

5
height
Size of the canvas in the Y dimension.

6
highlightcolor
Color shown in the focus highlight.

7
relief
Relief specifies the type of the border. Some of the values are SUNKEN, RAISED, GROOVE,
and RIDGE.

8
scrollregion
A tuple (w, n, e, s) that defines over how large an area the canvas can be scrolled, where w is the
left side, n the top, e the right side, and s the bottom.

9
width
Size of the canvas in the X dimension.

10
xscrollincrement
If you set this option to some positive dimension, the canvas can be positioned only on multiples
of that distance, and the value will be used for scrolling by scrolling units, such as when the user
 clicks on the arrows at the ends of a scrollbar.

11
xscrollcommand
If the canvas is scrollable, this attribute should be the .set() method of the horizontal scrollbar.

12
yscrollincrement
Works like xscrollincrement, but governs vertical movement.

13
yscrollcommand
If the canvas is scrollable, this attribute should be the .set() method of the vertical scrollbar.

The Canvas widget can support the following standard items −

arc − Creates an arc item, which can be a chord, a pieslice or a simple arc.
coord = 10, 50, 240, 210
arc = canvas.create_arc(coord, start=0, extent=150, fill="blue")
image − Creates an image item, which can be an instance of either the BitmapImage or the
PhotoImage classes.
filename = PhotoImage(file = "sunshine.gif")
image = canvas.create_image(50, 50, anchor=NE, image=filename)
line − Creates a line item.
line = canvas.create_line(x0, y0, x1, y1, ..., xn, yn, options)
oval − Creates a circle or an ellipse at the given coordinates. It takes two pairs of coordinates;
the top left and bottom right corners of the bounding rectangle for the oval.
oval = canvas.create_oval(x0, y0, x1, y1, options)
polygon − Creates a polygon item that must have at least three vertices.
oval = canvas.create_polygon(x0, y0, x1, y1,...xn, yn, options)

Example

Try the following example yourself −

importTkinter

top =Tkinter.Tk()

C =Tkinter.Canvas(top,bg="blue", height=250, width=300)

coord=10,50,240,210
arc =C.create_arc(coord, start=0, extent=150, fill="red")

C.pack()
top.mainloop()
When the above code is executed, it produces the following result −

The Checkbutton widget is used to display a number of options to a user as toggle buttons. The
user can then select one or more options by clicking the button corresponding to each option.
You can also display images in place of text.

Syntax

Here is the simple syntax to create this widget −

w = Checkbutton ( master, option, ... )

Parameters

 master − This represents the parent window.

 options − Here is the list of most commonly used options for this widget. These options
can be used as key-value pairs separated by commas.

Sr.No. Option & Description

1
activebackground
Background color when the checkbutton is under the cursor.

2
activeforeground
Foreground color when the checkbutton is under the cursor.

3
bg
The normal background color displayed behind the label and indicator.

4
bitmap
To display a monochrome image on a button.

5
bd
The size of the border around the indicator. Default is 2 pixels.

6
command
A procedure to be called every time the user changes the state of this checkbutton.

7
cursor
If you set this option to a cursor name (arrow, dot etc.), the mouse cursor will change to that
pattern when it is over the checkbutton.

8
disabledforeground
The foreground color used to render the text of a disabled checkbutton. The default is a stippled
version of the default foreground color.

9
font
The font used for the text.

10
fg
The color used to render the text.
11
height
The number of lines of text on the checkbutton. Default is 1.

12
highlightcolor
The color of the focus highlight when the checkbutton has the focus.

13
image
To display a graphic image on the button.

14
justify
If the text contains multiple lines, this option controls how the text is justified: CENTER, LEFT,
or RIGHT.

15
offvalue
Normally, a checkbutton's associated control variable will be set to 0 when it is cleared (off). You
can supply an alternate value for the off state by setting offvalue to that value.

16
onvalue
Normally, a checkbutton's associated control variable will be set to 1 when it is set (on). You can
supply an alternate value for the on state by setting onvalue to that value.

17
padx
How much space to leave to the left and right of the checkbutton and text. Default is 1 pixel.

18
pady
How much space to leave above and below the checkbutton and text. Default is 1 pixel.

19
relief
With the default value, relief=FLAT, the checkbutton does not stand out from its background. You
may set this option to any of the other styles

20
selectcolor
 The color of the checkbutton when it is set. Default is selectcolor="red".

21
selectimage
If you set this option to an image, that image will appear in the checkbutton when it is set.

22
state
The default is state=NORMAL, but you can use state=DISABLED to gray out the control and
make it unresponsive. If the cursor is currently over the checkbutton, the state is ACTIVE.

23
text
The label displayed next to the checkbutton. Use newlines ("\n") to display multiple lines of text.

24
underline
With the default value of -1, none of the characters of the text label are underlined. Set this option
to the index of a character in the text (counting from zero) to underline that character.

25
variable
The control variable that tracks the current state of the checkbutton. Normally this variable is
an IntVar, and 0 means cleared and 1 means set, but see the offvalue and onvalue options above.

26
width
The default width of a checkbutton is determined by the size of the displayed image or text. You
can set this option to a number of characters and the checkbutton will always have room for that
many characters.

27
wraplength
Normally, lines are not wrapped. You can set this option to a number of characters and all lines
will be broken into pieces no longer than that number.

Methods

Following are commonly used methods for this widget −

Sr.No. Method & Description

1
deselect()
Clears (turns off) the checkbutton.

2
flash()
Flashes the checkbutton a few times between its active and normal colors, but leaves it the way it
started.

3
invoke()
You can call this method to get the same actions that would occur if the user clicked on the
checkbutton to change its state.

4
select()
Sets (turns on) the checkbutton.

5
toggle()
Clears the checkbutton if set, sets it if cleared.

Example

Try the following example yourself −

fromTkinterimport*
importtkMessageBox
importTkinter

top =Tkinter.Tk()
CheckVar1=IntVar()
CheckVar2=IntVar()
C1 =Checkbutton(top, text ="Music", variable =CheckVar1, \
onvalue=1,offvalue=0, height=5, \
width =20)
C2 =Checkbutton(top, text ="Video", variable =CheckVar2, \
onvalue=1,offvalue=0, height=5, \
width =20)
C1.pack()
C2.pack()
top.mainloop()
When the above code is executed, it produces the following result −
The Entry widget is used to accept single-line text strings from a user.
 If you want to display multiple lines of text that can be edited, then you should use
the Text widget.
 If you want to display one or more lines of text that cannot be modified by the user, then
you should use the Label widget.

Syntax

Here is the simple syntax to create this widget −

w = Entry( master, option, ... )

Parameters

 master − This represents the parent window.

 options − Here is the list of most commonly used options for this widget. These options
can be used as key-value pairs separated by commas.

Sr.No. Option & Description

1
bg
The normal background color displayed behind the label and indicator.

2
bd
The size of the border around the indicator. Default is 2 pixels.

3
command
A procedure to be called every time the user changes the state of this checkbutton.
4
cursor
If you set this option to a cursor name (arrow, dot etc.), the mouse cursor will change to that
pattern when it is over the checkbutton.

5
font
The font used for the text.

6
exportselection
By default, if you select text within an Entry widget, it is automatically exported to the clipboard.
To avoid this exportation, use exportselection=0.

7
fg
The color used to render the text.

8
highlightcolor
The color of the focus highlight when the checkbutton has the focus.

9
justify
If the text contains multiple lines, this option controls how the text is justified: CENTER, LEFT,
or RIGHT.

10
relief
With the default value, relief=FLAT, the checkbutton does not stand out from its background. You
may set this option to any of the other styles

11
selectbackground
The background color to use displaying selected text.

12
selectborderwidth
The width of the border to use around selected text. The default is one pixel.

13
selectforeground
 The foreground (text) color of selected text.

14
show
Normally, the characters that the user types appear in the entry. To make a .password. entry that
echoes each character as an asterisk, set show="*".

15
state
The default is state=NORMAL, but you can use state=DISABLED to gray out the control and
make it unresponsive. If the cursor is currently over the checkbutton, the state is ACTIVE.

16
textvariable
In order to be able to retrieve the current text from your entry widget, you must set this option to
an instance of the StringVar class.

17
Width
The default width of a checkbutton is determined by the size of the displayed image or text. You
can set this option to a number of characters and the checkbutton will always have room for that
many characters.

18
xscrollcommand
If you expect that users will often enter more text than the onscreen size of the widget, you can
link your entry widget to a scrollbar.

Methods

Following are commonly used methods for this widget −

Sr.No. Method & Description

1
delete ( first, last=None )
Deletes characters from the widget, starting with the one at index first, up to but not including the
character at position last. If the second argument is omitted, only the single character at position
first is deleted.
2
get()
Returns the entry's current text as a string.

3
icursor ( index )
Set the insertion cursor just before the character at the given index.

4
index ( index )
Shift the contents of the entry so that the character at the given index is the leftmost visible
character. Has no effect if the text fits entirely within the entry.

5
insert ( index, s )
Inserts string s before the character at the given index.

6
select_adjust ( index )
This method is used to make sure that the selection includes the character at the specified index.

7
select_clear()
Clears the selection. If there isn't currently a selection, has no effect.

8
select_from ( index )
Sets the ANCHOR index position to the character selected by index, and selects that character.

9
select_present()
If there is a selection, returns true, else returns false.

10
select_range ( start, end )
Sets the selection under program control. Selects the text starting at the start index, up to but not
including the character at the end index. The start position must be before the end position.

11
select_to ( index )
Selects all the text from the ANCHOR position up to but not including the character at the given
 index.

12
xview ( index )
This method is useful in linking the Entry widget to a horizontal scrollbar.

13
xview_scroll ( number, what )
Used to scroll the entry horizontally. The what argument must be either UNITS, to scroll by
character widths, or PAGES, to scroll by chunks the size of the entry widget. The number is
positive to scroll left to right, negative to scroll right to left.

Example

Try the following example yourself −

fromTkinterimport*

top =Tk()
L1 =Label(top, text="User Name")
L1.pack( side = LEFT)
E1 =Entry(top, bd =5)
E1.pack(side = RIGHT)

top.mainloop()
When the above code is executed, it produces the following result −

The Frame widget is very important for the process of grouping and organizing other widgets in
a somehow friendly way. It works like a container, which is responsible for arranging the
position of other widgets.
It uses rectangular areas in the screen to organize the layout and to provide padding of these
widgets. A frame can also be used as a foundation class to implement complex widgets.

Syntax

Here is the simple syntax to create this widget −

w = Frame ( master, option, ... )
Parameters

 master − This represents the parent window.

 options − Here is the list of most commonly used options for this widget. These options
can be used as key-value pairs separated by commas.

Sr.No. Option & Description

1
bg
The normal background color displayed behind the label and indicator.

2
bd
The size of the border around the indicator. Default is 2 pixels.

3
cursor
If you set this option to a cursor name (arrow, dot etc.), the mouse cursor will change to that
pattern when it is over the checkbutton.

4
height
The vertical dimension of the new frame.

5
highlightbackground
Color of the focus highlight when the frame does not have focus.

6
highlightcolor
Color shown in the focus highlight when the frame has the focus.

7
highlightthickness
Thickness of the focus highlight.

8
relief
With the default value, relief=FLAT, the checkbutton does not stand out from its background. You
may set this option to any of the other styles
9
width
The default width of a checkbutton is determined by the size of the displayed image or text. You
can set this option to a number of characters and the checkbutton will always have room for that
many characters.

Example

Try the following example yourself −

fromTkinterimport*

root =Tk()
frame =Frame(root)
frame.pack()

bottomframe=Frame(root)
bottomframe.pack( side = BOTTOM )

redbutton=Button(frame, text="Red",fg="red")
redbutton.pack( side = LEFT)

greenbutton=Button(frame, text="Brown",fg="brown")
greenbutton.pack( side = LEFT )

bluebutton=Button(frame, text="Blue",fg="blue")
bluebutton.pack( side = LEFT )

blackbutton=Button(bottomframe, text="Black",fg="black")
blackbutton.pack( side = BOTTOM)

root.mainloop()
When the above code is executed, it produces the following result −

Label :This widget implements a display box where you can place text or images. The text
displayed by this widget can be updated at any time you want.
It is also possible to underline part of the text (like to identify a keyboard shortcut) and span the
text across multiple lines.
Syntax

Here is the simple syntax to create this widget −

w = Label ( master, option, ... )

Parameters

 master − This represents the parent window.

 options − Here is the list of most commonly used options for this widget. These options
can be used as key-value pairs separated by commas.

Sr.No. Option & Description

1
anchor
This options controls where the text is positioned if the widget has more space than the text needs.
The default is anchor=CENTER, which centers the text in the available space.

2
bg
The normal background color displayed behind the label and indicator.

3
bitmap
Set this option equal to a bitmap or image object and the label will display that graphic.

4
bd
The size of the border around the indicator. Default is 2 pixels.

5
cursor
If you set this option to a cursor name (arrow, dot etc.), the mouse cursor will change to that
pattern when it is over the checkbutton.

6
font
If you are displaying text in this label (with the text or textvariable option, the font option specifies
in what font that text will be displayed.
7
fg
If you are displaying text or a bitmap in this label, this option specifies the color of the text. If you
are displaying a bitmap, this is the color that will appear at the position of the 1-bits in the bitmap.

8
height
The vertical dimension of the new frame.

9
image
To display a static image in the label widget, set this option to an image object.

10
justify
Specifies how multiple lines of text will be aligned with respect to each other: LEFT for flush left,
CENTER for centered (the default), or RIGHT for right-justified.

11
padx
Extra space added to the left and right of the text within the widget. Default is 1.

12
pady
Extra space added above and below the text within the widget. Default is 1.

13
relief
Specifies the appearance of a decorative border around the label. The default is FLAT; for other
values.

14
text
To display one or more lines of text in a label widget, set this option to a string containing the text.
Internal newlines ("\n") will force a line break.

15
textvariable
To slave the text displayed in a label widget to a control variable of class StringVar, set this option
to that variable.
16
underline
You can display an underline (_) below the nth letter of the text, counting from 0, by setting this
option to n. The default is underline=-1, which means no underlining.

17
width
Width of the label in characters (not pixels!). If this option is not set, the label will be sized to fit
its contents.

18
wraplength
You can limit the number of characters in each line by setting this option to the desired number.
The default value, 0, means that lines will be broken only at newlines.

Example

Try the following example yourself −

fromTkinterimport*

root =Tk()
var=StringVar()
label =Label( root,textvariable=var, relief=RAISED )

var.set("Hey!? How are you doing?")

label.pack()
root.mainloop()
When the above code is executed, it produces the following result −

Let us study these widgets in detail −

Standard attributes

Let us take a look at how some of their common attributes.such as sizes, colors and fonts are
specified.
 Dimensions
 Colors
  Fonts
 Anchors
 Relief styles
 Bitmaps
 Cursors
Let us study them briefly −

Geometry Management

All Tkinter widgets have access to specific geometry management methods, which have the
purpose of organizing widgets throughout the parent widget area. Tkinter exposes the following
geometry manager classes: pack, grid, and place.
 The pack() Method − This geometry manager organizes widgets in blocks before placing
them in the parent widget.
 The grid() Method − This geometry manager organizes widgets in a table-like structure
in the parent widget.
 The place() Method − This geometry manager organizes widgets by placing them in a
specific position in the parent widget.
Let us study the geometry management methods briefly −Python provides multiple frameworks
for the web development. This tutorial covers five most commonly used python libraries which
are used for web development. All the mentioned libraries in this tutorial are the first choice in
certain project-specific conditions/requirements. Also, while trying to select the libraries, the
developers interest (based on their queries and community support) is considered.

Audience
This tutorial is designed to compare the basic features of some of the most commonly used
python frameworks. The target audience for this tutorial is −
 Anyone who wants to have a basic understanding of Django, Flask, Pyramid, Web2py
and Dash libraries.
 Anyone who wants to compare different Python frameworks and choose the most
suitable framework for their projects.
 Anyone who wants to explore Python web technologies in detail.

Prerequisites
Though there is no mandatory requirement for this tutorial, any prior knowledge on below
mentioned technologies will be an added advantage −
 Knowledge of any web related technologies
 Python language
  Developers/users who have previously worked on any python framework will definitely
find it easier to understand
Whenever a user opens any web browser like Google Chrome or Mozilla and search for ‘Web
development’, thousands of results appear in no time. What makes this possible? Web
development! It broadly refers to the work associated with building, creating and maintaining
websites for hosting via intranet or internet. The work associated in website design contain
multiple areas: web programming, database management, web design, web publishing, etc.
Web development includes all the codes that influence a website to run. We can separate the
whole process of web development into two categories −

 Front-end
 Back-end
Though frontend and backend web development are certainly distinct from each other, they are
also like two sides of the same coin. A complete website relies on each side communicating and
operating effectively with the other as a single unit. Both front-end and back-end are equally
important in web development.
The front-end or client-side of an application is the code responsible for everything the user
directly experiences on screen from text colors to buttons, images and navigation menus. Some
of the common skills and tools which are used by front-end developers are listed below −

 HTML/CSS/JavaScript
 CSS preprocessors
 Frameworks
 Libraries
 Git and Github
Generally, the back-end/server-side of an application is responsible for managing information
within the database and serving that information to the front-end. The back-end of a website
consists of a server, application, and database. In general, it involves everything that happens
before hitting your browser. The tools required in back-end web development are −
 Programming language − Ruby, PHP, Python, etc.
 Database − MySQL, PostgreSQL, MongoDB, Oracle, etc.
Why Web Development?
In today’s world, there are multiple choices to promote your business or skills and share your
ideas. Some of them are promotion through websites, native applications in marketplaces, etc.
The trend of creating new website as a tool of business development is rapidly gaining
momentum around the world. But, some of us may be unaware of the importance a website
holds in the growth of a business.
Currently there are numerous start-ups struggling to establish their presence in the open market.
However, it is also true that most of them fail to gain as much targeted audience as they want.
One primary reason which is bringing them down is that they underestimate the potential of a
full feature developed website to earn business for them. Website development for business or
any other purpose can prove quite fruitful.
Let us look at some of the important reasons why website development is important for business
growth −

Reaching out to your audiences

Online websites can reach to broadest audience and are not restricted to a platform that
constrain native applications. The viewers or customers can have easy access i.e. from a
desktop/laptop to a mobile device, as websites have the capability to display the content through
web browser.
In comparison to native application, web browsing is much simpler because it does not require
users to access the app store on their devices or download their app (which may include one or
more processes in accessing to your contents). Distribution of your data is much more flexible
and agile with web-based application than native ones as there are no stringent app store
requirements and content restrictions to follow.
Another tool which is very helpful for web development is the power of utilizing SEO
techniques to target your audiences.

24/7 Accessible
Instead of establishing a physical outlet for the company, if the business owner develops a
website as an online forum or similar, there will be good chance of gaining larger audience
online to link up. This is because, most of the people are wired up with the Internet all day.
Generally, people prefer to go for the smartest way to check online first and then take a
decision. So if the business owner fills all the basic details of the product and make a secure
way to get the product to the customer in timely manner, then people will prefer buying online
instead of visiting the outlet physically. This also allows people to access it even in the oddest
hour of the day.

Convenience
A fully functional website provides greater advantage for users where they can always tune in
and look for what they need. Generally, users avoid going to stores physically if they have an
option to get it online. So, if you are a smart businessman, you would prefer to have all the
details of your products or your stores on the website only to earn business, which you
otherwise might not.

Global Marketing
With an online website, you can link up to social forums and market your product/service to a
huge audience all around the globe. With this, you can regularly advertise and share your work
on social forums to gain much higher footprints of targeted audience.
Credible Source
An online portal is the most trustworthy platform for any company/organization. Sometimes
official websites can even function as their sole office. Consider a scenario, where it is not easy
to get access to a company’s physical location. In such case, you can overcome this worry by
focusing on their website.
In short, by developing a website, you can promote your services by a few clicks and you can
grab the attention of consumers from various parts of the world. The website of a company can
prove remarkable to gain business not only in a shorter time but also with a much bigger
audience.
Python is one of the most acceptable languages among web and application developers because
of its strong emphasis on efficiency and readability. There are numerous outstanding Python
web frameworks, each with their own specialities and features.
Django
Here, we will outline some necessary details and features of Django framework.
Category − Django belongs to the full-stack Python framework.
Release − Latest release – 2.1 version, commonly used release – 1.8, 1.6 version.
About − Built by experienced developers, Django is a high level Python web framework which
allows rapid, clean and pragmatic design development. Django handles much of the
complexities of web development, so you can focus on writing your app without a need to
reinvent the wheel. It’s free and open source.
To map objects to database table, Django uses ORM and the same is used to transfer from one
database to other.
It works with mostly all important databases like Oracle, MySQL, PostgreSQL, SQLite, etc.
There are numerous websites in the industry which uses Django as their primary framework for
backend development.

Features of Django
Some of the exemplary features of this Python web framework are −

 URL routing
 Authentication
 Database schema migrations
 ORM (Object-relational mapper)
 Template engine
The Official Website for Django framework is −https://www.djangoproject.com/
Flask
Category − Flask belongs to Non Full-stack frameworks.
Release − 1.0.2 released on 2018-05-02
About − It is classified as a micro-framework as we don’t require any particular libraries or
tools. It has no form validation or database abstraction layer or any other components where
pre-existing third party libraries provide common functions. However, flask support multiple
extensions which extended the application features as if they were implemented in Flask itself.
Extensions exist for object-relational mappers, form validation, upload handling, various open
authentication technologies and several common frameworks related tools.

Features of Flask

 Integrated support for unit testing

 Restful request dispatching
 Contains development server and debugger
 Uses Jinja2 templating
 Support for secure cookies
 Unicode-based
 100% WSGI 1.0 compliant
 Extensive documentation
 Google App Engine compatibility
 Extensions available to enhance features desired
Web2py
Category − Web2py belongs to Full-stack framework family.
Release − 2.17.1, released on 2018-08-06
About − Python 2.6, 2.7 to Python 3.x version. With no further dependencies, it’s a complete
package in itself. Development, database administration, debugging, deployment, testing, and
maintenance of applications all can be done through web interface, but generally not required. It
is a scalable open source framework that comes with its own web-based IDE alongside a code
editor, one-click deployment and debugger.

Features of Web2py
This framework comes with many developing tools and built-in features that eliminate the
hassle of complexity to the developers.
 With no installation and configuration, it is easy to run.
 Supports almost all major operating system, like Windows, Unix/Linux, Mac, Google
App Engine and almost all web hosting platform through Python 2.7/3.5/3.6/ version.
 Easy to communicate with MySQL, MSSQL, IBM DB2, Informix, Ingres, MongoDB,
SQLite, PostgreSQL, Sybase, Oracle and Google App Engine.
 It prevents the most common types of vulnerabilities including Cross Site Scripting,
Injection Flaws, and Malicious File Execution.
 Supports error tracking and internationalization.
 Multiple protocols readability.
 Employs successful software engineering practices that makes code easy to read and
maintain.
  Ensure user-oriented advancements through backward compatibility.

Pyramid
Category − Pyramid is a non-Full Stack Frameworks
Release − 1.9.2, released on 2018-04-23
About − Pyramid is a small, fast, down-to-earth Python web framework. It is developed as part
of the Pylons Project. It is licensed under a BSD-like license. It makes real-world web
application development and deployment more fun, more predictable and more productive.

Features of Pyramid
Python Pyramid is an open sourced framework with the following features −
 Simplicity − Anyone can start to work with it without any prior knowledge about it.
 Minimalism − Quite out of the box, Pyramid comes with only some important tools,
which are needed for almost every web application, may it be security or serving static
assets like JavaScript and CSS or attaching URLs to code.
 Documentation − Includes exclusive and up to date documentation.
 Speed − Very fast and accurate.
 Reliability − It is developed, keeping in mind that it is conservative and tested
exhaustively. If not tested properly, it will be considered as broke.
 Openness − It’s sold with a permissive and open license.
Dash
Category − The Dash framework belongs to “other” Python web frameworks.
Release − 0.24.1, core dash backend.
About − Dash as an open source library for creating interactive web-based visualizations. The
plotly team created Dash – an open source framework that leverages Flask, React.js and
plotly.js to build custom data visualization apps. Key highlight of this library is that you can
build highly interactive web application only through Python code. Data scientists love dash
framework, specially all those who are less familiar with web development.
With Dash, developers get access to all the configurable properties and underlying Flask
instance. The applications developed using Dash framework can be deployed to servers and are
eventually rendered in the web browser.
Dash applications are inherently cross-platform (Linux/Win/Mac) and mobile friendly and the
capabilities of applications can be extended by the rich set of Flask Plugins.
Features of Dash

 Provides access to configurable properties and Flask instance

 Through Flash plugins, we can extend the capabilities of the Dash application
 Mobile-ready
Django is an MVT web framework that is used to build web applications. The huge Django
web-framework comes with so many “batteries included” that developers often get amazed as to
how everything manages to work together. The principle behind adding so many batteries is to
have common web functionalities in the framework itself instead of adding latter as a separate
library.
One of the main reasons behind the popularity of Django framework is the huge Django
community. The community is so huge that a separate website was devoted to it where
developers from all corners developed third-party packages including authentication,
authorization, full-fledged Django powered CMS systems, e-commerce add-ons and so on.
There is a high probability that what you are trying to develop is already developed by
somebody and you just need to pull that into your project.
Why should you use Django?
Django is designed in such a way that encourages developers to develop websites fast, clean and
with practical design. Django’s practical approach to getting things done is where it stands out
from the crowd.
If you’re planning to build a highly customizable app, such as social media website, Django is
one of the best frameworks to consider. Django strength lies in its interaction between users or
its ability to share different types of media. One of the great advantage of django is its ability to
utilize large community-based support which gives you highly customizable third-party ready to
use plugins in your applications.
Below are the top ten reasons to choose Django for web development −

Python
Python is arguably one of the easiest programming languages to learn because of its simple
language constructs, flow structure and easy syntax. It is versatile and runs websites, desktop
applications and mobile applications embedded in many devices and is used in other
applications as a popular scripting language.

Batteries Included
Django comes with common libraries which are essential to build common functionalities like
URL routing, authentication, an object-relational mapper (ORM), a templating system and db-
schema migrations.
Built-in admin
Django has an in-built administration interface which lets you handle your models, user/ group
permissions and to manage users. With model interface in place, there is no need for a separate
database administration program for all but advanced database functions.

Doesn’t get in your way

Creating a Django application adds no boilerplate and no unnecessary functions. There’s no
mandatory imports, third-party libraries and no XML configuration files.

Scalable
Django is based on MVC design pattern. It means that all the entities like db (database), back-
end and front-end code are individual entity. Django allows us to separate code from the static
media including pictures, files, CSS and JavaScript that make up your site.
Django supports a full list of third-party libraries for web servers, caching, performance
management, clustering and balancing. One of the advantages Django provides is the support
for major email and messaging applications and services like ReST and OAuth.

Battle tested
Django was first open-sourced in 2005. After 12 years of growth, Django now not only runs
news publishing websites but also runs all or part of major global enterprise like Pinterest,
Instagram, Disqus, Bitbucket, EventBrite and Zapier. This makes it a robust and reliable web
framework to work with.

Huge package support

Because of its large community support and huge developers network, there is a high possibility
that whatever you intend to do might have been done before. Large international community of
developers contribute to the community by releasing their projects as open-source packages.
One such repository of these projects is Django Package site. Currently, Django packages list
over 3400 plus reusable Django apps, sites and tools to use in our Django projects.

Actively developed
One of the biggest risks associated with open source project is its sustainability. We cannot be
sure if it lasts long.
There is no such risk with Django as it is 12 years old. Its consistent releases, newer/better
versions and active community is growing every-day with a large core team of voluntary
contributors who maintains and improve the code base every-day.

Stable releases
Open-source software projects like Django are, in many cases, actively developed and more
secure than competing proprietary software as many developers are developing and testing it
every day. However, the drawback of an open-source software project is the absence of a stable
codebase to commercially viable development.
In Django, we have Long Term Support (LTS) versions of the software and a defined release
process as shown in the below image −

First Class Documentation

From the very first release, Django developers made sure that there must be proper
comprehensive documents available and the tutorials are easily understand.
Django Models
We are trying to create a Django model that will store all the posts in our blog. But to make sure
it works, we need to learn about objects.

Objects
Objects is a collection of properties and actions. Let us understand this with an example.
Suppose we want to model a cat, we will create an object called Cat that has properties such as
color, age, mood (good/bad/sleepy) and owner.
Then the Cat has some actions: purr, scratch or feed.
Cat
--------
color
age
mood
owner
purr()
scratch()
feed(cat_food)
CatFood
--------
taste
So basically we are trying to describe real things in code with properties (called object
properties) and actions (called methods).
As we are building a blog, we need some text content and a title. It is also necessary to have the
author’s name, date of creation and also the date of publication.
So our blog will have the following objects −
Post
--------
title
text
author
created_date
published_date
Also we need to have some method that publishes that post. As we are now aware of what an
object is, we can create a Django model for our blog post.
A model is a special kind of object in Django and is saved in the database. We will store our
data in SQLite database.
Creating an application
To keep everything clear, we will create a separate application inside our project. Below, we’ll
try to create a blog web application by running below mentioned simple command.
Now we will notice that a new myBlog directory is created and it contains a number of files
now. The directories and files in our project should be as following −
After creating an application, we also need to tell Django to use it. We do that in the file
myFirstDjangoProject/settings.py.
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'myBlog',
]
Creating a blog post model
In the myBlog/models.py file, we define all objects called Models – this is a place in which we
will define our blog post.
Let’s open myBlog/models.py, remove everything from it, and write code as follows −
Firstly, we import some functionalities from other files by calling from or import. So with this,
instead of copying and pasting the same things in every file, we can include some parts
with from and import.
class Post(models.Model) − This line defines our model (it is an object).
 class is a special keyword that indicates that we are defining an object.
 Post is the name of our model. Always start a class name with an uppercase letter.
 models.Model means that the Post is a Django Model, so Django knows that it should be
saved in the database.
Now let us discuss about the properties we defined above: title, text, created_date,
published_date and author. To do that, we need to define the type of each field.
 models.CharField − This is how you define text with a limited number of characters.
 Models.TextField − This is for long text without a limit.
 Models.DateTimeField − This is for date and time.
 Models.ForeignKey − This is a link to another model.
We define a function/method with def and publish is the name of the method.
Methods often return something. Here when we call __str__(), we will get a text (string) with a
Post title.
Creating tables for models in your database
The final step is to add a new model to our database. First, we have to let Django understand
that we have made some changes in our model. Let us do the same in our console window with
command python manage.py make migrations myBlog as shown below −

Then, Django prepares a migration file that we now have to apply to our database. In our
console we can type: python manage.py migrate myBlog and the output should be as follows
−

Our Post model is now in our database.

Django Admin
To add, edit and delete the posts we’ve just modelled, we are using Django admin.
So let us open the myBlog/admin.py file and put below contents in this −

Firstly, we import (include) the Post model defined in the previous chapter. To make our model
visible on the admin page, we need to register the model with admin.site.register (Post).
To login to an admin site, you need to create a superuser – a user account that has control over
everything on the site. So stop the webserver and type in command line python manage.py
createsuperuser, and press enter.
Ok, so it’s time to look at our Post model. Remember to run python manage.py run server in
console to run the web server. Go to your browser and type the
address https://127.0.0.1:8000/admin/. Log in with the credentials we have just chosen. Then
you should see the Django admin dashboard as below −

Go to Posts and experiment a little with it. You can add many blog posts, and contents from
anywhere. Your blog will look something like this −

This is just an overview of Django and we are able to create a blog with just a few lines of code
 What is a Python Web App?

A Python web application is a web page, where the instructions/actions are created using python

programming. For example, the following snippet of code creates a drop down option where the

user can select multiple wave heights as a filter:

wave_height = st.sidebar.multiselect("Wave height", df['wave_height'].unique())
st.write("Wave height selected", wave_height)

This is a creative way to showcase what your python script can do without needing others to

install python on their computers — all they require is an internet connection.

While python helps define your actions on your web app, a web framework will take care of the

details. This is where Streamlit comes in. Streamlit does the dashboarding design for you.

“Streamlit turns data scripts into shareable web apps in minutes.

All in Python. All for free. No front-end experience required.” — Streamlit Inc., 2021

This tutorial will make use of Streamlit as the web framework of choice, however, it is good to

understand the alternative options out there

Web Frameworks

Popular web frameworks used to build web applications using Python based programming

include: Dash, Streamlit, Voila , Flask and Django.

Flask is a web framework that has no data analysis capabilities. It is more of a ‘building from

scratch’ engineering tool that allows more customisation. It is better equipped at building smaller

custom applications and websites e.g. Reddit and Airbnb. Whereas, Django is geared to help

create more complex sites that receive a lot of traffic e.g. Spotify and Instagram.

Where Flask is a web framework, Streamlit and Dash would, for example, run on Flask, as they

are considered to be ‘dashboarding tools’. Streamlit is a great way to showcase a ‘Proof of

Value’ to stakeholders, whereas, Dash is if you require your dashboard to be put into production

and available for company use.

If you are coding in Jupyter Notebook and want to visualise your findings on a web page, then

you would use an add-on like Viola. Streamlit cannot, yet, be used in Jupyter Notebooks, but

rather runs your .py file in PyCharm, for example.

After choosing the best suited web framework, you would also need to choose a web server to

host your website and there are a variety of free and paid options available.

A popular example is Google App Engine, which is created under your Google Cloud project.

Other popular cloud providers include Amazon Web Services (AWS) and Microsoft Azure.

Here is a cheat sheet comparing these three cloud services:

(Trevor Jones, 2020, Source)

Working Example

The data

You can download the publicly available dataset, that I use in this working example,

from kaggle (Author: Guilherme Loureiro). The df_surf.csv dataset was an online survey

answered by surfers. Each row represents an individual surfer and the columns refer to the

questions answered. These questions relate to the surfboard, wave, surfer, manoeuvres and the

surfer’s performance.
Creating your project

By creating a new project in PyCharm, it will create a ‘main.py’ file for you. Save

the df_surf.csv in the same location as this main.py file. You can rename the .py file to anything

you like, but make sure you adapt my code to import your renamed file.

In the PyCharm Terminal, install the following packages:

pip install streamlit
pip install pandas
pip install numpy

In the main.py script, insert the following code and see my comments to follow along. If you are

new to Streamlit, here is a cheat sheet that is available on the streamlit site, created by Daniel

Lewis, 2021.
#import the packagesimport pandas as pd
import numpy as np
import streamlit as st#import the dataset and create a checkbox to shows the data on your website
df1 = pd.read_csv("df_surf.csv")
if st.checkbox('Show Dataframe'):
st.write(df1)#Insert some sentences in markdown format'''
# Surfing App
'''#Read in data again, but by using streamlit's caching aspect. Our whole app re-runs every time we
make small changes, which is infeasible the more complicated our code becomes. So this is the
recommended way to import the data.df = st.cache(pd.read_csv)("df_surf.csv")#create side bars that
allow for multiple selections:age = st.sidebar.multiselect("Select age", df['surfer_age'].unique())
st.write("Age selected", age)weekly_surfs = st.sidebar.multiselect("Surfer Exercise frequency",
df['surfer_exercise_frequency'].unique())
st.write("Frequency selected", weekly_surfs)surfer_experience = st.sidebar.multiselect("Surfer
Experience", df['surfer_experience'].unique())
st.write("Experience selected", surfer_experience)surfer_gender = st.sidebar.multiselect("Surfer
Gender", df['surfer_gender'].unique())
st.write("Gender selected", surfer_gender)wave_height = st.sidebar.multiselect("Wave height",
df['wave_height'].unique())
st.write("Wave height selected", wave_height)board_type = st.sidebar.multiselect("Board Type",
df['board_type'].unique())
st.write("Board type selected", board_type)#create a sidebar to select the variable output you want to
see after making your multiple selectionsvariables = st.sidebar.multiselect("Select what you want to
see for your selection", df.columns)
st.write("You selected these variables", variables)# return the sub-set data of variables you want to
seeselected_data = df[(df['surfer_age'].isin(age))]
subset_data = selected_data[variables]
data_is_check = st.checkbox("Display the data of selected variables")
if data_is_check:
st.write(subset_data)
Web Surfing with Python: Creating Simple Web Clients
One thing to keep in mind is that a browser is only one type of Web client. Any application that
makes a request for data from a Web server is considered a "client." Yes, it is possible to create
other clients that retrieve documents or data off the Internet. One important reason to do this is
that a browser provides only limited capacity, i.e., it is used primarily for viewing and interacting
with Web sites. A client program, on the other hand, has the ability to do moreit can not only
download data, but it can also store it, manipulate it, or perhaps even transmit it to another
location or application.

Applications that use the urllib module to download or access information on the Web [using
either urllib.urlopen() or urllib.urlretrieve()] can be considered a simple Web client. All you need
to do is provide a valid Web address.

Uniform Resource Locators

Simple Web surfing involves using Web addresses called URLs (Uniform Resource Locators).
Such addresses are used to locate a document on the Web or to call a CGI program to generate a
document for your client. URLs are part of a larger set of identifiers known as URIs (Uniform
Resource Identifiers). This superset was created in anticipation of other naming conventions that
have yet to be developed. A URL is simply a URI which uses an existing protocol or scheme
(i.e., http, ftp, etc.) as part of its addressing. To complete this picture, we'll add that non-URL
URIs are sometimes known as URNs (Uniform Resource Names), but because URLs are the
only URIs in use today, you really don't hear much about URIs or URNs, save perhaps as XML
identifiers.

Like street addresses, Web addresses have some structure. An American street address usually is
of the form "number street designation," i.e., 123 Main Street. It differs from other countries,
which have their own rules. A URL uses the format:

prot_sch://net_loc/path;params?query#frag
Table 20.1 describes each of the components.

Table 20.1. Web Address Components

URL ComponentDescription
prot_sch Network protocol or download scheme
net_loc Location of server (and perhaps user information)
path Slash ( / ) delimited path to file or CGI application
params Optional parameters
query Ampersand ( & ) delimited set of "key=value" pairs
frag Fragment to a specific anchor within document
net_loc can be broken down into several more components, some required, others optional.
The net_loc string looks like this:

user:passwd@host:port
20.2. Web Surfing with Python: Creating Simple Web Clients
One thing to keep in mind is that a browser is only one type of Web client. Any application that
makes a request for data from a Web server is considered a "client." Yes, it is possible to create
other clients that retrieve documents or data off the Internet. One important reason to do this is
that a browser provides only limited capacity, i.e., it is used primarily for viewing and interacting
with Web sites. A client program, on the other hand, has the ability to do moreit can not only
download data, but it can also store it, manipulate it, or perhaps even transmit it to another
location or application.

Applications that use the urllib module to download or access information on the Web [using
either urllib.urlopen() or urllib.urlretrieve()] can be considered a simple Web client. All you need
to do is provide a valid Web address.

20.2.1. Uniform Resource Locators

Simple Web surfing involves using Web addresses called URLs (Uniform Resource Locators).
Such addresses are used to locate a document on the Web or to call a CGI program to generate a
document for your client. URLs are part of a larger set of identifiers known as URIs (Uniform
Resource Identifiers). This superset was created in anticipation of other naming conventions that
have yet to be developed. A URL is simply a URI which uses an existing protocol or scheme
(i.e., http, ftp, etc.) as part of its addressing. To complete this picture, we'll add that non-URL
URIs are sometimes known as URNs (Uniform Resource Names), but because URLs are the
only URIs in use today, you really don't hear much about URIs or URNs, save perhaps as XML
identifiers.

Like street addresses, Web addresses have some structure. An American street address usually is
of the form "number street designation," i.e., 123 Main Street. It differs from other countries,
which have their own rules. A URL uses the format:

prot_sch://net_loc/path;params?query#frag

Table 20.1 describes each of the components.

Table 20.1. Web Address Components

URL ComponentDescription
prot_sch Network protocol or download scheme
net_loc Location of server (and perhaps user information)
path Slash ( / ) delimited path to file or CGI application
params Optional parameters
query Ampersand ( & ) delimited set of "key=value" pairs
frag Fragment to a specific anchor within document

net_loc can be broken down into several more components, some required, others optional.
The net_loc string looks like this:
user:passwd@host:port

These individual components are described in Table 20.2.

Table 20.2. Network Location Components

net_loc ComponentDescription
user User name or login
passwd User password
host Name or address of machine running Web server [required]
port Port number (if not 80, the default)

Of the four, the host name is the most important. The port number is necessary only if the Web
server is running on a different port number from the default. (If you aren't sure what a port
number is, go back to Chapter 16.)

User names and perhaps passwords are used only when making FTP connections, and even then
they usually aren't necessary because the majority of such connections are "anonymous."

Python supplies two different modules, each dealing with URLs in completely different
functionality and capacities. One is urlparse, and the other is urllib. We will briefly introduce
some of their functions here.
0.2.2. urlparse Module
The urlparse module provides basic functionality with which to manipulate URL strings. These
functions include urlparse(), urlunparse(), and urljoin().

urlparse.urlparse()
urlparse() breaks up a URL string into some of the major components described above. It has the
following syntax:

urlparse(urlstr, defProtSch=None, allowFrag=None)

urlparse() parses urlstr into a 6-tuple (prot_sch, net_loc, path, params, query, frag). Each of these
components has been described above. defProtSch indicates a default network protocol or
download scheme in case one is not provided in urlstr.allowFrag is a flag that signals whether or
not a fragment part of a URL is allowed. Here is what urlparse() outputs when given a URL:

>>>urlparse.urlparse('http://www.python.org/doc/FAQ.html') ('http', 'www.python.org',

'/doc/FAQ.html', '', '', '')
urlparse.urlunparse()
urlunparse() does the exact opposite of urlparse()it merges a 6-tuple
(prot_sch, net_loc, path, params, query, frag)urltup, which could be the output of urlparse(), into
a single URL string and returns it. Accordingly, we state the following equivalence:

urlunparse(urlparse(urlstr)) You may have already surmised that the syntax of urlunparse() is
as follows:

urlunparse(urltup)
Web Surfing with Python: Creating Simple Web Clients
One thing to keep in mind is that a browser is only one type of Web client. Any application that
makes a request for data from a Web server is considered a "client." Yes, it is possible to create
other clients that retrieve documents or data off the Internet. One important reason to do this is
that a browser provides only limited capacity, i.e., it is used primarily for viewing and interacting
with Web sites. A client program, on the other hand, has the ability to do moreit can not only
download data, but it can also store it, manipulate it, or perhaps even transmit it to another
location or application.

Applications that use the urllib module to download or access information on the Web [using
either urllib.urlopen() or urllib.urlretrieve()] can be considered a simple Web client. All you need
to do is provide a valid Web address.

Uniform Resource Locators

Simple Web surfing involves using Web addresses called URLs (Uniform Resource Locators).
Such addresses are used to locate a document on the Web or to call a CGI program to generate a
document for your client. URLs are part of a larger set of identifiers known as URIs (Uniform
Resource Identifiers). This superset was created in anticipation of other naming conventions that
have yet to be developed. A URL is simply a URI which uses an existing protocol or scheme
(i.e., http, ftp, etc.) as part of its addressing. To complete this picture, we'll add that non-URL
URIs are sometimes known as URNs (Uniform Resource Names), but because URLs are the
only URIs in use today, you really don't hear much about URIs or URNs, save perhaps as XML
identifiers.

Like street addresses, Web addresses have some structure. An American street address usually is
of the form "number street designation," i.e., 123 Main Street. It differs from other countries,
which have their own rules. A URL uses the format:

prot_sch://net_loc/path;params?query#frag

Table describes each of the components.

Table Web Address Components

URL ComponentDescription
prot_sch Network protocol or download scheme
net_loc Location of server (and perhaps user information)
Path Slash ( / ) delimited path to file or CGI application
params Optional parameters
Query Ampersand ( & ) delimited set of "key=value" pairs
Frag Fragment to a specific anchor within document

net_loc can be broken down into several more components, some required, others optional.
The net_loc string looks like this:
user:passwd@host:port

These individual components are described in Table

Table Network Location Components

net_loc ComponentDescription
user User name or login
passwd User password
host Name or address of machine running Web server [required]
port Port number (if not 80, the default)

Of the four, the host name is the most important. The port number is necessary only if the Web
server is running on a different port number from the default. (If you aren't sure what a port
number is, go back to Chapter 16.)

User names and perhaps passwords are used only when making FTP connections, and even then
they usually aren't necessary because the majority of such connections are "anonymous."

Python supplies two different modules, each dealing with URLs in completely different
functionality and capacities. One is urlparse, and the other is urllib. We will briefly introduce
some of their functions here.

urlparse Module
The urlparse module provides basic functionality with which to manipulate URL strings. These
functions include urlparse(), urlunparse(), and urljoin().

urlparse.urlparse()
urlparse() breaks up a URL string into some of the major components described above. It has the
following syntax:

urlparse(urlstr, defProtSch=None, allowFrag=None)

urlparse() parses urlstr into a 6-tuple (prot_sch, net_loc, path, params, query, frag). Each of these
components has been described above. defProtSch indicates a default network protocol or
download scheme in case one is not provided in urlstr.allowFrag is a flag that signals whether or
not a fragment part of a URL is allowed. Here is what urlparse() outputs when given a URL:

>>>urlparse.urlparse('http://www.python.org/doc/FAQ.html') ('http', 'www.python.org',

'/doc/FAQ.html', '', '', '')

urlparse.urlunparse()
urlunparse() does the exact opposite of urlparse()it merges a 6-tuple
(prot_sch, net_loc, path, params, query, frag)urltup, which could be the output of urlparse(), into
a single URL string and returns it. Accordingly, we state the following equivalence:

urlunparse(urlparse(urlstr)) You may have already surmised that the syntax of urlunparse() is
as follows:

urlunparse(urltup)

urlparse.urljoin()

The urljoin() function is useful in cases where many related URLs are needed, for example, the
URLs for a set of pages to be generated for a Web site. The syntax for urljoin() is:

urljoin(baseurl, newurl, allowFrag=None)

urljoin() takes baseurl and joins its base path (net_loc plus the full path up to, but not including, a
file at the end) with newurl. For example:

>>>urlparse.urljoin('http://www.python.org/doc/FAQ.html', \ ... 'current/lib/lib.htm')

'http://www.python.org/doc/current/lib/lib.html'

A summary of the functions in urlparse can be found in Table 20.3.

Core urlparse Module Functions

urlparse Functions Description

urlparse(urlstr,defProtSch=None,allowFrag=None)Parses urlstr into separate components, using
defProtSch if the protocol or scheme is not
given in urlstr; allowFrag determines whether
a URL fragment is allowed
urlunparse(urltup) Unparses a tuple of URL data (urltup) into a
single URL string
urljoin(baseurl,newurl,allowFrag=None) Merges the base part of the baseurl URL with
newurl to form a complete URL; allowFrag is
the same as for urlparse()

urllib Module
Core Module: urllib

Unless you are planning on writing a more lower-level network client, the
urllib module provides all the functionality you need. urllib provides a high-
level Web communication library, supporting the basic Web protocols,
HTTP, FTP, and Gopher, as well as providing access to local files.
Specifically, the functions of the urllib module are designed to download data
(from the Internet, local network, or local host) using the aforementioned
protocols. Use of this module generally obviates the need for using the
httplib, ftplib, and gopherlib modules unless you desire their lower-level
functionality. In those cases, such modules can be considered as alternatives.
(Note: Most modules named *libare generally for developing clients of the
corresponding protocols. This is not always the case, however, as perhaps
urllib should then be renamed "internetlib" or something similar!)

The urllib module provides functions to download data from given URLs as well as encoding
and decoding strings to make them suitable for including as part of valid URL strings. The
functions we will be looking at in this upcoming section include: urlopen(), urlretrieve(), quote(),
unquote(), quote_plus(), unquote_plus(), and urlencode(). We will also look at some of the
methods available to the file-like object returned by urlopen(). They will be familiar to you
because you have already learned to work with files back in Chapter 9.

urllib.urlopen()

urlopen() opens a Web connection to the given URL string and returns a file-like object. It has
the following syntax:

urlopen(urlstr, postQueryData=None)

urlopen() opens the URL pointed to by urlstr. If no protocol or download scheme is given, or if a
"file" scheme is passed in, urlopen() will open a local file.

For all HTTP requests, the normal request type is "GET." In these cases, the query string
provided to the Web server (key-value pairs encoded or "quoted," such as the string output of the
urlencode() function [see below]), should be given as part of urlstr.

If the "POST" request method is desired, then the query string (again encoded) should be placed
in the postQueryData variable. (For more information regarding the GET and POST request
methods, refer to any general documentation or texts on programming CGI applicationswhich we
will also discuss below. GET and POST requests are the two ways to "upload" data to a Web
server.

When a successful connection is made, urlopen() returns a file-like object as if the destination
was a file opened in read mode. If our file object is f, for example, then our "handle" would
support the expected read methods such as f.read(), f.readline(), f.readlines(), f.close(), and
f.fileno().

In addition, a f.info() method is available which returns the MIME (Multipurpose Internet Mail
Extension) headers. Such headers give the browser information regarding which application can
view returned file types. For example, the browser itself can view HTML (HyperText Markup
Language), plain text files, and render PNG (Portable Network Graphics) and JPEG (Joint
Photographic Experts Group) or the old GIF (Graphics Interchange Format) graphics files. Other
files such as multimedia or specific document types require external applications in order to
view.

Finally, a geturl() method exists to obtain the true URL of the final opened destination, taking
into consideration any redirection that may have occurred. A summary of these file-like object
methods is given in Table 20.4.

Table .urllib.urlopen() File-like Object Methods

urlopen() Object MethodsDescription
f.read([bytes]) Reads all or bytesbytes from f
f.readline() Reads a single line from f
f.readlines() Reads a all lines from f into a list
f.close() Closes URL connection for f
f.fileno() Returns file number of f
f.info() Gets MIME headers of f
f.geturl() Returns true URL opened for f

If you expect to be accessing more complex URLs or want to be able to handle more complex
situations such as basic and digest authentication, redirections, cookies, etc., then we suggest
using the urllib2 module, introduced back in the 1.6 days (mostly as an experimental module). It
too, has a urlopen() function, but also provides other functions and classes for opening a variety
of URLs. For more on urllib2, see the next section of this chapter.

urllib.urlretrieve()

urlretrieve() will do some quick and dirty work for you if you are interested in working with a
URL document as a whole. Here is the syntax for urlretrieve():

urlretrieve(urlstr, localfile=None, downloadSta- tusHook=None)

Rather than reading from the URL like urlopen() does, urlretrieve() will simply download the
entire HTML file located at urlstr to your local disk. It will store the downloaded data into
localfile if given or a temporary file if not. If the file has already been copied from the Internet or
if the file is local, no subsequent downloading will occur.

The downloadStatusHook, if provided, is a function that is called after each block of data has
been downloaded and delivered. It is called with the following three arguments: number of
blocks read so far, the block size in bytes, and the total (byte) size of the file. This is very useful
if you are implementing "download status" information to the user in a text-based or graphical
display.

urlretrieve() returns a 2-tuple, (filename, mime_hdrs). filename is the name of the local file
containing the downloaded data. mime_hdrs is the set of MIME headers returned by the
responding Web server. For more information, see the Message class of the mimetools module.
mime_hdrs is None for local files.

For a simple example using urlretrieve(), take a look at Example 11.4 (grabweb.py). A larger
piece of code using urlretrieve() can be found later in this chapter in Example 20.2.

urllib.quote() and urllib.quote_plus()

The quote*() functions take URL data and "encodes" them so that they are "fit" for inclusion as
part of a URL string. In particular, certain special characters that are unprintable or cannot be
part of valid URLs acceptable to a Web server must be converted. This is what the quote*()
functions do for you. Both quote*() functions have the following syntax:

quote(urldata, safe='/')

Characters that are never converted include commas, underscores, periods, and dashes, as well as
alphanumerics. All others are subject to conversion. In particular, the disallowed characters are
changed to their hexadecimal ordinal equivalents prepended with a percent sign (%), i.e., "%xx"
where "xx" is the hexadecimal representation of a character's ASCII value. When calling
quote*(), the urldata string is converted to an equivalent string that can be part of a URL string.
The safe string should contain a set of characters which should also not be converted. The default
is the slash ( / ).

quote_plus() is similar to quote() except that it also encodes spaces to plus signs ( + ). Here is an
example using quote() vs. quote_plus():

>>> name = 'joe mama' >>> number = 6 >>> base = 'http://www/~foo/cgi-bin/s.py'

>>> final = '%s?name=%s&num=%d' % (base, name, number) >>> final
'http://www/~foo/cgi-bin/s.py?name=joe mama&num=6' >>>>>>urllib.quote(final)
'http:%3a//www/%7efoo/cgi-bin/s.py%3fname%3djoe%20mama%26num%3d6'
>>>>>>urllib.quote_plus(final) 'http%3a//www/%7efoo/cgi-bin/s.py%3fname%3dj
oe+mama%26num%3d6'

urllib.unquote() and urllib.unquote_plus()

As you have probably guessed, the unquote*() functions do the exact opposite of the
quote*()functionsthey convert all characters encoded in the "%xx" fashion to their ASCII
equivalents. The syntax of unquote*() is as follows:

unquote*(urldata)

Calling unquote() will decode all URL-encoded characters in urldata and return the resulting
string. unquote_plus() will also convert plus signs back to space characters.

urllib.urlencode()

urlencode(), added to Python back in 1.5.2, takes a dictionary of key-value pairs and encodes
them to be included as part of a query in a CGI request URL string. The pairs are in "key=value"
format and are delimited by ampersands ( & ). Furthermore, the keys and their values are sent to
quote_plus() for proper encoding. Here is an example output from urlencode():

>>>aDict = { 'name': 'Georgina Garcia', 'hmdir': '~ggarcia' } >>>urllib.urlencode(aDict)

'name=Georgina+Garcia&hmdir=%7eggarcia'

There are other functions in urllib and urlparse which we did not have the opportunity to cover
here. Refer to the documentation for more information.
Secure Socket Layer support

The urllib module was given support for opening HTTP connections using the Secure Socket
Layer (SSL) in 1.6. The core change to add SSL is implemented in the socket module.
Consequently, the urllib and httplib modules were updated to support URLs using the "https"
connection scheme. In addition to those two modules, other protocol client modules with SSL
support include: imaplib, poplib, and smtplib.

A summary of the urllib functions discussed in this section can be found in Table 20.5.

Table 20.5. Core urllib Module Functions

urllib Functions Description
urlopen(urlstr, postQueryData=None) Opens the URL urlstr, sending the query data in
postQueryData if a POST request
urlretrieve(urlstr, Downloads the file located at the urlstr URL to
localfile=None,downloadStatusHook=None) localfile or a temporary file if localfile not given;
if present, downloaStatusHook is a function that
can receive download statistics
quote(urldata, safe='/') Encodes invalid URL characters of urldata;
characters in safe string are not encoded
quote_plus(urldata, safe='/') Same as quote() except encodes spaces as plus
(+) signs (rather than as %20)
unquote(urldata) Decodes encoded characters of urldata
unquote_plus(urldata) Same as unquote() but converts plus signs to
spaces
urlencode(dict) Encodes the key-value pairs of dict into a valid
string for CGI queries and encodes the key and
value strings with quote_plus()

20.2.4. urllib2 Module

As mentioned in the previous section, urllib2 can handle more complex URL opening. One
example is for Web sites with basic authentication (login and password) requirements. The most
straightforward solution to "getting past security" is to use the extended net_loc URL component
as described earlier in this chapter, i.e., http://user:[email protected]. The problem with
this solution is that it is not programmatic. Using urllib2, however, we can tackle this problem in
two different ways.

We can create a basic authentication handler (urllib2.HTTPBasicAuthHandler) and "register" a

login password given the base URL and perhaps a realm, meaning a string defining the secure
area of the Web site. (For more on realms, see RFC 2617 [HTTP Authentication: Basic and
Digest Access Authentication]). Once this is done, you can "install" a URL-opener with this
handler so that all URLs opened will use our handler.

The other alternative is to simulate typing the username and password when prompted by a
browser and that is to send an HTTP client request with the appropriate authorization headers. In
Example 20.1 we can easily identify each of these two methods.

he "handler" version of the code allocates a basic handler class as described earlier, then adds the
authentication information. The handler is then used to create a URL-opener that is then installed
so that all URLs opened will use the given authentication. This code was adapted from the
official Python documentation for the urllib2 module.

Lines 1722

The "request" version of our code just builds a Request object and adds the simple base64-
encoded authentication header into our HTTP request. This request is then used to substitute the
URL string when calling urlopen() upon returning back to "main." Note that the original URL
was built into the Request object, hence the reason why it was not a problem to replace it in the
subsequent call to urllib2.urlopen(). This code was inspired by Mike Foord's and Lee Harr's
recipes in the Python Cookbook located at:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/305288

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/267197

It would have been great to have been able to use Harr's HTTPRealmFinder class so that we do
not need to hardcode it in our example.

Lines 2429

The rest of this script just opens the given URL using both techniques and displays the first line
(dumping the others) of the resulting HTML page returned by the server once authentication has
been validated. Note that an HTTP error (and no HTML) would be returned if the authentication
information is invalid.

The output should look something like this:

 $ python urlopen-auth.py Using handler: <html> Using request: <html>

In addition to the official Python documentation for urllib2, you may find this companion piece
useful: http://www.voidspace.org.uk/python/articles/urllib2.shtml.

Related Modules:

Web Programming Related Modules

Module/Package Description
Web Applications
cgiGets Common Gateway Interface (CGI) form data
cgitb
[c] Handles CGI tracebacks
htmllibOlder HTML parser for simple HTML files; HTML-Parser class extends from
sgmllib.SGMLParser
HTMLparser
[c] Newer non-SGML-based parser for HTML and XHTML
htmlentitydefsHTML general entity definitions
Cookie Server-side cookies for HTTP state management
cookielib
[e] Cookie-handling classes for HTTP clients
webbrowser
[b] Controller: launches Web documents in a browser
sgmllibParses simple SGML files
robotparser
[a] Parses robots.txt files for URL "fetchability" analysis
httplib
[a] Used to create HTTP clients
XML Processing
xmllib(Outdated/deprecated) original simple XML parser
xml
[b] XML package featuring various parsers (some below)
xml.sax
[b] Simple API for XML (SAX) SAX2-compliant XML parser
xml.dom
[b] Document Object Model [DOM] XML parser
xml.etree
[f] Tree-oriented XML parser based on the Element flexible container object

xml.parsers.expat
[b] Interface to the non-validating Expat XML parser
xmlrpclib
[c] Client support for XML Remote Procedure Call (RPC) via HTTP
XML Processing
SimpleXMLRPCServer
[c] Basic framework for Python XML-RPC servers
DocXMLRPCServer
[d] Framework for self-documenting XML-RPC servers
Web Servers
BaseHTTPServerAbstract class with which to develop Web servers
SimpleHTTPServerServe the simplest HTTP requests (HEAD and GET)
CGIHTTPServerIn addition to serving Web files like SimpleHTTPServers, can also process CGI
(HTTP POST) requests
wsgiref
[f] Standard interface between Web servers and Python Web application
3rd party packages (not in standard library)
HTMLgenCGI helper converts Python objects into valid HTML http://starship.python.
net/crew/friedrich/HTMLgen/html/main.html
BeautifulSoupHTML and XML parser and screen-scraper http://crummy.com/software/
BeautifulSoup
Mail Client Protocols
poplibUse to create POP3 clients
imaplibUse to create IMAP4 clients
Mail and MIME Processing and Data Encoding Formats
email
[c] Package for managing e-mail messages, including MIME and other RFC2822-
based message
mailbox Classes for mailboxes of e-mail messages
mailcapParses mailcap files to obtain MIME application delegations
Mail and MIME Processing and Data Encoding Formats
mimetoolsProvides functions for manipulating MIME-encoded messages
mimetypesProvides MIME-type associations
MimeWriterGenerates MIME-encoded multipart files
multifile Can parse multipart MIME-encoded files
quopriEn-/decodes data using quoted-printable encoding
rfc822 Parses RFC822-compliant e-mail headers
smtplibUses to create SMTP (Simple Mail Transfer Protocol) clients
base64 En-/decodes data using base64 encoding
binasciiEn-/decodes data using base64, binhex, or uu (modules)
binhexEn-/decodes data using binhex4 encoding
uuEn-/decodes data using uuencode encoding
Internet Protocols
httplib
[a] Used to create HTTP clients
ftplibUsed to create FTP (File Transfer Protocol) clients
gopherlibUsed to create Gopher clients
telnetlibUsed to create Telnet clients
nntplibUsed to create NNTP (Network News Transfer Protocol [Usenet]) clients