Introduction to python programming (BPLCK205B) Module 4

MODULE 4

3.1 ORGANIZING FILES

The shutil Module, Walking a Directory Tree, Compressing Files with the zipfile Module, Project:
Renaming Files with American-Style Dates to European-Style Dates, Project: Backing Up a
Folder intoa ZIP File

3.2 DEBUGGING
Raising Exceptions, Getting the Traceback as a String, Assertions, Logging, IDLE’s Debugger

Department of CS&E, KVGCE, SULLIA Page 1

 Introduction to python programming (BPLCK205B) Module 4

3.1 ORGANIZING FILES

❖ Introduction
• We learnt how to create and write to new files in Python.
• Python programs can also organize preexisting files on the hard drive.
• Consider some of the tasks such as these:
• Making copies of all PDF files (and only the PDF files) in every subfolder of a folder
• Removing the leading zeros in the filenames for every file in a folder of hundreds
of filesnamed spam001.txt, spam002.txt, spam003.txt, and so on
• Compressing the contents of several folders into one ZIP file (which could be a simple
backupsystem).
• All this boring can be automated in Python

❖ The shutil Module

• The shutil (or shell utilities) module has functions to copy, move, rename, and delete
files inPython programs.
• Functions are:
➢ Copying Files and Folders
➢ Moving and Renaming Files and Folders
➢ Permanently Deleting Files and Folders
➢ Safe Deletes with the send2trash Module

→ Copying Files and Folders

• The shutil module provides functions for copying files, as well as entire folders.
• Calling shutil.copy(source, destination) will copy the file at the path source to the folder at
the pathdestination. (Both source and destination are strings.)
• If destination is a filename, it will be used as the new name of the copied file.
• This function returns a string of the path of the copied file.
• Example:

Department of CS&E, KVGCE, SULLIA Page 2

 Introduction to python programming (BPLCK205B) Module 4

• The shutil.copy() will copy a single file, shutil.copytree() will copy an entire folder and
everyfolder and file contained in it.
• Calling shutil.copytree(source, destination) will copy the folder at the path source, along
with all ofits files and subfolders, to the folder at the path destination.
• The source and destination parameters are both strings.
• The function returns a string of the path of the copied folder.
• Example:

→ Moving and Renaming Files and Folders

• Calling shutil.move(source, destination) will move the file or folder at the path source to
the pathdestination and will return a string of the absolute path of the new location.
• If destination points to a folder, the source file gets moved into destination and keeps its
currentfilename.
• Example:

Department of CS&E, KVGCE, SULLIA Page 3

 Introduction to python programming (BPLCK205B) Module 4

→ Permanently Deleting Files and Folders

• To delete a folder and all of its contents, we can use the following functions of shutil
module.
• Calling os.unlink(path) will delete the file at path.
• Calling os.rmdir(path) will delete the folder at path. This folder must be empty of any
files orfolders.
• Calling shutil.rmtree(path) will remove the folder at path, and all files and folders it
containswill also be deleted.
• Example

→ Safe Deletes with the send2trash Module

• A much better way to delete files and folders is with the third-party send2trash module.
• Install this module by running pip install send2trash from a Terminal window.
• Using send2trash is much safer than Python’s regular delete functions, because it will send
foldersand files to the computer’s trash or recycle bin instead of permanently deleting them.
• Use the send2trash.send2trash() function to have safe deletes

• Example:

→ Walking a Directory Tree

• Suppose, we want to rename every file in some folder and also every file in every subfolder

Department of CS&E, KVGCE, SULLIA Page 4

 Introduction to python programming (BPLCK205B) Module 4
of thatfolder.
• To do this, we want to walk through the directory tree, referring to each file. Writing a
program todo this could get tricky;
• But Python provides a function to handle this process more easily
• Use the os.walk() function on the directory tree
• The os.walk() function is passed a single string value: the path of a folder.
• We can use os.walk() in a for loop statement to walk a directory tree, much like how we
use therange() function to walk over a range of numbers.
• Unlike range(), the os.walk() function will return three values on each iteration through the
loop:
1. A string of the current folder’s name
2. A list of strings of the folders in the current folder
3. A list of strings of the files in the current folder
• Example: The below program will print names of the current folder along with its
subfolder andfile

import os

for folderName, subfolders, filenames in os.walk('C:\\delicious'):

print('The current folder is ' + folderName)

for subfolder in subfolders:

print('SUBFOLDER OF ' + folderName + ': ' + subfolder)
for filename in filenames:
print('FILE INSIDE ' + folderName + ': '+ filename)

print('')

Department of CS&E, KVGCE, SULLIA Page 5

 Introduction to python programming (BPLCK205B) Module 4

• Output:

→ Compressing Files with the zipfile Module

• We all are familiar with ZIP files (with the .zip file extension), which can hold the
compressedcontents of many other files.
• Compressing a file reduces its size, which is useful when transferring it over the Internet.

• Since a ZIP file can also contain multiple files and subfolders, it’s a handy way to
package several

files into one. This single file, called an archive file, can then be, say, attached to an email.

Python programs can both create and open (or extract) ZIP files using functions in
the zipfilemodule.

Department of CS&E, KVGCE, SULLIA Page 6

 Introduction to python programming (BPLCK205B) Module 4
→ Reading ZIP Files
• To read the contents of a ZIP file, first create a ZipFile object
• ZipFile objects are conceptually similar to the File objects
• They are values through which the program interacts with the file.
• To create a ZipFile object, call the zipfile.ZipFile() function, passing it a string of the
.zip file’sfilename.
• Note that zipfile is the name of the Python module, and ZipFile() is the name of the
function.
• Example:

• A ZipFile object has a namelist() method that returns a list of strings for all the files and
folderscontained in the ZIP file.

• These strings can be passed to the getinfo() ZipFile method to return a ZipInfo object about
thatparticular file.
• ZipInfo objects have their own attributes, such as file_size and compress_size in bytes, which
holdintegers of the original file size and compressed file size, respectively.
• While a ZipFile object represents an entire archive file, a ZipInfo object holds useful
informationabout a single file in the archive.
• Example:

Department of CS&E, KVGCE, SULLIA Page 7

 Introduction to python programming (BPLCK205B) Module 4
→ Extracting from ZIP Files
• The extractall() method for ZipFile objects extracts all the files and folders from a ZIP
file into thecurrent working directory.
• Example:

• After running this code, the contents of of the zip file will be extracted to the current
workingdirectory.
• Optionally, we can pass a folder name to extractall() to have it extract the files into a
folder otherthan the current working directory.
• If the folder passed to the extractall() method does not exist, it will be created.
• For instance, if we replaced the call at with exampleZip.extractall('C:\ delicious'), the
code wouldextract the files from zip file into a newly created C:\delicious folder.
• The extract() method for ZipFile objects will extract a single file from the ZIP file.

• Example:

• The string we pass to extract() must match one of the strings in the list returned by
namelist()
• In the above example, it will extract to the current working dir
ectoryC:\\Users\\Dell\\automate_online-materials\\yourname2.py
• Optionally, we can pass a second argument to extract() to extract the file into a folder
other than thecurrent working directory.
• If this second argument is a folder that doesn’t yet exist, Python will create the folder.
• The value that extract() returns is the absolute path to which the file was extracted.

→ Creating and Adding to ZIP Files

Department of CS&E, KVGCE, SULLIA Page 8

 Introduction to python programming (BPLCK205B) Module 4
• To create your own compressed ZIP files, we must open the ZipFile object in write mode
by passing 'w' as the second argument. (This is similar to opening a text file in write mode
by passing 'w' to the open() function.)
• When we pass a path to the write() method of a ZipFile object, Python will compress the
file at thatpath and add it into the ZIP file.
• The write() method’s first argument is a string of the filename to add. The second
argument is the compression type parameter, which tells the computer what algorithm it
should use to compress the files; we can always just set this value to
zipfile.ZIP_DEFLATED. (This specifies the deflate compression algorithm, which works
well on all types of data.)
• Example:

• This code will create a new ZIP file named new.zip that has the compressed contents of
spam.txt
• writing to files, write mode will erase all existing contents of a ZIP file.
• If you want to simply add files to an existing ZIP file, pass 'a' as the second argument to
zipfile.ZipFile() to open the ZIP file in append mode.

Department of CS&E, KVGCE, SULLIA Page 9

 Introduction to python programming (BPLCK205B) Module 4
3.2 DEBUGGING

Raising Exceptions:

• Python’s exceptions with try and except statements so that your program can recover from
exceptions that you anticipated.
• But you can also raise your own exceptions in your code.
• Raising an exception is a way of saying, “Stop running the code in this function and move
the program execution to the except statement.”
Exceptions are raised with a raise statement. In code, a raise statement consists of the
following:
• The raise keyword
• A call to the Exception() function
• A string with a helpful error message passed to the Exception() function

• If there are no try and except statements covering the raise statement that raised the
exception, the program simply crashes and displays the exception’s error message.
• The code that calls the function, not the fuction itself, that knows how to handle an
expection.
• So you will commonly see a raise statement inside a function and the try and except
statements in the code calling the function.

Department of CS&E, KVGCE, SULLIA Page 10

 Introduction to python programming (BPLCK205B) Module 4

Example:
def boxPrint(symbol, width, height):
if len(symbol) != 1:
raise Exception('Symbol must be a single character string.')
if width <= 2:
raise Exception('Width must be greater than 2.')
if height <= 2:
raise Exception('Height must be greater than 2.')

print(symbol * width)
for i in range(height - 2):
print(symbol + (' ' * (width - 2)) + symbol)
print(symbol * width)

for sym, w, h in (('*', 4, 4), ('O', 20, 5), ('x', 1, 3), ('ZZ', 3, 3)):
try:
boxPrint(sym, w, h)
except Exception as err:
print('An exception happened: ' + str(err))

Output:

Getting the Traceback as a String:

• When Python encounters an error, it produces a treasure trove of error information
called the traceback.
• The traceback includes the error message, the line number of the line that caused the
error, and the sequence of the function calls that led to the error.
• This sequence of calls is called the call stack.

Department of CS&E, KVGCE, SULLIA Page 11

 Introduction to python programming (BPLCK205B) Module 4

Traceback (most recent call last):

File "errorExample.py", line 7, in <module>
spam()
File "errorExample.py", line 2, in spam
bacon()
File "errorExample.py", line 5, in bacon
raise Exception('This is the error message.')
Exception: This is the error message.

• From the traceback, you can see that the error happened on line 5, in the bacon()
function. This particular call to bacon() came from line 2, in the spam() function, which
in turn was called on line 7.
• In programs where functions can be called from multiple places, the call stack can help
you determine which call led to the error.
Assertions:
• An assertion is a sanity check to make sure your code isn’t doing something obviously
wrong. These sanity checks are performed by assert statements.
• If the sanity check fails, then an AssertionError exception is raised. In code, an assert
statement consists of the following:
• The assert keyword
• A condition (that is, an expression that evaluates to True or False)
• A comma
• A string to display when the condition is False
Example:

def KelvinToFahrenheit(Temperature):
assert (Temperature >= 0),"Colder than absolute zero!"
return ((Temperature-273)*1.8)+32

print KelvinToFahrenheit(273)
print int(KelvinToFahrenheit(505.78))
print KelvinToFahrenheit(-5)

Department of CS&E, KVGCE, SULLIA Page 12

 Introduction to python programming (BPLCK205B) Module 4
Ouput:
32.0
451
Traceback (most recent call last):
File "test.py", line 9, in <module>
print KelvinToFahrenheit(-5)
File "test.py", line 4, in KelvinToFahrenheit
assert (Temperature >= 0),"Colder than absolute zero!"
AssertionError: Colder than absolute zero!

Logging:
• Logging is a great way to understand what’s happening in your program and in what
order its happening.
• Python’s logging module makes it easy to create a record of custom messages that you
write.
• These log messages will describe when the program execution has reached the logging
function call and list any variables you have specified at that point in time.
• On the other hand, a missing log message indicates a part of the code was skipped and
never executed.
Using the logging Module:

• To enable the logging module to display log messages on your screen as your program runs,
• copy the following to the top of your program

• when Python logs an event, it creates a LogRecord object that holds information
about that event.
• The logging module’s basicConfig() function lets you specify what details about the
LogRecord object you want to see and how you want those details displayed.
Example:

Department of CS&E, KVGCE, SULLIA Page 13

 Introduction to python programming (BPLCK205B) Module 4

• the logging.debug() function when we want to print log information.

• This debug() function will call basicConfig(), and a line of information will be printed.
• This information will be in the format we specified in basicConfig() and will include
the messages we passed to debug().
• The print(factorial(5)) call is part of the original program, so the result is displayed
even if logging messages are disabled.
Output:

• The factorial() function is returning 0 as the factorial of 5, which isn’t right.

• The for loop should be multiplying the value in total by the numbers from 1 to 5.
• But the log messages displayed by logging.debug() show that the i variable is starting
at 0 instead of 1.
• Since zero times anything is zero, the rest of the iterations also have the wrong value
for total
• Change the for i in range(n + 1): line to for i in range(1, n + 1):, and run the program
again. The output will look like this:

Department of CS&E, KVGCE, SULLIA Page 14

 Introduction to python programming (BPLCK205B) Module 4

Logging Levels:
• Logging levels provide a way to categorize your log messages by importance.
• Messages can be logged at each level using a different logging function.
LEVEL LOGGING DESCRIPTION
FUNCTION

DEBUG logging.debug() The lowest level. Used for small details. Usually, you care about these
messages only when diagnosing problems.

INFO logging.info() Used to record information on general events in your program or

confirm that things are working at their point in the program.

WARNING logging.warning() Used to indicate a potential problem that doesn’t prevent the program
from working but might do so in the future.

ERROR logging.error() Used to record an error that caused the program to fail to do
something.

CRITICAL logging.critical() The highest level. Used to indicate a fatal error that has caused or is
about to cause the program to stop running entirely.

• Your logging message is passed as a string to these functions. The logging levels are
suggestions. Ultimately, it is up to you to decide which category your log message falls into.

Disabling Logging:
• The logging.disable() function disables these so that you don’t have to go into your

Department of CS&E, KVGCE, SULLIA Page 15

 Introduction to python programming (BPLCK205B) Module 4
program and remove all the logging calls by hand.
• if you want to disable logging entirely, just add logging.disable(logging.CRITICAL)
to your program

>>> import logging

>>> logging.basicConfig(level=logging.INFO, format=' %(asctime)s - %(levelname)s - %(message)s')
>>> logging.critical('Critical error! Critical error!')
2015-05-22 11:10:48,054 - CRITICAL - Critical error! Critical error!
>>> logging.disable(logging.CRITICAL)
>>> logging.critical('Critical error! Critical error!')
>>> logging.error('Error! Error!')

IDLE ’s Debugger:
• The debugger is a feature of IDLE that allows you to execute your program one line at a
time.
• The debugger will run a single line of code and then wait for you to tell it to continue. By
running your program “under the debugger” like this, you can take as much time as you want
to examine the values in the variables at any given point during the program’s lifetime.
• This is a valuable tool for tracking down bugs. To enable IDLE’s debugger, click
Debug4Debugger in the interactive shell window.
• When the Debug Control window appears, select all four of the Stack, Locals, Source, and
Globals checkboxes so that the window shows the full set of debug information.
• While the Debug Control window is displayed, any time you run a program from the file
editor, the debugger will pause execution before the first instruction and display the
following:
▪ The line of code that is about to be executed
▪ A list of all local variables and their values
▪ A list of all global variables and their values

Department of CS&E, KVGCE, SULLIA Page 16

 Introduction to python programming (BPLCK205B) Module 4

The program will stay paused until you press one of the five buttons in the Debug Control window:
Go, Step, Over, Out, or Quit.
Go
Clicking the Go button will cause the program to execute normally until it terminates or reaches a
breakpoint. (Breakpoints are described later in this chapter.) If you are done debugging and want
the program to continue normally, click the Go button.
Step
Clicking the Step button will cause the debugger to execute the next line of code and then pause
again. The Debug Control window’s list of global and local variables will be updated if their values
change. If the next line of code is a function call, the debugger will “step into” that function and
jump to the first line of code of that function.
Over
Clicking the Over button will execute the next line of code, similar to the Step button. However, if
the next line of code is a function call, the Over button will “step over” the code in the function. The
function’s code will be executed at full speed, and the debugger will pause as soon as the function
call returns. For example, if the next line of code is a print() call, you don’t really care about code
inside the built-in print() function; you just want the string you pass it printed to the screen. For this
reason, using the Over button is more common than the Step button.
Out
Clicking the Out button will cause the debugger to execute lines of code at full speed until it returns
from the current function. If you have stepped into a function call with the Step button and now
simply want to keep executing instructions until you get back out, click the Out button to “step out”

Department of CS&E, KVGCE, SULLIA Page 17

 Introduction to python programming (BPLCK205B) Module 4
of the current function call.
Quit
If you want to stop debugging entirely and not bother to continue executing the rest of the program,
click the Quit button. The Quit button will immediately terminate the program. If you want to run
your program normally again, select Debug4Debugger again to disable the debugger

Department of CS&E, KVGCE, SULLIA Page 18