Module -3

Organizing Fi les
 Organizing Fi les

• Your programs can also organize preexisting files on the hard drive. Maybe
you’ve had the experience of going through a folder full of dozens, hundreds,
or even thousands of files and copying, renaming, moving, or compressing
them all by hand.
The shutil Module:

The shutil (or shell utilities) module has functions to let you copy, move,
rename, and delete files in your Python programs.

To use the shutil functions, you will first need to use import shutil.
 Copying Files and Folders
The shutil module provides functions for copying files, as well as entire
folders.

Calling shutil.copy(source, destination) will copy the file at the path source to
the folder at the path destination. (Both source and destination are strings.)

If destination is a filename, it will be used as the new name of the copied file.
This function returns a string of the path of the copied file.
 Cont..
• Enter the following into the interactive shell to see how shutil.copy() works:

>>> import shutil, os

>>> os.chdir('C:\\')

>>> shutil.copy('C:\\spam.txt', 'C:\\delicious')

'C:\\delicious\\spam.txt'

>>> shutil.copy('eggs.txt', 'C:\\delicious\\eggs2.txt')

'C:\\delicious\\eggs2.txt'

The first shutil.copy() call

 Cont..

The first shutil.copy() call copies the file at C:\spam.txt to the folder C:\delicious. The return
value is the path of the newly copied file.

Note that since a folder was specified as the destination, the original spam.txt filename is
used for the new, copied file’s filename.

The second shutil.copy() call also copies the file at C:\eggs.txt to the folder C:\delicious but
gives the copied file the name eggs2.txt.

While shutil.copy() will copy a single file, shutil.copytree() will copy an entire folder and
every folder and file contained in it. Calling shutil.copytree(source, destination) will copy
the folder at the path source, along with all of its files and subfolders, to the folder at the
path destination.
 Cont..

The source and destination parameters are both strings. The function returns a string of the
path of the copied folder.

>>> import shutil, os

>>> os.chdir('C:\\')

>>> shutil.copytree('C:\\bacon', 'C:\\bacon_backup')

'C:\\bacon_backup‘

The shutil.copytree() call creates a new folder named bacon_backup with the same content
as the original bacon folder. You have now safely backed up your precious, precious bacon.
 Moving and Renaming Files and Folders

• Calling shutil.move(source, destination) will move the file or folder at the path
source to the path destination and will return a string of the absolute path of the new
location.

• If destination points to a folder, the source file gets moved into destination and
keeps its current filename. For example, enter the following into the interactive
shell:

>>> import shutil

>>> shutil.move('C:\\bacon.txt', 'C:\\eggs')

'C:\\eggs\\bacon.txt'
Assuming a folder named eggs already exists in the C:\ directory, this
shutil.move() calls says, “Move C:\bacon.txt into the folder C:\eggs.”

If there had been a bacon.txt file already in C:\eggs, it would have been
overwritten. Since it’s easy to accidentally overwrite files in this way, you
should take some care when using move().

The destination path can also specify a filename. In the following example,

the source file is moved and renamed.

>>> shutil.move('C:\\bacon.txt', 'C:\\eggs\\new_bacon.txt')

'C:\\eggs\\new_bacon.txt‘
 Cont..
• Both of the previous examples worked under the assumption that there was a
folder eggs in the C:\ directory. But if there is no eggs folder, then move() will
rename bacon.txt to a file named eggs.

>>> shutil.move('C:\\bacon.txt', 'C:\\eggs')

'C:\\eggs'

• Here, move() can’t find a folder named eggs in the C:\ directory and so
assumes that destination must be specifying a filename, not a folder. So the
bacon.txt text file is renamed to eggs.
 Permanently Deleting Files and Folders

You can delete a single file or a single empty folder with functions in the os module,
whereas to delete a folder and all of its contents, you use the shutil module.

• Calling os.unlink(path) will delete the file at path.

• Calling os.rmdir(path) will delete the folder at path. This folder must be empty of

any files or folders.

• Calling shutil.rmtree(path) will remove the folder at path, and all files and folders

it contains will also be deleted.

Be careful when using these functions in your programs!

 Cont..
A Python program that was intended to delete files that have the .txt
file extension but has a typo (highlighted in bold) that causes it to
delete .rxt files instead:
 import os
 for filename in os.listdir():
 if filename.endswith('.rxt'):
 os.unlink(filename)
• If you had any important files ending with .rxt, they would

have been accidentally, permanently deleted. Instead, you

should have first run the program like this:

 import os
 for filename in os.listdir():
 if filename.endswith('.rxt'):
 #os.unlink(filename)
 print(filename)
 Safe Deletes with the send2trash Module.

Since Python’s built-in shutil.rmtree() function irreversibly deletes files

and folders, it can be dangerous to use.

A much better way to delete files and folders is with the third-party
send2trash module. You can install this module by running pip install
send2trash from a Terminal window.

Using send2trash is much safer than Python’s regular delete functions,

because it will send folders and files to your computer’s trash or recycle bin
instead of permanently deleting them.
 Cont..

• If a bug in your program deletes something with send2trash you didn’t

intend to delete, you can later restore it from the recycle bin.

• After you have installed send2trash, enter the following into the interactive
shell:
>>> import send2trash
>>> baconFile = open('bacon.txt', 'a') # creates the file
>>> baconFile.write('Bacon is not a vegetable.')
25
>>> baconFile.close()
>>> send2trash.send2trash('bacon.txt')
 Walking a Directory Tree
• Say you want to rename every file in some folder
and also every file in every subfolder of that
folder. That is, you want to walk through the
directory tree, touching each file as you go.
Writing a program to do this could get tricky;
fortunately, Python provides a function to handle
this process for you.

• Here is an example program that uses the

os.walk() function on the directory tree
 Cont..
import os
for folderName, subfolders, filenames in os.walk('C:\\delicious'):
print('The current folder is ' + folderName)
for subfolder in subfolders:
print('SUBFOLDER OF ' + folderName + ': ' + subfolder)
for filename in filenames:
print('FILE INSIDE ' + folderName + ': '+ filename)
print('')

• The os.walk() function is passed a single string value: the path of a folder. You
can use os.walk() in a for loop statement to walk a directory tree, much like
how you can use the range() function to walk over a range of numbers.
 Cont..
Unlike range(), the os.walk() function will return three values on each iteration
through the loop:

1. A string of the current folder’s name

2. A list of strings of the folders in the current folder

3. A list of strings of the files in the current folder

When you run this program, it will output the following:

Output for above program:
The current folder is C:\delicious
SUBFOLDER OF C:\delicious: cats
SUBFOLDER OF C:\delicious: walnut
FILE INSIDE C:\delicious: spam.txt
The current folder is C:\delicious\cats
FILE INSIDE C:\delicious\cats: catnames.txt
FILE INSIDE C:\delicious\cats: zophie.jpg
The current folder is C:\delicious\walnut
SUBFOLDER OF C:\delicious\walnut: waffles
The current folder is C:\delicious\walnut\waffles
FILE INSIDE C:\delicious\walnut\waffles: butter.txt.
 Compressing Files with the zipfile Module
You may be familiar with ZIP files (with the .zip file extension), which
can hold the compressed contents of many other files.

Compressing a file reduces its size, which is useful when transferring it

over the Internet. And since a ZIP file can also contain multiple files
and subfolders, it’s a handy way to package several files into one. This
single file, called an archive file, can then be, say, attached to an email.
 Cont..

• since a ZIP file can also contain multiple

files and subfolders, it’s a handy way to
package several files into one.

• This single file, called an archive file,

can then be, say, attached to an email.
 Reading ZIP Files
• To read the contents of a ZIP file, first you must create a ZipFile object (note
the capital letters Z and F).

• ZipFile objects are conceptually similar to the File objects you saw returned
by the open() function in the previous chapter: They are values through
which the program interacts with the file.

• To create a ZipFile object, call the zipfile.ZipFile() function, passing it a

string of the .zip file’s filename. Note that zipfile is the name of the Python
module, and ZipFile() is the name of the function.
For example, enter the following into the interactive shell:
>>> import zipfile, os
>>> os.chdir('C:\\') # move to the folder with example.zip
>>> exampleZip = zipfile.ZipFile('example.zip')
>>> exampleZip.namelist()
['spam.txt', 'cats/', 'cats/catnames.txt', 'cats/zophie.jpg']
>>> spamInfo = exampleZip.getinfo('spam.txt')
>>> spamInfo.file_size
13908
>>> spamInfo.compress_size
3828
>>> 'Compressed file is %sx smaller!' % (round(spamInfo.file_size / spamInfo
.compress_size, 2))
'Compressed file is 3.63x smaller!'
>>> exampleZip.close()
• A ZipFile object has a namelist() method that returns a list of strings for all the
files and folders contained in the ZIP file. These strings can be passed to the
getinfo() ZipFile method to return a ZipInfo object about that particular file.

• ZipInfo objects have their own attributes, such as file size and compress_size in
bytes, which hold integers of the original file size and compressed file size,
respectively.
 Extracting from ZIP Files
• The extractall() method for ZipFile objects extracts all the files and
folders from a ZIP file into the current working directory.

>>> import zipfile, os

>>> os.chdir('C:\\') # move to the folder with example.zip

>>> exampleZip = zipfile.ZipFile('example.zip')

>>> exampleZip.extractall()

>>> exampleZip.close()
After running this code, the contents of example.zip will be extracted to C:\.
Optionally, you can pass a folder name to extractall() to have it extract the
files into a folder other than the current working directory.

If the folder passed to the extractall() method does not exist, it will be created.
For instance, if you replaced the call at with
exampleZip.extractall('C:\\delicious'), the code would extract the files from
example.zip into a newly created C:\delicious folder.
The extract() method for ZipFile objects will extract a single file from the
ZIP file. Continue the interactive shell example:
>>> exampleZip.extract('spam.txt')
'C:\\spam.txt'
>>> exampleZip.extract('spam.txt', 'C:\\some\\new\\folders')
'C:\\some\\new\\folders\\spam.txt'
>>> exampleZip.close()

The string you pass to extract() must match one of the strings in the list
returned by namelist(). Optionally, you can pass a second argument to
extract() to extract the file into a folder other than the current working
directory.
 Creating and Adding to ZIP Files
To create your own compressed ZIP files, you must open the ZipFile object
in write mode by passing 'w' as the second argument.

When you pass a path to the write() method of a ZipFile object, Python
will compress the file at that path and add it into the ZIP file.

 The write() method’s first argument is a string of the filename to add. The
second argument is the compression type parameter, which tells the
computer what algorithm it should use to compress the files; you can always
just set this value to zipfile.ZIP_DEFLATED.
• Enter the following into the interactive shell:

>>> import zipfile

>>> newZip = zipfile.ZipFile('new.zip', 'w')

>>> newZip.write('spam.txt', compress_type =zipfile. ZIP_ DEFLATED)

>>> newZip.close()

• This code will create a new ZIP file named new.zip that has the compressed
contents of spam.txt.
 Debugging
• The debugger is a feature of IDLE that executes a program one
instruction at a time, giving you a chance to inspect the values in
variables while your code runs, and track how the values change over
the course of your program.
 Raising Exceptions
• Python raises an exception whenever it tries to execute invalid code.

• To handle Python’s exceptions with try and except statements so that

your program can recover from exceptions that you anticipated. But you
can also raise your own exceptions in your code.

• Raising an exception is a way of saying, “Stop running the code in this

function and move the program execution to the except statement.”
Cont..

• Exceptions are raised with a raise statement. In code, a raise statement

consists of the following:

The raise keyword

A call to the Exception() function

A string with a helpful error message passed to the Exception()

function
 Cont…
• For example, enter the following into the interactive shell:
>>> raise Exception('This is the error message.')

If there are no try and except statements covering the raise statement that
raised the exception, the program simply crashes and displays the
exception’s error message.

For example, open a new file editor window, enter the following code, and
save the program as boxPrint.py:
def boxPrint(symbol, width, height):
if len(symbol) != 1:
raise Exception('Symbol must be a single character string.')
if width <= 2:
raise Exception('Width must be greater than 2.')
if height <= 2:
raise Exception('Height must be greater than 2.')
print(symbol * width)
for i in range(height - 2):
print(symbol + (' ' * (width - 2)) + symbol)
print(symbol * width)
for sym, w, h in (('*', 4, 4), ('O', 20, 5), ('x', 1, 3), ('ZZ', 3, 3)):
try:
boxPrint(sym, w, h)
except Exception as err:
print('An exception happened: ' + str(err))
 Getting the Traceback as a String

When Python encounters an error, it produces a treasure trove of

error information called the traceback.

The traceback includes the error message, the line number of the line
that caused the error, and the sequence of the function calls that led to
the error. This sequence of calls is called the call stack.
 Cont..
Open a new file editor window in IDLE, enter the

following program, and save it as errorExample.py:

def spam():

bacon()

def bacon():

raise Exception('This is the error message.')

spam()

When you run errorExample.py, the output will look

like this:
• The traceback is displayed by Python whenever a raised exception goes
unhandled. But you can also obtain it as a string by calling
traceback.format_exc(). This function is useful if you want the information
from an exception’s traceback but also want an except statement to
gracefully handle the exception.

• For example, instead of crashing your program right when an exception

occurs, you can write the traceback information to a log file and keep your
program running. You can look at the log file later, when you’re r eadyto
debug your program.
• Enter the following into the interactive shell:

The 116 is the return value from the write() method, since 116 characters were
written to the file
 Assertions
An assertion is a sanity check to make sure your code isn’t doing something
obviously wrong. These sanity checks are performed by assert statements. If
the sanity check fails, then an Assertion Error exception is raised. In code, an
assert statement consists of the following:

• The assert keyword

• A condition (that is, an expression that evaluates to True or False)

• A comma

• A string to display when the condition is False

• For example, enter the following into the interactive shell:
 Using an Assertion in a Traffic Light Simulation
• Say you’re building a traffic light simulation program. The data structure
representing the stoplights at an intersection is a dictionary with keys 'ns' and
'ew', for the stoplights facing north-south and east-west, respectively. The
values at these keys will be one of the strings 'green', 'yellow', or 'red'. The code
would look something like this:
• market_2nd = {'ns': 'green', 'ew': 'red'}
• mission_16th = {'ns': 'red', 'ew': 'green'}

These two variables will be for the intersections of Market Street and 2nd Street,

and Mission Street and 16th Street.

• To start the project, you want to write a switchLights() function, which will
take an intersection dictionary as an argument and switch the lights. At first,
you might think that switchLights() should simply switch each light to the next
color in the sequence: Any 'green' values should change to 'yellow', 'yellow'
values should change to 'red', and 'red' values should change to 'green'. The
code to implement this idea might look like this:
 Logging
• Logging is a great way to understand what’s happening in your program and
in what order its happening.

• Python’s logging module makes it easy to create a record of custom

messages that you write.

• These log messages will describe when the program execution has reached
the logging function call and list any variables you have specified at that
point in time. On the other hand, a missing log message indicates a part of
the code was skipped and never executed.
 Using the logging Module
• To enable the logging module to display log messages on your screen as
your program runs, copy the following to the top of your program

import logging

logging.basicConfig(level=logging.DEBUG, format=' %(asctime)s - %

(levelname)s - %(message)s')

• The logging module’s basicConfig() function specify the details about the
LogRecord object.
factorialLog.py
The output of this program looks like this:
• Change the for i in range(n + 1): line to for i in range(1, n + 1):, and run the
program again. The output will look like this:
 Logging Levels

• Logging levels provide a way to categorize your log messages by importance.

There are five logging levels. Messages can be logged at each level using a
different logging function.
Tables: Logging Levels in Python
 Cont..
• The benefit of logging levels is that you can change what priority of logging

message you want to see.

• Passing logging.DEBUG to the basicConfig() function’s level keyword

argument will show messages from all the logging levels (DEBUG being the

lowest level).

• In case, you can set basicConfig()’s level argument to logging.ERROR. This

will show only ERROR and CRITICAL messages and skip the DEBUG, INFO,

and WARNING messages.

 Disabling Logging
• After you’ve debugged your program, you probably don’t want all these
log messages cluttering the screen. The logging.disable() function disables
these so that you don’t have to go into your program and remove all the
logging calls by hand.

• You simply pass logging.disable() a logging level, and itwill suppress all
log messages at that level or lower. So if you want to disable logging
entirely, just add logging.disable(logging.CRITICAL) to your program.
 Cont..
>>> import logging
>>> logging.basicConfig(level=logging.INFO, format=' %(asctime)s -
%(levelname)s - %(message)s')
>>> logging.critical('Critical error! Critical error!')
2015-05-22 11:10:48,054 - CRITICAL - Critical error! Critical error!
>>> logging.disable(logging.CRITICAL)
>>> logging.critical('Critical error! Critical error!')
>>> logging.error('Error! Error!')
 Logging to a File
• Instead of displaying the log messages to the screen, you can write them to a text
file. The logging.basicConfig() function takes a filename keyword argument.

>>import logging

>>logging.basicConfig(filename='myProgramLog.txt', level=logging. DEBUG,

format=‘ %(asctime)s - %(levelname)s - %(message)s')

The log messages will be saved to myProgramLog.txt.

 Buttons in the Debug Control window
There are five buttons in the Debug Control window: Go, Step, Over, Out, or Quit.
Go
Clicking the Go button will cause the program to execute normally until it terminates or reaches a breakpoint.
If you are done debugging and want the program to continue normally, click the Go button.

Step

Clicking the Step button will cause the debugger to execute the next line of code and then pause again. The

Debug Control window’s list of global and local variables will be updated if their values change. If the next line

of code is a function call, the debugger will “step into” that function and jump to the first line of code of that

function.
 Cont..

Over

• Clicking the Over button will execute the next line of code, similar to the Step
button. However, if the next line of code is a function call, the Over button will
“step over” the code in the function.

• The function’s code will be executed at full speed, and the debugger will pause as
soon as the function all returns. For example, if the next line of code is a print()
call, you don’t really care about code inside the built-in print() function;
Out

• Clicking the Out button will cause the debugger to execute lines of code at full speed until

it returns from the current function. If you have stepped into a function call with the Step

button and now simply want to keep executing instructions until you get back out, click the

Out button to “step out” of the current function call.

Quit

• If you want to stop debugging entirely and not bother to continue executing the rest of the

program, click the Quit button. The Quit button will immediately terminate the program. If

you want to run your program normally again, select Debug4Debugger again to disable the

debugger.
Practice Questions
1. What is the difference between shutil.copy() and shutil.copytree()?
2. What function is used to rename files?
3. What is the difference between the delete functions in the send2trash
and shutil modules?
4. ZipFile objects have a close() method just like File objects’ close()
method. What ZipFile method is equivalent to File objects’ open()
method?