DATA FILE HANDLING

Know Python Bytes

www.knowpythonbytes.blogspot.com

Mrs. Payal Bhattacharjee, PGT(C.Sc.)

K V No.1 Kanchrapara
KVS-RO(Kolkata)
 UNIT 1
For XII CSc:
DATA FILE HANDLING
IN PYTHON
Unit I: Computational Thinking and Progamming-2
DATA FILE HANDLING
❖INTRODUCTION
TO FILES

❖TYPES OF FILES

❖RELATIVE AND
ABSOLUTE PATHS
 INTRODUCTION TO FILES
• Files are used to store data in a storage device permanently.
• File handling provides a mechanism to store the output of a program in
a file and to perform various operations on it.
• The data stored in Secondary Memory storage in a file is given a file name using
which it is accessed.
• The data stored within the file can be read, modified and deleted through
program using different File I/O functions or methods provided by the language.

NEED FOR FILE:

• For future reference, it is essential to keep the huge amount of data in a file on
any storage medium permanently. Especially for research, data analysis and
business applications.
• Till now, in programs, we were accessing and using the variables by taking input
and displaying output. And these variables get lost once program execution
stops. But, the question arises that , if we need to store the input and output
data then where can we store them permanently so that, we can get them later
for future use.
• For example, If you want to store the marks of all the subjects of all the students
of a class. OR Storing the admission data of students in a school.
 TYPES OF DATA FILES
CONTENTS TEXT FILES:
Text files are structured as a sequence of
➢ INTRODUCTION lines, where each line includes a sequence of
▪ Introduction to Files characters (i.e. stored based on ASCII value).
▪Types of Files It can be edited in any text editor program.
Eg.Notepad
▪ Relative and Absolute
Path BINARY FILES:
➢ TEXT FILE A binary file is a file stored in binary format
➢ BINARY FILE (i.e. combination of one’s and zero's). Binary
➢ CSV FILE files typically contain bytes that are intended
to be interpreted as something other than
text characters. Binary files are a computer-
readable form of storing data.
CSV FILES:
A comma-separated values (CSV) file is a delimited text file that uses a
comma to separate values. Each line of the file is a data record. Each record
consists of one or more fields, separated by commas. The use of the
comma as a field separator is the source of the name for this file format.
 CONCEPT OF PATH
Full name of a File or Directory or Folder
consists of path\primaryname.extension Root Directory
C:\
ABSOLUTE PATH for Subject
accessing the file Circle.txt
C:\Subject\Maths\
Geometry\Circle.txt Maths CS English

Geometry Python Grammar Literature

DFH Poem

Let, DFH is our current Circle.txt

working directory.

RELATIVE PATH for accessing the file Circle.txt

from the current working directory(DFH) Kite.txt
..\Maths\Geometry\Circle.txt List.py
 RELATIVE AND ABSOLUTE PATH OF A FILE
CONTENTS ABSOLUTE PATH RELATIVE PATH
➢INTRODUCTION A path related to the root A relative file path is
▪ Introduction to Files directory is called Absolute Path. interpreted from the
▪ Types of Files an absolute path is a complete perspective of your current
▪Relative and path from start of actual file working directory.
system from / (root)directory.
Absolute Path
An absolute file path describes The relative path is
➢TEXT FILE how to access a given file or the path to some file with
➢BINARY FILE directory, starting from the root respect to your current
➢CSV FILE of the file system. working directory (PWD).
The absolute path is the A relative path points to a
full path to some place on your location that is relative to a
computer. Absolute file paths current directory. Relative
start with a leading forward paths make use of two special
slash and describe how to symbols, a dot (.) and
access a given file or directory, a double-dot (..), which
starting from the root of the file translate into the current
system. directory and the parent
directory.
 LET’S UNDERSTAND PICTORIALLY
Eg. Prog1.py
HARDDISK
Write in the File
on the Disk

READ from the

File on the Disk

Python file being

executed(run) in HARDDISK containing
the memory multiple files of different
types eg. Text file, Binary File,
(RAM)
CSV File etc.
 TEXT FILE
CONTENTS Text files that contain plain-text data are also known as
ASCII (American Standard Code for Information
➢ INTRODUCTION Interchange) formatted files, a popular format based on the
➢TEXT FILE English alphabet. There are many advantages to saving data
▪ File Modes in ASCII format, such as allowing the data to be easily read
▪Opening a Text File and compatibility between other applications and operating
▪Closing a File systems.
▪Writing/
Below is the most common file extensions used
appending data to
a text file with text files and documents.
▪Reading from a .doc and .docx - Microsoft Word file.
text file .odt - OpenOffice Writer document file.
▪seek and tell .pdf - PDF file.
methods
.rtf - Rich Text Format.
▪manipulation of
data in a text file .tex - A LaTeX document file.
➢ BINARY FILE .txt - Plain text file.
➢ CSV FILE .wpd - WordPerfect document.
 TEXT FILE
CONTENTS TEXT DESCRIPTION
FILE
➢ INTRODUCTION
MODES
➢TEXT FILE
Opens a file for reading only. The file
▪ File Modes
▪Opening a Text File
▪Closing a File
r pointer is placed at the beginning of the
file. This is the default mode.
▪Writing/ Opens a file for writing only. Overwrites
appending data to
a text file
▪Reading from a
w the file if the file exists. If the file does
not exist, creates a new file for writing.
text file Opens a file for appending. The file
▪seek and tell
methods
▪manipulation of
a pointer is at the end of the file if the file
exists. That is, the file is in the append
data in a text file
mode (write from end). If the file does
➢ BINARY FILE not exist, it creates a new file for
➢ CSV FILE writing.
 TEXT FILE
CONTENTS TEXT FILE
MODES
DESCRIPTION

➢ INTRODUCTION
Opens a file for both reading and
➢TEXT FILE
▪ File Modes
▪Opening a Text File
r+ writing. The file pointer placed at the
beginning of the file.

▪Closing a File Opens a file for both writing and

▪Writing/
appending data to
a text file
w+ reading. Overwrites the existing file if
the file exists. If the file does not exist,
creates a new file for writing and
▪Reading from a reading.
text file
▪seek and tell Opens a file for both appending and
methods
▪manipulation of
data in a text file
a+ reading. The file pointer is at the end of
the file if the file exists. The file opens in
the append mode. If the file does not
➢ BINARY FILE exist, it creates a new file for writing
➢ CSV FILE and reading.
STEPS TO WORK WITH A TEXT FILE
• OPEN a file
• (READ from/WRITE in/APPEND in/MANIPULATE in )the file
• CLOSE a file
Eg. Prog1.py
HARDDISK
Write in the File
on the Disk

READ from the

File on the Disk
STEP-1 TO WORK WITH A TEXT FILE
using Python Program
•OPEN a file (Here, text file) using the required
mode in the program
To open a file in python, use open( ) function.
SYNTAX:→ file_objectname= open(filename, mode)

Ex 1:→ fin = open(“Forest.txt")

Ex 2:→ fout = open(“My_Notes.txt", "w")

TIPS: Here, in the above examples, fin and fout are the file objects. The naming of a
file_object (/File_handler) follows the rule of an identifier. The File_object points to
the text file which is being opened using open() function.
 STEP-2 TO WORK WITH A TEXT FILE
using Python Program
• (READ from /WRITE in/APPEND in/ MANIPULATE in/
SEARCH in ) the file
One can work with a text file present on the HardDisk.
The Python program executes(runs) in the memory and after
opening the file, the Python program can do various kind of
operations on a text file viz.
• READ from the Text File and display output on the
output screen using print() function in Python program.

• WRITE or APPEND any text on the Text File/ Or take

input from the user through input() function and then
write on the Text File on the Disk.

• MANIPULATE or SEARCH any text on the Text File.

STEP-3 TO WORK WITH A TEXT FILE using Python Program
• CLOSE a file (Here, text file) using the required mode in the program
To close a file in python, use close( ) function.
SYNTAX:→ file_objectname.close( )
Ex 1:→ fin.close( )
Ex 2:→ fout.close( )
TIPS: Here, in the above examples, fin and fout are the file objects. So, after the
manipulation on the File, it is a good programming practice to close the file. We can
skip writing the method close() by using with open statements also. With open
statements automatically close the file after the nested block of code. **We will
learn in the following slides about with open statements.
REASONS for closing the open Files are:
i) In absence of close(), sometimes the data written onto files using write functions is held in
memory untill the next write or the file is not closed.
ii) Operating System restrictions on no. of open files at a time.
iii) Some OS treats open files as locked & private.
iv) Unnecessary blockage of memory by the files which are open, but, no longer in use and
not closed.
 CREATION OF TEXT FILE from Python IDLE
1. Open an untitled File as you open a file for a script mode File
2. Write some text on the File and SAVE
3. Give a File_Name and SAVE AS type .txt instead of .py
4. Choose the folder in which you want to Save or you may SAVE in the
same folder where you save your *.py files inside the Python Folder
Saving a File Friend.txt inside the Python310 Folder
The contents of Friend.txt file for our
reference for the example programs
FUNCTION / SYNTAX Description
METHOD
read( ) <Filehandle>.read(<n>) Reads at the most n bytes;
If no n is specified, reads the entire file.
Returns the read bytes in the form of a string.
readline( ) <Filehandle>.readline(<n>) Reads a single line of input;
If n is specified, it reads at the most n bytes.
Returns the read bytes in the form of a string
ending with ‘\n’ (new line character) or
returns a blank string if no more bytes are left
for reading in the file.
readlines( ) <Filehandle>.readlines() Reads all lines and returns them in the form
of a List. Every element of the list is the line
of the file. For example, the first line of the
text file is at List[0] position, 2nd Line at List[1]
and so on. Every element of the List is a string
ending with a ‘\n’ new line character.
READING the contents of Friend.txt file
using read() method

# The output of read() function from a file is of type STRING

READING the contents of Friend.txt file
using read() method contd..

# The output of read() function from a file is of type STRING

PROGRAM CODE

Here, the no. of characters from

the beginning is 10.

OUTPUT
READING the contents of Friend.txt file
using readline() method

# The output of read() function from a file is of type STRING

PROGRAM CODE

OUTPUT

Here, readline() method reads one line at a time.

READING the contents of Friend.txt file
using readline() method contd..
Reading 1st line
Reading 2nd line and storing it in variable/object var.
Reading 3rd line

Reading 4th line

Printing var which is holding the 2nd line

PROGRAM CODE

OUTPUT

Here, readline() method reads one line at a time.

READING the contents of Friend.txt file
using readline(<n>) method contd..

Reading 1st line’s 19 characters only.

Here, the value of n=19

PROGRAM CODE

OUTPUT
 READING the contents of Friend.txt file
using readlines() method

PROGRAM CODE

OUTPUT

Here, readlines() method reads the full file in the form of a LIST.
Every element of the list is the line of the file. For example, the first line of the text file is at List[0] position,
2nd Line at List[1] and so on. Every element of the List is a string ending with a ‘\n’ new line character.
 STANDARD FILE STREAMS:
INPUT, OUTPUT AND ERROR STREAMS
The interpreter provides three standard file objects, known as standard input, standard
output, and standard error, which are available in the sys module as sys. stdin,
sys. stdout, and sys. stderr, respectively.
Standard streams in Python are File Objects, which get automatically connected to your program’s
standard device(s) when we start Python. The standard streams available in Python are:
❑ Standard input stream
❑ Standard output stream
❑ Standard error stream
In order to work with I/O stream, we need to import sys module. The methods which are available for
I/O operations in it are:
➢ read() for reading a byte at a time from keyboard
➢ write() for writing data on console, i.e.,monitor
The three standard streams are described as follows:
i. sys.stdin: When a stream reads from standard input.
ii. sys.stdout: Data written to sys.stdout appears on the screen, but can be linked to the standard
input of another program too.
iii. sys.stderr: Error messages are written to sys.stderr. It is similar to sys.stdout as it also prints
directly to the console but with the difference that it also prints exceptions,error messages
along with debugging comments.
WORKING WITH BINARY FILES
 BINARY FILE
• Binary Files stores data in the form of BYTES
• Binary Files are in Binary form (0’s and 1’s)
• There is no EOL(End of Line) character in Binary Files
• Example of Binary Files are- image files, Audio files, Video files
• Binary Files don’t require any language translator as they are in Binary Level (0’s and 1’s)
which the computer understands. Hence, any kind of computation and working on
Binary Files is faster.

A Dat file is a data file that contains specific information about the program
used to create it. This file always has the .dat file extension, which is a generic
format that can contain any information – video, audio, PDF, and virtually any
other type of file.
Most of the files that we see in our computer system are called Binary Files.
Examples:
Document Files: .pdf,.doc,.xls etc.
Image Files: .png,.jpg,.bmp etc.
Video files: .mp4,.3gp,.mkv,.avi etc.
Audio Files: .mp3,.wav,.mka,.aac etc.
Database Files: .mdb,.frm,sqlite etc.
Archive Files: .zip,.rar,.iso etc.
Executable Files: .exe,.dll,.class etc.
 BINARY FILE MODES
CONTENTS BINARY FILE DESCRIPTION
MODES
➢ INTRODUCTIO
Opens a file for reading only. The file pointer is placed at the
N
➢ TEXT FILE
rb beginning of the file. This is the default mode.

▪ File Opens a file for writing only. Overwrites the file if the file exists. If
Modes
wb the file does not exist, creates a new file for writing.
▪Opening a Opens a file for appending. The file pointer is at the end of the file
Text File
ab if the file exists. That is, the file is in the append mode (write from
▪Closing a File end). If the file does not exist, it creates a new file for writing.
▪Writing/ Opens a file for both reading and writing. The file pointer placed at
appending rb+ or the beginning of the file.
data to a text
file r+b
▪Reading from
Opens a file for both writing and reading. Overwrites the existing
a text file
▪seek and tell
wb+ or file if the file exists. If the file does not exist, creates a new file for
writing and reading.
methods
▪manipulation
w+b
of data in a Opens a file for both appending and reading. The file pointer is at
text file ab+ or the end of the file if the file exists. The file opens in the append
➢ BINARY FILE mode. If the file does not exist, it creates a new file for writing and
➢ CSV FILE a+b reading.
READING and WRITING DATA from/to a BINARY FILE
➢ Suppose, you want to write a Python List or Dictionary to a Binary File and then
intend to read it. For that, you need to use the Python module Pickle.
➢ Pickle module is used for Serialization(Pickling) and Deserialization(Unpickling)

SERIALIZATION or PICKLING: DESERIALIZATION or UNPICKLING:

It is the process of transforming data or an object It is the process of transforming the
in memory(RAM) to a stream of bytes called byte byte stream to the original
streams. These byte streams can then be stored on structure(Python object)
a disk or in a database or sent through a network.
Pickiling, is the process of converting the structure Unpickling refers to the conversion of
(Python object like List,Dictionary etc.) to a byte byte stream to the original structure
stream before writing to the file. (Python objects ).
LIST LIST
DICTIONARY Byte Byte DICTIONARY
Other Python structures Stream Other Python
Stream
structures
Write on the Read from the
Pickle.dump( ) Binary File Binary File Pickle.load( )
 Pickle.dump( ) Pickle.load( )
SYNTAX: SYNTAX:
Pickle.dump(PythonObject,FileObject) PythonObject=Pickle.load(FileObject)

NOTE: Here, PythonObject is the List,Dictionary or any Python structure.

FileObject is the object through which you have opened your file in a specific File mode.

STEPS TO WORK WITH BINARY FILE

1. Import pickle
2. Open Binary File in the required file mode
3. Process binary file by writing/reading objects
4. Close the file at the last.
WRITING ON A BINARY FILE

Here, PythonObject are the Dictionaries -

stu1,stu2,stu3,stu4.
fout is the File object.
Mode is “wb” that means writing on a binary file.
Pickle module is imported for working with Binary Files.
Open the “Stu.dat” File which has been created in the
same directory.
WRITING ON A BINARY FILE contd..

Binary File when opened in an editor

shows some garbled values which is
not in the human readable form.
APPENDING ON A BINARY FILE
 READING FROM A BINARY FILE

NOTE:
Here, look at the Exception given as EOFError that
means once the pointer started reading the file and
encountered the End of the File. It didn’t stop
automatically rather showed the EOFError
 Before READ Operation… let us recap the
concept of exceptions (try and except blocks)
Even if a statement or expression is syntactically correct, it may cause
an error when an attempt is made to execute it.
Errors detected during execution are called exceptions and are not
unconditionally fatal.
Most exceptions are not handled by programs, and result in error
messages.  As you have noticed in the previous program.

SOLUTION to EXCEPTIONS (EXCEPTION HANDLING):

When an error occurs, or exception , program normally stops and
generate an error message. These exceptions can be handled with the
try statements.
Whenever the try block raises an error, the except block gets executed.
In absence of try block, program crashes and raises an error.
The try block lets you test a block of code for errors.
READING FROM A BINARY FILE with try and except blocks

NOTE:
Here, look at the try block to execute the code and except block to handle the exception.
SEARCHING FROM A BINARY FILE
UPDATING A BINARY FILE
It is a 3 step process:
(i) Locate the record to be updated by
searching for it
(ii) Make changes in the loaded record in
memory( the read record)
(iii)Write back onto the file at the exact
location of old record.

For the third step you need to understand File-

pointer and its functions seek() and tell().
 tell() function of the File-pointer
tell() function returns the current position of the file pointer
in the file.
SYNTAX:
<fileobject>.tell()

Recall the FRIEND.TXT File done earlier.

tell() function of the File-pointer contd.

Recall the FRIEND.TXT File done earlier.

 seek() function of the File-pointer
seek() function changes the position of the file-pointer by placing the file-pointer
at the specified position in the open file.
SYNTAX:
<fileobject>.seek(offset[,mode])
where,
offset is a number specifying number-of-bytes

mode is a number 0 or 1 or 2 signifying

0 for beginning of file ( to move file-pointer with respect to
beginning of file). It is the default position(i.e., when no
mode is specified)
1 for current position of the file-pointer w.r.t current
position of it)
2 for end of file (to move file-pointer w.r.t end of file)

NOTE: seek() function changes the File pointer’s position to a

new file position=start+offset
with respect to the start position as specified by the mode specified.
NOTE: You can move the file in FORWARD direction (by giving positive value for bytes)
as well as the BACKWARD direction (by giving negative value for bytes)
 NOTE:
Seek() function with
negative offset only works
when file is opened in
binary mode

NOTE:
Backward movement of file-pointer is not
possible from the beginning of the file (BOF).
Forward movement of file-pointer is not
possible from the end of file (EOF).
NOTE:
EOL character used in Windows OS is
Carriage Return \r and Line Feed \n \r is treated as one character and \n is treated as one character.
 USING WITH STATEMENT
The with statement is a compact statement
which combines the opening of file and
processing of file along with inbuilt exception
handling. The with statement automatically
closes the file too after the execution of the
with block.
SYNTAX:
with open(<filename>,<mode>) as <filehandle>:
# use pickle.load() here inside this with block
# you can perform other file handling operations
UPDATING BINARY FILE RECORD
WORKING WITH CSV FILES
 What are CSV files (.CSV files)?
CSV FILES: A comma-separated values (CSV) file is a delimited text file that uses a
comma to separate values. The CSV format refers to a tabular data that has been saved as
plain text where data is separated by commas. The separator character of CSV Files is
known as delimiter. Default and most popular delimiter is comma ,
Other popular delimiters are- tab (\t) , colon (:), pipe (|) and semi-colon (;)

ADMNO NAME STREAM MARKS Tabular Data

ADMNO,NAME,STREAM,MARKS
4132 Sindhu Science 99.8 when
converted to 4132,Sindhu,Science,99.8
5680 Mary Commerc 96.5 5680,Mary,Commerce,96.5
e CSV

Data in Table CSV Data

NOTE:
❑ Each row of the table is stored in one row i.e. the number of rows in a CSV file are equal to number
of rows in the table( or database table or excel sheet)
❑ The field values of a row are stored together with commas after every field value; but after the last
field’s value in a line/row, no comma is given.
ADVANTAGES in working with CSV file formats:
• Simple and compact format for data storage and Common format for data interchange
• Cabable of storing huge amount of data
• Nearly all spreadsheets and databases support import/export to csv format.
• It can be opened in popular spreadsheet packages like MS-Excel,Calc etc.
 WORKING WITH A CSV FILE

Eg. Prog1.py

csv.writer object
converts the user data into
csv writable form (i.e. in
delimited string form)

ADMNO,NAME,STREAM,MARKS
4132,Sindhu,Science,99.8
csv.reader() returns a reader 5680,Mary,Commerce,96.5

object which loads data from CSV file

into an iterable(List/tuple etc.) after
converting the delimited data.
 READING and WRITING from CSV File
Writing and reading operation from text file is very easy.
1) import csv module for file operation/method call.
2) For writing, we open file in ‘w’ writing mode using open () method which create newFile like object.
3) Through csv.writer() method ,we create writer object to call writerow() method to write objects.
4) For reading, we open the file in ‘r’ mode and create newFile like object,further we create
newfilereader object using csv.reader() method to read each row of the file.
5) Three file opening modes are there ‘w’,’r’,’a’ for write,read,append.
6) After processing, use close () method to close the file

import csv module

The CSV module of Python provides functionality to read/write tabular data in CSV format. This
module provides two types of objects-

PURPOSE Functions Description

csv.writer() Returns a writer object which writes data into CSV

WRITE on File
to a CSV <writerobject>.writerow() Write one row of data onto the writer object
File
<writerobject>.writerows() Writes multiple rows of data onto the writer object

READ from csv.reader() Returns a reader object which loads data from CSV
a CSV File file into an iterable after parsing delimited data
 SOME IMPORTANT NOTES:
Q) What is the use of writerow() function in Q) What is the use of writerows()
csv? function in CSV?
Ans) writer() function is used to create a writer.writerows() is used to write
writer object. multiple rows from a buffer that
The writer. writerow() function is then used contains one or more lists at once.
to write single rows to the CSV file.
IMPORTANT NOTE:
csv.reader() and csv.writer() both work with List/Tuple.
** EXTRA KNOWLEDGE
csv.DictReader() and csv.DictWriter() methods work with dictionary.
csv.DictReader and csv.DictWriter take additional argument fieldnames that are used as dictionary keys.
** EXTRA KNOWLEDGE
Reading CSV File Into A Dictionary: To read a CSV file into a dictionary can be done by using DictReader()
class of csv module which works similar to the reader( ) class but creates an object which maps data to
a dictionary.

**HINT: DictReader works by reading the first line of the CSV and using each comma separated value in
this line as a dictionary key. The columns in each subsequent row then behave like dictionary values and
can be accessed with the appropriate key (i.e. fieldname).
 Program to READ and WRITE onto a CSV FILE

nfWriter and nfReader are the writer and reader objects.

Program to WRITE using wrtierows() and then READ
The fieldnames
parameter is a
sequence of keys
that identify the
order in which
values in the
dictionary passed to
the writerow()
method are written
to file

writeheader() method takes the first entry

as header in the File

OUTPUT

CSV File content

 Bibliograhy and References

• Google.com
• Quora.com
• StackOverflow.com
• Geekforgeeks.com
• Sarthaks.com
• Wikipedia
• XII CS , By Sumita Arora, Dhanpat Rai Publication 2020 Edition
• XII IP , By Sumita Arora, Dhanpat Rai Publication 2020 Edition
Stay safe. Stay aware. Stay healthy. Stay alert.