Data File Handling

Definition:

The programs that we have seen so far are transient i.e. temporary in nature that run for a short
period of time. But in a business environment, many contents are there that required to be saved
permanently. That’s why this file handling operations are used.

The files are stream of bytes that are persistent i.e. permanent in nature, they run for a long time (or
all time); they keep at least some of their data in permanent storage; and if they shut down or
restart, they execute from the same point.

Data File Operations:

Before, performing any operation on the file, the file needs to open it. After performing the required
operation, it needs to be closed so that resources that are tied with file are freed.

Thus, Python File Handling takes place in the following order:

i. Opening a File.
ii. Performing Operations or Processing Data
iii. Closing the file

Using the above mentioned basic operations, we can process file in several ways, such as:

i. Creating a file
ii. Traversing a file for displaying data on screen
iii. Appending data in a file
iv. Inserting data in a file
v. Deleting data from a file
vi. Creating a copy of a file
vii. Updating data in a file.etc.

File Types:

Python allows us to create and manage two types of files:

i. Text File: A text file consists of a sequence of lines. A line is a sequence of characters, stored
on permanent storage media. In text file, each line is terminated by a special character,
known as EOL(End of line). By default, this EOL character is the newline character(‘\n’). So, at
the lowest level, text file will be a collection of bytes. Text file are store in human readable
form and can be created using any text editor. Text files are used to store the character data.

ii. Binary File: Binary files are used to store binary data such as images, video files audio files
etc. A binary file contains arbitrary binary data, usually numbers stored in the file which can
be used for numerical operation(s).
In binary file, there is no delimiter for a line. Also, no character translations can be carried
out in binary file. As a result, the operations performed on binary files are much faster than
text files.

Opening and Closing Files: For performing any type of operation on data files, first we need to have
a file object or a file variable. Object can be created by using open() function or file() function. Using
this function, a file object is created which is then used for accessing various methods and functions
available for file manipulation.

open() -Opening a File:

open() function takes the name of the file as the first argument. The second argument indicates the
mode of accessing the file.

Syntax for opening a file:

<file variable/object name> =open(file name, access mode)

Modes for opening a file:

i. read(r): to read a file

ii. write(w): to write the content to the file
iii. append(a): To write at the end of the file.

When a file is opened, a buffer(area in memory where data is temporarily stored before being
written to the file) is directly associated with the file when we open it. While writing the content to
the file, first it goes to buffer, and once the buffer is full, data is written to the file. Also, when the file
is closed, any unsaved data is transferred to the file. flush() function is used to force transfer of data
form buffer to file.

example:

fl=open(‘abc.txt’, ‘r’)

print(fl)

Output:

<_io.TextIOWrapper name='abc.txt' mode='r' encoding='cp1252'>

example:

fl=open('abc.txt')

rd=fl.read()

print(rd)

Output:

Chinmaya Vidyalaya, Sector-5, Bokaro

Content in Text Editor:

Note: When you open a file in read mode, the given file must exist in the folder, otherwise Python
will raise FileNotFoundError

File Modes:
 Mode Description
r It opens a text file in reading mode. The file
pointer is placed at the beginning of the file.
This is the default mode, If the specified file
does not exist, it will generate
FileNotFoundError.
rb It opens a file for reading only in binary format.
The file pointer is placed at the beginning of the
file. This is the default mode.
r+ Open a file for both reading and writing. The
file pointer will be placed at the beginning of
the file.
rb+ Opens a file for both reading and writing in
binary format. The file pointer will be at the
beginning of the file.
w Opens a file for writing only. It overwrites the
content, if the file already exists otherwise
creates a new file for writing
wb Opens a file for writing only in binary format. It
overwrites the content, if the file already exists
otherwise creates a new file for writing.
w+ Opens a file for both reading and writing. It
overwrites the content, if the file already exists
otherwise creates a new file for reading and
writing.
wb+ Opens a file for both reading and writing in
binary format. It overwrites the content, if the
file already exists otherwise creates a new file
for writing.
a Opens a file for appending. The file pointer will
be placed at the end of the file, if the file exists.
If the file does not exists, then it creates a new
file for writing.
ab Opens a file for appending in binary format. The
file pointer will be placed at the end of the file,
if the file exists. The file opens in append mode.
If the file does not exist, it creates a new file for
writing.
a+ Opens a file for appending and reading. The file
pointer will be placed at the end of the file, if
the file exists. If the file does not exists, then it
creates a new file for writing and reading.
ab+ Opens a file for appending and reading in
binary format. The file pointer will be placed at
the end of the file, if the file exists. The file
opens in append mode. If the file does not
exist, it creates a new file for writing and
reading.

Note:
 i. Text file uses .txt extension
ii. Whereas binary file uses .dat extension.

close()- Closing a file

The close() method of a file object flushes any unwritten information and closes the file object, after
which no more writing can be done. Python automatically closes a file when the reference object of
a file reassigned to another file. It is good practice to close the file by using the method close().

Syntax:

fileobject.close()

example:

fl=open(‘abc.txt’, ‘r’)

rd=fl.read()

print(rd)

fl.close()

Properties of file Object

When a file get opened and the file object gets created, we can retrieve some of the related details
to that file using its associated properties:

The properties are:

1. name: Name of the opened file.

2. mode: Mode in which the file gets opened.

3. closed: returns Boolean value, which indicates whether the file is closed or not.

4. readable: returns Boolean value, which indicates whether the file is readable or not.

Example:

fl=open('abc.txt', 'r')

print("File Name:", fl.name)

print("File Mode:", fl.mode)

print("Is the file get closed:", fl.closed)

print("Is the file is readble:", fl.readable())

fl.close()

print("Is the file get closed:", fl.closed)

Output:

File Name: abc.txt

File Mode: r
Is the file get closed: False

Is the file is readble: True

Is the file get closed: True

Output:

File Name: abc.txt

File Mode: r

Is the file get closed: False

Is the file is readble: True

Is the file get closed: True

Reading from a file:

Python Provides various methods to read data from a file. Following are the methods to retrieve
data from a file:

i. read(): This method reads the entire content from the file. It starts reading from the
beginning of the file.
example:
fobj=open("sample_text.txt")
ln=fobj.read()
print("A single line from the file:\n", ln)
Output:
A single line from the file:
Name Roll Class
Abhinav 1 12/E
Aditya 2 12/E
ii. read(n): This method reads up to n number of characters specified at the time of calling of
the method.
example:
fobj=open("sample_text.txt")
ln=fobj.read(5)
print("A single line from the file:\n", ln)
Output:
A single line from the file:
Name
iii. readline(): This method reads only one line from the file.
example:
fobj=open("sample_text.txt")
ln=fobj.readline()
print("A single line from the file:\n", ln)
Output:
A single line from the file:
Name Roll Class
iv. readlines(): This method reads all lines from the file into a list. It returns a list of lines.
example:
 fobj=open("sample_text.txt")
ln=fobj.readlines()
print("A single line from the file:\n", ln)
Output:
A single line from the file:
['Name\tRoll\tClass\n', 'Abhinav\t1\t12/E\n', 'Aditya \t2\t12/E']

Writing to a file:
Folowing are the methods to write content to the file:

i. write(String): This method requires a string as parameter and write this string to a file. For
storing data with end of line character, we will have to add ‘\n’ character to the end of the
string.
Example:
fobj=open("samp.txt", "w")
fobj.write("Hi! Good Morning\nWelcome to the world of Python. Python Programming")
print("Data successfully saved to the file")
fobj.close()
Output:
Data successfully saved to the file
Screenshot of the text file containing data:

ii. writelines(Sequence of lines): For writing a string at a time, we use write() method. It can’t
be used for writing a list, tuple etc. into a file. Sequence data type including strings can be
written using writelines() in the file.
So, whenever we have to write a sequence of string/data type, we must use writelines()
instead of write().
writelines() method does not add any EOL(End of line) character to the end of string. We
have to do it ourselves.
Example:
fobj=open("samp.txt", "w")
lst=["Python \tProgramming Language\n", "C++\n", "Java\n"]
fobj.writelines(lst)
print("Data successfully saved to the file")
fobj.close()
Output:
Data successfully saved to the file
Screenshot of the text file containing data:
with Statement:

Apart from using open() and file () function for creation of a file, with statement can also be used for
the same purpose. We can use this statement to group the file operation statements within block.
with statement ensures that all the resources allocated to the file get deallocated automatically once
we stop using the file.

Example:

with open("samp.txt", 'w') as fobj:

fobj.write("Python\n")

fobj.write("\tProgramming\n")

fobj.write("\t\tLanguage\n")

fobj.write("!!Thank You!!")

print("Is the File operation completed\n", fobj.closed)

print("Is the File operation completed\n", fobj.close())

Output:

Is the File operation completed

False

Is the File operation completed

None

Appending to File:

Append means “to add”. If we have a requirement to add the content at the end of the existing file
without erasing the previous content. In such situation, the file needs to be opened in append mode
i.e ‘a’ mode.

In python, we can use the ‘a’ mode to open output file in append mode. this means that:
  If the file already exists, it will not be erased. If the file does not exist, it will be created.
 When data is written to the file, it will be written at the end of the file’s current content

example:

fob=open("sample_text.txt", "a")

nm=input("Enter your Name:")

roll=input("Enter your roll no:")

c_s=input("Enter your class and section(e.g. 12/E)")

fob.write("\n")

fob.write(nm)

fob.write("\t")

fob.write(roll)

fob.write("\t")

fob.write(c_s)

print("Content added to the file")

fob.close()

Output:

Enter your Name:Anubhav

Enter your roll no:4

Enter your class and section(e.g. 12/E)12/E

Content added to the file

Content in the file:

Binary File Operations:

If a user have requirement to write the content as a list or a dictionary to the file, and read it
subsequently, then there is a module named pickle that needs to be imported. While reading the
contents of the file, a reverse process called unpickling is done that converts the byte stream into
user-readable format(original structure).
We know that the methods provided for reading and writing to the file work with string type
parameters. So, when we work with binary file, the data needs to be converted to byte stream at the
time of writing. Pickle module can be used to store any type of data into the file.

So, for performing operations on binary file, first, we need to import the module pickle. It provides
two methods for this purpose- dump and load. For creation of a binary file, we will use
pickle.dump() to write the object in the file. And for reading data from a file, we have to use
pickle.load() to read object from the pickle file.

Example:

 To write the content to the file:

import pickle
lst=[100, 200, 300, 400, 500]
fobj=open('bin_fl.dat', 'wb')
pickle.dump(lst, fobj)
print("Content written successfully to the file")
fobj.close()
Output:
Content written successfully to the file
 To read the content from the file:
import pickle
fobj=open('bin_fl.dat', 'rb')
lst=pickle.load(fobj)
print("Content stored in the file\n", lst)
fobj.close()
Output:
Content stored in the file
[100, 200, 300, 400, 500]

Relative and Absolute Path:

Files are organised into directories/folders. Every running program has a current directory.” which is
by default directory for most operations. Python looks for the current directory.

There is a module defined in Python, named OS module, that performs operations/functions for
working with files and directories. (OS stands for Operating System”).

os.getcwd() is the function that returns the name of the current working directory.

The os module of Python provides various methods to work with file and folder. for using these
functions, we have to import os module in our program.

Relative path: A relative starts from the current directory.

example:

fobj=open(“sample.txt”)

Absolute Path: An absolute path starts from the topmost directory in the file system.

example:

fobj=open(“c:\\Documents\\my files\\sample.txt”)
*chdir(): To change current working directory(CWD) this method is used to. This method changes
the CWD to a specified path. It only takes a single argument as a new directory path.

Example:

import os

fobj=open("c:\\/Users\\Jai Mata Di\\Documents\\my files\\sample.txt")

rd=fobj.read()

print(rd)

fobj.close()

dir=os.getcwd()

print("Current directory:", dir)

os.chdir("c:\\/Users\\Jai Mata Di\\Documents\\my files")

dir=os.getcwd()

print("Current directory:", dir)

fobj=open("SS.txt", "w")

fobj.write("Hello! file in my file folder")

print("Content saved")

fobj.close()

*mkdir(): This method is used to create a directory named path with the specified numeric mode.
This method raise FileExistsError if the directory to be created is already exists.

Example:

import os

dir="my dir"

parent_dir="c:\\Users\\Jai Mata Di\\Documents"

path=os.path.join(parent_dir, dir)

os.mkdir(path)

print("Directory", dir, "is successfully created")

Output:

Directory my dir is successfully created

Screenshot:

Example:

# to create a directory

import os

dir="my directory"

parent_dir="c:\\Users\\Jai Mata Di\\Documents"

path=os.path.join(parent_dir, dir)

os.mkdir(path)

print("Directory", dir, "is successfully created")

cur_dir=os.getcwd()

print("Current directory:", cur_dir)

os.chdir("c:\\Users\\Jai Mata Di\\Documents\\my directory")

dir=os.getcwd()

print("Changed directory:", dir)

fobj=open("SS.txt", "w")

fobj.write("Hello! file in my dir folder")

print("Content saved")

fobj.close()
Output:

Directory my directory is successfully created

Current directory: C:\Users\Jai Mata Di\Documents\my programs

Changed directory: c:\Users\Jai Mata Di\Documents\my directory

Content saved

Screenshot

*makedirs(): This method is used to create a directory recursively. That means while making leaf
directory if any intermediate-level directory is missing, os.makedirs() method will create them all.

Example:

# To create the directories

import os

dir="leaf dir"

parent_dir="C:\\My Documents\\parent dir"

path=os.path.join(parent_dir, dir)

os.makedirs(path)

print("Directory", dir, "is created")

#To read the current working directory

cur_dir=os.getcwd()

print("Current directory:", cur_dir)

#To change the current working directory

os.chdir("c:\\My Documents\\parent dir\\leaf dir")

dir=os.getcwd()

print("Changed directory:", dir)

#To create a file inside the current working directory

fobj=open("SS.txt", "w")

fobj.write("Hello! file in my dir folder")

print("Content saved")

fobj.close()

Output:

Directory leaf dir is created

Current directory: C:\Users\Jai Mata Di\Documents\my programs

Changed directory: c:\My Documents\parent dir\leaf dir

Content saved

Screenshot:

A Sample Program to perform reading and writing both operations in a binary file:
import pickle

fobj=open("emp_data.dat", "wb")

print("File Opened")

while True:

empcode=int(input("Enter Employee code:"))

empnm=input("Enter Employee Name:")

emp=[empcode, empnm]

pickle.dump(emp, fobj)

print("One record inserted")

ch=input("Do you want to insert more records?(y/n)")

if ch=='n' or ch=='N':

break

fobj.close()

print("File closed")

fobj=open("emp_data.dat", "rb")

print("File Opened")

print("Records in file are:")

try:

while True:

emp=pickle.load(fobj)

print(emp)

except EOFError:

pass

fobj.close()

print("File closed")

Output:

File Opened

Enter Employee code:1002

Enter Employee Name:Sumit

One record inserted

Do you want to insert more records?(y/n)y

Enter Employee code:1003

Enter Employee Name:Rohit

One record inserted

Do you want to insert more records?(y/n)1004

Enter Employee code:1004

Enter Employee Name:Raj

One record inserted

Do you want to insert more records?(y/n)n

File closed

File Opened

Records in file are:

[1002, 'Sumit']

[1003, 'Rohit']

[1004, 'Raj']

File closed

Questions:

Q1. Indian library has a wide collection of books for readers. this library provides a soothing
environment for reading. The library wishes to update its record for any new books being purchased
or added to the library. Write a python program to add the record of new books in a file so that the
records are stored for future retrieval and can be accessed whenever required.

Note:
i. Take the required the input as many times as the user wants.
ii. The details of books that are to be taken form the user are:
a. Book Code
b. Book Name
c. Author Name

Q2. Write a user-defined function in Python that displays the number of lines starting with ‘H’ in the
file para.txt.
fobj=open("Paragraph.txt", "r")
cnt=0
try:
 while True:
rd=fobj.readline()
if rd[0]=='H':
cnt+=1
except IndexError:
pass
print(cnt, 'lines are starting with \'H\':')
fobj.close()
Output:
2 lines are starting with 'H':
Q3. Write a user-defined function in Python that displays the number of word starting with ‘I’ or ‘i’ in
the file para.txt.
fobj=open("Paragraph.txt", "r")
cnt=0
try:
while True:
rd=fobj.readline()
rd=rd.split(" ")
for i in rd:
if i[0]=='I' or i[0]=='i':
cnt+=1
except IndexError:
pass
print(cnt, 'words are starting with \'I\':')
fobj.close()
Output:
4 words are starting with 'I':
Q4. Write a user-defined function in Python that displays the number of words starting with ‘I’ or
‘i’ in the file para.txt.
def words():
fobj=open("Paragraph.txt", "r")
cnt=0
try:
while True:
rd=fobj.readline()
print(rd)
rd=rd.split(" ")
print(rd)
for i in rd:
if i[0]=='I' or i[0]=='i':
cnt+=1
except IndexError:
pass
print(cnt, 'words are starting with \'I\':')
fobj.close()
words()
Output:
Whose woods these are I think I know.
['Whose', 'woods', 'these', 'are', 'I', 'think', 'I', 'know.\n']
His House is in the vilage though;

['His', 'House', 'is', 'in', 'the', 'vilage', 'though;\n']

He will not see me stopping here.

['He', 'will', 'not', 'see', 'me', 'stopping', 'here.\n']

To watch his woods fill up with snow.

['To', 'watch', 'his', 'woods', 'fill', 'up', 'with', 'snow.\n']

['']
4 words are starting with 'I':
OR
def words():
fobj=open("Paragraph.txt", "r")
cnt=0
try:
rd=fobj.read()
print(rd)
rd=rd.split(" ")
print(rd)
for i in rd:
if i[0]=='I' or i[0]=='i':
cnt+=1
except IndexError:
pass
print(cnt, 'words are starting with \'I\':')
fobj.close()
words()
Output:
Whose woods these are I think I know.
His House is in the vilage though;
He will not see me stopping here.
To watch his woods fill up with snow.

['Whose', 'woods', 'these', 'are', 'I', 'think', 'I', 'know.\nHis', 'House', 'is', 'in', 'the', 'vilage', 'though;\
nHe', 'will', 'not', 'see', 'me', 'stopping', 'here.\nTo', 'watch', 'his', 'woods', 'fill', 'up', 'with', 'snow.\n']
4 words are starting with 'I':

Standard File Streams:

Standard file streams are the streams used to work with standard input/output devices. There is an
object that performs standard I/O stream operations. Since, we use high-level functions for
performing input/output through keyboard and monitor such as eval(), input() and print() statement,
we are not required to explicitly use I/O object.
The standard file streams are:
1. Standard Input Stream
 2. Standard Output Stream
3. Standard Error Stream
These standard streams are automatically get connected to the programs standard device such
keyboard and monitor. In order to work with standard file stream, we need to import a module
named sys. The methods that are available for I/O operations in it are read() for reading a byte
at a time from keyboard, write() for writing data on console i.e. monitor.
The three standard streams are described below:
1. sys.stdin: When a stream reads from standard input
2. sys.stdout: Data written to sysy.stdout typically appears on the screen, but can be linked
to the standard input of another program with pipe.
3. sys.stderr: error messages are written to sys.stderr
Program to copy the content of one file to another file:
#COPY CONTENT OF ONE FILE TO ANOTHER
import os
def filecopy(fl1, fl2):
f1=open(fl1, "r")
f2=open(fl2, "w")
line=f1.readline()
while line != '':
f2.write(line)
line=f1.readline()
print("One line written successfully:\n")
f1.close()
f2.close()
def main():
flnm=input("enter a file name:")
clnm=input("enter a file name, where you want to copy the content:")
filecopy(flnm, clnm)

main()

Output:
enter a file name:example.txt
enter a file name, where you want to copy the content:copy.txt
One line written successfully:

One line written successfully:

One line written successfully:

Screenshot of the file:
Binary File Operations:
Basic Operations that can be performed using binary file are:
1. Inserting/appending a record in a binary file
2. Reading/retrieving records from a binary file
3. Searching a record from a binary file
4. Updating a record in a binary file
5. Removing a record from a binary file.
Program:
import pickle
import sys
import os
#Inserting/Appending Records in a binary file
def insert():
fobj=open("Stud_data.dat", "ab")
while True:
roll=int(input("Enter the Roll No.:->"))
nm=input("Enter the Name:->")
cls_sec=input("Enter the Class & Section:->")
marks=float(input("Enter the Marks:->"))
val=[roll, nm, cls_sec, marks]
pickle.dump(val, fobj)
print("One Record inserted successfully")
choice=input("Do you want add more records?(Y/N)")
if choice=='N' or choice=='n':
break
fobj.close()
print("File Closed")

#Reading records from a binary file

def retrieve():
fobj=open("Stud_data.dat", "rb")
print("Students records are:")
try:
while True:
record=pickle.load(fobj)
print(record)
except :
pass
fobj.close()

def search(rno):
fob=open("Stud_data.dat", "rb")
try:
while True:
rec=pickle.load(fob)
if rec[0]==rno:
print("Record Found!")
print(rec)
break
except:
pass
fob.close()

def update(rno):
fob=open("Stud_data.dat", "rb+")
try:
while True:
cur=fob.tell()
rec=pickle.load(fob)
if rec[0]==rno:
rec[1]=input("Enter new name:")
rec[2]=input("Enter new class and section:")
rec[3]=float(input("Enter new marks:"))
fob.seek(cur, 0)
pickle.dump(rec, fob)
print("Record updated successfully")
break
except:
pass
fob.close()

def delete(rno):
fob1=open("Stud_data.dat", "rb")
fob2=open("Temp.dat", "wb")
try:
while True:
rec=pickle.load(fob1)
 if rec[0]==rno:
print("Record deleted successfully")
continue
else:
pickle.dump(rec, fob2)
except:
pass
fob1.close()
fob2.close()
os.remove("Stud_data.dat")
os.rename("Temp.dat", "Stud_data.dat")

#main Menu
while True:
print("Press 1 to insert\nPress 2 to read")
print("Press 3 to search\nPress 4 to Update")
print("Press 5 to delete")
ch=int(input("Enter your choice:->"))
if ch==1:
insert()
elif ch==2:
retrieve()
elif ch==3:
rno=int(input("Enter a roll no that is to be search:"))
search(rno)
elif ch==4:
rno=int(input("Enter a roll no that is to be updated:"))
update(rno)
elif ch==5:
rno=int(input("Enter a roll no that is to be deleted:"))
delete(rno)
else:
break
print("Program closed")
Output:
Press 1 to insert
Press 2 to read
Enter your choice:->2
Students records are:
Roll Name Class & Sec Marks
[1, 'Abhinav', '12/E', 345.0]
Press 1 to insert
Press 2 to read
Enter your choice:->

Random Access in Files using tell() and seek():

Introduction to CSV: