ED5340 - Data Science: Theory apa th y

and Practise an
h ug
u t
M
a n
L12 - File Input and Output n a th
a
Ram

Ramanathan Muthuganapathy (https://ed.iitm.ac.in/~raman)

Course web page: https://ed.iitm.ac.in/~raman/datascience.html
Moodle page: Available at https://courses.iitm.ac.in/
File I/O
Sequence of operations

• Open a file h y
a t
ap
• Read / Write data to it an
h ug
u t
• Close the file a n
M
th
n a
a
Ram

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

File modes
L12_File1.py

• f = open(‘filename’, ‘mode’) e.g. f = open(‘messages.txt’,

y
‘w’)
th
a
• ‘w’ - opens the file for writing in text mode nap
a g
h u
• ‘r’ - opens the file for reading in text mode u t
M
a n
• t h
‘a’ - opens the file for appendingnain text mode
a
a m
• R in binary mode
‘wb’ - opens the file for writing

• ‘rb’ - opens the file for reading in binary mode

• ‘ab’ - opens the file for appending in binary mode
• Check for other available modes
Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras
Strings
L12_File1.py

• Note that both reading and writing are in the form

h y of strings
a t
ap
msg1 = 'This is message 1 \n’ an
h ug
u t
f = open('messages.txt', 'w') #Opens an file
M for writing
h a
a t
a n
f.write(msg1)
Ram
f.close()

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

Strings
L12_File1.py

• Note that both reading and writing are in the form

h y of strings
a t
ap
f = open('messages.txt', 'r') #Opens a file for ga nreading
h u
u t
data = f.read() #Reads ALL lines into data n
M
h a
a t
a n
print(data) m
R a
f.close()

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

‘with’ open
L12_File1.py

• Open but automatically closes the file. h y

a t
ap
with open('messages.txt', 'r') as f: #with closes ga n the file automatically
h u
u t
data = f.read() n
M
h a
a t
a n
a m
R

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

f.seek - moving within a file

• f.seek(offset, reference) h y
a t
ap
• ‘reference’ can take 0 (beginning of a file),ga1n (current position) and 2 (end of
file) h u
u t
M
a n
• f.seek(12, 0) - moves to 12th position
n a t h from bof.
a
a m
• R
f.seek(-15, 2) - moves 15 positions to the left from eof.

• f.seek(0, 2)

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

File and Directory Operations
L12_FileDirs.py

• File operations - Creation, deletion, renaming etc.

h y
a t
ap
• Directory operations - creation (recursive),gadeletion, n listing etc.
h u
u t
• Path operations - absolute and relative
a n
M paths, splitting / joining etc..
t h
n a
• a m
a
. - current dir, .. - parent of current directory.
R

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

 h y
a t
p

HW: Use file input / output for n a

a
h ug
u t

complex and matrix classes.

M
a n
th
n a
a
Ram

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

Comma separated values (csv)
L12_CSV.py

• Format is the most common import and exporthyformat for spreadsheets and
databases a t
ap
an
name1,name2, name3 h ug
u t
ram1,ram2, ram3 M
a n
th
cam1,cam2, cam3 n a
a
Ram

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

Reading / Writing other datatypes

• Currently, all are strings! h y

a t
ap
• Tuple, dictionaries etc an
h ug
u t
• Using JSON module a n
M
th
n a
• a
JSON - JavaScript ObjectmNotation
a
R

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

JSON Format

• Web development, configuration / settings

h y
a t
• JSON format ap
n a
• sequence of key-value pairs surrounded by curly u g
brackets
th
u
M
• Each key is mapped to a particular value using a n this format.
t h
n a
• Key-value pairs are separated by a m a
comma. Only the last pair is not followed by a comma
R a
• Keys must be strings.
• Values can be either a string, a number, an array, a boolean value or a JSON object
{
"name": "Raman",
"languages": ["C", "C++"]
}

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

Some rules for JSON

• Always choose meaningful names. y

th
• Array types should have plural key names. All other key
p anames should be singular. For
a
an
example: use "orders" instead of "order" if thegcorresponding value is an array.
h u
• There should be no comments in JSON objects. u t
M
a n
t h
n a
a
a m
R

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

Relation to python

• JSON and Dictionaries might look very similar hy

a t
a p
• JSON is a file format used to represent and ga n store data.
h u
u t
• Python Dictionary is a data structuren (object)
a
M that is kept in memory.
t h
n a
• a m
a
We can’t read the JSON files directly (as entire file is a single string, and
R
individual key-value pairs cannot be accessed individually).

• A dictionary can be created using Key-value pair

• NOTE: JSON is the string rep and dictionaries are data structures in
python
Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras
JSON module

• Python string in JSON format h y

a t
ap
• per = '{"name": "Ram", "languages": g["Python", an “C++"]}'
h u
u t
• To convert this to a python dictionaryn M
a
t h
n a
• per_dct = json.loads(per) a
#per_dct
a m is a dictionary
R
• json.loads(per) — JSON string to a dictionary

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

Reverse - Python object to JSON format
e.g. dictionary to JSON format - L13_json1.py and L13_json2.py

• dct = {'name': 'Ram', 'languages': ['Python', 'C++']}

h y
a t
ap
• str1 = json.dumps(dct) #This function returns ga n a string
h u
u t
• print(str1) a n
M
t h
n a
a
a m
R

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras

Python object to JSON equivalent

Python JSON Equivalent

dict object
list, tuple array
str string
int, float, int number
TRUE true
FALSE false
None null
For User-defined datatypes
L12_Complex_JSON.py

• Complex object into JSON, encode_complex( )hyin json.dump( )

a t
ap
• For load( ), decode_complex( ) through the ga nobject_hook parameter.
h u
u t
M
a n
t h
n a
a
R am

Ramanathan Muthuganapathy, Department of Engineering Design, IIT Madras