Introduction

and lists
DATA T YP E S F O R DATA S C I E N C E

I NP Y T H O N

Jason Myers
Instructor
Data
types

Data type system sets the stage for the capabilities of the
language

Understanding data types empowers you as a data scientist

DATA TYPES FOR DATA SCIENCE IN

PYTHON
Hold other types of data Container
Used for aggregation, sorting, and more sequences
Can be mutable (list, set) or immutable (tuple)

Iterable

DATA TYPES FOR DATA SCIENCE IN

Lists

Hold data in order it was added

Mutable

Index

DATA TYPES FOR DATA SCIENCE IN

 Accessing single items in
list
cookies = ['chocolate chip', 'peanut butter', 'sugar']

cookies.append('Tirggel')
sugar', 'Tirggel']

print(cookies)

print(cookies[2])

DATA TYPES FOR DATA SCIENCE IN

Combining Lists
Using operators, you can combine two lists into a new one

cakes = ['strawberry',

'vanilla'] desserts = cookies +

cakes print(desserts)
['chocolate chip', 'peanut butter', 'sugar',
'Tirggel', 'strawberry', 'vanilla']

.extend() method merges a list into another list at the end

DATA TYPES FOR DATA SCIENCE IN

Finding Elements in a
List
.index() method locates the position of a data element in a
list

position = cookies.index('sugar')

print(position)

cookies[3]

'sugar'

DATA TYPES FOR DATA SCIENCE IN

Removing Elements in a
List
.pop() method removes an item from a list and allows you
to save it

name = cookies.pop(position)

print(name)

sugar

print(cookies)

['chocolate chip', 'peanut butter', 'Tirggel']

DATA TYPES FOR DATA SCIENCE IN

Iterating over
lists
for loops are the most common way of iterating over a list

for cookie in cookies:

print(cookie)

chocolate chip
peanut butter
Tirggel

DATA TYPES FOR DATA SCIENCE IN

Sorting
lists
sorted() function sorts data in numerical or alphabetical
order and returns a new list

print(cookies)

['chocolate chip', 'peanut butter', 'Tirggel']

sorted_cookies = sorted(cookies)

print(sorted_cookies)

['Tirggel', 'chocolate chip', 'peanut butter']

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Meet the Tuples
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON

Jason Myers
Instructor
Tuple,
Tuple
Hold data in order

Index
Immutable

Pairing

Unpackable

DATA TYPES FOR DATA SCIENCE IN

Zipping
tuples
Tuples are commonly created by zipping lists together with
zip()

Two lists: us_cookies , in_cookies

top_pairs = list(zip(us_cookies, in_cookies))

print(top_pairs)

[('Chocolate Chip', 'Punjabi'), ('Brownies', 'Fruit Cake Rusk'),

('Peanut Butter', 'Marble Cookies'), ('Oreos', 'Kaju Pista Cookies'),
('Oatmeal Raisin', 'Almond Cookies')]

DATA TYPES FOR DATA SCIENCE IN

Unpacking
tuples
Unpacking tuples is a very expressive way for working with
data

us_num_1, in_num_1 =
top_pairs[0] print(us_num_1)

Chocolate Chip

print(in_num_1)

Punjabi

DATA TYPES FOR DATA SCIENCE IN

More unpacking in
Loops
Unpacking is especially powerful in loops

for us_cookie, in_cookie in

top_pairs: print(in_cookie)
print(us_cookie)

Punjabi
Chocolate Chip
Fruit Cake Rusk
Brownies
# ..etc..

DATA TYPES FOR DATA SCIENCE IN

Enumerating
positions
Another useful tuple creation method is the enumerate()
function

Enumeration is used in loops to return the position and the

data in that position while looping

for idx, item in enumerate(top_pairs):

us_cookie, in_cookie = item
print(idx, us_cookie, in_cookie)

(0, 'Chocolate Chip', 'Punjabi')

(1, 'Brownies', 'Fruit Cake Rusk')
# ..etc..

DATA TYPES FOR DATA SCIENCE IN

Be careful when making
tuples
Use zip() , enumerate() , or () to make
tuples

item = ('vanilla', 'chocolate')

print(item)

('vanilla', 'chocolate')

Beware of tailing commas!

item2 = 'butter',
print(item2)

('butter',)

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Sets for
unordered and
unique data
DATA T YP E S F O R DATA S C I E N C E I
N PYTHON
Jason Myers
Instructor
Se
t Unique
Unordered

Mutable

Python's implementation of Set Theory from Mathematics

DATA TYPES FOR DATA SCIENCE IN

Creating
Sets
Sets are created from a list

cookies_eaten_today = ['chocolate chip', 'peanut butter',

...: 'chocolate chip', 'oatmeal cream', 'chocolate

chip'] types_of_cookies_eaten = set(cookies_eaten_today)

print(types_of_cookies_eaten)

set(['chocolate chip', 'oatmeal cream', 'peanut

butter'])

DATA TYPES FOR DATA SCIENCE IN

Modifying
Sets
.add() adds single elements

.update() merges in another set or list

types_of_cookies_eaten.add('biscotti')

types_of_cookies_eaten.add('chocolate chip')

print(types_of_cookies_eaten)

set(['chocolate chip', 'oatmeal cream', 'peanut butter', 'biscotti'])

DATA TYPES FOR DATA SCIENCE IN

Updating
Sets
cookies_hugo_ate = ['chocolate chip', 'anzac']

types_of_cookies_eaten.update(cookies_hugo_ate

) print(types_of_cookies_eaten)

set(['chocolate chip', 'anzac', 'oatmeal

cream', 'peanut butter', 'biscotti'])

DATA TYPES FOR DATA SCIENCE IN

Removing data from
sets
.discard() safely removes an element from the set by value

.pop() removes and returns an arbitrary element from the

set (KeyError when empty)

types_of_cookies_eaten.discard('biscotti')
print(types_of_cookies_eaten)

set(['chocolate chip', 'anzac', 'oatmeal cream', 'peanut butter'])

types_of_cookies_eaten.pop()

types_of_cookies_eaten.pop()

'chocolate chip'
'anzac'

DATA TYPES FOR DATA SCIENCE IN

Set Operations -
Similarities
.union() set method returns a set of all the names ( or )
.intersection() method identifies overlapping data ( and )
cookies_jason_ate = set(['chocolate chip', 'oatmeal
cream', 'peanut butter'])

cookies_hugo_ate = set(['chocolate chip',

'anzac'])

cookies_jason_ate.union(cookies_hugo_ate)
set(['chocolate chip', 'anzac', 'oatmeal cream', 'peanut butter'])

cookies_jason_ate.intersection(cookies_hugo_ate)

set(['chocolate chip'])

DATA TYPES FOR DATA SCIENCE IN

Set Operations -
Differences
.difference() method identifies data present in the set on
which the method was used that is not in the arguments ( - )
Target is important!

cookies_jason_ate.difference(cookies_hugo_ate)

set(['oatmeal cream', 'peanut butter'])

cookies_hugo_ate.difference(cookies_jason_ate)

set(['anzac'])

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Using dictionaries
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON

Jason Myers
Instructor
Creating and looping through
dictionaries
Hold data in key/value pairs

Nestable (use a dictionary as the value of a key within a

dictionary)

Iterable

Created by dict() or {}

art_galleries = {}

for name, zip_code in galleries:

art_galleries[name] = zip_code

DATA TYPES FOR DATA SCIENCE IN

Printing in the
loop
for name in art_galleries:
print(name)

Zwirner David Gallery

Zwirner & Wirth
Zito Studio Gallery
Zetterquist Galleries
Zarre Andre Gallery

DATA TYPES FOR DATA SCIENCE IN

Safely finding by
key
art_galleries['Louvre']

|
KeyError Traceback (most recent call last)
<ipython-input-1-4f51c265f287> in <module>()
--> 1 art_galleries['Louvre']

KeyError: 'Louvre'

Geting a value from a dictionary is done using the key as an

index

If you ask for a key that does not exist that will stop your
program from running in a KeyError

DATA TYPES FOR DATA SCIENCE IN

Safely finding by key
(cont
.get().)method allows you to safely access a key without
error or exception handling

If a key is not in the dictionary, .get() returns None by

default or you can supply a value to return

art_galleries.get('Louvre', 'Not Found')

'Not Found'

art_galleries.get('Zarre Andre Gallery')

'10011'

DATA TYPES FOR DATA SCIENCE IN

Working with nested
dictionaries
art_galleries.keys()

dict_keys(['10021', '10013', '10001', '10009', '10011',

...: '10022', '10027', '10019', '11106', '10128'])

print(art_galleries['10027'])

{"Paige's Art Gallery": '(212) 531-1577',

'Triple Candie': '(212) 865-0783',
'Africart Motherland Inc': '(212) 368-6802',
'Inner City Art Gallery Inc': '(212) 368-4941'}

The method shows the keys for a given dictionary

.keys()

DATA TYPES FOR DATA SCIENCE IN

Accessing nested
data
art_galleries['10027']['Inner City Art Gallery Inc']

'(212) 368-4941'

Common way to deal with repeating data structures Can

be accessed using multiple indices or the .get()

method

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Altering
dictionaries
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON

Jason Myers
Instructor
Adding and extending
dictionaries
Assignment to add a new key/value to a dictionary

.update() method to update a dictionary from another

dictionary, tuples or keywords

print(galleries_10007)

{'Nyabinghi Africian Gift Shop': '(212) 566-3336'}

art_galleries['10007'] = galleries_10007

DATA TYPES FOR DATA SCIENCE IN

Updating a
dictionar y= [
galleries_11234
('A J ARTS LTD', '(718) 763-5473'),
('Doug Meyer Fine Art', '(718) 375-8006'),
('Portrait Gallery', '(718) 377-8762')]
art_galleries['11234'].update(galleries_11234
) print(art_galleries['11234'])

{'Portrait Gallery': '(718) 377-8762',

'A J ARTS LTD': '(718) 763-5473',
'Doug Meyer Fine Art': '(718) 375-8006'}

DATA TYPES FOR DATA SCIENCE IN

Popping and deleting from
dictionaries
del instruction deletes a key/value

.pop() method safely removes a key/value from a

dictionary.

del art_galleries['11234']
galleries_10310 =
art_galleries.pop('10310')
print(galleries_10310)
{'New Dorp Village Antiques Ltd': '(718) 815-2526'}

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Pythonical y
using
dictionaries
DATA T YP E S F O R DATA S C I E N C E IN
PYTHON

Jason Myers
Instructor
Working with dictionaries more
py.items()
thonicall
methody
returns an object we can iterate over

for gallery, phone_num in art_galleries.items():

print(gallery)
print(phone_num)

'Miakey Art Gallery'

'(718) 686-0788'
'Morning Star Gallery Ltd'
'(212) 334-9330'}
'New York Art Expo Inc'
'(212) 363-8280'

DATA TYPES FOR DATA SCIENCE IN

Checking dictionaries for
data
.get() does a lot of work to check for a key

in operator is much more efficient and clearer

'11234' in art_galleries

False

if '10010' in art_galleries:

print('I found: %s' % art_galleries['10010'])

else:
print('No galleries found.')

I found: {'Nyabinghi Africian Gift Shop': '(212) 566-3336'}

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Working with
CSV files
DATA T YP E S F O R DATA S C I E N C E I
N PYTHON

Jason Myers
Instructor
CSV
Files
NAME,TEL,ADDRESS1,ADDRESS2,CITY,ZIP
O'reilly William & Co Ltd,(212) 396-1822,52 E 76th St,,New York,10021

DATA TYPES FOR DATA SCIENCE IN

Reading from a file using CSV
reader
Python csv module

open() function provides a variable that represents a file,

takes a path and a mode

csv.reader() reads a file object and returns the lines from

the file as tuples

.close() method closes file objects

import csv
csvfile = open('ART_GALLERY.csv',
'r') for row in csv.reader(csvfile):
print(row)

DATA TYPES FOR DATA SCIENCE IN

Reading from a CSV -
Res ults
['NAME', 'the_geom', 'TEL', 'URL',
'ADDRESS1', 'ADDRESS2', 'CITY', 'ZIP']
["O'reilly William & Co Ltd",
'POINT (-73.96273074561996 40.773800871637576)',
'(212) 396-1822', '52 E 76th St', '', 'New York',
'10021']

csvfile.close()

DATA TYPES FOR DATA SCIENCE IN

Creating a dictionary from a
file
Often we want to go from CSV file to dictionary

DictReader does just that

If data doesn't have a header row, you can pass in the

column names

for row in csv.DictReader(csvfile):

print(row)

OrderedDict([('NAME', 'Odyssia Gallery'),

('the_geom', 'POINT (-73.96269813635554 40.7618747512849)'),
('TEL', '(212) 486-7338'),
('URL', '
http://www.livevillage.com/newyork/art/odyssia-gallery.html'),
('ADDRESS1', '305 E 61st St'), ...

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Counting made
easy
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON

Jason Myers
Instructor
Collections
Module
Part of Standard Library Advanced
data containers

DATA TYPES FOR DATA SCIENCE IN

Counte
r Special dictionary used for counting data, measuring
frequency

from collections import Counter

nyc_eatery_count_by_types = Counter(nyc_eatery_types)
print(nyc_eatery_count_by_type)

Counter({'Mobile Food Truck': 114, 'Food Cart': 74, 'Snack Bar': 24,
'Specialty Cart': 18, 'Restaurant': 15, 'Fruit & Vegetable Cart': 4})

print(nyc_eatery_count_by_types['Restaurant'])

15

DATA TYPES FOR DATA SCIENCE IN

Counter to find the most
common
.most_common() method returns the counter values in
descending order

print(nyc_eatery_count_by_types.most_common(3)
)

[('Mobile Food Truck', 114), ('Food Cart', 74), ('Snack Bar',

24)]

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Dictionaries of
unknown
structure -
defaultdict
DATA T YP E S F O R DATA S C I E N C E I NP Y
Jason Myers THON
Instructor
Dictionary
Handling
for park_id, name in
nyc_eateries_parks: if park_id not
in eateries_by_park:
eateries_by_park[park_id] = []
eateries_by_park[park_id].append(name
)
print(eateries_by_park['M010'])
{'MOHAMMAD MATIN','PRODUCTS CORP.', 'Loeb Boathouse
Restaurant', 'Nandita Inc.', 'SALIM AHAMED', 'THE NY PICNIC
COMPANY',
'THE NEW YORK PICNIC COMPANY, INC.', 'NANDITA,
INC.', 'JANANI FOOD SERVICE, INC.'}

DATA TYPES FOR DATA SCIENCE IN

Using
defa altdict
Pass itu default type that every key will have even if it
doesn't currently exist

Works exactly like a dictionary

from collections import defaultdict

eateries_by_park = defaultdict(list)
for park_id, name in
nyc_eateries_parks:

eateries_by_park[park_id].append(name)

print(eateries_by_park['M010'])
{'MOHAMMAD MATIN','PRODUCTS CORP.', 'Loeb Boathouse Restaurant',
'Nandita Inc.', 'SALIM AHAMED', 'THE NY PICNIC COMPANY',
'THE NEW YORK PICNIC COMPANY, INC.', 'NANDITA, INC.', ...}

DATA TYPES FOR DATA SCIENCE IN

defaultdict
cont
(from .)
collections import defaultdict
eatery_contact_types = defaultdict(int)
for eatery in nyc_eateries:
if eatery.get('phone'):
eatery_contact_types['phones'] += 1
if eatery.get('website'):
eatery_contact_types['websites'] += 1
print(eatery_contact_types)

defaultdict(<class 'int'>, {'phones': 28, 'websites': 31})

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Maintaining
Dictionary
Order with
OrderedDict
DATA T YP E S F O R DATA S C I E N C E
I NP Y T H O N
Jason Myers
Instructor
Order in Python
dictionaries
Python version < 3.6 NOT ordered Python version
> 3.6 ordered

DATA TYPES FOR DATA SCIENCE IN

Getting started with
OrderedDict
from collections import OrderedDict
nyc_eatery_permits = OrderedDict()
for eatery in nyc_eateries:
nyc_eatery_permits[eatery['end_date']] = eatery
print(list(nyc_eatery_permits.items())[:3]

('2029-04-28', {'name': 'Union Square Seasonal Cafe',

'location': 'Union Square Park', 'park_id': 'M089',
'start_date': '2014-04-29', 'end_date': '2029-04-28',
'description': None, 'permit_number': 'M89-SB-R', ...})

DATA TYPES FOR DATA SCIENCE IN

OrderedDict power
feat ure method returns items in reverse insertion order
.popitem()

print(nyc_eatery_permits.popitem())

('2029-04-28', {'name': 'Union Square Seasonal Cafe',

'location': 'Union Square Park', 'park_id': 'M089',
'start_date': '2014-04-29', 'end_date': '2029-04-28',
'description': None, 'permit_number': 'M89-SB-R', ...})

print(nyc_eatery_permits.popitem())

('2027-03-31', {'name': 'Dyckman Marina Restaurant',

'location': 'Dyckman Marina Restaurant', 'park_id': 'M028',
'start_date': '2012-04-01', 'end_date': '2027-03-
31', ...})

DATA TYPES FOR DATA SCIENCE IN

OrderedDict power feature (2)
You can use the last=False keyword argument to return the
items in insertion order

print(nyc_eatery_permits.popitem(last=False))

('2012-12-07', {'name': 'Mapes Avenue Ballfields Mobile Food Truck',

'location': 'Prospect Avenue, E. 181st Street', 'park_id': 'X289',
'start_date': '2009-07-01', 'end_date': '2012-12-07',
'description': None, 'permit_number': 'X289-MT', 'phone': None,
'website': None, 'type_name': 'Mobile Food Truck'})

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 namedtuple
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON

Jason Myers
Instructor
What is a namedtuple?
A tuple where each position (column) has a name
Ensure each one has the same properties Alternative to
a pandas DataFrame row

DATA TYPES FOR DATA SCIENCE IN

Creating a
namedt
Pass a nameuple
and a list of fields

from collections import namedtuple

Eatery = namedtuple('Eatery', ['name', 'location', 'park_id',
...: 'type_name'])
eateries = []
for eatery in nyc_eateries:
details = Eatery(eatery['name'],
eatery['location'],
eatery['park_id'],
eatery['type_name'])
eateries.append(details)

DATA TYPES FOR DATA SCIENCE IN

Print the first
element
print(eateries[0])

Eatery(name='Mapes Avenue Ballfields Mobile Food Truck',

location='Prospect Avenue, E. 181st Street',
park_id='X289', type_name='Mobile Food Truck')

DATA TYPES FOR DATA SCIENCE IN

Leveraging
namedt avples
Each field is u ailable as an atribute of the namedtuple

for eatery in eateries[:3]:

print(eatery.name)
print(eatery.park_id)
print(eatery.location)

Mapes Avenue Ballfields Mobile Food Truck

X289
Prospect Avenue, E. 181st Street

Claremont Park Mobile Food Truck

X008
East 172 Street between Teller & Morris avenues ...

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 There and Back
Again a
DateTime
Journey
DATA T YP E S F O R DATA S C I E N C E I
N PYTHON
Jason Myers
Instructor
From string to
datetime
The datetime module is part of the Python standard library

Use the datetime type from inside the datetime module

.strptime() method converts from a string to a datetime

object

from datetime import datetime

print(parking_violations_date)

06/11/2016

DATA TYPES FOR DATA SCIENCE IN

Parsing strings into
datetimes
date_dt = datetime.strptime(parking_violations_date,
'%m/%d/%Y')
print(date_dt)

2016-06-11 00:00:00

DATA TYPES FOR DATA SCIENCE IN

Time Format
Strings
Directive Meaning Example

%d Day of the month as a zero- 01, 02, ..., 31

padded decimal number.
%m Month as a zero-padded 01, 02, ..., 12
decimal number.
%Y Year with century as a 0001, 0002, ..., 2013,
decimal number. 2014, ..., 9998, 9999
Full list available in the Python documentation

DATA TYPES FOR DATA SCIENCE IN

Datetime to
String
.strftime() method uses a format string to convert a
datetime object to a string

date_dt.strftime('%m/%d/%Y')

'06/11/2016'

isoformat() method outputs a datetime as an ISO standard

string

date_dt.isoformat()

'2016-06-11T00:00:00'

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Working with
Datetime
Components
and current
time
Jason Myers DATA T YP E S F O R DATA S C I E N C E I NP Y T
Instructor HON
Datetime
Components
day , month , year , hour , minute , second , and more
are available from a datetime instance

Great for grouping data

daily_violations = defaultdict(int)
for violation in parking_violations:
violation_date = datetime.strptime(violation[4],
'%m/%d/%Y')
daily_violations[violation_date.day] += 1

DATA TYPES FOR DATA SCIENCE IN

Datetime Components -
Res ults
print(sorted(daily_violations.items()))

[(1, 80986), (2, 79831), (3, 74610), (4, 69555),

(5, 68729), (6, 76232),(7, 82477), (8, 72472),
(9, 80415), (10, 75387), (11, 73287), (12, 74614),
(13, 75278), (14, 81803), (15, 79122), (16, 80692),
(17, 73677), (18, 75927), (19, 80813), (20, 80992),
(21, 78138), (22, 81872), (23, 78104), (24, 63490),
(25, 78898), (26, 78830), (27, 80164), (28, 81954),
(29, 80585), (30, 65864), (31, 44125)]

DATA TYPES FOR DATA SCIENCE IN

What is the deal with
no.now()
w method returns the current local datetime
.utcnow() method returns the current UTC datetime

from datetime import

datetime local_dt =
datetime.now()
print(local_dt)

2017-05-05 12:30:00.740415

DATA TYPES FOR DATA SCIENCE IN

What is the deal with
tcno= w
uutc_dt datetime.utcnow()
print(utc_dt)

2017-05-05 17:30:05.467221

DATA TYPES FOR DATA SCIENCE IN

Timezone
s Naive datetime objects have no timezone data
Aware datetime objects have a timezone

Timezone data is available via the pytz module via the

timezone object

Aware objects have .astimezone() so you can get the time

in another timezone

DATA TYPES FOR DATA SCIENCE IN

Timezones in
action
from pytz import timezone
record_dt = datetime.strptime('07/12/2016
04:39PM',
...: '%m/%d/%Y %H:%M%p')
ny_tz = timezone('US/Eastern')
a_tz = timezone('US/Pacific')
ny_dt =
record_dt.replace(tzinfo=ny_tz) la_dt
= ny_dt.astimezone(la_tz)

DATA TYPES FOR DATA SCIENCE IN

Timezones in action -
res ults
print(ny_dt)

2016-07-12 04:39:00-04:00

print(la_dt)

2016-07-12 01:39:00-07:00

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Time Travel
(Adding and
Subtracting
Time)
DATA T YP E S F O R DATA S C I E N C E I NP Y
Jason Myers THON
Instructor
Incrementing through
time
timedelta is used to represent an amount of change in time

Used to add or subtract a set amount of time from a datetime

object

from datetime import timedelta

flashback =
timedelta(days=90)
print(record_dt)

2016-07-12 04:39:00

DATA TYPES FOR DATA SCIENCE IN

Adding and subtracting
timedeltas
print(record_dt - flashback)

2016-04-13 04:39:00

print(record_dt + flashback)

2016-10-10 04:39:00

DATA TYPES FOR DATA SCIENCE IN

Datetime
differences
Use the - operator to calculate the diference

Returns a timedelta with the diference

time_diff = record_dt - record2_dt

type(time_diff)

datetime.timedelta

print(time_diff)

0:00:04

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 HELP! Libraries
to make it
easier
DATA T YP E S F O R DATA S C I E N C E I
N PYTHON

Jason Myers
Instructor
Parsing time with
pend uluwillmatempt to convert a string to a pendulum
.parse()
datetime object without the need of the format string

import pendulum

occurred = violation[4] + ' ' + violation[5] +'M'

occurred_dt = pendulum.parse(occurred, tz='US/Eastern')

print(occured_dt)

'2016-06-11T14:38:00-04:00'

DATA TYPES FOR DATA SCIENCE IN

Timezone hopping with
pend ulum method converts a pendulum time object to
.in_timezone()
a desired timezone.

.now() method accepts a timezone you want to get the

current time in

print(violation_dts)

[<Pendulum [2016-06-11T14:38:00-04:00]>,
<Pendulum [2016-04-25T14:09:00-04:00]>,
<Pendulum [2016-04-23T07:49:00-04:00]>,
<Pendulum [2016-04-26T07:09:00-04:00]>,
<Pendulum [2016-01-04T09:52:00-05:00]>]

DATA TYPES FOR DATA SCIENCE IN

More timezone
hopping
for violation_dt in violation_dts:
print(violation_dt.in_timezone('Asia/Tokyo'))

2016-06-12T03:38:00+09:00
2016-04-26T03:09:00+09:00
2016-04-23T20:49:00+09:00
2016-04-26T20:09:00+09:00
2016-01-04T23:52:00+09:00

print(pendulum.now('Asia/Tokyo'))

<Pendulum [2017-05-06T08:20:40.104160+09:00]>

DATA TYPES FOR DATA SCIENCE IN

Humanizing
differences
.in_XXX() methods provide the diference in a chosen
metric

.in_words() provides the diference in a nice expressive

form

diff = violation_dts[3] - violation_dts[2]

diff

<Period [2016-04-26T07:09:00-04:00 ->

2016-04-23T07:49:00-04:00]>

print(diff.in_words())

'2 days 23 hours 20 minutes'

DATA TYPES FOR DATA SCIENCE IN

More human than
hprint(diff.in_days())
uman
2

print(diff.in_hours())

71

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Case Study -
Counting
Crimes
DATA T YP E S F O R DATA S C I E N C E
I NP Y T H O N
Jason Myers
Instructor
Data Set
Overview
Date,Block,Primary Type,Description,
Location Description,Arrest,Domestic, District

05/23/2016 05:35:00 PM,024XX W DIVISION ST,ASSAULT,SIMPLE,

STREET,false,true,14

03/26/2016 08:20:00 PM,019XX W HOWARD

ST,BURGLARY,FORCIBLE ENTRY, SMALL RETAIL
STORE,false,false,24

Chicago Open Data Portal htps://data.cityofchicago.org/

DATA TYPES FOR DATA SCIENCE IN

Part 1 - Step
1 Read data from CSV
import csv

csvfile = open('ART_GALLERY.csv',

'r') for row in csv.reader(csvfile):

print(row)

DATA TYPES FOR DATA SCIENCE IN

Part 1 - Step
2 Create and use a Counter with a slight twist
from collections import Counter

nyc_eatery_count_by_types = Counter(nyc_eatery_types)

Use date parts for Grouping like in Chapter 4

daily_violations = defaultdict(int)

for violation in parking_violations:

violation_date = datetime.strptime(violation[4],
'%m/%d/%Y')
daily_violations[violation_date.day] += 1

DATA TYPES FOR DATA SCIENCE IN

Part 1 - Step
3 Group data by Month
The date components we learned about earlier.

from collections import defaultdict

eateries_by_park = defaultdict(list)

for park_id, name in

nyc_eateries_parks:
eateries_by_park[park_id].append(name)

DATA TYPES FOR DATA SCIENCE IN

Part 1 -
Final
Find 5 most common locations for crime each month.

print(nyc_eatery_count_by_types.most_common(3))

[('Mobile Food Truck', 114), ('Food Cart', 74), ('Snack Bar',

24)]

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Case Study -
Crimes by District
and Differences
by Block
DATA T YP E S F O R DATA S C I E N C E I NP Y T
Jason Myers HON
Instructor
Part 2 - Step
1 Read in the CSV data as a dictionary
import csv

csvfile = open('ART_GALLERY.csv',

'r') for row in

csv.DictReader(csvfile):
print(row)

Pop out the key and store the remaining dict

galleries_10310 = art_galleries.pop('10310')

DATA TYPES FOR DATA SCIENCE IN

Part 2 - Step
2 Pythonically iterate over the Dictionary
for zip_code, galleries in art_galleries.items():
print(zip_code)
print(galleries)

DATA TYPES FOR DATA SCIENCE IN

Wrapping
Up
Use sets for uniqueness
cookies_eaten_today = ['chocolate chip', 'peanut butter',
'chocolate chip', 'oatmeal cream', 'chocolate chip']

types_of_cookies_eaten = set(cookies_eaten_today)

print(types_of_cookies_eaten)

set(['chocolate chip', 'oatmeal cream', 'peanut butter'])

difference() set method as at the end of Chapter 1

cookies_jason_ate.difference(cookies_hugo_ate)
set(['oatmeal cream', 'peanut butter'])

DATA TYPES FOR DATA SCIENCE IN

 Let's practice!
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON
 Final thoughts
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON

Jason Myers
Instructor
 Congratulations
DATA T YP E S F O R DATA S C I E N C E I NP Y T
HON