George Mason University

Python: Generators & Regular Expressions

▪ Iterators, Iterables, Generators

▪ Regular Expressions (re Python library)

▪ Final Exam format

2
5
Iterable Objects, Iterators, Generators
❑ What is an iterable objects? Iterators? Generators?
❑ How do iterators differ from generators?
❑ How to use them in Python – manually and automatically
❑ Why use generators in Digital Forensics?

Regular Expressions
❑ What is a regular expression
❑ How can we search for specific items or patterns

6
7
 ▪ Iterable objects are like containers (in memory) that you
are able step through to access its content.
▪ Lists, Sets, Tuples, Dictionaries are all iterable objects
▪ Iterable objects invokes the iter() method to get an iterator

Lists/Tuples Dictionaries

8
▪ Iterators remember the state of the last item accessed in a
memory container (iterable object)
▪ Iterators use next() method to get the next item in the
container

▪ When the iterator has accessed the last element, it will

raise the StopIteration exception – indicating there are
no more elements in the container

9
 StopIteration Exception

10
http://nvie.com/posts/iterators-vs-generators/
‘for’ loop What are:
─ listWeek15?
─ iteratorList?
─ week15item?
─ listWeek15a?

iter() method

11
▪ Generators – special functions that produce a special iterator
called a generator
▪ They can generate new data values when called
▪ Can be written as a single-lined expression or as a function
▪ Use special method yield() which holds the generator’s state for
the next time next() is called
▪ Best to use when dealing with data that takes up vast amounts of
memory
Why would this be useful in digital forensics?
12
13
RECALL
▪ Iterables
▪ Memory containers where you can access elements within memory
▪ Calls iter() method to get an iterator object

▪ Iterators
▪ Object that holds internal state; allows you to retrieve next item in
our iterable container by calling next() method
▪ Generators
▪ Special functions that produce special iterators (for lazy evaluation)
▪ Lazy evaluation – only done when specifically called
▪ Used to generator data or process large volumes of data
▪ If you see yield() you are working with a generator

14
ITERABLES, ITERATORS & GENERATORS
From BB → Course Content → Class15… → Exercises

Download:
❑ iterGen (.ipynb or .py)

Run through the cell in note any questions/issues/comments you

have for each step.

Complete the QUICK CHECK with your group. 15

QUICK CHECK 1
a. Calls iter() method to get an
iterator object
_ Generator

b. Object that holds internal

_ Iterable Object state; allows you to retrieve
next item in our iterable
container by calling next()
_ Iterator method

c. Special functions that

produce special iterators (for
lazy evaluation) 16
 QUICK CHECK 1
____( )?
____( )?
___________? ‘this’
This is the
x12=[‘this’, ‘is’, ‘the’] _________ ‘is’
which
‘the’
calls…

• Generators are special methods that produce _____________________
and uses the _________ method which will hold the generator’s state until
_________ is called?
• Generators are best used for? 18
20

re library
▪ Regular expressions are used to search for patterns in datasets.
What information might we want to quickly triage for in an
investigation?

21
 ▪ Python re library
▪ Allows user to search for Unicode or 8-byte character strings

Useful re methods:
re.compile(pattern, flags=0) compiles a sequence into a regex object to perform searches directly on
the object
re.match(pattern, string, flags=0) returns a match object* if pattern is found at the beginning of string
re.search(pattern, string, flags=0) returns a match object if pattern is found in string
re.findall(pattern, string, flags=0) returns a list containing all patterns found in string
re.finditer(pattern, string, flags=0) return an iterator yielding match object over all non-overlapping matches
for the RE pattern in string.
* match object = object containing information about the search and the result.
23
▪ Match object:
▪ .span() – returns a tuple containing the start and end positions of
the match
▪ .string() – returns the string passed into the function
▪ .group() – returns the part of the string where there was a match

24
 25
Source: https://www.w3schools.com/python/python_regex.asp
 26
Source: https://www.w3schools.com/python/python_regex.asp
 27
Source: https://www.w3schools.com/python/python_regex.asp
REGULAR EXPRESSIONS
From BB → Course Content → Class15… → Exercises

Download:
❑ reExamples (.ipynb or .py)

Run through the cell in note any questions/issues/comments you

have for each step.

Complete the QUICK CHECK with your group. 28

QUICK CHECK 2
1. What information can one triage data for with regular expressions?
2. Which re statement is used to find if a pattern is at the beginning of
a sequence?
3. What information does the Match Object return?
4. Would the following statement return a Match Object or None:
pat2find = “MU”
str2Search = “Mason Patriots at GMU”
re.search(pat2find,str2Search)

a. If a Match Object is returned, what is given in the span() field?

29
▪ 12-May: Final Exam
▪ Test will be available starting at 12AM (EST) on 12-May and open all day
▪ Timed exam (90 minutes)
▪ Covers the 2nd half of the semester
▪ Similar format to Midterm exam (e.g. multiple choice/answer, short answer, etc.)
▪ Be sure you have examined (static and dynamically)
DFOR510_Spring22_FINAL
▪ Password will be available tonight (10PM EST) for you to access the file
▪ Conduct your analysis prior to final
▪ The final WILL contain questions about the file

31
32
33
 34
https://clipartfox.com/categories/view/8afc348758b19996c75d34ea32d903b144a67dca/spring-break-2017.html
Iterators, Iterators, Generators
https://nvie.com/posts/iterators-vs-generators/
https://realpython.com/introduction-to-python-generators/
https://realpython.com/python-for-loop/#the-python-for-loop
https://www.dataquest.io/blog/python-generators-tutorial/
https://towardsdatascience.com/pythons-list-generators-what-when-how-and-why-2a560abd3879

Regular Expressions
https://docs.python.org/3/library/re.html
https://docs.python.org/3/howto/regex.html#regex-howto
https://www.regular-expressions.info/tutorial.html
https://www.w3schools.com/python/python_regex.asp

35