演化式計算

Evolutionary Computation
EE-6950, Fall 2023
Lecture # 2 - Python Review & SW Development Best Practices
Outline
Note: Today we will do a very rapid review of Python. Will highlight language
peculiarities, especially for those coming from C++, Java, etc.

I. Python Programming Fundamentals

II. Object-oriented Python

III. Software Development Best Practices

IV. Homework #1

2
I. Python programming fundamentals
What is Python?

• Python:
• A high-level, general purpose interpreted programming language
• Object-oriented
• Dynamically typed
• Automatic memory management (garbage collection)
• Highly introspective and malleable
• In addition to OO, also supports Functional style programming
• Cross platform
• Everything is an object!

4
How To Run Python Programs
• Interactive interpreter:
python3
Python 3.7.8 (v3.7.8:4b47a5b6ba, Jun 27 2020, 04:47:50)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print('Hello World!')
Hello World!
>>>

• Run les or modules from command line (batch mode):

python3 Hello.py
Hello World!

python3 –m Hello
Hello World!

• Shell scripts:
#!/usr/bin/env python
import Hello
Hello.main()

Hello World!

5
fi
Getting Help: The interactive interpreter is your friend

• dir(obj) command: Can nd list of commands and attributes associated with

any object or object type (or current scope, if use dir() without parameters

Example:
>>> dir(list)
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__',
'__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__',
'__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__',
'__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__',
'__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop',
'remove', 'reverse', 'sort']
>>>

fi
Getting Help: The interactive interpreter is your friend

• help(obj) command: Can nd detailed help for most classes, modules,

objects, functions, and object types

Example:
>>> from random import Random
>>> help(Random.shuffle)
Help on function shuffle in module random:

shuffle(self, x, random=None)
Shuffle list x in place, and return None.

Optional argument random is a 0-argument function returning a random float in [0.0, 1.0);
if it is the default None, the standard random.random will be used.
(END)

fi
Getting Help: The online Python documentation

• https://docs.python.org

8
Installing Packages/Libraries – pip

• Search for available packages:

pip search some_name

• List currently installed packages/versions:

pip list

• Install a package (latest version) & dependencies:

pip install some_package

• Check for outdated packages, update to latest version:

pip list --outdated
pip install –U some_package

9
Python Code Structure - Whitespace

• Whitespace is signi cant!

- This is one of the weirdest (and most controversial) aspects of Python
- Code blocks are delineated by indentation, not {}’s
- I strongly suggest you do not mix spaces and tabs for indentation in Python les, this can
sometimes result in hard to debug problems

10
fi
Python Code Structure – Comments & Help

• Basic code comment are indicated using the “#” character

• PyDoc self-documenting code/help comments are in triple-quote blocks (“””
This is help text”””) at the beginning of a function or class

>>>help(TestFunction)

11
Code Organization – Modules & Packages

• Modules:
- Each le with a .py suf x is a module
- Each module has its own namespace

• Packages are collections of modules (libraries, in e ect)

- Packages contain multiple modules (.py les or subdirs containing .py les)
- Package directory must contain a special le: __init__.py (can be empty)

• PYTHONPATH
- When importing modules and packages, python interpreter will search local path, the python
interpreter system path, and all paths listed in the PYTHONPATH environment variable
- This allows you to better organize your python les and packages (for example, no need to
keep them all in the current working directory)

12
fi
fi
fi
fi
Modules & Packages -- Importing

• To use code in modules and packages we use the “import” keyword

• If the previous TestFunction is in a le TestModule.py, we can import it using
any of the following
- import TestModule
- from TestModule import TestFunction
- from TestModule import *

• If TestModule.py is in a package directory named TestPackage/,

then we can import as follows
- import TestPackage.TestModule
- from TestPackage.TestModule import TestFunction
- from TestPackage.TestModule import *

13

fi
Module & Package Import Examples
>>> import TestModule
>>> TestModule.TestFunction(123)

>>> from TestModule import TestFunction

>>> TestFunction(-123)

>>> from TestModule import *

>>> TestFunction(0)

>>> import TestPackage.TestModule

>>> TestPackage.TestModule.TestFunction(456)

>>> from TestPackage import TestModule

>>> TestModule.TestFunction(-456)

>>> from TestPackage.TestModule import *

>>> TestFunction(0)

14
Variables & Dynamic Typing
• Variables are not declared, just assigned eevee$ python3
>>> x
• A variable is created the rst time you Traceback (most recent call last):
assign it a value
File "<stdin>", line 1, in <module>
• Variables are references to objects NameError: name 'x' is not defined
>>> x="hello"
• Type information is with the object, not >>> print(x)
the reference hello
• Everything in Python is an object >>> y=123.4
>>> print(y)
- Basic datatypes
- Functions 123.4
- Class instances >>> y+10
- Classes 133.4
- Modules >>> x+" world"
- Heck, even Types! 'hello world'
- Yup…pretty much everything! >>>

15
fi
Everything is an object
eevee$ python3
>>> x=1.2e4
>>> type(x)
<class 'float'>
>>>
>>> y=[1,2,3]
>>> type(y)
<class 'list'>
>>>
>>> def TestFunction(x):
... return x
...
>>> z=TestFunction
>>> type(z)
<class 'function'>
>>>
>>> a=float
>>> type(a)
<class 'type'>
>>>
>>> import random
>>> b=random
>>> type(b)
<class 'module'>
>>>

16
Basic datatypes

• Basic types generally similar to C/C++/Java, with a few notable exceptions

• Floating point
• Integers
• Complex numbers
• Booleans
• The None type
• Also includes support for a few specialized types
- Fixed-precision “Decimal” type
- Rational number “Fraction” type

17
Basic datatypes generally work exactly the way you’d expect them to!

• Int’s, oats: x=123, x=1.2e6, standard operators (+,-,*,%,**)

• Also support for bitwise operators: <<, >>, |, &, ^
• Booleans:
- True and False
• Complex numbers:
- x=1.0+2.0j
- x.real, x.imag
• None type is the Null object, if None -> False
• Integers support unlimited length
• Python has “ oor” division operator //:
• 3//4=0
• 3/4=0.75
18
fl
fl
Datatypes: Boolean, identity, and comparison operators

• Logical comparison operators

- and
- or
- not
• Equality comparison operators
- ==
- !=
• Identity and inclusion operators
- is
- in

19
Strings

• Rich support for operations on strings – quite convenient!

• Python3 natively supports unicode
• Three forms for string literals, single, double, and triple quotes.
• We’ve already seen triple-quoted blocks in PyDoc help comments

20
String operators, functions

• There are many, many useful built-in methods for working with strings. Here
are a few…

21
Lists

• Ordered collections of data >>> x = [1,'hello', (3 + 2j)]

>>> x
• Data can be heterogeneous [1, 'hello', (3+2j)]
>>> x[2]
• Lists are mutable (3+2j)
>>> x[0:2]
• Supports rich access using indexes [1, 'hello']
and “slicing”

22
Lists: “Backwards” indexing

• Don’t try this in C/C++!

>>> x = [1,2,3]
• You can index from the “back” of the >>> x[-1]
list using negative numbers 3

• a = x[-1]
• b = x[-2]

23
List operators, functions

• There are many useful built-in methods for working with lists.

24
Tuples

• Tuples are immutable versions of >>> x = (1,2,3)

lists >>> x[1:]
(2, 3)
>>> y = (2,)
>>> y
(2,)
>>>

25
Dictionaries

• A set of key-value pairs >>> d = {1: 'hello', 'two': 42, 'blah': [1,2,3]}
>>> d
• Dictionaries are mutable {1:
>>>
'hello', 'two': 42, 'blah': [1, 2, 3]}
d['blah']
[1, 2, 3]

26
Dictionary operators, functions

• There are many useful built-in

methods for working with dict’s.

27
Copying Dictionaries and Lists

>>> l1 = [1] >>> d1 = {1 : 10}

• Can use built-in list or dict functions (ctor's) >>> l2 = list(l1) >>> d2 = dict(d1)
>>> l3 = l1.copy() >>> d3 = d.copy()
• Can also use the copy method >>> l1[0] = 22 >>> d1[1] = 22
>>> l2[0] = 33 >>> d2[1] = 33
>>> l1 >>> d1
[22] {1: 22}
>>> l2 >>> d2
[33] {1: 33}
>>> l3 >>> d3
[1] {1: 10}

28
Sets

• Sets are unordered collections of data >>> s = {123, ‘abc’}

>>> s
• Data can be heterogeneous {123, ‘abc’}

29
Files

30
Data Type Summary
• Integers: 2323
• Floating Point: 32.3, 3.1E2
• Complex: 3 + 2j, 1j
• Booleans: True, False
• None type: None
• Strings: “this is a string”
• Lists: l = [ 1,2,3]
• Tuples: t = (1,2,3)
• Dictionaries: d = {‘hello’ : ‘there’, 2 : 15}
• Sets: s ={123, ‘abc’}
• Files: f=open(‘data.txt’, ’r’)
31
Next up, control and loop statements…

• if, elif, else

• while
• break, continue, pass, else
• for

32
If statements

33
while statements

34
for loops

35
for loops: the range function

>>> for i in range(10):

... print(i)
...
0
1
2
3
4
5
6
7
8
9

36
Next up, functions

• def

37
Function Basics

• We already had a preview of this earlier

• def de nes a new function
• Optionally can return a value

38
fi
Functions are first-class objects

>>> def x(who):

• Functions are objects ... print('hello '+str(who))
>>> x
• The same reference rules hold for <function x at 0x619f0>
them as for other objects >>> x('you')
hello you
>>> y = x
>>> y
<function x at 0x619f0>
>>> y('me')
hello me

39
Function Parameters: Defaults

• Parameters can be assigned default >>> def foo(x = 3) :

values ... print x
...
• They are overridden if a parameter is >>> foo()
given for them 3
>>> foo(10)
• The type of the default doesn’t limit 10
the type of a parameter >>> foo('hello')
hello

40
Function Parameters: Named

• Pay attention, you will nd this very >>> def foo (a,b,c) :
useful! ... print a, b, c
...
• Call by name >>> foo(c = 10, a = 2, b = 14)
2 14 10
• Any positional arguments must come >>> foo(3, c = 2, b = 19)
before named ones in a call 3 19 2

41
fi
Exceptions: Hierarchical stack traces
• Exceptions automatically generate hierarchical stack traces

• Extremely useful for debugging complex code!

#exception hierarchical stack traces

Traceback (most recent call last):
#
File "/…/Excpt_exmpl.py", line 51, in <module>
def LowerFunction(a,b):
#if b=0, we'll get a ZeroDivisionError UpperFunction(10,0)
# exception on the next line File "/…/Excpt_exmpl.py", line 47, in UpperFunction
return a/b LowerFunction(x,y)
File "/…/Excpt_exmpl.py", line 44, in LowerFunction
def UpperFunction(x,y): return a/b
LowerFunction(x,y) ZeroDivisionError: division by zero

#calling UpperFunction with y=0, will

trigger
# ZeroDivisionError exception in
# LowerFunction
UpperFunction(10,0)

42
Some useful additional packages and libraries.
• System, os, and le-system: import os, import sys, import shutil
• Serialization: YAML, JSON
• Random numbers – import random
• Graphs – networkx
• Scienti c data visualization – matplotlib
• Matrices and linear algebra – numpy
• Scienti c number crunching – scipy
• Machine Learning – scikit-learn, Tensor ow, PyTorch
• Image processing – Pillow (formerly PIL), OpenCV
• Many more available…
43
fi
fi
fi
A few notes on Python computational efficiency

• Yes, it is true that Python is an interpreted language, and natively is “slower”

than compiled strongly typed languages like C++

• However, that is not the whole story!

- Most computationally intensive libraries in Python actually are written and compiled in C++ or
C, and are wrapped/exposed as Python classes. So these libraries generally are just as fast as
C++ (numpy, scipy, etc.)
- There are a number of associated technologies available to allow more ef cient computational
and numerical Python code, or interaction with compiled code – including cython, SWIG, and
PyPy.

44
II. Object-oriented Python
Classes and Object-oriented programming

• Python is an object-oriented (OO) language, similar to other OO languages

like C++ and Java

• Just as in other OO languages, classes:

• Support encapsulation of both instance data and behavior
• Support inheritance (multiple inheritance)
• Support both instance and class variables and methods (attributes)

46
Objects and Classes – remember?

Classes Instances

Class name

Data

Functions

(Two instances of the Student class)

47
A simple example: Point class
# A Simple Point class (represents an x,y coordinate)
#
class Point:
# __init__ function is the constructor
def __init__(self,x=0,y=0):
self.x=x
self.y=y

#move the point by x,y

def moveBy(self,x,y):
self.x+=x
self.y+=y

#move the point to x,y

def moveTo(self,x,y):
self.x=x
self.y=y

#calculate Hamming distance between two points

def distanceTo(self, p2):
return abs(self.x-p2.x) + abs(self.y-p2.y)

# __str__ generates string representation of objects

def __str__(self):
return '['+str(self.x)+','+str(self.y)+']'

48
The Constructor Method

• There is a special method known as the “constructor”

- In Python the constructor is: __init__(self…)
- The constructor is automatically called every time we create a new
instance
- It is used to initialize the state (variables) of each new object instance

• In Python the rst parameter on all class instance methods,

including the constructor, must be the variable “self”
- The ”self” variable always references the current instance
- If you know C++ or Java, it is similar to the “this” pointer

49
fi
Point class example: Object instances
• Point class de nition can be used to
generate multiple object instances
- Each instance has its own data
- Each instance can be manipulated
independently

• In this example, we create and operate

on three Point instances: p1, p2 and p3

• The “.” (dot) operator can be used to

access object methods and variables
(also known as “attributes”)

(Example continued next page…)

50
fi
Point class example: Object instances

• Using the “.” operator to access

object variables

• Also, remember we said classes

were like user-de ned types?
>>> type(p1)
>>> <class
’SimplePoint.Point'>

51
fi
Class Inheritance

• In Python you can de ne an inheritance hierarchy for classes

• Sub-classes inherit both state (variables) and behavior
(methods) from their super-classes
• Why would we want to do this?
- Better code organization -- functionality common to a related set of classes can be
shared/reused
- Can de ne common interface across set of related classes, so functionality can be
accessed in a uniform manner

• This will be clearer if we take a look at some speci c

examples…

52
fi
fi
Class inheritance – Python syntax

Python syntax for class derivation/inheritance

class DerivedClassName(SuperClassName): Superclass or Base class in parentheses ()

def __init__(self):
do_something_interesting_here…

53
Class inheritance – Animals

54
Class inheritance – Animals

55
Class inheritance – Super-class access

• Python constructors do not automatically chain up the class hierarchy

• To access base constructor or overridden methods from derived classes
-> Use super() or Base.xxx
• An example:
class Base:
def __init__(self):
print(”This is the Base ctor!”)

class Derived(Base):
def __init__(self):
super().__init__() #this will call Base’s __init__
print(“This is the derived ctor!”)

56
Class inheritance – Super-class access

• Use super() if you want constructor chaining like C++

class A:
def __init__(self):
super().__init__()
print("Called Init A!")

class B(A):
def __init__(self):
super().__init__()
print("Called Init B!")

class C(B):
def __init__(self):
super().__init__()
print("Called Init C!")
Called Init A!
c=C() Called Init B!
Called Init C!

57
III. Software Development Best Practices
Software Development Best Practices

I. Software Development Cycle

II. Modularization
III. Version Control
IV. Testing
V. Repeatability
VI. Debugging
VII. Documentation
VIII.Bug tracking

59
Software Development Cycle

Production software also has:

Continuous automated nightly testing!

60
Modularization

• Adopt good code structure & architecture habits!

• Break larger complex tasks/code into smaller independent

components with (narrower) well-de ned purpose

• Depending on size and complexity of program, you can leverage:

- Functions
- Classes
- Modules
- Packages (Libraries)

61
Modularization: Benefits

• Why Modularization?
- Facilities code re-use (components can be used in many di erent programs)
- Much easier to debug!
- Much easier to test
- Much easier to maintain
- Much easier to collaborate on larger projects with multiple developers

62
Version Control

• Is this how you manage le versions?

63

fi
Version Control: Overview

• Version control tools “automatically” maintain/track all changes to your les over time

• A few bene ts of version control:

- Easily revert back to older le versions (long-term undo)
- Enables comparisons between di erent le versions
- Allows multiple developers to work on same les at same time in an orderly manner
- Version history (what changes were made when)
- Ownership history (who made changes)
- Management of multiple simultaneous releases of production software (branches)

• A few widely-used version control systems:

- GIT  Open-source, and currently the most popular version-control system
- Perforce (p4, Helix)  Commercial software (large projects pay $$, small projects are free)

64
fi
fi
ff
fi
fi
Version Control: File change timeline
• Version control system maintains a timeline for all changes in
all versions of your les and software releases

65
fi
Git Demo/Tutorial
• This grad student (Brian Yu) in Harvard's CS Dep't made a
nice YouTube tutorial on Git & GitHub
• https://youtu.be/MJUJ4wbFm_A

66
Testing: Overview
• Algorithm and functional veri cation testing
- Test that your algorithms and computational code give correct results and reasonable
performance
• Unit testing
- Suites of smaller, more focused tests that check behavior of speci c aspects of
functions, classes, etc.
- Usually run by an automated framework
- Python has built-in unit-test framework called PyUnit. This can be accessed with
import unittest
• Nightly automated regression testing
- For production software usually there is a test harness that runs suites of regression
tests nightly, and reports any failures back to the developers
- Nightly tests may work in conjunction with version control system to automate traceback
to nd source of failures (which check-in or changelist)

67
fi
fi
Testing: Python unittest

68
Testing: unittest output (one test fails)

• python testMytest.py
F..
======================================================================
FAIL: test_isupper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
File ”testMytest.py", line 9, in test_isupper
self.assertTrue('FOo'.isupper())
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (failures=1)

69
Repeatability: Overview

• Always strive to make your code results repeatable!

• That means for the same input conditions, your code should
give the same output results over multiple runs

Same Input Program Same Output

Multiple Runs

Note: Yes, there are a few applications that need true randomness (e.g.,
cryptography, uuid’s), but these are generally the exception
70
Repeatability: Determinism is good!

• Why is repeatability important?

- Easier debugging and development
- Reliable and sane testing (remember unit tests?)
- Reliable interpretability
- Trust factor (does your code really work properly?)
- Customers expect it, particularly in commercial software!

71
Debugging: A few tips…

• Inspection and exploration of program & variable state:

- Always add methods to your classes that allow easy inspection of internal state
of objects (__str__ method, for example)
- Interactive interpreter is useful for exploration
- print() is your friend
• Interactive step-debuggers are available, if needed (Eclipse + PyDev)
• Make and respect your unit tests!
• Sensible modularization is your ally
- Easier to debug component by component or function by function
- Divide and conquer!

72
Documentation: Overview

• Particularly on large projects, maintaining good

documentation is essential
- Code comments
- PyDoc comments
- User Guides
- Developer Guides
- Speci cation documentation
• Vision/Architectural
• Functional
• Implementation

73
fi
Bug Tracking: Overview

• If working on a larger team-based project, or any commercially

released software, it is advisable to use a bug-tracking system

• Allows you to capture, prioritize, and track detailed status and

resolution of software bugs and problems

• Allows you to maintain a searchable history of resolved problems

• There are many, many bug tracking systems available

• Bugzilla is one popular open-source platform used by many teams and
companies: http://bugzilla.org

74
IV. Homework #01
- Due by Sept 21 @ noon
- Submit to iLearning
Homework #1: Welcome to Madagascar

• Submit to iLearning:
• Due by Sept 21 @ noon
• Submit only your python program

• Important  Please name as follows: hw1_your_id_num.py

76
Homework #1: Welcome to Madagascar
For this assignment, use input data le: assign1_input.txt
- This le contains a collection of animals (zebra, lion, hippo, gira e, penguin, gorilla)
that want to escape from NCHU’s Central Park!
- But they are in a hurry, and have cluttered up the park with integers everywhere!
- Some of penguins have even hurt themselves, so they are misspelled as "pengiun"
- There is also a hidden treasure, but we only know it’s a very very large integer.
- Animal names are separated either by new lines or %’s

- Example:
zebra%zebra%zebra
lion
1234
penguin
pengiun%penguin%pengiun
54687

77
fi
fi
Homework #1: Welcome to Madagascar
Start with hw1_skeleton.py. There is a “Decoder” in it to help you discover
the treasure. The contents of the le are shown below:

78

fi
Homework #1: Welcome to Madagascar - Criteria
1. Find all integers in the le:
•Add them to a Python list, sort largest to smallest (example: [10, 4, 1]), and you’ll
notice that there is only one very large number (compared to others) in your list.
That’s our encoded treasure!
•Use the decoder (we have already given in the code) to reveal it, then replace the
encoded number with the treasure. (Hint: It might be something related to our
course.) After that, write the Python list out to le ‘animals.out’
•Add all the integers ( except the encoded treasure) together & take the natural log of
the total. Write this value to le ‘animals.out’ using exponential format with 2 digits
of precision (example: 1.23e+05)

2. Some penguin’s have been spelled incorrectly as ‘pengiun’. Fix the spelling,
and write out a corrected version of the le to ‘ xed.out’

79
fi
fi
Homework #1: Welcome to Madagascar - Criteria
3. Create a Python dictionary with animal names as keys and a count of
occurrences as values. Example: {‘lion’: 5, ‘hippo’:11}. Write the nal
dictionary out to le ‘animals.out’

(note, the penguin count should include the xed "pengiun's")

4. Calculate the total number of animals encountered in the le, then write out
the total as a dictionary, as follows:

{'total_count': the_result} (Example:{'total_count': 1234})

80
fi
Homework #1: Welcome to Madagascar - Result Example
Final output le ‘animals.out’ should look something like this:

[Decode_Treasure,101, 80, 34, 5, …]

1.23e+05
{‘zebra’: 11, ‘hippo’: 145, …}
{‘total_count’: 1234}

81
fi
Next week:
- Back to EC!