CM2015 – Programming with Data [SIM – UOL]

Topic 1
Introduction to Data Programming

Learning Outcomes
After completing this topic and the recommended reading, you should be able
to:
• Set up and run Jupyter Notebook on a Windows, Mac or Linux operating
system.
• Use Jupyter Notebook to write and edit code.
• Write and explain simple Python programs using variables and
mathematical operators.

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 1

 CM2015 – Programming with Data [SIM – UOL]

1. Introduction to Data Programming

Data (definition)
• “Facts and statistics collected together for reference or analysis.”
[Oxford English Dictionary]

• “Information, especially facts or numbers, collected to be examined and

considered and used to help decision-making, or information in an
electronic form that can be stored and used by a computer.”
[Cambridge Dictionary]

• “Factual information (such as measurements or statistics) used as a basis

for reasoning, discussion, or calculation.”
[Merriam-Webster]

• “Data are individual facts, statistics, or items of information, often

numeric, that are collected through observation. In a more technical
sense, data are a set of values of qualitative or quantitative variables
about one or more persons or objects.”
[Wikipedia]

Information (definition)
• “Facts provided or learned about something or someone.”
[Oxford English Dictionary]

• “Facts or details about a situation, person, event, etc.”

[Cambridge Dictionary]

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 2

 CM2015 – Programming with Data [SIM – UOL]

• “Knowledge obtained from investigation, study, or instruction.”

[Merriam-Webster]

• “Knowledge communicated or received concerning a particular fact or

circumstance; knowledge gained through study, communication,
research, instruction, etc.”
[Dictionary.com]

Data vs. Information

• Data
o Raw, unorganised facts that need to be processed.
o Unusable until it is organised.
• Information
o Created when data is processed, organised, and structured.
o Needs to be put in an appropriate context in order to become
useful.

Data Science

Data Processing Information

Programming and Data

• Tasks to undertake for data programming
o Data collection
o Data processing (wrangling)
o Data visualisation

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 3

 CM2015 – Programming with Data [SIM – UOL]

o Train and apply algorithms from fields such as machine learning,

statistics, data mining, optimisation, image processing, etc.

• Programming
o The process of producing an executable computer program that
performs a specific task.
o The purpose is to find a sequence of instructions that automate the
implementation of the task for solving a given problem.

• Programming Language
o The source code of a program is written in one or more languages
that are intelligible to humans, rather than machine code, which is
directly executed by the CPU.

o Python
§ https://www.python.org/

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 4

 CM2015 – Programming with Data [SIM – UOL]

2. Introduction to Development Environments

Source-code Editors
• Source-code editor, or programming text editor, is a fundamental
programming tools designed specifically for editing source code of
computer programs.
• It highlights the syntax elements of your programs, and provides many
features that aid in your program development.
• Examples:
o Visual Studio Code [https://code.visualstudio.com/]
o Notepad++ (Windows only) [https://notepad-plus-plus.org/]
o Vim [https://www.vim.org/]
o Sublime Text (not open source) [https://www.sublimetext.com/]
o Atom [https://atom.io/]
o Emacs [https://www.gnu.org/software/emacs/]
o TextMate (Macs only) [https://macromates.com/]

o Jupyter
§ https://jupyter.org/

Integrated Development Environments (IDEs)

• Integrated development environment is a software application that
provides comprehensive facilities to computer programmers for software
development.
• An IDE normally consists of at least a source code editor, build
automation tools and a debugger.
• Examples:
o Spyder [https://www.spyder-ide.org/]

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 5

 CM2015 – Programming with Data [SIM – UOL]

o RStudio [https://rstudio.com]
o Eclipse [https://www.eclipse.org/]
o Microsoft Visual Studio [https://visualstudio.microsoft.com/vs/]
o Wing Python IDE [https://wingware.com]

Markdown / Markup Languages

• Markdown is a markup language that consists of a set of rules for adding
formatting elements to plain text documents
o Boldface, italics, headers, paragraphs, lists, code blocks, images,
etc.
o https://www.markdownguide.org/
• Invented by John Gruber
o The overriding design goal for Markdown’s formatting syntax is to
make it as readable as possible.
o The idea is that a Markdown-formatted document should be
publishable as-is, as plain text, without looking like it’s been
marked up with tags or formatting instructions
• Examples
o HTML; XML; LaTeX

Version Control Systems

• Version Control is a class of systems responsible for managing changes
to computer programs, documents, large websites, or other collections of
information.
• Version Control Systems (VCS) are software tools that help software
teams manage changes to source code over time.
o Undertakes the tedious task of keeping track of the changes to all
project’s files and who made them

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 6

 CM2015 – Programming with Data [SIM – UOL]

o Allows users to recover any previous version at any given time

• Examples:
o Subversion [https://subversion.apache.org]

o Git
§ https://git-scm.com/

o GitHub
§ https://github.com/

Package/Environment Manager
• Package manager, or package management system, is a collection of
software tools that automates the process of installing, upgrading,
configuring, and removing computer programs for a computer in a
consistent manner. Also deals with packages, distributions of software
and data in archive files.
• Environment manager enables personalised, consistent desktop
environments without cumbersome roaming profiles or scripts.
• Example:

o Anaconda
§ https://www.anaconda.com/

Installing Anaconda
• Go to Anaconda, download Anaconda Individual Edition
o https://www.anaconda.com/products/distribution
• Packages include
o conda
§ package management system

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 7

 CM2015 – Programming with Data [SIM – UOL]

o pandas, scikit-learn, nltk

§ packages for data science
o Anaconda Navigator
§ a graphical user interface
o QtConsole
§ an interactive Python environment
o Spyder
§ a standard cross-platform IDE for Python
o Jupyter Notebook
§ an interactive web-browser based application for creating
and sharing code

Package Installer for Python (pip)

• pip is the de facto and recommended package-management system
written in Python and is used to install and manage software packages.
• It connects to an online repository of public packages, called the Python
Package Index.
• We use pip to install packages from the Python Package Index
• Examples
o pip install beautifulsoup4
o pip install -r dependencies
§ Install packages based on dependencies in code
o pip freeze
§ See all the packages installed

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 8

 CM2015 – Programming with Data [SIM – UOL]

3. Introduction to Python

• Open-source, interpreted, high-level, object-oriented, general-purpose,

easy to download, write and read
• Named for the British comedy group Monty Python
• Simpler language, allow us to focus less on the language and more on
problem solving
• Many of the best parts of other languages are included
o Data structures
o Controls
o Many packages for common tasks

Variables
• Variable is a named piece of memory whose value can change during the
running of the program; constant is a value which cannot change as the
program runs.
o Python doesn’t use constant
• We use variable names to represent objects (number, data structures,
functions, etc.) in our program, to make our program more readable.
o All variable names must be one word, spaces are never allowed.
o Can only contain alpha-numeric characters and underscores.
o Must start with a letter or the underscores character.
o Cannot begin with a number.
o Case-sensitive
o Standard way for most things named in Python is lower with under
§ Lower case with separate words joined by an underscore

Comments

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 9

 CM2015 – Programming with Data [SIM – UOL]

• Not processed by the computer, valued by other programmers.

• Header comments
o Appear at beginning of a program or a module
o Provide general information
• Step comments or in-line comments
o Appear throughout program
o Explain the purpose of specific portion of code
• Often comments delineated by
o // comment goes here
o /* comment goes here */
o # Python uses this

Python Operations
• Assignment Operator
o “=”
o Example:
§ a = 67890/12345
# compute the ratio, store the result in ram, assign to a
# the value of a is 5.499392
§ b=a
# b pointing to value of a

• Output
o “print()”
o Example:
§ print(‘Hello World!’) # print the string literals
§ print(a) # print the value of a

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 10

 CM2015 – Programming with Data [SIM – UOL]

Data Types in Python

• Declaration of variables in Python is not needed
o Use an assignment statement to create a variable

• Float
o Stores real numbers
o a = 4.6
o print(type(a))

• Integer
o Stores integers
o b = 10
o print(type(b))

• Conversion
o int(a) # convert float to int => 4
o float(b) # convert int to float => 10.0

• Basic arithmetic operators

o 3+2 # Addition => 5
o 5–2 # Subtraction => 3
o 5 * -2 # Multiplication => -10
o 5 / 2.5 # Division => 2.0
o 2**2 # Exponentiation => 4
o 10 % 3 # Modulus => 1

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 11

 CM2015 – Programming with Data [SIM – UOL]

o 10 // 3 # Floor Division => 3

• String
o Stores strings
o phrase = ‘All models are wrong, but some are useful.’
o phrase[0:3] # slicing character 0 up to 2
=> All
o phrase.find(‘models’) # find the starting index of word
=> 4
o phrase.find(‘right’) # word not found
=> -1
o phrase.lower() # set to lower case
=> ‘all models are wrong, but
some are useful.’
o phrase.upper() # set to upper case
=> ‘ALL MODELS ARE
WRONG, BUT SOME ARE
USEFUL.’
o phrase.split(‘,’) # split strings into list, base on delimiter
=> [‘All models are wrong’,
‘ but some are useful.’]

• Boolean
o Stores logical or Boolean values of TRUE or FALSE
o k=1>3
o print(k)

o print(type(k))

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 12

 CM2015 – Programming with Data [SIM – UOL]

• Logical operators
o Conjunction (AND): “and”
o Disjunction (OR): “or”
o Negation (NOT): “not”
a b a and b a or b not a
T T T T F
T F F T F
F T F T T
F F F F T

Data Structures in Python

• Tuples
o Store ordered collection of objects
o Immutable: elements cannot be modified, added or deleted
o Written with round brackets “( )”
§ tuple1 = (“apple”, “banana”, “cherry”, “orange”, “kiwi”,
“melon”, “mango”)
§ tuple2 = (“Handsome Koh”, 4896, 13.14, True)
o Accessing elements by indexing
§ tuple1[0] # first element index => ‘apple’
§ tuple1[-1] # last element index => ‘mango’
§ tuple1[2:5] # range of elements => (‘cherry’, ‘orange’,
‘kiwi’)

• Lists

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 13

 CM2015 – Programming with Data [SIM – UOL]

o Store ordered collection of objects; mutable

o Written with square brackets “[ ]”
§ list1 = [“apple”, “banana”, “cherry”]
§ list2 = [“Handsome Koh”, 4896, 13.14, True]
o Changing elements
§ list1.append(“orange”) # add to last position
=> [‘apple’, ‘banana’, ‘cherry’,
‘orange’]
§ list1[2] = “coconut” # modify index element
=> [‘apple’, ‘banana’, ‘coconut’,
‘orange’]
§ list1.remove(“apple”) # delete elements
=> [‘banana’, ‘coconut’, ‘orange’]
§ list1.insert(2, “durian”) # insert element at position
=> [‘banana’, ‘coconut’, ‘durian’,
‘orange’]

• Sets
o Store unordered, unindexed, nonduplicates collection of objects
o Written with square brackets “{ }”
§ set1 = {“apple”, “banana”, “cherry”}
§ set2 = {“apple”, “samsung”}
o Set operations
§ set1.union(set2) # Union both sets
=> {‘apple’, ‘banana’, ‘cherry’,
‘samsung’}
§ set1.intersection(set2) # Intersect both sets
=> {‘apple’}

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 14

 CM2015 – Programming with Data [SIM – UOL]

• Dictionaries
o Store unordered collection of objects
o Written with square brackets “{ }”, and “key:value” pair
§ thisdict = {“brand”: “Ford”, “model”: “Mustang”,
“year”: 1964}
o Accessing/modifying elements by key name
§ thisdict[“model”] => ‘Mustang’
§ thisdist[“year”] = 2018 => {‘brand’: ‘Ford’,
thisdist[“color”] = “red” ‘model’: ‘Mustang’,
‘year’: 2018,
‘color’: ‘red’}

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 15

 CM2015 – Programming with Data [SIM – UOL]

4. Introduction to Jupyter Notebook

• Jupyter Notebook is a web-based interactive computing platform.

• “Julia” + “Python” + “R”
• Integrate code and output into a single document contains:
o Live code, mathematical equations, visualisations, and
explanatory/narrative text, interactive dashboards and other media
• Can be easily shared
o Notebook files have “.ipynb” extension
o Export to “.html” and “.pdf” extensions

• Launch “Jupyter Notebook” from “Anaconda Navigator”

• Create new notebook
o “File” à “New Notebook” à “Python 3”
• Exporting notebook
o “File” à “Download as” à “HTML (.html)”
o “File” à “Print Preview” (for PDF)
• Shutting Down Jupyter
o “File” à “Close and Halt”
o Quit

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 16

 CM2015 – Programming with Data [SIM – UOL]

5. Exercises

1.301 Practice Exercises (Coursera)

• Refers to “1.301 part-1.html”

1.302 A bit more Python – our first downloadable notebook!

(Coursera)
• Refers to “1.302 pythonPractice.html”

1.304 World’s Population

• Refers to “1.304 Topic 1 - Lab.html”

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 17

 CM2015 – Programming with Data [SIM – UOL]

6. Practice Quiz
• Work on Practice Quiz 01 posted on Canvas.

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 18

 CM2015 – Programming with Data [SIM – UOL]

Useful Resources
•
o http://

Prepared by Koh Chung Haur @ 2022 (version 2022.1) 19