02 - The Python Ecosystem For ML
02 - The Python Ecosystem For ML
Hands on Python
for Machine Learning
25/01/2022
The Python “Ecosystem” for ML
# Table of Contents
› The Python Ecosystem
› Anaconda Python - Introduction
› Anaconda Python (IE) - Installation and components overview
L02 – The Python Ecosystem for ML
M. Zappatore
› Conclusive Remarks
3
M. Zappatore
› Library ~= package = collection of code written by someone else
L02 – The Python Ecosystem for ML
#
Scientific Python Stack
5
M. Zappatore
6
› Enthought Canopy
(end of life)
M. Zappatore
› WinPython:
(Windows only)
7
M. Zappatore
› Google colaboratory:
(Cloud-based, Google-powered environment for Jupyter notebooks)
8
# Python Ecosystem
› In this course, we are going to use Ananconda Python and Google
Colaboratory. Why?
L02 – The Python Ecosystem for ML
M. Zappatore
9
Anaconda Python
Introduction
1
0
# Anaconda Python Individual Edition (IE)
› Open source
› Multi-platform
› Modular
L02 – The Python Ecosystem for ML
M. Zappatore
1
1
# How to get it
› https://www.anaconda.com/products/individual
L02 – The Python Ecosystem for ML
M. Zappatore
1
2
# Some clarifications before installing it
› Definition: Anaconda is a complete distribution that includes Python along with
binaries for hundreds of third-party open-source libraries and projects. It comes
with a package management system as well as an environment management
system.
L02 – The Python Ecosystem for ML
M. Zappatore
1
3
# What is a packet management system?
› It is a tool used to download, install, manage and remove Python packages.
› It also keeps an accurate track of package versions and dependencies.
› Each package manager comes with a simple command structure ( almost all
L02 – The Python Ecosystem for ML
package manager share the same working principles, even if their commands can
vary).
› Every Python distribution is bundled with a package manager.
› The two most widely adopted package managers are conda and pip.
M. Zappatore
1
4
# What is an environment management system?
› It is a tool used to create/clone, manage and remove Conda environments.
M. Zappatore
either clone it or create an empty new one that you are going to populate with
new installaed Python packages as soon as you need them.
1
5
# Conda
› Open-source, multi-platform package management system and environment
management system.
› It quickly installs, runs and updates packages and their dependencies.
› It easily creates, saves, loads and switches between environments on your local
L02 – The Python Ecosystem for ML
computer.
› As a package manager, it helps you find and install packages. If you need a
package that requires a different version of Python, you do not need to switch to a
different environment manager, because conda is also an environment manager.
› With just a few commands, you can set up a totally separate environment to run
that different version of Python, while continuing to run your usual version of
Python in your normal environment.
› NOTE (for legacy Python developers): conda works and improves the traditional
M. Zappatore
combined usage of pip and virtualenv
1
6
# Miniconda
› It is a slimmed-down distribution version of Anaconda. It has all the
components of the Anaconda distribution, except the 200+ pre-installed data
science applications.
› Following a successful installation of the Miniconda distribution, you can use the
L02 – The Python Ecosystem for ML
command line to install the packages you need individually. As such, Miniconda
offers all the benefits of the Anaconda distribution with minimal space
requirements.
› Owing to the smaller file size and reduced disk space requirements, Miniconda is
relatively faster to install as compared to the Anaconda distribution.
M. Zappatore
1
7
# Anaconda (1)
› Anaconda is an open-source python distribution. It is purpose-built for such
applications as machine learning, data science, and large-scale data
processing.
› It includes the core Python language, along with more than 350+ packages and a
L02 – The Python Ecosystem for ML
M. Zappatore
1
8
# Anaconda (2)
› These are some of the Extra packages you will get on your Anaconda distribution,
depending on the installer you choose:
– 350+ packages will be automatically installed with every Anaconda installation
– More than 7,500 open-source packages, including R, will be available at the
L02 – The Python Ecosystem for ML
Anaconda repository. You may download and install any of these additional
packages individually using the Conda install command feature
– Access to thousands of packages from the Anaconda Cloud as well.
– Using the conda build command feature, you can customize your packages,
and then share them by uploading to such repositories as PyPI or Anaconda
Cloud.
– If you have the pip package manager installed with your Anaconda distribution,
you can download additional packages through the pip install command.
However, you should always give preference to conda packages as opposed to
pip packages.
M. Zappatore
1
9
# Which one should I install and use?
› Choose Miniconda if you
– Only need to install just a few specific packages
– Require fast access to Python and the Conda commands, and would rather sort
the other packages later on
L02 – The Python Ecosystem for ML
– Do not have the extra time and disk space required to install the 350+ packages
on the Anaconda distribution (typically, you will not need most of them)
M. Zappatore
– Need to address several activities (ML/DL, DS, etc.)
2
0
# What should I expect after installing one of them?
L02 – The Python Ecosystem for ML
M. Zappatore
Source: Whimsical.com (2018)
L02 – The Python Ecosystem for ML
M. Zappatore
L02 – The Python Ecosystem for ML
M. Zappatore
23
M. Zappatore
L02 – The Python Ecosystem for ML
M. Zappatore
L02 – The Python Ecosystem for ML
#
Installation
6
2
M. Zappatore
L02 – The Python Ecosystem for ML
#
Anaconda Navigator
7
2
M. Zappatore
L02 – The Python Ecosystem for ML
#
Test the installation via terminal/prompt
8
2
M. Zappatore
2
9
# (env_name)conda list
› Name: package name
› Version: version number of the package
› Build: Python version the package is made for
L02 – The Python Ecosystem for ML
M. Zappatore
3
0
# (env_name)python
› Show the running version of Python as well as whether
Anaconda is available
L02 – The Python Ecosystem for ML
Python Anaconda is
version available
M. Zappatore
Write exit()here to
leave the Python prompt
3
1
# (env_name)conda env list
› Show list of available environments
L02 – The Python Ecosystem for ML
M. Zappatore
name environment path
3
2
# Anaconda Navigator is definitely easier
L02 – The Python Ecosystem for ML
Environment
manager
M. Zappatore
3
3
# Anaconda Navigator is definitely easier
Package
filtering options
L02 – The Python Ecosystem for ML
List of
environments
List of
packages
M. Zappatore
3
4
# How to install new packages
2. Refine the
search by
adding some
meaningful
1. Search for terms (if
not installed available)
packages only
L02 – The Python Ecosystem for ML
M. Zappatore
4. Click on APPLY
3
5
# Anaconda useful tools
› Jupyter Notebook: allows you creating and editing documents that
display the input and output of a Python / R script. Once saved,
notebooks can be shared.
L02 – The Python Ecosystem for ML
M. Zappatore
Explorer, Variable Explorer, IPython Console, Plotting Window, etc.).
36
3. Assign a name to
1. Open the the new environment
environments
L02 – The Python Ecosystem for ML
tab
4. Check the
environment’s path
5. Specify Python
version
M. Zappatore
2/A. Click on CREATE
to start a new empty
environment with core
Python packages only
3
8
# 2. Check the new environment
2. Select INSTALLED
Python packages only
L02 – The Python Ecosystem for ML
1. The new
environment
is already
selected
M. Zappatore
3. There are only 11 core
Python packages pre-installed
3
9
# 3. Install additional Packages
› You should install the following packages:
1. NumPy: N-dimensional array for numerical computation
2. Pandas: data structures and data analysis toolkit
L02 – The Python Ecosystem for ML
M. Zappatore
4
0
# 4. Install additional tools
2. Ensure the new 3. Install Jupyter
environment is selected Notebook
1. Open the
HOME tab
L02 – The Python Ecosystem for ML
3. Install
JupyterLab
M. Zappatore
4. Install
Spyder IDE
4
1
# 5. Start programming…
› …by using either Spyder, Jupyter Notebook or JupyterLab
L02 – The Python Ecosystem for ML
M. Zappatore
4
2
# 5. Start programming…
› For all the following tests we are going to use this Python script, which plots
a sine wave defined within the time window [0;1] with step 0.1.
› The script requires importing two packages:
L02 – The Python Ecosystem for ML
– Numpy: to define the random values of the sine wave (X axis) and its
values on the Y axis
– Matplotlib: to plot the curve
› Therefore, in Anaconda, we must check whether these two packages are
already installed in the active environment
M. Zappatore
4
3
# 5. Start programming…
import numpy as np
Required import matplotlib.pyplot as plot
packages # Simple textual output
print("\nThis simple Python script plots a sine wave\ndefined within a time window [0;1] with step 0.1 \n")
# Plot a sine wave using time and amplitude obtained for the sine wave
plot.plot(time, amplitude)
M. Zappatore
plot.axhline(y=0, color='k')
plot.show()
M. Zappatore
1. Install NumPy
2. Install Matplotlib
4
5
# 5. Start programming…
L02 – The Python Ecosystem for ML
M. Zappatore
dependencies and
click on Apply
46
Jupyter Notebook
Overview, setup and test
4
7
# Overview
› Jupyter Notebook is an interface that you will use to
interact with data files and Python software libraries, much
like you use a web browser to access files and software on
L02 – The Python Ecosystem for ML
web servers.
› Jupyter Notebook runs inside of your web browser (though
you will use it to access programs and data files on your
local hard drive, not the Web).
› In order to use Jupyter Notebook, just open Anaconda
Navigator, select one environment, check whether it is
already installed in the selected environment and press
M. Zappatore
launch.
4
8
# Notebooks
› A “Notebook” or “Notebook Document” is a file that contains both code and
rich text elements, such as figures, links, equations, ...
› Each notebook element is called a cell (or a block)
L02 – The Python Ecosystem for ML
› The combination of code and text, allows putting together an analysis description
and its results.
› Each notebook can be executed to visualise the results of the code snippets it
hosts, provided that a Python Kernel has been defined.
› Kernels are programming language specific processes that run independently and
interact with the Jupyter Notebook Applications and their user interfaces.
ipykernel is the reference Jupyter kernel built on top of IPython, providing a
powerful environment for interactive computing in Python.
M. Zappatore
4
9
# Summary
› Jupyter Notebook is a server-client app that allows you
editing and running your notebooks via a Web
browser.
L02 – The Python Ecosystem for ML
M. Zappatore
5
0
# Notebooks + Kernels
› Jupyter Notebook is a server-client app that allows you
editing and running your notebooks via a Web
browser. It can be executed offline or installed on a
L02 – The Python Ecosystem for ML
M. Zappatore
5
1
# Jupyter Notebook UI
› When you launch Jupyter Notebook, a new browser
window opens the Jupyter dashboard. The dashboard
shows you the files in the current folder (under the Files
L02 – The Python Ecosystem for ML
M. Zappatore
lecture.
5
2
# Create a course-dedicated folder
Notice the URL pointing
to the localhost
L02 – The Python Ecosystem for ML
3. Click on
New and
2. Ensure the root/home then select
folder is selected Folder
M. Zappatore
5
3
# Create a course-dedicated folder
5. Select
Rename
L02 – The Python Ecosystem for ML
4. Select the
untitled folder
just created 6. Rename it into
something meaningful
7. Click on
Rename
M. Zappatore
5
4
# Create a Notebook
L02 – The Python Ecosystem for ML
M. Zappatore
5
5
# Create a Notebook
Assign a Name to
the notebook
L02 – The Python Ecosystem for ML
M. Zappatore
L02 – The Python Ecosystem for ML
#
Execute a code block in your Notebook
6
5
M. Zappatore
L02 – The Python Ecosystem for ML
#
Toolbar commands
7
5
M. Zappatore
L02 – The Python Ecosystem for ML
#
Cell menu
8
5
M. Zappatore
5
9
# Cell/Block types
› Code: contains code snippet. Once compiled produces the
corresponding output.
› Markdown: normal text. Once compiled produces a textual block.
L02 – The Python Ecosystem for ML
M. Zappatore
6
0
# Execute a markdown block in your Notebook
Execute all cells by
clicking on
Cell Run All
L02 – The Python Ecosystem for ML
M. Zappatore
L02 – The Python Ecosystem for ML
#
Execute a markdown block in your Notebook
1
6
M. Zappatore
6
2
# Useful tips & tricks
› Click TAB after one object to receive suggestions about object’s
attributes (TAB completion function)
› How to trigger: object.<click TAB>
L02 – The Python Ecosystem for ML
M. Zappatore
6
3
# Useful tips & tricks
› Place a ? after an object and run the cell to receive help about that
object. This function is called Object Introspection
› How to trigger: object?
L02 – The Python Ecosystem for ML
M. Zappatore
6
4
# Useful tips & tricks
› Write %whos and run in order to have a list of currently active
variables
L02 – The Python Ecosystem for ML
M. Zappatore
6
5
# Useful tips & tricks
› Other usefule tips:
› %run <script filename>: execute Python script
› %cd: change the current working directory
L02 – The Python Ecosystem for ML
M. Zappatore
6
6
# Special characters for instructions
› #: indicate comments
› ;: used to suppress automatic output printing (do not use it to
separate multiple statements on the same line!)
L02 – The Python Ecosystem for ML
M. Zappatore
L02 – The Python Ecosystem for ML
#
File downloading
7
6
M. Zappatore
L02 – The Python Ecosystem for ML
#
Kernel restart, output cleansing
8
6
M. Zappatore
6
9
# Cell deletion
To edit an already
executed cell, just
double-click on it
L02 – The Python Ecosystem for ML
M. Zappatore
L02 – The Python Ecosystem for ML
#
File uploading
0
7
M. Zappatore
L02 – The Python Ecosystem for ML
#
Closing a notebook
1
7
M. Zappatore
7
2
# Preliminary Note
› Without installing the required
packages (or checking for their
availability), the output of the
script execution on Jupyter
Notebook will provide a
L02 – The Python Ecosystem for ML
ModuleNotFoundError
M. Zappatore
7
3
# Test
› By exploiting
Markdown
cells, we can
avoid putting
print lines
L02 – The Python Ecosystem for ML
within the
script, thus
dedicating code
cells to
computational
activities only.
› Now let us write
the script and
execute it.
M. Zappatore
7
4
# Test
L02 – The Python Ecosystem for ML
M. Zappatore
75
JupyterLab
Overview, setup and test
7
6
# Overview
› JupyterLab provides flexible building blocks for interactive, exploratory computing.
While JupyterLab has many features found in traditional integrated development
environments (IDEs), it remains focused on interactive, exploratory computing.
› The JupyterLab interface consists of a main work area containing tabs of
L02 – The Python Ecosystem for ML
documents and activities, a collapsible left sidebar, and a menu bar. The left
sidebar contains a file browser, the list of running kernels and terminals, the
command palette, the notebook cell tools inspector, and the tabs list.
› JupyterLab sessions always reside in a workspace. Workspaces contain the state
of JupyterLab: the files that are currently open, the layout of the application areas
and tabs, etc. Workspaces can be saved on the server with named workspace
URLs. To learn more about URLs in Jupyterlab, visit JupyterLab URLs.
M. Zappatore
7
7
# Overview Menu bar
Main work area
https://youtu.be/Ka8qS7CO1XQ
File
explorer
List of
running
kernels
L02 – The Python Ecosystem for ML
Command
palette
Cell tool
inspector
Tabs list
Left
sidebar
M. Zappatore
L02 – The Python Ecosystem for ML
File
explorer
Overview
Tabs list
8
7
M. Zappatore
7
9
# Test
1. Select the
folder where we
are going to place
2/B. ...or creating
all the scripts for
L02 – The Python Ecosystem for ML
a new notebook or
this course
another file type
2/A. Decide
whether opening
an already
existing script...
M. Zappatore
L02 – The Python Ecosystem for ML
#
Test
3/A. Execute cells one by one
0
8
M. Zappatore
L02 – The Python Ecosystem for ML
#
Test
M. Zappatore
L02 – The Python Ecosystem for ML
#
Test
2
8
M. Zappatore
83
Spyder IDE
Overview and test
8
4
# Overview
› Spyder is a free and open source scientific environment written in
Python, for Python, and designed by and for scientists, engineers
and data analysts.
L02 – The Python Ecosystem for ML
M. Zappatore
Dark Theme is available, too.
8
5
# Test
Command bar
Additional Tabs
(Plots, variable
explorer, file explorer)
L02 – The Python Ecosystem for ML
Console
(code output will be
shown here)
Main working area
M. Zappatore
(write the code here)
8
6
# Test
3. Execute
the script
2. Save the
script
L02 – The Python Ecosystem for ML
1. Write the
script
4. Check the
console
output
M. Zappatore
8
7
# Test
6. Check variables
L02 – The Python Ecosystem for ML
and their
type/size/values
M. Zappatore
Delete current plot Delete all plots 8
8
# Test
Previews
of all
generated
plots
L02 – The Python Ecosystem for ML
M. Zappatore
to delete output from console
89
Google Colab
Overview and test
9
0
# Overview
› Google colab or Colaboratory is a freemium product by Google
research, based on Jupyter Notebook.
› Almost all important Python libraries are pre-installed with Colab, so
L02 – The Python Ecosystem for ML
M. Zappatore
To open Google Colab, just reach the URL:
https://colab.research.google.com/?hl=en 9
1
#
L02 – The Python Ecosystem for ML
Tutorial
Colab features:
https://www.youtube.com/watch?v=rNgswRZ2C1Y
M. Zappatore
Jupyter-like files
9
2
# Cell Types
M. Zappatore
9
3
# Note
M. Zappatore
L02 – The Python Ecosystem for ML
#
Test
1.
File New notebook
4
9
M. Zappatore
9
5
# Test
2.
Assign a name to the colab
notebook (autosave is enabled
by default as in all applications
from Google Workspace)
L02 – The Python Ecosystem for ML
3.
Move the notebook on a
dedicated folder in your
Google Drive account
(suggested choice to make your
workspace orderly)
M. Zappatore
9
6
# Test
Text cell
menu bar
L02 – The Python Ecosystem for ML
Default markwodn
for headings and
bold type
Live preview
of the text
4. cell content
Start writing
in a Text cell
M. Zappatore
9
7
# Test
6.
Execute the
L02 – The Python Ecosystem for ML
code cell
5.
Write the script
in a code cell
M. Zappatore
9
8
# Test
M. Zappatore
9
9
# Test
L02 – The Python Ecosystem for ML
M. Zappatore
notebook in .ipynb (default for
Jupyter Notebooks) or .py file
format (default for Python scripts)
1
0
https://colab.research.google.com/notebooks/
L02 – The Python Ecosystem for ML
M. Zappatore
101
Conclusive Remarks
Final notes
on the discussed environment
1
0
› Use Spyder IDE if you are going to code and implement Python
scripts essentially locally (i.e., offline) and you need a complete UI to
control code writing, variables, plots, console outputs, etc.
L02 – The Python Ecosystem for ML
M. Zappatore
1
0
› Use Spyder IDE if you are going to code and implement Python
scripts essentially locally (i.e., offline) and you need a complete UI to
control code writing, variables, plots, console outputs, etc.
L02 – The Python Ecosystem for ML
M. Zappatore
1
0
M. Zappatore
Source: O.G.
1
0
M. Zappatore
Source: O.G.
L02 – The Python Ecosystem for ML
#
6
0
1
M. Zappatore
L02 – The Python Ecosystem for ML
#
7
0
1
M. Zappatore
108
End of lesson.