Scribd
Upload
28 views42 pages

Chapter 2

The document discusses several topics related to developing Python packages, including: - Installing your own package by placing code in a directory with an __init__.py file and importing it in another script. - The typical directory structure for a package with subpackages, including the placement of setup.py. - What setup.py is used for and the basic information it contains. - How to specify dependencies in setup.py using install_requires and control dependency versions. - Creating an environment file with pip freeze to specify package requirements for developers.

Uploaded by

2022ac05722
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
28 views42 pages

Chapter 2

The document discusses several topics related to developing Python packages, including: - Installing your own package by placing code in a directory with an __init__.py file and importing it in another script. - The typical directory structure for a package with subpackages, including the placement of setup.py. - What setup.py is used for and the basic information it contains. - How to specify dependencies in setup.py using install_requires and control dependency versions. - Creating an environment file with pip freeze to specify package requirements for developers.

Uploaded by

2022ac05722
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 42

Installing your own

package
D E V E L O P I N G P Y T H O N PA C K A G E S

James Fulton
Climate informatics researcher
Why should you install your own package?
Inside example_script.py Directory tree for package with subpackages

import mysklearn home/


|-- mysklearn <-- in same directory
| |-- __init__.py
| |-- preprocessing
| | |-- __init__.py
| | |-- normalize.py
| | |-- standardize.py
| |-- regression
| | |-- __init__.py
| | |-- regression.py
| `-- utils.py
|-- example_script.py <-- in same directory

DEVELOPING PYTHON PACKAGES


Why should you install your own package?
Inside example_script.py Directory tree

import mysklearn home/


|-- mypackages
| |-- mysklearn <---
Traceback (most recent call last):
| |-- __init__.py
File "<stdin>", line 1, in <module>
| |-- preprocessing
ModuleNotFoundError: No module named 'mysklearn'
| | |-- __init__.py
| | |-- normalize.py
| | |-- standardize.py
| |-- regression
| |-- __init__.py
| |-- regression.py
`-- myscripts
`-- example_script.py <---

DEVELOPING PYTHON PACKAGES


setup.py
Is used to install the package
Contains metadata on the package

DEVELOPING PYTHON PACKAGES


Package directory structure
Directory tree for package with subpackages

mysklearn/
|-- __init__.py
|-- preprocessing
| |-- __init__.py
| |-- normalize.py
| |-- standardize.py
|-- regression
| |-- __init__.py
| |-- regression.py
|-- utils.py

DEVELOPING PYTHON PACKAGES


Package directory structure
Directory tree for package with subpackages

mysklearn/ <-- outer directory


|-- mysklearn <--- inner source code directory
|-- __init__.py
|-- preprocessing
| |-- __init__.py
| |-- normalize.py
| |-- standardize.py
|-- regression
| |-- __init__.py
| |-- regression.py
|-- utils.py

DEVELOPING PYTHON PACKAGES


Package directory structure
Directory tree for package with subpackages

mysklearn/ <-- outer directory


|-- mysklearn <--- inner source code directory
| |-- __init__.py
| |-- preprocessing
| | |-- __init__.py
| | |-- normalize.py
| | |-- standardize.py
| |-- regression
| | |-- __init__.py
| | |-- regression.py
| |-- utils.py
|-- setup.py <-- setup script in outer

DEVELOPING PYTHON PACKAGES


Inside setup.py
# Import required functions
from setuptools import setup

# Call setup function


setup(
author="James Fulton",
description="A complete package for linear regression.",
name="mysklearn",
version="0.1.0",
)

version number = (major number) . (minor number) . (patch number)

DEVELOPING PYTHON PACKAGES


Inside setup.py
# Import required functions
from setuptools import setup, find_packages

# Call setup function


setup(
author="James Fulton",
description="A complete package for linear regression.",
name="mysklearn",
version="0.1.0",
packages=find_packages(include=["mysklearn", "mysklearn.*"]),
)

DEVELOPING PYTHON PACKAGES


Editable installation
pip install -e . Directory tree for package with subpackages

. = package in current directory mysklearn/ <-- navigate to here


|-- mysklearn
-e = editable | |-- __init__.py
| |-- preprocessing
| | |-- __init__.py
| | |-- normalize.py
| | |-- standardize.py
| |-- regression
| | |-- __init__.py
| | |-- regression.py
| |-- utils.py
|-- setup.py

DEVELOPING PYTHON PACKAGES


Let's practice!
D E V E L O P I N G P Y T H O N PA C K A G E S
Dealing with
dependencies
D E V E L O P I N G P Y T H O N PA C K A G E S

James Fulton
Climate informatics researcher
What are dependencies?
Other packages you import inside your package
Inside mymodule.py :

# These imported packages are dependencies


import numpy as np
import pandas as pd
...

DEVELOPING PYTHON PACKAGES


Adding dependencies to setup.py
from setuptools import setup, find_packages

setup(
...
install_requires=['pandas', 'scipy', 'matplotlib'],
)

DEVELOPING PYTHON PACKAGES


Controlling dependency version
from setuptools import setup, find_packages

setup(
...
install_requires=[
'pandas>=1.0',
'scipy==1.1',
'matplotlib>=2.2.1,<3'
],
)

DEVELOPING PYTHON PACKAGES


Controlling dependency version
from setuptools import setup, find_packages

setup(
...
install_requires=[
'pandas>=1.0', # good
'scipy==1.1', # bad
'matplotlib>=2.2.1,<3' # good
],
)

Allow as many package versions as possible


Get rid of unused dependencies

DEVELOPING PYTHON PACKAGES


Python versions
from setuptools import setup, find_packages

setup(
...
python_requires='>=2.7, !=3.0.*, !=3.1.*',
)

DEVELOPING PYTHON PACKAGES


Choosing dependency and package versions
Check the package history or release notes
e.g. the NumPy release notes

Test different versions

DEVELOPING PYTHON PACKAGES


Making an environment for developers
pip freeze

alabaster==0.7.12
appdirs==1.4.4
argh==0.26.2
...
wrapt==1.11.2
yapf==0.29.0
zipp==3.1.0

DEVELOPING PYTHON PACKAGES


Making an environment for developers
Save package requirements to a file mysklearn/
|-- mysklearn
pip freeze > requirements.txt | |-- __init__.py
| |-- preprocessing
| | |-- __init__.py
Install requirements from file
| | |-- normalize.py
| | |-- standardize.py
pip install -r requirements.txt
| |-- regression
| | |-- __init__.py
| | |-- regression.py
| |-- utils.py
|-- setup.py
|-- requirements.txt <-- developer environment

DEVELOPING PYTHON PACKAGES


Let's practice!
D E V E L O P I N G P Y T H O N PA C K A G E S
Including licences
and writing
READMEs
D E V E L O P I N G P Y T H O N PA C K A G E S

James Fulton
Climate informatics researcher
Why do I need a license?
To give others permission to use your code

DEVELOPING PYTHON PACKAGES


Open source licenses
Find more information here
Allow users to
use your package

modify your package

distribute versions of your package

1 https://choosealicense.com

DEVELOPING PYTHON PACKAGES


What is a README?
The "front page" of your package
Displayed on Github or PyPI

DEVELOPING PYTHON PACKAGES


What to include in a README
README sections

Title

Description and Features


Installation

Usage examples

Contributing

License

DEVELOPING PYTHON PACKAGES


README format
Markdown (commonmark) reStructuredText

Contained in README.md file Contained in README.rst file

Simpler More complex

Used in this course and in the wild Also common in the wild

DEVELOPING PYTHON PACKAGES


Commonmark
Contents of README.md What it looks like when rendered

DEVELOPING PYTHON PACKAGES


Commonmark
Contents of README.md What it looks like when rendered

# mysklearn mysklearn
mysklearn is a package for complete
mysklearn is a package for complete linear
**linear regression** in Python.
regression in python.
You can find out more about this package
on [DataCamp](https://datacamp.com) You can find out more about this package on
DataCamp

DEVELOPING PYTHON PACKAGES


Commonmark
Contents of README.md What it looks like when rendered

# mysklearn mysklearn
mysklearn is a package for complete
mysklearn is a package for complete linear
**linear regression** in Python.
regression in python.
You can find out more about this package
on [DataCamp](https://datacamp.com) You can find out more about this package on
DataCamp
## Installation
You can install this package using
Installation
You can install this package using

DEVELOPING PYTHON PACKAGES


Commonmark
Contents of README.md What it looks like when rendered

# mysklearn mysklearn
mysklearn is a package for complete
mysklearn is a package for complete linear
**linear regression** in Python.
regression in python.
You can find out more about this package
on [DataCamp](https://datacamp.com) You can find out more about this package on
DataCamp
## Installation
You can install this package using
Installation
``` You can install this package using
pip install mysklearn
``` pip install mysklearn

DEVELOPING PYTHON PACKAGES


Adding these files to your package
Directory tree for package with subpackages

mysklearn/
|-- mysklearn
| |-- __init__.py
| |-- preprocessing
| | |-- ...
| |-- regression
| | |-- ...
| |-- utils.py
|-- setup.py
|-- requirements.txt
|-- LICENSE <--- new files
|-- README.md <--- added to top directory

DEVELOPING PYTHON PACKAGES


MANIFEST.in
Lists all the extra files to include in your package distribution.

DEVELOPING PYTHON PACKAGES


MANIFEST.in
Contents of MANIFEST.in mysklearn/
|-- mysklearn
include LICENSE | |-- __init__.py
include README.md | |-- preprocessing
| | |-- ...
| |-- regression
| | |-- ...
| |-- utils.py
|-- setup.py
|-- requirements.txt
|-- LICENSE
|-- README.md
|-- MANIFEST.in <---

DEVELOPING PYTHON PACKAGES


Let's practice!
D E V E L O P I N G P Y T H O N PA C K A G E S
Publishing your
package
D E V E L O P I N G P Y T H O N PA C K A G E S

James Fulton
Climate informatics researcher
PyPI
Python Package Index

pip installs packages from here

Anyone can upload packages


You should upload your package as soon as it might be useful

1 https://pypi.org/

DEVELOPING PYTHON PACKAGES


Distributions
Distribution package - a bundled version of your package which is ready to install.
Source distribution - a distribution package which is mostly your source code.

Wheel distribution - a distribution package which has been processed to make it faster to
install.

DEVELOPING PYTHON PACKAGES


How to build distributions
python setup.py sdist bdist_wheel mysklearn/
|-- mysklearn
sdist = source distribution |-- setup.py
bdist_wheel = wheel distribution |-- requirements.txt
|-- LICENSE
|-- README.md
|-- dist <---
| |-- mysklearn-0.1.0-py3-none-any.whl
| |-- mysklearn-0.1.0.tar.gz
|-- build
|-- mysklearn.egg-info

DEVELOPING PYTHON PACKAGES


Getting your package out there
Upload your distributions to PyPI mysklearn/
|-- mysklearn
twine upload dist/* |-- setup.py
|-- requirements.txt
Upload your distributions to TestPyPI |-- LICENSE
|-- README.md
twine upload -r testpypi dist/* |-- dist
| |-- mysklearn-0.1.0-py3-none-any.whl
| |-- mysklearn-0.1.0.tar.gz
|-- build
|-- mysklearn.egg-info

DEVELOPING PYTHON PACKAGES


How other people can install your package
Install package from PyPI

pip install mysklearn

Install package from TestPyPI

pip install --index-url https://test.pypi.org/simple


--extra-index-url https://pypi.org/simple
mysklearn

DEVELOPING PYTHON PACKAGES


Let's practice!
D E V E L O P I N G P Y T H O N PA C K A G E S

You might also like