Programming in Python

(23DCE2101)

(As per NEP 2023 pattern)

Prof. S.M. Sabale
Head of Computer Engineering (Diploma)
Dr. Babasaheb Ambedkar Technological University, Lonere
 Unit VI
Introduction to Libraries and
Modules
• Overview of Python Standard Library
• Importing and using modules
• Exploring common modules (e.g., math, random, datetime)
• Basic Data Analysis with NumPy and Pandas
• Introduction to NumPy for numerical computing
• Introduction to Pandas for data manipulation and analysis
• Loading and exploring datasets
 Libraries in Python
• Normally, a library is a collection of books or is a room or place where many
books are stored to be used later. Similarly, in the programming world, a library is
a collection of precompiled codes that can be used later on in a program for some
specific well-defined operations. Other than pre-compiled codes, a library may
contain documentation, configuration data, message templates, classes, and
values, etc.

• A Python library is a collection of related modules. It contains bundles of code

that can be used repeatedly in different programs. It makes Python Programming
simpler and convenient for the programmer. As we don’t need to write the same
code again and again for different programs. Python libraries play a very vital role
in fields of Machine Learning, Data Science, Data Visualization, etc.
• The “Python library” contains several different kinds of components.
• It contains data types that would normally be considered part of the “core” of a
language, such as numbers and lists. For these types, the Python language core
defines the form of literals and places some constraints on their semantics, but
does not fully define the semantics. (On the other hand, the language core does
define syntactic properties like the spelling and priorities of operators.)
• The library also contains built-in functions and exceptions — objects that
can be used by all Python code without the need of an import statement.
Some of these are defined by the core language, but many are not
essential for the core semantics and are only described here.

• The bulk of the library, however, consists of a collection of modules. There

are many ways to dissect this collection. Some modules are written in C
and built in to the Python interpreter; others are written in Python and
imported in source form. Some modules provide interfaces that are highly
specific to Python, like printing a stack trace; some provide interfaces that
are specific to particular operating systems, such as access to specific
hardware; others provide interfaces that are specific to a particular
application domain, like the World Wide Web. Some modules are available
in all versions and ports of Python; others are only available when the
underlying system supports or requires them; yet others are available only
when a particular configuration option was chosen at the time when
Python was compiled and installed.
Core Built-ins and Standard Library Modules
• The term built-in has more than one meaning in Python. In most contexts, a built-
in means an object directly accessible to Python code without an import
statement. “Python built-ins” shows the mechanism that Python uses to allow
this direct access. Built-in types in Python include numbers, sequences,
dictionaries, sets, functions, classes, standard exception classes, and modules.
“The io Module” covers the (built-in, in v2) file type, and “Internal Types” covers
some other built-in types intrinsic to Python’s internal operation. core built-in
types (in “Built-in Types”) and built-in functions available in the module builtins
(named __builtins__ in v2) in “Built-in Functions” need to understand.

• As mentioned in “Python built-ins”, some modules are known as “built-in”

because they are an integral part of the Python standard library (even though it
takes an import statement to access them), as distinguished from separate,
optional add-on modules, also called Python extensions.
• Core built-in modules: namely, the modules sys in “The sys Module”, copy in “The
copy Module”, collections in “The collections Module”, functools in “The
functools Module”, heapq in “The heapq Module”, argparse in “The argparse
Module”, and itertools in “The itertools Module”. ...
 The Python Standard Library
• The Python Standard Library is a collection of exact syntax, token, and semantics
of Python. It comes bundled with core Python distribution.
• It is written in C, and handles functionality like I/O and other core modules. All
this functionality together makes Python the language it is.
• More than 200 core modules sit at the heart of the standard library. This library
ships with Python.
• But in addition to this library, you can also access a growing collection of several
thousand components from the Python Package Index (PyPI: repository of
software for the Python programming language.)
• Python Standard Library plays a very important role. Without it, the programmers
can’t have access to the functionalities of Python. But other than this, there are
several other libraries in Python that make a programmer’s life easier.
• Python standard library includes built-in functions like max(), min(), open(),
print() etc; dictionary methods like keys(), pop(), etc; list methods like append(),
index(), sort(), etc.; set methods like add(), difference, intersection(), etc; string
methods like lower(), replace, rfind(), etc; tuple methods like len(), max(), count(),
etc.
 Built-ins vs Python standard Library
• In Python, the main difference between built-ins and the standard library is that built-ins are
directly accessible to Python code, while the standard library requires an import statement
• Built-ins:
These functions are part of the interpreter and are directly accessible to Python code without
an import statement. For example, the print command is a built-in function that allows you to
print something to a terminal prompt. Some core built-in modules include sys, copy, collections,
and functools.
• Standard library:
This is a collection of built-in modules and other modules that provide standardized solutions
for common programming problems. The standard library includes modules written in C and
Python. For example, the xmlrpc.client and xmlrpc.server modules make it easy to implement
remote procedure calls. The email package provides tools for managing email messages.
• You don't need to download the standard library, but you do need to import it. The Python
installers for Windows usually include the entire standard library. For Unix-like operating systems,
Python is normally provided as a collection of packages.
 Commonly Used Python Libraries
• TensorFlow: This library was developed by Google in collaboration with the Brain
Team. It is an open-source library used for high-level computations. It is also used
in machine learning and deep learning algorithms. It contains a large number of
tensor operations. Researchers also use this Python library to solve complex
computations in Mathematics and Physics.
• Matplotlib: This library is responsible for plotting numerical data. And that’s why
it is used in data analysis. It is also an open-source library and plots high-defined
figures like pie charts, histograms, scatterplots, graphs, etc.
• Pandas: Pandas are an important library for data scientists. It is an open-source
machine learning library that provides flexible high-level data structures and a
variety of analysis tools. It eases data analysis, data manipulation, and cleaning of
data. Pandas support operations like Sorting, Re-indexing, Iteration,
Concatenation, Conversion of data, Visualizations, Aggregations, etc.
 Commonly Used Python Libraries
• Numpy: The name “Numpy” stands for “Numerical Python”. It is the commonly
used library. It is a popular machine learning library that supports large matrices
and multi-dimensional data. It consists of in-built mathematical functions for easy
computations. Even libraries like TensorFlow use Numpy internally to perform
several operations on tensors. Array Interface is one of the key features of this
library.
• SciPy: The name “SciPy” stands for “Scientific Python”. It is an open-source library
used for high-level scientific computations. This library is built over an extension
of Numpy. It works with Numpy to handle complex computations. While Numpy
allows sorting and indexing of array data, the numerical data code is stored in
SciPy. It is also widely used by application developers and engineers.
• Scrapy: It is an open-source library that is used for extracting data from websites.
It provides very fast web crawling and high-level screen scraping. It can also be
used for data mining and automated testing of data.
 Commonly Used Python Libraries
• Scikit-learn: It is a famous Python library to work with complex data. Scikit-learn
is an open-source library that supports machine learning. It supports variously
supervised and unsupervised algorithms like linear regression, classification,
clustering, etc. This library works in association with Numpy and SciPy.
• PyGame: This library provides an easy interface to the Standard Directmedia
Library (SDL) platform-independent graphics, audio, and input libraries. It is used
for developing video games using computer graphics and audio libraries along
with Python programming language.
• PyTorch: PyTorch is the largest machine learning library that optimizes tensor
computations. It has rich APIs to perform tensor computations with strong GPU
acceleration. It also helps to solve application issues related to neural networks.
• PyBrain: The name “PyBrain” stands for Python Based Reinforcement Learning,
Artificial Intelligence, and Neural Networks library. It is an open-source library
built for beginners in the field of Machine Learning. It provides fast and easy-to-
use algorithms for machine learning tasks. It is so flexible and easily
understandable and that’s why is really helpful for developers that are new in
research fields.
 Importing and using modules
• A module is a file consisting of Python code. It can define functions, classes, and
variables and can also include runnable code. Any Python file can be referenced
as a module. A file containing Python code, for example: test.py, is called a
module, and its name would be test.
• First, let's see how to use the standard library modules.

import math #You need to put this command,`import` keyword along with the name of the module you want to import
num = 4
print(math.sqrt(num)) #Use dot operator to access sqrt() inside module "math"

Output:
2
# my_rand_int.py Output:
import random 20
for i in range(10): 8
15
print(random.randint(1, 25))
18
11
# dateModule.py 20
23
import datetime
13
x = datetime.datetime.now() 6
print(x) 9
Output:
2024-12-05 22:17:10.382522
#Year-month-date HH:MM:SS.microseconds
 Writing Modules in Python
• Writing a module is just like writing any other Python file. And saving this file with .py
extension.
#calculation.py
def add(x,y):
return (x+y)
def sub(x,y):
return (x-y)

#TestModule1.py
import calculation #Importing calculation module
print(calculation.add(1,2)) #Calling function defined in add module.
print(calculation.sub(7,3))
Output:
3
4
 Additional Ways to Import Modules in Python
There are more ways to import modules:
1) from .. import statement:-
from calculation import add
print(add(1,2))
#You can now access it directly without using the module name.
from calculation import add,sub # You can import multiple attributes

2) from .. import * statement:-

# You can import all attributes of a module using this statement. This will #make all attributes of imported
module visible in your code.
from calculation import *
print(add(1,2))
print(sub(3,2))

3) renaming the imported module:-

import calculation as cal
print(cal.add(1,2))
 NumPy
• NumPy stands for Numerical Python, is an open-source Python library that
provides support for large, multi-dimensional arrays and matrices.
• It also have a collection of high-level mathematical functions to operate on
arrays. It was created by Travis Oliphant in 2005.
• Features of NumPy: NumPy has various features which make them popular over
lists.
• Some of these important features include:
1. A powerful N-dimensional array object
2. Sophisticated (broadcasting) functions
3. Tools for integrating C/C++ and Fortran code
4. Useful linear algebra, Fourier transform, and random number capabilities
• Besides its obvious scientific uses, NumPy in Python can also be used as an
efficient multi-dimensional container of generic data.
• Arbitrary data types can be defined using Numpy which allows NumPy to
seamlessly and speedily integrate with a wide variety of databases.
Why Use NumPy?
• In Python we have lists that serve the purpose of arrays, but they are slow to process.
• NumPy aims to provide an array object that is up to 50x faster than traditional Python
lists.
• The array object in NumPy is called ndarray, it provides a lot of supporting functions that
make working with ndarray very easy.
• Arrays are very frequently used in data science, where speed and resources are very
important. (Data Science: is a branch of computer science where we study how to store,
use and analyze data for deriving information from it.)
• NumPy is a Python library and is written partially in Python, but most of the parts that
require fast computation are written in C or C++.
Why is NumPy Faster Than Lists?
• NumPy arrays are stored at one continuous place in memory unlike lists, so processes
can access and manipulate them very efficiently.
• This behavior is called locality of reference in computer science.
• This is the main reason why NumPy is faster than lists. Also it is optimized to work with
latest CPU architectures.
 Install Python NumPy
c\users\sabalesm> pip install numpy
c\users\sabalesm> python -c "import numpy; print(numpy.__version__)“
2.1.3
• NumPy is the fundamental package for scientific computing in
Python. It is a Python library that provides a multidimensional array
object, various derived objects (such as masked arrays and matrices),
and an assortment of routines for fast operations on arrays, including
mathematical, logical, shape manipulation, sorting, selecting, I/O,
discrete Fourier transforms, basic linear algebra, basic statistical
operations, random simulation and much more.
 What are NumPy Commands?
• NumPy commands refer to the functions and methods available in the NumPy library
that operate on arrays. Here are a few commonly used NumPy commands:
• np.array(): Create an array.
• np.arange(): Return evenly spaced values within a given interval.
• np.zeros(), np.ones(), np.full(): Create new arrays filled with zeros, ones, or a specified
value, respectively.
• np.dot(): Dot product of two arrays.
• np.reshape(): Gives a new shape to an array without changing its data.
• np.mean(), np.median(), np.std(): Compute the mean, median, and standard deviation of
array elements.
• np.linalg.inv(): Compute the (multiplicative) inverse of a matrix.
These commands and many others make NumPy a versatile tool for numerical computing
in Python
 Arrays in NumPy
NumPy’s main object is the homogeneous multidimensional array.
• It is a table of elements (usually numbers), all of the same type, indexed by a
tuple of positive integers.
• In NumPy, dimensions are called axes. The number of axes is rank.
• NumPy’s array class is called ndarray. It is also known by the alias array.

Example:
In this example, we are creating a two-dimensional array that has the rank of 2 as it
has 2 axes.
The first axis(dimension) is of length 2, i.e., the number of rows, and the second
axis(dimension) is of length 3, i.e., the number of columns. The overall shape of the
array can be represented as (2, 3)
# Program: numpy_array-example.py

import numpy as np
#Creating array object
arr = np.array( [[ 1, 2, 3],
[ 4, 2, 5]] )

# Printing type of arr object Output:

print("Array is of type: ", type(arr)) Array is of type: <class 'numpy.ndarray'>
No. of dimensions: 2
# Printing array dimensions (axes) Shape of array: (2, 3)
print("No. of dimensions: ", arr.ndim) Size of array: 6
Array stores elements of type: int64
# Printing shape of array
print("Shape of array: ", arr.shape)

# Printing size (total number of elements) of array

print("Size of array: ", arr.size)

# Printing type of elements in array

print("Array stores elements of type: ", arr.dtype)
 pandas
• pandas is a fast, powerful, flexible and easy to use open source data
analysis and manipulation tool, built on top of the Python
programming language.
• Pandas is a powerful and versatile library that simplifies the tasks of
data manipulation in Python. Pandas is well-suited for working with
tabular data, such as spreadsheets or SQL tables.
• The Pandas library is an essential tool for data analysts, scientists, and
engineers working with structured data in Python.
 Uses of pandas
Here is a list of things that we can do using Pandas.
• Data set cleaning, merging, and joining.
• Easy handling of missing data (represented as NaN) in floating point as well as non-
floating point data.
• Columns can be inserted and deleted from DataFrame and higher-dimensional objects.
• Powerful group by functionality for performing split-apply-combine operations on data
sets.
• Data Visualization.
• What Can Pandas Do?
Pandas gives you answers about the data. Like:
• Is there a correlation between two or more columns?
• What is average value?
• Max value?
• Min value?
Pandas are also able to delete rows that are not relevant, or contains wrong values, like
empty or NULL values. This is called cleaning the data.
• pip install pandas
• import pandas as pd
• Data Structures in Pandas Library:
Pandas generally provide two data structures for manipulating data. They are:
• Series
• DataFrame
• Pandas Series
• A Pandas Series is a one-dimensional labeled array capable of holding data of any type (integer,
string, float, Python objects, etc.). The axis labels are collectively called indexes.
• The Pandas Series is nothing but a column in an Excel sheet. Labels need not be unique but must
be of a hashable type.
• The object supports both integer and label-based indexing and provides a host of methods for
performing operations involving the index.
 Creating a Series
Pandas Series is created by loading the datasets from
existing storage (which can be a SQL database, a CSV
file, or an Excel file).
Pandas Series can be created from lists, dictionaries,
scalar values, etc.
Example: Creating a series using the Pandas Library.
#pandas1.py Output:
import pandas as pd Pandas Series: Series([], dtype: float64)
import numpy as np Pandas Series:
0 g
# Creating empty series 1 e
2 e
ser = pd.Series()
3 k
print("Pandas Series: ", ser) 4 s
dtype: object
# simple array
data = np.array(['g', 'e', 'e', 'k', 's'])

ser = pd.Series(data)
print("Pandas Series:\n", ser)
 Pandas DataFrame
Pandas DataFrame is a two-dimensional data structure with labeled axes (rows and
columns).
Creating DataFrame
Pandas DataFrame is created by loading the datasets from existing storage (which
can be a SQL database, a CSV file, or an Excel file).
Pandas DataFrame can be created from lists, dictionaries, a list of dictionaries, etc.

Example: Creating a DataFrame Using the Pandas Library

#pandas2.py
import pandas as pd

# Calling DataFrame constructor

df = pd.DataFrame()
print(df)

# list of strings
lslst = ['Python', 'Language', 'is','the','easiest', 'Language', 'I', 'studied']
# Calling DataFrame constructor on list
df = pd.DataFrame(lst)
print(df)