PYTHON

Programming
Programming is the process of creating instructions for a computer to
execute. It involves writing code using a programming language to solve
problems, automate tasks, or build software applications. Programmers use
a combination of syntax and logic to communicate with computers and
achieve desired outcomes.
There are numerous programming languages, each with its own syntax,
purpose, and strengths. Some popular programming languages include:
1.Python
2.Java
3.JavaScript
4.C++
5. C
 Many programming languages support
data processing, but some are
particularly well-suited for it due to
their features and libraries tailored for
handling data efficiently. Some popular
languages for data processing include:
 Python
Why Python?  SQL
R
 Scala
 Julia
 MATLAB
 Python:
Evolution

Origin: Guido van Rossum's creation in the late 1980s, Python 2.x Series: Introduced in 2000, with
aimed at readability and ease of use, inspired by enhancements like list comprehensions and garbage
Monty Python. collection, remained dominant for years.

late 1980s 20 Feb. 2000 2008

First Release: Python 0.9.0 in February,20 1991, Python 3.x Series: Python 3.0 (2008) brought major
featuring exception handling, functions, and modules. changes for better design, though adoption was
initially slow.
Features of Python

• Simple and Readable Syntax: Easy to learn and write, emphasizing readability.
• Versatile: Suitable for various applications, from web development to data science.
• Interpreted and Interactive: Code is executed line by line, allowing for quick
testing and debugging.
• Dynamic Typing: Variables are dynamically typed, providing flexibility and rapid
development.
• Extensive Standard Library: Comprehensive built-in modules for various tasks.
• Large Ecosystem: Vast collection of third-party libraries and frameworks available
via PyPI.
• Cross-platform: Runs on major operating systems without modification.
• Active Community: Large and supportive community offering resources and
collaboration, As per 2024 Q1, there 130K libraries available to support various tasks
Drawbacks

• Performance: Slower compared to compiled languages like C/C+

+.
• Mobile Development: Not as commonly used for mobile app
development as other languages.
• Packaging and Dependency Management: Complex ecosystem
with potential for dependency conflicts.
• Runtime Errors: Dynamic typing can lead to runtime errors that
may be harder to detect.

Despite these drawbacks, Python remains a popular and widely-used

programming language due to its simplicity, readability, versatility,
and vibrant ecosystem
Python Usage

 ETL/Big data processing + testing automation using Python

 Web development – Django and Desktop apps
 ML algorithm ( Deep leaning , predictive modelling, etc. )
 Networking
 Data analysis ( Matplot lib, Seaborn, pandas profiling )
 AI development
 Big data pipeline
Flavors of python

• Cpython
• Jpython
• Pyspark
print
In Python, the print() function is used to display output to the console or standard
output device. It allows you to print text, variables, and expressions for debugging,
logging, or providing information to the user.

Here's how you can use print() with various examples:

print(*objects, sep=' ', end='\n', file=file, flush=False)
print("Hello, world!")
print("Name:", name)
print("Sum:", x + y)
print(f"Name: {name}, Age: {age}")
Print(source_cnt, target_cnt, sep=‘-’,end=‘\t’)
Identifier
In Python, identifiers are names used to identify variables, functions, classes, modules, or
any other objects. Here are the rules for naming identifiers in Python

 Must begin with a letter (a-z, A-Z) or an underscore (_)

 Python identifiers are case-sensitive. For example, “Source_count” and “source_count” are different identifiers
 There is no specific length for variable name/class name/ function name/module
 Cannot be a Python keyword. Keywords are reserved words that have special meaning in Python, like if, else,
for, while, etc.
 It shouldn't start with number
 No spaces or special characters are allowed except underscores (_) within the identifier name.
 Should be meaningful
 As per PEP 8 standards constants should be represented in upper cases characters
Examples of Identifiers

count
total_count
firstName
total_amount_3
source_table_name
1_transformed_records
Target count
Else
Print
source_schema!
from_source
Datatypes
Data type is a classification of data that specifies which type of value a variable can hold,
how it behaves, and what operations can be performed on it. Python, being a dynamically typed
language, automatically determines the data type of a variable based on the value assigned to it.
Here are some of the common data types in Python:
 Int 1,2, -1,-2,0
 Float 1.1, 1.5 , 1.e4, -1.1,-1e5)
 Str“etl”, “etl2”, ‘etl’, ‘[email protected]’, “””etl”””, ‘’’etl’’’, “etl automation”, “I don’t care”
 Bool  ( True , False)
 Complex – 1+2j, 1-2j, -1+2j, -1-2j, 0+2j
 None  Null value representation
 List  [1,2,-4,-5, 1+2j, true, false, ‘etl’]
 Tuple  (1,2,-4,-5, 1+2j, true, false, ‘etl’)
 Dict  {1:’sreeni’, 2:’ravi’}
 Set  { 1,2,3,4,’etl’, 1+2j, true}
 Frozenset  frozenset(1,2,3,4,’etl’, 1+2j, true)
 Datatypes

Numeric Types:
• int: Integer values, e.g., 5, -10, 100.
product_id, employee_id, source_count, target_count, sequence generator, etc
• float: Floating-point values, e.g., 3.14, -0.5, 2.0
Sales_amount, marks_percentage, temperature, avg_cal
• Complex :
complex type is used to represent complex numbers. A complex number consists of a
real part and an imaginary part,both of which are floating-point numbers. E.g. 1+2j
Mathematics expression calculation, Tele communication
Datatypes continuation
Set Types:
• set: Unordered collection of unique items, e.g., {1, 2, 3}, {'apple', 'banana', 'cherry'}.
• frozenset: Immutable set, e.g., frozenset({1, 2, 3}).
Boolean Type:
• bool: Boolean values representing True or False.
None Type:
• None: A special type representing the absence of a value.
str: String values, e.g., "hello", 'Python', "123".
Collection Datatype - List

List:
A list in Python is a collection of items, separated by commas and enclosed
within square brackets [ ] and Lists can contain elements of different data
types, including numbers, strings, other lists, tuples, dictionaries, etc.
• Mutability: Lists are mutable, meaning you can change, add, and
remove elements after the list has been created.
• Order: Lists maintain the order of elements as they are inserted.
• Elements: Indexing and Slicing: Elements in a list can be accessed
using indices. Python also supports slicing, which allows you to extract a
portion of the list.
e.g., first_list = [1, 2, 'hello',1+2j,10.3]
Collection Datatypes - Tuple

Tuple:
A tuple in Python is similar to a list, but it's enclosed within parentheses ( ).
• Immutability: Tuples are immutable, meaning once created, their contents
cannot be changed (you cannot add, remove, or modify elements).
• Order: Similar to lists, tuples also maintain the order of elements.
• Elements: Tuples can contain elements of different data types, just like
lists.
• Indexing and Slicing: Similar to lists, elements in a tuple can be accessed
using indices. Tuple slicing is also supported.
e.g., first_tuple = (1, 2, 'hello', [3, 4], (5, 6), 1+2j,10.3)
Location : (lat, lang)
Difference between list and tuple

Feature Lists Tuples

Mutability Mutable Immutable
Syntax Defined with square brackets [ ] Defined with parentheses ( )
More memory efficient,
Performance Less memory efficient, slower
faster
Usage Suitable for dynamic collections Suitable for static collections
Collection type - Dictionary

Python dictionary is a built-in data type that allows you to store a collection of key-value pairs. It's
similar to a real-world dictionary where you look up a word (the key) to find its definition (the value)
1.Mutability: Dictionaries are mutable, meaning you can modify their contents after creation. You
can add new key-value pairs, remove existing ones, or modify the values associated with keys.
2.No Order: Unlike sequences such as lists and tuples, dictionaries do not maintain any order
among their elements. The order in which key-value pairs are stored internally may not match the
order in which they were added.
3.Key-Value Pairs: Dictionaries store data in the form of key-value pairs. Each element in a
dictionary is accessed by its associated key rather than by its position. Keys are unique within a
dictionary, but values can be duplicated.
4.Indexing and Access: Elements in a dictionary are accessed using keys rather than indices. You
use the key to retrieve the corresponding value. This provides a fast way to look up values based
on known keys.
5.Dynamic Size: Dictionaries can grow or shrink dynamically as key-value pairs are added or
removed. They are very efficient for tasks that involve dynamically changing collections of data.
Collection type - Set
Python, a set is a collection of unique elements. It is an unordered collection of
distinct hashable objects. Sets are commonly used for membership testing, removing
duplicates from a sequence, and mathematical operations like union, intersection,
difference, and symmetric difference.

1.Unordered Collection: Unlike lists or tuples, sets are unordered. This means that
the items in a set do not have a defined order.
2.Unique Elements: A set cannot contain duplicate elements. If you try to add an
element that already exists in the set, it won't be added again.
3.Mutable: Sets are mutable, meaning you can add or remove elements from them.
4.Set Operations: Sets support various operations like union, intersection,
difference, and symmetric difference. These operations allow you to combine,
compare, or manipulate sets in different ways.
Set = {1,2,3,4,1}
Datatype Description Is Immutable Example
Int We can use to represent the whole/integral numbers Immutable >>> a=10
>>> type(a)
<class 'int'>
Float We can use to represent the decimal/floating Immutable >>> b=10.5
Point Numbers >>> type(b)
<class 'float'>
Complex We can use to represent the complex numbers Immutable >>> c=10+5j
>>> type(c)
<class 'complex'>
>>> c.real 10.0
>>> c.imag 5.0

Bool We can use to represent the logical values(Only allowed values Immutable >>> flag=True
are True and False) >>> flag=False
>>> type(flag)
<class 'bool'>
Str To represent sequence of Characters Immutable >>> s='durga'
>>> type(s)
<class 'str'>
>>> s="durga"
>>> s='''Durga Software
Solutions
... Ameerpet'''
>>> type(s)
<class 'str'>
Datatype Description Is Immutable Example
List To represent an ordered collection of objects Mutable/Changeable >>> l=[10,11,12,13,14,15]
insertion order is preserved >>> type(l)
heterogeneous objects are allowed <class 'list'>
duplicates are allowed
Growable in nature
values should be enclosed within square brackets.
tuple To represent an ordered collections of objects Immutable/Can't change >>> t=(1,2,3,4,5)
insertion order is preserved >>> type(t)
heterogeneous objects are allowed <class 'tuple'>
duplicates are allowed
Not Growable in nature
values should be enclosed within circular brackets.

Set To represent an unordered Mutable >>> s={1,2,3,4,5,6}

collection of unique objects >>> type(s)
insertion order is not preserved
heterogeneous objects are allowed
duplicates are not allowed
Growable in nature
values should be enclosed within {} brackets.

frozenset To represent an unordered collection of unique objects Immutable >>> s={11,2,3,'Durga',100,'Ramu'}

insertion order is not preserved >>> fs=frozenset(s)
heterogeneous objects are allowed >>> type(fs)
duplicates are not allowed <class 'frozenset'>
Not Growable in nature
values should be enclosed within {} brackets.

dict To represent a group of key value pairs Mutable >>>

d={101:'durga',102:'ramu',103:'hari'}
>>> type(d)
<class 'dict'>
List comprehension

List comprehension is a concise and expressive way to

create lists in Python. It allows you to generate a new list
by applying an expression to each item in an iterable
[expression for item in iterable if condition]