python unit 3 module 1 2
python unit 3 module 1 2
UNIT 3
Module 1
# Multithreading: Types of Multi-Threading in
python , process base tasking , Thread base tasing ,
creating thread by using extending thread class.
In Python, multithreading allows multiple threads (smaller
units of a process) to run concurrently, enabling efficient CPU
utilization and faster execution for certain tasks.
1. Process-based Tasking
Definition: Process-based tasking (also known as
multiprocessing) involves running separate processes in
parallel. Each process has its own memory space and
runs independently of others. In Python, the
multiprocessing module is used for process-based
parallelism.
Best Use Case: Useful when the tasks are CPU-bound
(i.e., they require a lot of computation). Since each
process runs in its own memory space, Python’s Global
Interpreter Lock (GIL) doesn’t affect them, making them
efficient for CPU-intensive tasks.
Example:
Python code
from multiprocessing import Process
def task():
print("Process running")
if __name__ == "__main__":
process = Process(target=task)
process.start()
process.join()
2. Thread-based Tasking
Definition: Thread-based tasking (multithreading)
involves multiple threads within the same process.
Threads share the same memory space and resources, so
they can communicate more easily but also require
careful handling to avoid issues like data races.
Best Use Case: Useful for I/O-bound tasks, such as
network requests or file I/O, where threads can be
suspended while waiting for external resources. The GIL
limits true parallelism for CPU-bound tasks in Python, so
multithreading is generally less effective for such tasks.
Example:
Python code
from threading import Thread
def task():
print("Thread running")
thread = Thread(target=task)
thread.start()
thread.join()
3. Creating Threads by Extending the Thread Class
Definition: In this approach, you define a new class that
inherits from the Thread class, overriding its run()
method to define the thread's behavior.
Advantages: This method is useful if you want to create
specialized threads with additional attributes and
methods beyond a basic function.
Example:
Python code
from threading import Thread
class MyThread(Thread):
def run(self):
print("Thread running by extending Thread class")
my_thread = MyThread()
my_thread.start()
my_thread.join()
def task():
print(f"Thread ID: {threading.get_ident()}")
def task():
print(f"Running in Thread ID: {threading.get_ident()}")
# Daemon Threads .
In Python, a daemon thread is a thread that runs in the
background and is designed to terminate automatically when
all non-daemon threads (main threads or foreground threads)
have completed their work. Daemon threads are typically used
for background tasks that should not prevent a program from
exiting, such as monitoring services or periodic maintenance
tasks.
Characteristics of Daemon Threads
1. Background Execution: Daemon threads run in the
background, and Python does not wait for them to
complete before exiting the program.
2. Automatic Termination: When the main program exits,
any daemon threads are also terminated, regardless of
whether they have finished their tasks.
3. Use Case: Ideal for tasks like garbage collection,
logging, monitoring, and housekeeping that should not
keep the program alive.
Creating Daemon Threads
To make a thread a daemon, set its daemon attribute to True
before starting it. By default, threads are non-daemon (i.e.,
daemon=False).
Here’s how you can create a daemon thread:
Python code
import threading
import time
def background_task():
while True:
print("Background task running")
time.sleep(1)
def non_daemon_task():
print("Non-daemon task starting")
time.sleep(5)
print("Non-daemon task completed")
def daemon_task():
while True:
print("Daemon task running")
time.sleep(1)
# Synchronization of Threading .
Thread synchronization in Python is crucial for coordinating
access to shared resources among multiple threads, ensuring
that they don't interfere with each other and lead to
inconsistent or unexpected results. Python provides several
synchronization mechanisms, such as Locks, RLocks,
Semaphores, and Events.
1. Lock (Mutex)
A Lock (also known as a mutex) is the most common
synchronization primitive in Python's threading module. It
allows only one thread to access a critical section at a time,
preventing race conditions when multiple threads try to
modify shared data.
Usage: Use a Lock to wrap sections of code that should
only be executed by one thread at a time.
python code
import threading
# Shared resource
counter = 0
lock = threading.Lock()
def increment():
global counter
for _ in range(1000):
lock.acquire() # Acquire the lock
counter += 1 # Critical section
lock.release() # Release the lock
# Create threads
threads = [threading.Thread(target=increment) for _ in
range(10)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
lock = threading.RLock()
def task():
with lock:
print("First level lock acquired")
with lock:
print("Second level lock acquired")
thread = threading.Thread(target=task)
thread.start()
thread.join()
In this example, the same thread acquires lock twice without
issue, as RLock allows recursive acquisition within the same
thread.
3. Semaphore
A Semaphore allows controlling access to a resource by a
fixed number of threads. You initialize it with a count, and
each acquire() call decreases the count. When the count
reaches zero, other threads trying to acquire the semaphore
will block until the count is positive again.
Usage: Use a Semaphore when you need to limit access
to a shared resource by a certain number of threads.
Python code
import threading
import time
def task():
with semaphore:
print(f"{threading.current_thread().name} acquired the
semaphore")
time.sleep(2)
print(f"{threading.current_thread().name} released the
semaphore")
event = threading.Event()
def waiter():
print("Waiting for event to be set")
event.wait() # Block until the event is set
print("Event is set, continuing")
def setter():
time.sleep(2)
event.set() # Set the event after 2 seconds
print("Event has been set")
# Creating threads
thread1 = threading.Thread(target=waiter)
thread2 = threading.Thread(target=setter)
thread1.start()
thread2.start()
thread1.join()
thread2.join()
In this example, waiter() blocks at event.wait() until setter()
sets the event. This approach is useful when you want to delay
thread execution until a specific condition occurs.
5. Condition
A Condition variable allows threads to wait for certain
conditions to be met before they proceed. Conditions are often
used with Locks to allow more fine-grained control over when
threads are allowed to proceed.
Usage: Use a Condition when a thread needs to wait for
a specific condition to be met.
Python code
import threading
condition = threading.Condition()
shared_data = []
def consumer():
with condition:
print("Consumer waiting for data")
condition.wait() # Wait for data to be available
print("Consumer received data:", shared_data.pop())
def producer():
with condition:
shared_data.append("Data")
print("Producer added data")
condition.notify() # Notify waiting threads that data is
available
# Creating threads
consumer_thread = threading.Thread(target=consumer)
producer_thread = threading.Thread(target=producer)
consumer_thread.start()
producer_thread.start()
consumer_thread.join()
producer_thread.join()
Here, the consumer waits until the producer adds data to
shared_data and calls notify() to unblock it.
# Synchronization by using the lock concept .
Using the Lock mechanism in Python is one of the simplest
ways to achieve thread synchronization. A Lock ensures that
only one thread can access a shared resource or critical section
of code at any given time, preventing race conditions.
How Lock Works
1. Acquire: Before accessing a shared resource, a thread
calls lock.acquire() to obtain exclusive access. If another
thread has already acquired the lock, the requesting
thread will wait (or block) until the lock becomes
available.
2. Release: After finishing with the shared resource, the
thread calls lock.release() to release the lock, allowing
other waiting threads to proceed.
Example of Synchronization Using Lock
Suppose we have a shared resource, such as a counter, that
multiple threads want to increment. Without proper
synchronization, the result could be unpredictable, as threads
may read and modify the counter simultaneously. Using a
lock, we can ensure that only one thread increments the
counter at a time.
Python code
import threading
# Shared resource
counter = 0
# Starting threads
for thread in threads:
thread.start()
def increment_counters():
global counter1, counter2
for _ in range(1000):
with lock1:
counter1 += 1
with lock2:
counter2 += 1
Using lock1 and lock2 separately for counter1 and counter2
ensures independent access to each variable, enhancing
parallelism.
# Shared resource
counter = 0
lock = threading.Lock()
def increment():
global counter
for _ in range(1000):
lock.acquire() # Acquire the lock
try:
counter += 1 # Critical section
finally:
lock.release() # Release the lock
# Shared resource
counter = 0
rlock = threading.RLock()
def increment():
global counter
for _ in range(1000):
rlock.acquire() # Acquire the RLock
try:
counter += 1 # Critical section
finally:
rlock.release() # Release the RLock
def nested_increment():
rlock.acquire() # Acquire the RLock for the outer function
try:
increment() # Call a function that also acquires the same
RLock
finally:
rlock.release() # Release the RLock
def worker(thread_id):
print(f"Thread {thread_id} is waiting to acquire the
semaphore.")
with bounded_semaphore: # Acquire the semaphore
print(f"Thread {thread_id} has acquired the
semaphore.")
time.sleep(2) # Simulate some work
print(f"Thread {thread_id} has released the semaphore.")
# Shared resource
queue = []
condition = threading.Condition()
def producer():
global queue
for i in range(5):
time.sleep(1) # Simulate work
with condition:
queue.append(i)
print(f"Produced {i}")
condition.notify() # Notify one waiting thread
def consumer():
global queue
for _ in range(5):
with condition:
while not queue:
condition.wait() # Wait until an item is produced
item = queue.pop(0)
print(f"Consumed {item}")
producer_thread.start()
consumer_thread.start()
producer_thread.join()
consumer_thread.join()
event = threading.Event()
def worker():
print("Worker thread waiting for event...")
event.wait() # Block until the event is set
print("Worker thread proceeding after event is set.")
def main():
thread = threading.Thread(target=worker)
thread.start()
time.sleep(2)
print("Main thread setting event.")
event.set() # Set the event to let the worker thread proceed
main()
3. Using Queues
def producer(q):
for i in range(5):
time.sleep(1)
q.put(i) # Put an item in the queue
print(f"Produced {i}")
def consumer(q):
while True:
item = q.get() # Get an item from the queue
if item is None:
break # Exit if None is received
print(f"Consumed {item}")
q.task_done() # Signal that the item has been processed
# Create a Queue
q = queue.Queue()
producer_thread.start()
consumer_thread.start()
producer_thread.join()
q.put(None) # Send a signal to the consumer to exit
consumer_thread.join()
# Types Of Queues.
In Python, the queue module provides several types of queue
classes that are thread-safe and can be used for inter-thread
communication.
1. Queue (FIFO Queue)
The Queue class implements a first-in, first-out (FIFO) queue.
It is suitable for scenarios where the order of processing is
important. The first element added to the queue will be the
first one to be removed.
Key Methods:
def producer(q):
for i in range(5):
time.sleep(1)
q.put(i)
print(f"Produced {i}")
def consumer(q):
while True:
item = q.get()
if item is None:
break
print(f"Consumed {item}")
q.task_done()
# Create a FIFO queue
q = queue.Queue()
producer_thread.start()
consumer_thread.start()
producer_thread.join()
q.put(None) # Send signal to consumer to exit
consumer_thread.join()
# Create a SimpleQueue
simple_queue = queue.SimpleQueue()
# MySQL
conn = mysql.connector.connect(
host='localhost',
user='your_username',
password='your_password',
database='your_database'
)
# PostgreSQL
conn = psycopg2.connect(
dbname='your_database',
user='your_username',
password='your_password',
host='localhost'
)
4. Create a Cursor Object
A cursor object allows you to execute SQL queries and fetch
results.
python code
cursor = conn.cursor()
5. Execute SQL Queries
You can execute various SQL commands such as CREATE,
INSERT, SELECT, UPDATE, and DELETE.
python code
# Create a table
cursor.execute('''CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
name TEXT,
age INTEGER)''')
# Insert data
cursor.execute("INSERT INTO users (name, age) VALUES
(%s, %s)", ('Alice', 30))
# Select data
cursor.execute("SELECT * FROM users")
6. Fetch Results
Depending on the type of query, you can fetch results using
methods like fetchone(), fetchall(), or fetchmany(size).
python code
# Fetch all results
rows = cursor.fetchall()
for row in rows:
print(row)
7. Commit Changes (for Write Operations)
If you have executed INSERT, UPDATE, or DELETE
statements, make sure to commit the changes to the database.
python code
conn.commit() # Save changes
8. Handle Exceptions
Use try-except blocks to handle any potential errors that may
occur during database operations.
python code
try:
# Your database operations here
except Exception as e:
print(f"An error occurred: {e}")
finally:
# Clean up
cursor.close()
conn.close()
9. Close the Connection
Always ensure that you close the cursor and connection when
you are done to free up resources.
python code
cursor.close()
conn.close()
Full Example
Here’s a complete example that demonstrates the standard
steps in Python database programming using SQLite:
python code
import sqlite3
try:
# Step 4: Create a cursor object
cursor = conn.cursor()
except Exception as e:
print(f"An error occurred: {e}")
finally:
# Step 9: Clean up
cursor.close()
conn.close()