Scribd
Upload
285 views

Single Database Connection Throughout The Python Application

The question asks for the best way to maintain a single database connection throughout a Python application using the singleton pattern while handling multiple requests, closed connections, and connection timeouts. The poster provides a DBConnection class for creating connections but it opens a new connection for each query. Suggestions are made to use Django's connection management or implement a minimal backend. The poster decides to use a singleton class approach with a DBConnector for connections and DBConnection class for managing the singleton connection. Potential flaws in this approach could be issues sharing the connection between threads.

Uploaded by

leonard1971
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
285 views

Single Database Connection Throughout The Python Application

The question asks for the best way to maintain a single database connection throughout a Python application using the singleton pattern while handling multiple requests, closed connections, and connection timeouts. The poster provides a DBConnection class for creating connections but it opens a new connection for each query. Suggestions are made to use Django's connection management or implement a minimal backend. The poster decides to use a singleton class approach with a DBConnector for connections and DBConnection class for managing the singleton connection. Potential flaws in this approach could be issues sharing the connection between threads.

Uploaded by

leonard1971
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

single database connection throughout the

python application (following singleton pattern)


Ask Question

0
My Question is what is the best way to maintain the single database connection in the entire
application? Using Singleton Pattern? How?
Conditions that are needed to be taken care of:

1. In case of multiple requests, I should be using the same connection


2. In case connection is closed, create a new connection
3. If the connection has timed-out, on new request my code should create a new connection.

The driver to my Database is not supported by the Django ORM. And due to same driver related issues, I am
using pyodbc to connect to the database. Right now I am having below class for creating and managing the DB
connections:
class DBConnection(object):
def __init__(self, driver, serve,
database, user, password):

self.driver = driver
self.server = server
self.database = database
self.user = user
self.password = password

def __enter__(self):
self.dbconn = pyodbc.connect("DRIVER={};".format(self.driver) +\
"SERVER={};".format(self.server) +\
"DATABASE={};".format(self.database) +\
"UID={};".format(self.user) +\
"PWD={};".format(self.password) + \
"CHARSET=UTF8",
# "",
ansi=True)

return self.dbconn

def __exit__(self, exc_type, exc_val, exc_tb):


self.dbconn.close()
But the issue with this approach is that it will create new database connection for each query. What will be the
better way to do it following singleton pattern? The way I can think of will hold the reference to the
connection if the connection is closed. Something like:
def get_database_connection():
conn = DBConnection.connection
if not conn:
conn = DBConnection.connection = DBConnection.create_connection()
return conn
What will be the best way to achieve this? Any suggestion/ideas/examples?
PS: I was checking about using weakref which allows to create weak references to objects. I think it will be
good idea to use weakref along with singleton pattern for storing the connection variable. This way I won't
have to keep the connection alive when DB is not in use. What you guys say about this?
python django singleton database-connection pyodbc

shareimprove this question


edited Nov 10 '16 at 11:05
asked Nov 10 '16 at 10:40
Moinuddin Quadri
29.7k55082
 There seem to be a few third-party backends for pyocdb, such as django-pyocdb. Why not use one of those? –
knbk Nov 10 '16 at 11:20
 I am using IBM Netezza as a database which is not supported by django-pyodbc – Moinuddin Quadri Nov 10 '16 at
11:38
 You could implement a minimal database backend that only supports creating connections and cursors. That way
you can use Django's connection management to handle your connections. I guess that would be easier than
implementing the connection handling yourself. – knbk Nov 10 '16 at 11:51
 I tool a look at it for django.db.backends.mysql. There 6 files namely base.py, client.py, compiler.py, creation.py,
introspection.py and validation.py. I think it will a lot of overhead to implement. May be i didn't totally understood the
definition of minimal database backend. In my mind I was thinking of creating a Singleton class to maintain single
connection (already did), but your idea looks better to me. Do you know library, or blog which will give me idea of
how Django's backend works? – Moinuddin Quadri Nov 10 '16 at 21:42
 1
Most of that has to do with the ORM. I think if you implement the DatabaseWrapper class in base.py, with dummy
classes for the others, you can use connections and cursors to execute raw queries, and rely on Django's
connection management. I'm afraid I don't know of any guide or something, and I only know a bit about it myself. –
knbk Nov 10 '16 at 21:56
show 3 more comments
1 Answer
activeoldest votes

0
For now, I am going ahead with the singleton class approach. Anyone seeing the potential flaws in this, feel to
mention them :)

DBConnector class for creating a connection


class DBConnector(object):

def __init__(self, driver, server, database, user, password):

self.driver = driver
self.server = server
self.database = database
self.user = user
self.password = password
self.dbconn = None

# creats new connection


def create_connection(self):
return pyodbc.connect("DRIVER={};".format(self.driver) + \
"SERVER={};".format(self.server) + \
"DATABASE={};".format(self.database) + \
"UID={};".format(self.user) + \
"PWD={};".format(self.password) + \
"CHARSET=UTF8",
ansi=True)

# For explicitly opening database connection


def __enter__(self):
self.dbconn = self.create_connection()
return self.dbconn

def __exit__(self, exc_type, exc_val, exc_tb):


self.dbconn.close()
DBConnection class for managing the connections
class DBConnection(object):
connection = None
@classmethod
def get_connection(cls, new=False):
"""Creates return new Singleton database connection"""
if new or not cls.connection:
cls.connection = DBConnector().create_connection()
return cls.connection

@classmethod
def execute_query(cls, query):
"""execute query on singleton db connection"""
connection = cls.get_connection()
try:
cursor = connection.cursor()
except pyodbc.ProgrammingError:
connection = cls.get_connection(new=True) # Create new connection
cursor = connection.cursor()
cursor.execute(query)
result = cursor.fetchall()
cursor.close()
return result
shareimprove this answer
answered Nov 11 '16 at 6:55

Moinuddin Quadri
29.7k55082
 You should make sure that a connection is not shared between threads, or you might run into some surprising and
hard-to-debug behaviour. You can save the connection on a threading.local() object so that each thread has
it's own singleton connection. – knbk Nov 11 '16 at 11:12
 @knbk: I am just sharing the connection object, but creating a new local cursor for each query
in execute_query function. So, since query/result relationship are maintained by the cursor, I do not see any harm
even if execute_query is called asynchronously. That is again my assumption. Or I am wrong? – Moinuddin
Quadri Nov 11 '16 at 11:55
 1
From the pyodbc docs: "threadsafety The integer 1, indicating that threads may share the module but not
connections. Note that connections and cursors may be used by different threads, just not at the same time." -- So
you can't share the connection between threads that try to access the database concurrently. – knbk Nov 11 '16 at
12:42

You might also like