I will re-iterate the best practice at everyone who comes across the sql connection using MySQLdb or any other package to connect python2/3 needs to know this
(Following mock run assumes that you have a table named tablename in your sql database. It has got 4 columns/fields with names field1,field2,field3,field4). If your connection is local (same machine) then it is 127.0.0.1 also known as "localhost".
The process is to be simple 7 steps
- Create connection
- Create cursor
- Create Query string
- Execute the query
- Commit to the query
- Close the cursor
- Close the connection
Here is a simple step by stem mock run
mydb = MySQLdb.connect(host=host, user=user, passwd=passwd, db=database, charset="utf8")
cursor = mydb.cursor()
query = "INSERT INTO tablename (text_for_field1, text_for_field2, text_for_field3, text_for_field4) VALUES (%s, %s, %s, %s)"
cursor.execute(query, (field1, field2, field3, field4))
mydb.commit()
cursor.close()
mydb.close()
Connection and cursor are different. connection is at the SQL level while cursor can be considered as a data element. You can have multiple cursors on the same data within single connection. It is an unusual occurrence to have multiple connections to same data from the same computer.
More has been described here "The cursor paradigm is not specific to Python but are a frequent data structure in databases themselves.
Depending on the underlying implementation it may be possible to generate several cursors sharing the same connection to a database. Closing the cursor should free resources associated to the query, including any results never fetched from the DB (or fetched but not used) but would not eliminate the connection to the database itself so you would be able to get a new cursor on the same database without the need to authenticate again."
Answer from Mandar on Stack Overflowneed to close mysql connection? : Forums : PythonAnywhere
MySQL Connector/Python not closing connection explicitly - Stack Overflow
The Correct way to end a SQL connection with python?
Python Mysql-Connector. Which is better connection.close() or connection.disconnect() or connection.shutdown() - Stack Overflow
I will re-iterate the best practice at everyone who comes across the sql connection using MySQLdb or any other package to connect python2/3 needs to know this
(Following mock run assumes that you have a table named tablename in your sql database. It has got 4 columns/fields with names field1,field2,field3,field4). If your connection is local (same machine) then it is 127.0.0.1 also known as "localhost".
The process is to be simple 7 steps
- Create connection
- Create cursor
- Create Query string
- Execute the query
- Commit to the query
- Close the cursor
- Close the connection
Here is a simple step by stem mock run
mydb = MySQLdb.connect(host=host, user=user, passwd=passwd, db=database, charset="utf8")
cursor = mydb.cursor()
query = "INSERT INTO tablename (text_for_field1, text_for_field2, text_for_field3, text_for_field4) VALUES (%s, %s, %s, %s)"
cursor.execute(query, (field1, field2, field3, field4))
mydb.commit()
cursor.close()
mydb.close()
Connection and cursor are different. connection is at the SQL level while cursor can be considered as a data element. You can have multiple cursors on the same data within single connection. It is an unusual occurrence to have multiple connections to same data from the same computer.
More has been described here "The cursor paradigm is not specific to Python but are a frequent data structure in databases themselves.
Depending on the underlying implementation it may be possible to generate several cursors sharing the same connection to a database. Closing the cursor should free resources associated to the query, including any results never fetched from the DB (or fetched but not used) but would not eliminate the connection to the database itself so you would be able to get a new cursor on the same database without the need to authenticate again."
Closing the cursor as soon as you are done with it is probably the best bet, since you have no use for it anymore. However, I haven't seen anything where it's harmful to close it after the db connection. But since you can set it as:
cursor = conn.cursor()
I recommend closing it before, in case you accidentally assign it again and the DB connection is closed as this would throw an error. So you may want to close it first in order to prevent an accidental reassignment with a closed connection.
(Some don't even close it at all though as it gets collected by the garbage collector (see:In Python with sqlite is it necessary to close a cursor?))
References: When to close cursors using MySQLdb
In Python with sqlite is it necessary to close a cursor?
It's not a config issue. When you are done with a connection you should close it by explicitly calling close. It is generally a best practice to maintain the connection for a long time as creating one takes time. It's not possible to tell from your code snippet where would be the best place to close it - it's whenever you're "done" with it; perhaps at the end of your __main__ method. Similarly, you should close the cursor explicitly when your done with it. Typically that happens after each query.
So, maybe something like:
class FooData(object):
def __init__(self):
...
try:
self.my_cnf = os.environ['HOME'] + '/.my.cnf'
self.my_cxn = mysql.connector.connect(option_files=self.my_cnf)
def execute_some_query(self, query_info):
"""Runs a single query. Thus it creates a cursor to run the
query and closes it when it's done."""
# Note that cursor is not a member variable as it's only for the
# life of this one query
cursor = self.my_cxn.cursor(dictionary=True)
cursor.execute(...)
# All done, close the cursor
cursor.close()
def close():
"""Users of this class should **always** call close when they are
done with this class so it can clean up the DB connection."""
self.my_cxn.close()
You might also look into the Python with statement for a nice way to ensure everything is always cleaned up.
I rewrote my class above to look like this...
class FooData(object):
def __init__(self):
self.myconfig = {
'option_files': os.environ['HOME'] + '/.my.cnf',
'database': 'nsdata'
}
self.mysqlcxn = None
def __enter__(self):
try:
self.mysqlcxn = mysql.connector.connect(**self.myconfig)
except mysql.connector.Error as err:
if err.errno == 2003:
self.mysqlcxn = None
return self
def __exit__(self, exc_type, exc_value, traceback):
if self.mysqlcxn is not None and self.mysqlcxn.is_connected():
self.mysqlcxn.close()
def etl(self)
...
I can then use with ... as and ensure that I am cleaning up properly.
with FooData() as obj:
obj.etl()
The Aborted connection messages are thus properly eliminated.
Oliver Dain's response set me on the right path and Explaining Python's '__enter__' and '__exit__' was very helpful in understanding the right way to implement my Class.
So ive been doing a bit more reading and i use to do a MySQL query like so :-
cursor = db_connection5.cursor()
sql = "SELECT Price,Stock FROM Catalog WHERE ID=%s LIMIT 1"
val = (SKUC,)
cursor.execute(sql, val)
AddRecordCheck = cursor.fetchone()
db_connection5.commit() But now it has come to my attention i should be ending the connection like so:-
cursor = db_connection5.cursor()
sql = "SELECT Price,Stock FROM Catalog WHERE ID=%s LIMIT 1"
val = (SKUC,)
cursor.execute(sql, val)
AddRecordCheck = cursor.fetchone()
db_connection5.commit()
cursor.close() The only reason i ask is i have around 15 of the same scripts connected to my database working 24/7 and sometimes one of my other scripts connecting to the database hangs for an hour or two trying to connect at "cursor = db_connection5.cursor()" so i am starting to wonder if i have been missing something all along.