The pandas module imports read_sql from a submodule; you could try getting it from the submodule:

df = pd.io.sql.read_sql('select * from databasetable', engine, index_col = index)

You could also print pd.__dict__ to see what's available in the pd module. Do you get AttributeError if you try to use other things from the pd module, for example, pd.Series()?

Answer from Brian from QuantRocket on Stack Overflow
🌐
Reddit
reddit.com › r/learnpython › need help: use sql on a dataframe created through pandas
r/learnpython on Reddit: Need help: Use SQL on a dataframe created through Pandas
May 18, 2020 -

So I'm trying to create a python script that allows me to perform SQL manipulations on a dataframe (masterfile) I created using pandas. The dataframe draws its contents from the csv files found in a specific folder.

I was able to successfully create everything else, but I am having trouble with the SQL manipulation part. I am trying to use the dataframe as the "database" where I will pull the data using my SQL query but I am getting a "AttributeError: 'DataFrame' object has no attribute 'cursor' " error.

I'm not really seeing a lot of examples for pandas.read_sql_query() so I am having a difficult time on understanding how I will use my dataframe in it.

I appreciate any help I can get! I am a self-thought python programmer by the way, so I might be missing some general rule or something. Thanks!

import os
import glob
import pandas

os.chdir("SOMECENSOREDDIRECTORY")

all_csv = [i for i in glob.glob('*.{}'.format('csv')) if i != 'Masterfile.csv']

edited_files = []
for i in all_csv:
    df = pandas.read_csv(i)
    df["file_name"] = i.split('.')[0]
    edited_files.append(df)

masterfile = pandas.concat(edited_files, sort=False)

print("Data fields are as shown below:")
print(masterfile.iloc[0])

sql_query = '''
            SELECT Country, file_name as Year, Happiness_Score 
            
            FROM masterfile WHERE Country = 'Switzerland'
            
            '''
            
output = pandas.read_sql_query(sql_query, masterfile)

output.to_csv('data_pull')
Top answer
1 of 2
1
You'll need to install sqlalchemy and follow the example below: from sqlalchemy import create_engine engine = create_engine('sqlite:///:memory:') df.to_sql('data', engine) sql_df = pd.read_sql('data',con=engine) sql_df MORE INFO: (copy-pasted from a pandas course I took) The pandas.io.sql module provides a collection of query wrappers to both facilitate data retrieval and to reduce dependency on DB-specific API. Database abstraction is provided by SQLAlchemy if installed. In addition, you will need a driver library for your database. Examples of such drivers are psycopg2 for PostgreSQL or pymysql for MySQL. For SQLite, this is included in Python’s standard library by default. You can find an overview of supported drivers for each SQL dialect in the SQLAlchemy docs. If SQLAlchemy is not installed, a fallback is only provided for SQLite (and for MySQL for backward compatibility, but this is deprecated and will be removed in a future version). This mode requires a Python database adapter that respects the Python DB-API. See also some cookbook examples for some advanced strategies. The key functions are: read_sql_table(table_name, con[, schema, ...]) Read the SQL database table into a DataFrame. read_sql_query(sql, con[, index_col, ...]) Read SQL query into a DataFrame. read_sql(sql, con[, index_col, ...]) Read SQL query or database table into a DataFrame. DataFrame.to_sql(name, con[, flavor, ...]) Write records stored in a DataFrame to a SQL database.
2 of 2
1
I think you might be misunderstanding what pd.read_sql_query does. What it is: It is a method used to create a dataframe by reading data from a database based on a query and database connection object What it is not: A method for indexing/slicing an existing dataframe, using sql-like syntax If you really want to get the latter functionality you could potentially do 2 different things. (1) you could write a function that takes a sql query and parses it into pandas indexing commands and apply them to the df. Or (2) you could take your dataframe upload it to a SQLite table and then use pd.read_sql+the query to read the data out of the SQLite table. FWIW I don’t think either of these is a good idea, and that the best thing to do is get comfortable with pandas indexing.
🌐
GitHub
github.com › pandas-dev › pandas › issues › 24988
Version 0.24.0 breaks read_sql compatibility with ...
January 28, 2019 - import pandas as pd from sqlalchemy import create_engine engine = create_engine('mysql+pymysql://'+name+':'+pw+'@'+server+'/?charset=utf8') sql = 'select * from MyDatabase.my_temp_table' df = pd.read_sql_query(sql,engine)
Author   Shellcat-Zero
🌐
Reddit
reddit.com › r/learnpython › how to resolve the following error: attributeerror: module 'pandas' has no attribute 'read'
r/learnpython on Reddit: How to resolve the following error: Attributeerror: module 'pandas' has no attribute 'read'
August 18, 2021 -

I am trying to import a CSV file over to python but keep getting the following error. Any advice, I tried it on pycharm and juptyer notebook and get the same error. I am new to python

🌐
Stack Overflow
stackoverflow.com › questions › 47331410 › module-pandas-has-no-attribute-read-sql-query
python - module 'pandas' has no attribute 'read_sql_query' - Stack Overflow
As per the documentation, I have used the pandas module: import sqlite3 import pandas conn = sqlite3.connect('C:\sqlite\water-filter.db') query = pandas.read_sql_query('select * from clients;', c...
🌐
iheavy
iheavy.com › attribute-error-module-pandas-has-no-attribute-read-sql
Troubleshooting 'Attribute Error | Module 'pandas' has no Attribute 'read_sql' - iheavy
April 30, 2024 - If you suspect installation issues, uninstall Pandas using pip or conda and then reinstall it to ensure a clean installation. Review your code and ensure that the syntax for calling the ‘read_sql’ function is correct. The correct syntax is typically ‘pd.read_sql(sql_query, connection_object).’
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.read_sql.html
pandas.read_sql — pandas documentation - PyData |
This function is a convenience wrapper around read_sql_table and read_sql_query (for backward compatibility). It will delegate to the specific function depending on the provided input. A SQL query will be routed to read_sql_query, while a database table name will be routed to read_sql_table.
🌐
GitConnected
levelup.gitconnected.com › how-to-fix-attributeerror-optionengine-object-has-no-attribute-execute-in-pandas-eb635fbb89e4
How to Fix AttributeError: ‘OptionEngine’ object has no attribute ‘execute’ in Pandas | Level Up Coding
February 12, 2023 - Fix for AttributeError: ‘OptionEngine’ object has no attribute ‘execute’ in Pandas with SQLAlchemy 2.0.0 when calling read_sql_query() with pandas python
🌐
Stack Overflow
stackoverflow.com › questions › 75229333 › pandas-does-not-have-a-function-read-sql
python - Pandas does not have a function "read_sql" - Stack Overflow
Trying to use pandas to do a query, pd.read_sql(<my query>, conn).to_dict(orient="records") Error message is: AttributeError: module 'pandas' has no attribute 'read_sql' At the command line, python3 -c "import pandas; print(str(pandas.__dict__))" returns, among other things, 'read_sql': <function read_sql at 0x7eff9c08df28> so the function is obviously there and visible to Python.
Find elsewhere
🌐
Stack Overflow
stackoverflow.com › questions › 42697319 › unable-to-import-pandas-module-python-prints-attributeerror-from-older-python-s
Unable to import pandas module; python prints AttributeError from older python session - Stack Overflow
>>> import pandas as pd Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/tsd/p19/home/p19-jonasmst/.local/lib/python2.7/site-packages/pandas/__init__.py", line 13, in <module> __import__(dependency) File "/cluster/software/VERSIONS/python_packages-2.7_3/cluster/software/VERSIONS/python2-2.7.9/lib/python2.7/site-packages/pytz/__init__.py", line 29, in <module> from pkg_resources import resource_stream File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 74, in <module> File "parser.py", line 12, in <module> asid_occurrences = pd.read_sql_query(sql_find_asid_occurrences, db) AttributeError: 'module' object has no attribute 'read_sql_query' ... I'm on a cluster and have to load python2 as a module in order to import pandas.
🌐
GitHub
github.com › pandas-dev › pandas › issues › 57053
BUG: Pandas >= 2.2.0and SQLALCHEMY does not work together · Issue #57053 · pandas-dev/pandas
January 24, 2024 - Python==3.10.12 pandas==2.2.0 SQLAlchemy==1.4.51 denodo-sqlalchemy==20230818 def sqlalch(): connection_string = f"denodo://{username}:{urllib.parse.quote(password)}@{host}:{port}/{database}" denodo_engine = db.create_engine(connection_string, **kwargs) connection = denodo_engine.connect() return connection conn = sqlalch() pd.read_sql(query, conn) produces error: 2024-01-24T11:57:31 cursor = self.execute(sql, params) 2024-01-24T11:57:31 File "/usr/local/lib/python3.10/site-packages/pandas/io/sql.py", line 2673, in execute 2024-01-24T11:57:31 cur = self.con.cursor() 2024-01-24T11:57:31 AttributeError: 'Connection' object has no attribute 'cursor'
Author   jkorsvik
Top answer
1 of 10
113

Solution for 2024

I know this question is old, but a recent change to Pandas warrants a new solution

Pandas 2.2.0 appears to have modified how a SQLAlchemy Engine object operates when passed to the con argument for pd.read_sql, pd.to_sql, etc. Unsure if this is intentional.

My project worked perfectly fine with Pandas 2.1.4. When updated to 2.2.0, I receive the AttributeError due to the cursor not being present.

  • Solution: Use the connection attribute of a SQLAlchemy Connection object instead of the Engine object directly.

Peep this example:

from sqlalchemy import create_engine
import pandas as pd


# Engine for MS SQL Server using pymssql
conn_url = f"mssql+pymssql://{username}:{password}@{host}/{database}"
engine = create_engine(conn_url)

# Example query
query = 'select id from users'

# Works with pandas<2.2.0
df = pd.read_sql(
    sql=query,
    con=engine,
)

# Works with pandas==2.2.0
with engine.connect() as conn:
    df = pd.read_sql(
        sql=query,
        con=conn.connection
    )

As per comments below, this works with pandas 2.2 for .read_sql(), .read_sql_query(), etc., but does not work for .to_sql() with backends other than SQLite.

2 of 10
62

adding in a raw_connection() worked for me

from sqlalchemy import create_engine

sql_engine = create_engine('sqlite:///test.db', echo=False)
connection = sql_engine.raw_connection()
working_df.to_sql('data', connection, index=False, if_exists='append')

I had conda install sqlalchemy during my notebook session, so while it was accessible, since I already initiated pandas, it appeared as if there was no sqlalchemy. Restarting the session allowed me to remove the raw_connection().

from sqlalchemy import create_engine

sql_engine = create_engine('sqlite:///test.db', echo=False)
connection = sql_engine
working_df.to_sql('data', connection, index=False, if_exists='append')
🌐
Streamlit
discuss.streamlit.io › using streamlit
Unable to read a db whereas it was possible with Jupyter and streamlit run command - Using Streamlit - Streamlit
December 24, 2024 - Hi, I am trying to load a database from sql database. I first create this function to read my database. It works fine on Jupyter and in a local run of streamlit…(cf. the function down) Why did I get this error message and how to cancel it and read my database.
🌐
GitHub
github.com › dask › dask › issues › 2975
read_sql_table gives AttributeError when reading a database · Issue #2975 · dask/dask
October 22, 2017 - I get this error: AttributeError: 'Engine' object has no attribute '_instantiate_plugins' If I use the pandas read_sql_query function on the same db, it works just fine.
Author   gsavaia
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.read_sql_query.html
pandas.read_sql_query — pandas documentation
pandas.read_sql_query(sql, con, index_col=None, coerce_float=True, params=None, parse_dates=None, chunksize=None, dtype=None, dtype_backend=<no_default>)[source]#
🌐
GitHub
github.com › pandas-dev › pandas › issues › 23030
mssql pandas.DataFrame.to_sql AttributeError: 'Engine' object has no attribute 'cursor' · Issue #23030 · pandas-dev/pandas
July 4, 2018 - mssql pandas.DataFrame.to_sql AttributeError: 'Engine' object has no attribute 'cursor'#23030 · Copy link · Labels · IO SQLto_sql, read_sql, read_sql_queryto_sql, read_sql, read_sql_query · philiphoyos · opened · on Oct 7, 2018 · Issue body actions · # test data test = pd.DataFrame({'test':[1,2,3]}) #'te','te','te' # import modules import pyodbc from sqlalchemy import create_engine import urllib # test 1 params = urllib.parse.quote_plus(r'DRIVER={SQL Server};SERVER=server_name;DATABASE=db_name;Trusted_Connection=yes') engine = create_engine('mssql+pyodbc:///?odbc_connect={}'.format(par
Author   philiphoyos
🌐
TechOverflow
techoverflow.net › 2021 › 04 › 27 › how-to-fix-pandas-to_sql-attributeerror-dataframe-object-has-no-attribute-cursor
How to fix pandas to_sql() AttributeError: 'DataFrame' object has no attribute 'cursor' | TechOverflow
April 27, 2021 - import pandas as pd # Load pre-built time series example dataset df = pd.read_csv("https://datasets.techoverflow.net/timeseries-example.csv", parse_dates=["Timestamp"]) df.set_index("Timestamp", inplace=True) import sqlalchemy db = sqlalchemy.create_engine('sqlite:///timeseries.db') df.to_sql('timeseries', db, if_exists="replace")