🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.read_sql.html
pandas.read_sql — pandas 3.0.1 documentation - PyData |
>>> pd.read_sql("test_data", "postgres:///db_name") For parameterized query, using params is recommended over string interpolation.
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.read_sql_query.html
pandas.read_sql_query — pandas 3.0.1 documentation
pandas.read_sql_query(sql, con, index_col=None, coerce_float=True, params=None, parse_dates=None, chunksize=None, dtype=None, dtype_backend=<no_default>)[source]#
Discussions

python - Executing an SQL query on a Pandas dataset - Stack Overflow
I have a Pandas dataset called df. How can I do: df.query("select * from df") More on stackoverflow.com
🌐 stackoverflow.com
python - Read SQL query into pandas dataframe and replace string in query - Stack Overflow
In this way I'm not writing my query in pandas as it is a huge query and consumes lot of space. Can someone please tell me how to solve this and what is the correct syntax? Thank you very much! ... Here's how to read in a text file in Python. query_filename = 'C:\Users\Admin\Desktop\Python\Open Source\query.sql... More on stackoverflow.com
🌐 stackoverflow.com
python - Pandas read_sql with parameters - Stack Overflow
Are there any examples of how to pass parameters with an SQL query in Pandas? In particular I'm using an SQLAlchemy engine to connect to a PostgreSQL database. So far I've found that the following... More on stackoverflow.com
🌐 stackoverflow.com
Do folks ever use Pandas when they should use SQL?
SQL and Pandas are two very different things. I would argue that anyone who sees them as substitutes is probably misusing both of them. More on reddit.com
🌐 r/Python
155
95
April 7, 2024
🌐
Spark By {Examples}
sparkbyexamples.com › home › pandas › pandas read sql query or table with examples
Pandas Read SQL Query or Table with Examples - Spark By {Examples}
December 2, 2024 - # Insert Few Records ... (5,'PHP',28000,'30 Days', 800)''') # Commit inserts con.commit() Now by using Pandas read_sql() function load the table, as I said above, this can take either SQL query or table name as a ...
🌐
DataLemur
datalemur.com › blog › sql-pandas-read_sql
How to Use Pandas read_sql to Write and Run SQL?
April 21, 2025 - : This function executes a SQL query and reads results into a DataFrame from a database. : This function reads an entire SQL table or view into a DataFrame from a database. While Pandas simplifies the extraction of SQL databases within the Python library, it's essential to recognize its limitations.
🌐
TutorialsPoint
tutorialspoint.com › python_pandas › python_pandas_read_sql_method.htm
Python Pandas read_sql() Method
This method is a convenient wrapper ... and simplifies working with databases. Following is the syntax of the Python Pandas read_sql() method −...
🌐
Hex
hex.tech › blog › query-sql-database-pandas
How to query a SQL database from Pandas | Hex
February 12, 2023 - For simplicity's sake, we’ll pick a publicly available dataset from Kaggle, about Airbnb Data in New York City. Let's do this! Pandas has a built-in function to read from a database: read_sql.
🌐
Apache
spark.apache.org › docs › latest › api › python › reference › pyspark.pandas › api › pyspark.pandas.read_sql_query.html
pyspark.pandas.read_sql_query — PySpark 4.1.1 documentation
Read SQL query into a DataFrame. Returns a DataFrame corresponding to the result set of the query string. Optionally provide an index_col parameter to use one of the columns as the index, otherwise default index will be used. ... SQL query to be executed. ... A JDBC URI could be provided as str.
🌐
AskPython
askpython.com › home › pandas read_sql: read sql query/database table into a dataframe
Pandas read_sql: Read SQL query/database table into a DataFrame - AskPython
January 31, 2023 - You have learned how to use Pandas to read data from files in a variety of formats, including JSON, CSV, Excel, and others, and how to change data in our previous tutorials. One such way is Pandas read_sql(), which enables you to read a SQL ...
Find elsewhere
🌐
MyScale
myscale.com › blog › mastering-pandas-read-sql-query-step-by-step-guide
Mastering pandas read_sql_query: Your Ultimate Guide
This statement allows you to retrieve specific columns from a table in your database. By using the read_sql_query function, you can execute this query directly within Python and store the results in a DataFrame for further analysis.
🌐
Like Geeks
likegeeks.com › home › python › pandas › read sql query/table into dataframe using pandas read_sql
Read SQL Query/Table into DataFrame using Pandas read_sql
October 16, 2023 - In this example, we’re using the read_sql function with the ‘students’ table as the SQL query. The function fetches all rows from the ‘students’ table and converts them into a DataFrame.
🌐
Apache
spark.apache.org › docs › latest › api › python › reference › pyspark.pandas › api › pyspark.pandas.read_sql.html
pyspark.pandas.read_sql — PySpark 4.1.1 documentation
A SQL query will be routed to read_sql_query, while a database table name will be routed to read_sql_table. Note that the delegated function might have more specific notes about their functionality not listed here. ... SQL query to be executed or a table name.
🌐
DataCamp
datacamp.com › tutorial › how-to-use-sql-in-pandas-using-pandasql-queries
How to Use SQL in pandas Using pandasql Queries | DataCamp
May 11, 2023 - We can further increase its readability ... let's see how we can technically combine them both. The pandasql Python library allows querying pandas dataframes by running SQL commands without having to connect to any SQL server....
Top answer
1 of 8
155

This is not what pandas.query is supposed to do. You can look at package pandasql (same like sqldf in R )

Update: Note pandasql hasn't been maintained since 2017. Use another library from an answer below.

import pandas as pd
import pandasql as ps

df = pd.DataFrame([[1234, 'Customer A', '123 Street', np.nan],
               [1234, 'Customer A', np.nan, '333 Street'],
               [1233, 'Customer B', '444 Street', '333 Street'],
              [1233, 'Customer B', '444 Street', '666 Street']], columns=
['ID', 'Customer', 'Billing Address', 'Shipping Address'])

q1 = """SELECT ID FROM df """

print(ps.sqldf(q1, locals()))

     ID
0  1234
1  1234
2  1233
3  1233

Update 2020-07-10

update the pandasql

ps.sqldf("select * from df")
2 of 8
51

Much better solution is to use duckdb. It is much faster than sqldf because it does not have to load the entire data into sqlite and load back to pandas.

Update: duckdb is also faster than polars (which is also a very good solution and people are moving to polars for its performance), see https://benchmark.clickhouse.com/

pip install duckdb
import pandas as pd
import duckdb
test_df = pd.DataFrame.from_dict({"i":[1, 2, 3, 4], "j":["one", "two", "three", "four"]})

duckdb.query("SELECT * FROM test_df where i>2").df() # returns a result dataframe

Performance improvement over pandasql: test data NYC yellow cabs ~120mb of csv data

nyc = pd.read_csv('https://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2021-01.csv',low_memory=False)
from pandasql import sqldf
pysqldf = lambda q: sqldf(q, globals())
pysqldf("SELECT * FROM nyc where trip_distance>10")
# wall time 16.1s
duckdb.query("SELECT * FROM nyc where trip_distance>10").df()
# wall time 183ms

A improvement of speed of roughly 100x

This article gives good details and claims 1000x improvement over pandasql: https://duckdb.org/2021/05/14/sql-on-pandas.html

🌐
Pandas
pandas.pydata.org › pandas-docs › stable › reference › api › pandas.read_sql.html
pandas.read_sql — pandas 2.2.3 documentation - PyData |
>>> pd.read_sql("test_data", "postgres:///db_name") For parameterized query, using params is recommended over string interpolation.
🌐
Pandas
pandas.pydata.org › pandas-docs › version › 0.20 › generated › pandas.read_sql_query.html
pandas.read_sql_query — pandas 0.20.3 documentation
pandas.read_sql_query(sql, con, index_col=None, coerce_float=True, params=None, parse_dates=None, chunksize=None)[source]¶ · Read SQL query into a DataFrame. Returns a DataFrame corresponding to the result set of the query string. Optionally provide an index_col parameter to use one of the ...
🌐
MSSQLTips
mssqltips.com › home › read sql server data into a dataframe using python and pandas
Read SQL Server Data into a Dataframe using Python and Pandas
July 18, 2022 - Inside the query itself, we use ? (question mark) as placeholder indicators. They denote all places where a parameter will be used and should be familiar to you from working with pyodbc. Then, we use the params parameter of the read_sql function, to which we pass a list containing the parameter variables we defined.
Top answer
1 of 2
134

The read_sql docs say this params argument can be a list, tuple or dict (see docs).

To pass the values in the sql query, there are different syntaxes possible: ?, :1, :name, %s, %(name)s (see PEP249).
But not all of these possibilities are supported by all database drivers, which syntax is supported depends on the driver you are using (psycopg2 in your case I suppose).

In your second case, when using a dict, you are using 'named arguments', and according to the psycopg2 documentation, they support the %(name)s style (and so not the :name I suppose), see http://initd.org/psycopg/docs/usage.html#query-parameters.
So using that style should work:

df = psql.read_sql(('select "Timestamp","Value" from "MyTable" '
                     'where "Timestamp" BETWEEN %(dstart)s AND %(dfinish)s'),
                   db,params={"dstart":datetime(2014,6,24,16,0),"dfinish":datetime(2014,6,24,17,0)},
                   index_col=['Timestamp'])
2 of 2
-1

I was having trouble passing a large number of parameters when reading from a SQLite Table. Then it turns out since you pass a string to read_sql, you can just use f-string. Tried the same with MSSQL pyodbc and it works as well.

For SQLite, it would look like this:

# write a sample table into memory
from sqlalchemy import create_engine
df = pd.DataFrame({'Timestamp': pd.date_range('2020-01-17', '2020-04-24', 10), 'Value1': range(10)})
engine = create_engine('sqlite://', echo=False)
df.to_sql('MyTable', engine);

# query the table using a query
tpl = (1, 3, 5, 8, 9)
query = f"""SELECT Timestamp, Value1 FROM MyTable WHERE Value1 IN {tpl}"""
df = pd.read_sql(query, engine)

If the parameters are datetimes, it's a bit more complicated but calling the datetime conversion function of the SQL dialect you're using should do the job.

start, end = '2020-01-01', '2020-04-01'
query = f"""SELECT Timestamp, Value1 FROM MyTable WHERE Timestamp BETWEEN STRFTIME("{start}") AND STRFTIME("{end}")"""
df = pd.read_sql(query, engine)
🌐
Proclus Academy
proclusacademy.com › blog › practical › pandas-read-write-sql-database
Pandas: How to Read and Write Data to a SQL Database | Proclus Academy
April 16, 2023 - That is extremely dangerous as you’ll lose the current data in the table. Therefore use if_exists=‘replace’ with utmost caution! You can read an entire database table into a DataFrame using the method read_sql_table().