Reading all data in a csv to any other type than pl.String likely fails with a lot of null values. We can use expressions to declare how we want to deal with those null values.

If you read a csv with infer_schema_length=0, polars does not know the schema and will read all columns as pl.String as that is a super type of all polars types.

When read as String we can use expressions to cast all columns.

(pl.read_csv("test.csv", infer_schema_length=0)
   .with_columns(pl.all().cast(pl.Int32, strict=False))

Update: infer_schema=False was added in 1.2.0 as a more user-friendly name for this feature.

pl.read_csv("test.csv", infer_schema=False) # read all as pl.String
Answer from ritchie46 on Stack Overflow
🌐
Polars
docs.pola.rs › py-polars › html › reference › datatypes.html
Data types — Polars documentation
Base class for all Polars data types · Decimal 128-bit type with an optional precision and non-negative scale
🌐
Polars
docs.pola.rs › user-guide › concepts › data-types-and-structures
Data types and structures - Polars user guide
When creating a series, Polars will infer the data type from the values you provide. You can specify a concrete data type to override the inference mechanism: ... s1 = pl.Series("ints", [1, 2, 3, 4, 5]) s2 = pl.Series("uints", [1, 2, 3, 4, 5], dtype=pl.UInt64) print(s1.dtype, s2.dtype)
🌐
Polars
pola.rs › posts › understanding-polars-data-types
Polars — Understanding Polars data types
That is because Polars stores a validity mask with Boolean values indicating which items in a series are missing. This validity mask is bit-packed in the same efficient manner as Boolean columns, so the memory overhead is minimal: s1 = pl.Series([1, 2, 3, 4, 5, 6, 7, 8], dtype=pl.Int64) # 8 bytes x 8 integers = 64 bytes print(s1.estimated_size()) # 64 # 64 bytes for the 8 integers Int64, plus the validity mask: # 1 bit x 8 integers = 1 byte; total: 64 + 1 = 65 s2 = pl.Series([1, 2, 3, 4, None, 6, 7, 8], dtype=pl.Int64) print(s2.estimated_size()) # 65 print(s2.is_null())
🌐
Polars
docs.pola.rs › py-polars › html › reference › dataframe › api › polars.DataFrame.dtypes.html
polars.DataFrame.dtypes — Polars documentation
>>> df = pl.DataFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> df.dtypes [Int64, Float64, String] >>> df shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ ...
🌐
Rust
docs.rs › polars › latest › polars › datatypes › index.html
polars::datatypes - Rust
At the moment Polars doesn’t include all data types available by Arrow. The goal is to incrementally support more data types and prioritize these by usability. See the AnyValue variants for the data types that are currently supported. categorical · dtype-categorical ·
🌐
Rho Signal
rhosignal.com › posts › polars-nested-dtypes
Nested dtypes in Polars 1: the `pl.List` dtype | Rho Signal
December 22, 2024 - Polars uses Apache Arrow to store its data in-memory. One of the big advantages of Arrow is that it supports a variety of nested data types (or “dtypes”). In this post we look at the pl.List dtype in more detail: we start with an overview of the pl.List dtype we call expressions on each row of a pl.List column we do aggregations with neural network embeddings we do simple text analytics
🌐
Polars
docs.pola.rs › api › python › dev › reference › lazyframe › api › polars.LazyFrame.dtypes.html
polars.LazyFrame.dtypes — Polars documentation
>>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.dtypes [Int64, Float64, String]
Find elsewhere
🌐
O'Reilly
oreilly.com › library › view › python-polars-the › 9781098156077 › ch04.html
4. Data Structures and Data Types - Python Polars: The Definitive Guide [Book]
February 20, 2025 - Data comes in many shapes and sizes, all of which need to be stored in proper structures in order to work with them. To accommodate all the data you’ll be working with, Polars implements the Arrow memory specification, which provides a vast array of data types.
Authors   Jeroen JanssensThijs Nieuwdorp
Published   2025
Pages   504
🌐
Polars
docs.pola.rs › py-polars › html › reference › series › api › polars.Series.dtype.html
polars.Series.dtype — Polars documentation
Series.dtype[source]# Get the data type of this Series. Examples · >>> s = pl.Series("a", [1, 2, 3]) >>> s.dtype Int64 ·
🌐
Polars
docs.pola.rs › py-polars › html › reference › dataframe › api › polars.DataFrame.cast.html
polars.DataFrame.cast — Polars documentation
dtypes: Mapping[ColumnNameOrSelector | PolarsDataType, PolarsDataType] | PolarsDataType, *, strict: bool = True, ) → DataFrame[source]# Cast DataFrame column(s) to the specified dtype(s). Parameters: dtypes · Mapping of column names (or selector) to dtypes, or a single dtype to which all columns will be cast.
🌐
Rho Signal
rhosignal.com › posts › polars-list-dtypes
Don't fear the List dtype in Polars | Rho Signal
December 22, 2024 - Every row in a pl.Object dtype is just a python list under the hood. While they can hold arbitrary objects these Object columns won’t be so efficient for analysis. In summary, fear the lists (and maybe fear the pl.Object), but don’t fear the pl.Lists! Want to know more about Polars for high performance data science and ML?
🌐
Polars
docs.pola.rs › api › python › dev › reference › api › polars.datatypes.Struct.html
polars.datatypes.Struct — Polars documentation
class polars.datatypes.Struct(fields: Sequence[Field] | SchemaDict)[source]# Struct composite type. Parameters: fields · The fields that make up the struct. Can be either a sequence of Field objects or a mapping of column names to data types. Examples · Initialize using a dictionary: >>> dtype = pl.Struct({"a": pl.Int8, "b": pl.List(pl.String)}) >>> dtype Struct({'a': Int8, 'b': List(String)}) Initialize using a list of Field objects: >>> dtype = pl.Struct([pl.Field("a", pl.Int8), pl.Field("b", pl.List(pl.String))]) >>> dtype Struct({'a': Int8, 'b': List(String)}) When initializing a Series, Polars can infer a struct data type from the data.
🌐
GitHub
github.com › pola-rs › polars › issues › 4982
Get column dtype in expression · Issue #4982 · pola-rs/polars
September 25, 2022 - pl.DataFrame({'a': [date(2022, 1, 1)]}).select( (col('a') + pl.duration(days=1)).cast(col('a').dtype) # convert back to column a's dtype )
Author   pola-rs
🌐
Polars
docs.pola.rs › docs › python › dev › reference › api › polars.datatypes.Datetime.html
polars.datatypes.Datetime — Polars documentation
class polars.datatypes.Datetime(time_unit: TimeUnit = 'us', time_zone: str | timezone | None = None)[source]# Data type representing a calendar date and time of day. ... Unit of time. Defaults to 'us' (microseconds). ... Time zone string, as defined in zoneinfo (to see valid strings run import zoneinfo; zoneinfo.available_timezones() for a full list). When used to match dtypes, can set this to “*” to check for Datetime columns that have any (non-null) timezone.
🌐
Spark By {Examples}
sparkbyexamples.com › home › polars › how to select columns by data type in polars
How to Select Columns by Data Type in Polars - Spark By {Examples}
March 5, 2025 - In Polars, you can select columns by their data type using the select() method along with the pl.col() function. This is useful when you want to operate