Mapping a Python dict to a Polars series

stackoverflow.com › questions › 74779644 › mapping-a-python-dict-to-a-polars-series

Update 2024-07-09

Polars has dedicated replace and replace_strict expressions.

Old answer

Mapping a python dictionary over a polars Series should always be considered an anti-pattern. This will be terribly slow and what you want is semantically equal to a join.

Use joins. They are heavily optimized, multithreaded and don't use python.

Example

import polars as pl

dic = { 1: 'a', 2: 'b', 3: 'c' }

mapper = pl.DataFrame({
    "keys": list(dic.keys()),
    "values": list(dic.values())
})

pl.Series([1, 2, 3, 4]).to_frame("keys").join(mapper, on="keys", how="left").to_series(1)

Series: 'values' [str]
[
    "a"
    "b"
    "c"
    null
]

Answer from ritchie46 on Stack Overflow

Stack Overflow

stackoverflow.com › questions › 74779644 › mapping-a-python-dict-to-a-polars-series

dataframe - Mapping a Python dict to a Polars series - Stack Overflow

Update 2024-07-09

Polars has dedicated replace and replace_strict expressions.

Old answer

Mapping a python dictionary over a polars Series should always be considered an anti-pattern. This will be terribly slow and what you want is semantically equal to a join.

Use joins. They are heavily optimized, multithreaded and don't use python.

Example

import polars as pl

dic = { 1: 'a', 2: 'b', 3: 'c' }

mapper = pl.DataFrame({
    "keys": list(dic.keys()),
    "values": list(dic.values())
})

pl.Series([1, 2, 3, 4]).to_frame("keys").join(mapper, on="keys", how="left").to_series(1)

Series: 'values' [str]
[
    "a"
    "b"
    "c"
    null
]

2 of 5

Polars has the Expr.replace_strict method and the Series.replace_strict method which can be used as follows:

mapping_dict = {1: "a", 2: "b", 3: "c"}

# pl.Series.replace_strict
pl.Series([1, 2, 3, 4]).replace_strict(mapping_dict, default=None)

# pl.Expr.replace_strict
pl_df = pl.Series(name="to_map_col", values=[1, 2, 3, 4]).to_frame()

pl_df.with_columns(pl.col("to_map_col").replace_strict(mapping_dict, default=None))

GeeksforGeeks

geeksforgeeks.org › python › mapping-a-python-dict-to-a-polars-series

Mapping a Python Dict to a Polars Series - GeeksforGeeks

July 23, 2025 - Mapping a Python dictionary to a Polars Series is a straightforward process that can be done using the apply method in combination with a lambda function.

Discussions

Polars mapping

It's usually helpful if you can give a small runnable example. It sounds like you want .replace_strict() though. More on reddit.com

r/dataengineering

April 10, 2025

Map DataFrame Column using Python Dicts

I would love to see a map function in polars similar to pandas map function. I think this is a very useful and common operation. Maybe also add a default_value if the key is not found in the dict (although this can be fixed with an additional fill_null). More on github.com

github.com

December 15, 2022

Change default behavior `map_dict`

Problem description The current default behavior of map_dict is to only replace the matching values, if it's not in the dict, it will return Nones. With the possible renaming to replace_values #107... More on github.com

github.com

August 28, 2023

Dict/Hashmap lookup expression

With Pandas, I can use Series.map to create a series that maps the contents of the initial column with the key of a Python dictionary and contains the value. Using Polars, that's doable, but a fair amount more involved, because I need to cast both columns as Categorical and perform a join within ... More on github.com

github.com

June 23, 2022

Polars

docs.pola.rs › api › python › version › 0.19 › reference › series › api › polars.Series.map_dict.html

polars.Series.map_dict — Polars documentation

mapping: dict[Any, Any], *, default: Any = None, return_dtype: PolarsDataType | None = None, ) → Self[source]# Replace values in the Series using a remapping dictionary. Deprecated since version 0.19.16: This method has been renamed to replace(). The default behavior has changed to keep any values not present in the mapping unchanged.

reddit.com › r/dataengineering › polars mapping

r/dataengineering on Reddit: Polars mapping

April 10, 2025 -

I am relatively new to python. I’m trying to map a column of integers to string values defined in a dictionary.

I’m using polars and this is seemingly more difficult that I first anticipated. can anyone give advice on how to do this?

Top answer

1 of 1

It's usually helpful if you can give a small runnable example. It sounds like you want .replace_strict() though.

Polars

docs.pola.rs › api › python › version › 0.18 › reference › expressions › api › polars.Expr.map_dict.html

polars.Expr.map_dict — Polars documentation

Expr.map_dict( remapping: dict[Any, Any], *, default: Any = None, return_dtype: PolarsDataType | None = None, ) → Self[source]# Replace values in column according to remapping dictionary. Needs a global string cache for lazily evaluated queries on columns of type pl.Categorical.

GitHub

github.com › pola-rs › polars › issues › 5822

Map DataFrame Column using Python Dicts · Issue #5822 · pola-rs/polars

December 15, 2022 - COUNTRY_CODE_MAPPING = { "DE": "Germany", "FR": "France", "US": "United States", } STOCK_CODE_MAPPING = { -1: "On Request", 0: "Out of Stock", 1: "In Stock", } df = pl.DataFrame({ "country_code": ["DE", "FR", "US", "FR"], "stock_code": [1, 0, -1, 1], }) # using apply (works but not efficient/fast) df.with_columns([ pl.col("country_code").apply(COUNTRY_CODE_MAPPING.get).alias("country"), pl.col("stock_code").apply(STOCK_CODE_MAPPING.get).alias("stock"), ]) # using pandas df.assign( country=df["country_code"].map(COUNTRY_CODE_MAPPING), stock=df["stock_code"].map(STOCK_CODE_MAPPING), )

Author pola-rs

Polars

docs.pola.rs › api › python › dev › reference › api › polars.from_dict.html

polars.from_dict — Polars documentation

polars.from_dict( data: Mapping[str, Sequence[object] | Mapping[str, Sequence[object]] | Series], schema: SchemaDefinition | None = None, *, schema_overrides: SchemaDict | None = None, strict: bool = True, ) → DataFrame[source]# Construct a DataFrame from a dictionary of sequences.

GitHub

github.com › pola-rs › polars › issues › 10755

Change default behavior `map_dict` · Issue #10755 · pola-rs/polars

August 28, 2023 - The current default behavior of map_dict is to only replace the matching values, if it's not in the dict, it will return Nones. With the possible renaming to replace_values #10744 (comment), setting the default parameter, to pl.first() would make more sense. default=pl.first().

Author pola-rs

Find elsewhere

Google Bing Mojeek

Spark By {Examples}

sparkbyexamples.com › home › polars › mapping a python dict to a polars series

Mapping a Python Dict to a Polars Series - Spark By {Examples}

May 1, 2025 - Mapping a Python dictionary to a Polars Series allows you to replace specific values in the Series based on a key-value relationship defined in the

GitHub

github.com › pola-rs › polars › issues › 3789

Dict/Hashmap lookup expression · Issue #3789 · pola-rs/polars

June 23, 2022 - A new expression method, maybe something like Expr.lookup(map: dict[str | int, ...]) would make this sort of operation doable in a single step. An extra argument, like lookup(map, on_missing: Literal['omit','null','error']) could also be useful to specify the behavior when the hashmap does not contain anything.

Author pola-rs

Polars

docs.pola.rs › py-polars › html › reference › dataframe › api › polars.DataFrame.to_dict.html

polars.DataFrame.to_dict — Polars documentation

Convert DataFrame to a dictionary mapping column name to values.

Polars

docs.pola.rs › docs › python › dev › reference › api › polars.from_dicts.html

polars.from_dicts — Polars documentation

polars.from_dicts( data: Sequence[dict[str, Any]], schema: SchemaDefinition | None = None, *, schema_overrides: SchemaDict | None = None, strict: bool = True, infer_schema_length: int | None = 100, ) → DataFrame[source]# Construct a DataFrame from a sequence of dictionaries. This operation clones data. Parameters: data · Sequence with dictionaries mapping column name to value ·

GitHub

github.com › pola-rs › polars › issues › 7389

`map_dict` add rows if dtype and dict keys mismatch · Issue #7389 · pola-rs/polars

March 7, 2023 - In some cases, it will map a null to every value in the dictionary. import polars as pl s = pl.Series(values = [None, '0'], dtype=pl.Utf8) int_map = {0: 'a', 1: 'b'} bool_map = {True: 'a', False: 'b'} str_map = {'0': 'a', '1': 'b'} # Adds row when dtype of keys does not match dtype of column print( s.map_dict(int_map), # shape: (3,) Series: '' [str] [ "a" "b" null ] s.map_dict(bool_map), # shape: (3,) Series: '' [str] [ "a" "b" null ] ) # For each null in s, adds all keys of dict larger_int_map = {0: 'a', 1: 'b', 2: 'c', 3: 'd'} print( s.map_dict(larger_int_map), # shape: (5,) Series: '' [str]

Author pola-rs

Polars

docs.pola.rs › api › python › version › 0.19 › reference › expressions › api › polars.Expr.replace.html

polars.Expr.replace — Polars documentation

mapping: dict[Any, Any], *, default: Any = _NoDefault.no_default, return_dtype: PolarsDataType | None = None, ) → Self[source]# Replace values according to the given mapping. Needs a global string cache for lazily evaluated queries on columns of type Categorical.

Polars

docs.pola.rs › api › python › version › 0.18 › reference › expressions › api › polars.Expr.map.html

polars.Expr.map — Polars documentation

map_dict · Examples · >>> df = pl.DataFrame( ... { ... "sine": [0.0, 1.0, 0.0, -1.0], ... "cosine": [1.0, 0.0, -1.0, 0.0], ...

Polars

docs.pola.rs › api › python › version › 0.19 › reference › series › index.html

Series — Polars documentation

map_dict( mapping: dict[Any, Any], *, default: Any = None, return_dtype:

GitHub

github.com › pola-rs › polars › issues › 7077

Expr.map_dict doesn't work on categorical columns · Issue #7077 · pola-rs/polars

February 21, 2023 - ComputeError Traceback (most recent call last) Cell In[43], line 15 7 with pl.StringCache(): 8 df = pl.DataFrame( 9 { 10 "country_code": ["FR", None, "ES", "DE"], 11 }, 12 schema = {"country_code": pl.Categorical} 13 ) ---> 15 df.with_columns( 16 pl.col("country_code").map_dict(country_code_dict).alias("remapped") 17 ) File /usr/local/lib/python3.10/site-packages/polars/internals/dataframe/frame.py:6022, in DataFrame.with_columns(self, exprs, *more_exprs, **named_exprs) 5871 def with_columns( 5872 self, 5873 exprs: IntoExpr | Iterable[IntoExpr] = None, 5874 *more_exprs: IntoExpr, 5875 **named_exprs: IntoExpr, 5876 ) -> Self: 5877 """ 5878 Add columns to this DataFrame.

Author pola-rs

Ithy

ithy.com › article › polars-dict-mapping-p1dv7d0x

Ithy - Mapping a Dictionary to a DataFrame Column in Polars

This is frequently needed in machine ... of Polars, there are two primary methods to perform this mapping: using replace_strict and using the map expression....

GitHub

github.com › pola-rs › polars › issues › 12410

`.map_dict` ignores `return_dtype=` · Issue #12410 · pola-rs/polars

November 13, 2023 - import polars as pl mapping = {"a": {"b": "c"}} return_dtype = pl.Struct({"b": pl.Utf8}) pl.select( pl.lit("a").map_dict(mapping, return_dtype=return_dtype) .struct["b"] ) # polars.exceptions.StructFieldNotFoundError: b

Author pola-rs

Polars

docs.pola.rs › py-polars › html › reference › dataframe › api › polars.DataFrame.to_dicts.html

polars.DataFrame.to_dicts — Polars documentation

>>> df = pl.DataFrame({"foo": [1, 2, 3], "bar": [4, 5, 6]}) >>> df.to_dicts() [{'foo': 1, 'bar': 4}, {'foo': 2, 'bar': 5}, {'foo': 3, 'bar': 6}]