Mapping a Python dict to a Polars series

stackoverflow.com › questions › 74779644 › mapping-a-python-dict-to-a-polars-series

Update 2024-07-09

Polars has dedicated replace and replace_strict expressions.

Old answer

Mapping a python dictionary over a polars Series should always be considered an anti-pattern. This will be terribly slow and what you want is semantically equal to a join.

Use joins. They are heavily optimized, multithreaded and don't use python.

Example

import polars as pl

dic = { 1: 'a', 2: 'b', 3: 'c' }

mapper = pl.DataFrame({
    "keys": list(dic.keys()),
    "values": list(dic.values())
})

pl.Series([1, 2, 3, 4]).to_frame("keys").join(mapper, on="keys", how="left").to_series(1)

Series: 'values' [str]
[
    "a"
    "b"
    "c"
    null
]

Answer from ritchie46 on Stack Overflow

Polars

docs.pola.rs › api › python › version › 0.18 › reference › expressions › api › polars.Expr.map_dict.html

polars.Expr.map_dict — Polars documentation

) shape: (4, 3) ┌────────┬──────────────┬───────────────┐ │ row_nr ┆ country_code ┆ remapped │ │ --- ┆ --- ┆ --- │ │ u32 ┆ str ┆ str │ ╞════════╪══════════════╪═══════════════╡ │ 0 ┆ FR ┆ France │ │ 1 ┆ null ┆ Not specified │ │ 2 ┆ ES ┆ null │ │ 3 ┆ DE ┆ Germany │ └────────┴──────────────┴───────────────┘ · Set a default value for values that cannot be mapped… · >>> df.with_columns( ... pl.col("country_code") ... .map_dict(country_code_dict, default="unknown") ...

Polars

docs.pola.rs › docs › python › version › 0.19 › reference › series › api › polars.Series.map_dict.html

polars.Series.map_dict — Polars documentation

Replace values in the Series using a remapping dictionary. Deprecated since version 0.19.16: This method has been renamed to replace(). The default behavior has changed to keep any values not present in the mapping unchanged.

Discussions

Change default behavior `map_dict`

Problem description The current default behavior of map_dict is to only replace the matching values, if it's not in the dict, it will return Nones. With the possible renaming to replace_values #107... More on github.com

github.com

August 28, 2023

Polars mapping

It's usually helpful if you can give a small runnable example. It sounds like you want .replace_strict() though. More on reddit.com

r/dataengineering

April 10, 2025

Map DataFrame Column using Python Dicts

Problem description similar to here (#5815) but much more general it would be great if you could map column values of any type to any type with simple python dicts. A simple but unfortunately rathe... More on github.com

github.com

December 15, 2022

Expr.map_dict doesn't work on categorical columns

I have confirmed this bug exists on the latest version of Polars. ... I've been using polars for a while and find it amazingly useful :) However, I found some weird behavior I expect is not intended. More specifically, polars.Expr.map_dict doesn't work for Categorical type columns. More on github.com

github.com

February 21, 2023

Stack Overflow

stackoverflow.com › questions › 74779644 › mapping-a-python-dict-to-a-polars-series

dataframe - Mapping a Python dict to a Polars series - Stack Overflow

Update 2024-07-09

Polars has dedicated replace and replace_strict expressions.

Old answer

Mapping a python dictionary over a polars Series should always be considered an anti-pattern. This will be terribly slow and what you want is semantically equal to a join.

Use joins. They are heavily optimized, multithreaded and don't use python.

Example

import polars as pl

dic = { 1: 'a', 2: 'b', 3: 'c' }

mapper = pl.DataFrame({
    "keys": list(dic.keys()),
    "values": list(dic.values())
})

pl.Series([1, 2, 3, 4]).to_frame("keys").join(mapper, on="keys", how="left").to_series(1)

Series: 'values' [str]
[
    "a"
    "b"
    "c"
    null
]

2 of 5

Polars has the Expr.replace_strict method and the Series.replace_strict method which can be used as follows:

mapping_dict = {1: "a", 2: "b", 3: "c"}

# pl.Series.replace_strict
pl.Series([1, 2, 3, 4]).replace_strict(mapping_dict, default=None)

# pl.Expr.replace_strict
pl_df = pl.Series(name="to_map_col", values=[1, 2, 3, 4]).to_frame()

pl_df.with_columns(pl.col("to_map_col").replace_strict(mapping_dict, default=None))

Polars

docs.pola.rs › api › python › version › 0.18 › reference › expressions › api › polars.Expr.map.html

polars.Expr.map — Polars documentation

map_dict · Examples · >>> df = pl.DataFrame( ... { ... "sine": [0.0, 1.0, 0.0, -1.0], ... "cosine": [1.0, 0.0, -1.0, 0.0], ...

GitHub

github.com › pola-rs › polars › issues › 10755

Change default behavior `map_dict` · Issue #10755 · pola-rs/polars

August 28, 2023 - The current default behavior of map_dict is to only replace the matching values, if it's not in the dict, it will return Nones. With the possible renaming to replace_values #10744 (comment), setting the default parameter, to pl.first() would ...

Author pola-rs

GeeksforGeeks

geeksforgeeks.org › python › mapping-a-python-dict-to-a-polars-series

Mapping a Python Dict to a Polars Series - GeeksforGeeks

July 23, 2025 - In this article, we’ll explore how to map a Python dictionary to a Polars Series, which is a column in a Polars DataFrame.

reddit.com › r/dataengineering › polars mapping

r/dataengineering on Reddit: Polars mapping

April 10, 2025 -

I am relatively new to python. I’m trying to map a column of integers to string values defined in a dictionary.

I’m using polars and this is seemingly more difficult that I first anticipated. can anyone give advice on how to do this?

Top answer

1 of 1

It's usually helpful if you can give a small runnable example. It sounds like you want .replace_strict() though.

Polars

docs.pola.rs › py-polars › html › reference › dataframe › api › polars.DataFrame.to_dict.html

polars.DataFrame.to_dict — Polars documentation

Convert DataFrame to a dictionary mapping column name to values · True -> Values are Series False -> Values are List[Any]

Find elsewhere

Google Bing Mojeek

GitHub

github.com › pola-rs › polars › issues › 5822

Map DataFrame Column using Python Dicts · Issue #5822 · pola-rs/polars

December 15, 2022 - Problem description similar to here (#5815) but much more general it would be great if you could map column values of any type to any type with simple python dicts. A simple but unfortunately rather slow solution is using apply. A much f...

Author pola-rs

GitHub

github.com › pola-rs › polars › issues › 7077

Expr.map_dict doesn't work on categorical columns · Issue #7077 · pola-rs/polars

February 21, 2023 - ---Version info--- Polars: 0.16.7 Index type: UInt32 Platform: Linux-5.15.49-linuxkit-aarch64-with-glibc2.28 Python: 3.10.10 (main, Feb 9 2023, 07:44:20) [GCC 8.3.0] ---Optional dependencies--- pyarrow: 10.0.1 pandas: 1.5.3 numpy: 1.24.2 fsspec: <not installed> connectorx: <not installed> xlsx2csv: <not installed> deltalake: <not installed> matplotlib: <not installed>

Author pola-rs

Polars

docs.pola.rs › api › python › version › 0.18 › reference › expressions › api › polars.map.html

polars.map — Polars documentation

"b": [4, 5, 6, 7], ... } ... ) >>> >>> df.with_columns( ... ( ... pl.struct(["a", "b"]).map( ... lambda x: test_func(x.struct.field("a"), x.struct.field("b"), 1) ... ) ... ).alias("a+b+c") ...

Polars

docs.pola.rs › docs › python › dev › reference › api › polars.from_dicts.html

polars.from_dicts — Polars documentation

polars.from_dicts( data: Sequence[dict[str, Any]], schema: SchemaDefinition | None = None, *, schema_overrides: SchemaDict | None = None, strict: bool = True, infer_schema_length: int | None = 100, ) → DataFrame[source]# Construct a DataFrame from a sequence of dictionaries. This operation clones data. Parameters: data · Sequence with dictionaries mapping column name to value ·

Polars

docs.pola.rs › api › python › dev › reference › api › polars.from_dict.html

polars.from_dict — Polars documentation

Choose version · GitHub · Discord · X/Twitter · Bluesky · polars.from_dict( data: Mapping[str, Sequence[object] | Mapping[str, Sequence[object]] | Series], schema: SchemaDefinition | None = None, *, schema_overrides: SchemaDict | None = None, strict: bool = True, ) → DataFrame[source]# Construct a DataFrame from a dictionary of sequences.

Ithy

ithy.com › article › polars-dict-mapping-p1dv7d0x

Ithy - Mapping a Dictionary to a DataFrame Column in Polars

When working with categories, you might have a mapping dictionary, for example: ... Applying such a mapping on a DataFrame allows for swift conversion of textual or categorical data into numerical formats or other forms required for further analysis. This is frequently needed in machine learning pipelines, statistical analyses, or even for visual representations like bar charts and histograms. In the recent versions of Polars, there are two primary methods to perform this mapping: using replace_strict and using the map expression.

GitHub

github.com › pola-rs › polars › issues › 12410

`.map_dict` ignores `return_dtype=` · Issue #12410 · pola-rs/polars

November 13, 2023 - As .map_dict uses .map_batches internally, it seems like return_dtype needs to be passed along? ... shape: (1, 1) ┌─────┐ │ b │ │ --- │ │ str │ ╞═════╡ │ c │ └─────┘

Author pola-rs

Spark By {Examples}

sparkbyexamples.com › home › polars › mapping a python dict to a polars series

Mapping a Python Dict to a Polars Series - Spark By {Examples}

May 1, 2025 - Mapping a Python dictionary to a Polars Series allows you to replace specific values in the Series based on a key-value relationship defined in the

GitHub

github.com › MarcoGorelli › polars-upgrade

GitHub - MarcoGorelli/polars-upgrade: Automatically upgrade your Polars code to use the latest syntax available · GitHub

- pl.col('a').shift(periods=4) + pl.col('a').shift(n=4) - pl.col('a').shift_and_fill(periods=4) + pl.col('a').shift_and_fill(n=4) - pl.col('a').list.shift(periods=4) + pl.col('a').list.shift(n=4) - pl.col('a').map_dict(remapping={1: 2}) + pl.col('a').map_dict(mapping={1: 2})

Author MarcoGorelli

Polars

docs.pola.rs › api › python › version › 0.19 › reference › expressions › api › polars.Expr.replace.html

polars.Expr.replace — Polars documentation

mapping: dict[Any, Any], *, default: Any = _NoDefault.no_default, return_dtype: PolarsDataType | None = None, ) → Self[source]# Replace values according to the given mapping. Needs a global string cache for lazily evaluated queries on columns of type Categorical.

GitHub

github.com › pola-rs › polars › issues › 7389

`map_dict` add rows if dtype and dict keys mismatch · Issue #7389 · pola-rs/polars

March 7, 2023 - I guess technically it can make sense that null matches anything in the same way that the empty string matches everything, but I would always expect map_dict to keep the amount of rows stable. I'm not exactly sure what to expect or what the best solution is, but I would at least expect it to behave consistently no matter the type. ... ---Version info--- Polars: 0.16.11 Index type: UInt32 Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.35 Python: 3.9.14 (main, Sep 7 2022, 23:43:48) [GCC 11.2.0] ---Optional dependencies--- pyarrow: 10.0.0 pandas: 1.3.5 numpy: 1.23.5 fsspec: 2022.11.0 connectorx: <not installed> xlsx2csv: <not installed> deltalake: <not installed> matplotlib: 3.4.0

Author pola-rs

GitHub

github.com › pola-rs › polars › issues › 3789

Dict/Hashmap lookup expression · Issue #3789 · pola-rs/polars

June 23, 2022 - A new expression method, maybe something like Expr.lookup(map: dict[str | int, ...]) would make this sort of operation doable in a single step. An extra argument, like lookup(map, on_missing: Literal['omit','null','error']) could also be useful to specify the behavior when the hashmap does not contain anything.

Author pola-rs