When Polars assigns the pl.Object type it essentially means: "I do not understand what this is."
By the time you end up with this type, it is generally too late to do anything useful with it.
In this particular case, numpy.random.choice is creating a numpy array of dtype=object
>>> rng.choice([None, "foo"], 3)
array([None, None, 'foo'], dtype=object)
Polars has native .sample() functionality which you could use to create your data instead.
df = pl.select(date =
pl.Series([None, "03.04.1998", "03.05.1834", "05.06.2025"])
.sample(100, with_replacement=True)
)
# shape: (100, 1)
# โโโโโโโโโโโโโโ
# โ date โ
# โ --- โ
# โ str โ
# โโโโโโโโโโโโโโก
# โ null โ
# โ 05.06.2025 โ
# โ 03.05.1834 โ
# โ 03.04.1998 โ
# โ โฆ โ
# โ null โ
# โ 03.04.1998 โ
# โ 03.05.1834 โ
# โ 03.04.1998 โ
# โโโโโโโโโโโโโโ
Answer from jqurious on Stack OverflowWhen Polars assigns the pl.Object type it essentially means: "I do not understand what this is."
By the time you end up with this type, it is generally too late to do anything useful with it.
In this particular case, numpy.random.choice is creating a numpy array of dtype=object
>>> rng.choice([None, "foo"], 3)
array([None, None, 'foo'], dtype=object)
Polars has native .sample() functionality which you could use to create your data instead.
df = pl.select(date =
pl.Series([None, "03.04.1998", "03.05.1834", "05.06.2025"])
.sample(100, with_replacement=True)
)
# shape: (100, 1)
# โโโโโโโโโโโโโโ
# โ date โ
# โ --- โ
# โ str โ
# โโโโโโโโโโโโโโก
# โ null โ
# โ 05.06.2025 โ
# โ 03.05.1834 โ
# โ 03.04.1998 โ
# โ โฆ โ
# โ null โ
# โ 03.04.1998 โ
# โ 03.05.1834 โ
# โ 03.04.1998 โ
# โโโโโโโโโโโโโโ
Looks like a bug in Polars, could you report it to their GitHub please?
For now you can use tolist
In [24]: import numpy as np
...: import polars as pl
...:
...: rng = np.random.default_rng(12345)
...: df = pl.LazyFrame(
...: data={
...: "date": rng.choice(
...: [None, "03.04.1998", "03.05.1834", "05.06.2025"], 100
...: ).tolist(),
...: },
...: )
In [25]: df
Out[25]: <LazyFrame [1 col, {"date": Utf8}] at 0x7F6C4A588580>
In [26]: df.collect()
Out[26]:
shape: (100, 1)
โโโโโโโโโโโโโโ
โ date โ
โ --- โ
โ str โ
โโโโโโโโโโโโโโก
โ 03.05.1834 โ
โ null โ
โ 05.06.2025 โ
โ 03.04.1998 โ
โ โฆ โ
โ null โ
โ null โ
โ null โ
โ 03.05.1834 โ
โโโโโโโโโโโโโโ
I have a series with numbers that are coming back as str which I'm converting to int and it's showing a number with a comma as null. I'm not sure how to avoid this.
Example:
| 131 |
|---|
| 302 |
| 3,940 |
I've done the following to convert this data to int.
df.with_column(pl.col('column').cast(pl.Int64, strict=False))I get this as a result as int:
| 131 |
|---|
| 302 |
| null |