Stack Overflow
stackoverflow.com › questions › 54178349 › sample-values-until-getting-the-all-the-unique-values › 54179079
pandas - sample values until getting the all the unique values - Stack Overflow
df = pd.DataFrame([1,1,1,2,3,2,3,2,3,1,4,5,3,4,5,2,3,2], columns=['col']) positions = df['col'].value_counts(normalize=True).to_dict() print (positions) {3: 0.2777777777777778, 2: 0.2777777777777778, 1: 0.2222222222222222, 5: 0.1111111111111111, 4: 0.1111111111111111} def sample(obj, replace=False, total=20): return obj.sample(n=int(positions[obj.name] * total), replace=replace) N = 3 v = df["col"].drop_duplicates().sample(n=N) df1 = df[df['col'].isin(v)].groupby('col', group_keys=False).apply(sample).sort_index() print (df1) col 3 2 4 3 5 2 6 3 7 2 8 3 10 4 12 3 13 4 15 2 16 3 17 2
python - Get sample of Pandas dataframe but keep all unique values - Stack Overflow
Make sure that the sample will have at least 1 row of each unique value from y_true. More on stackoverflow.com
sample dataset with column unique values
Hello, I would like to sample the dataset based on the unique values from a particular column. More on community.dataiku.com
python - Sampling one record per unique value - Stack Overflow
Communities for your favorite technologies. Explore all Collectives · Stack Overflow for Teams is now called Stack Internal. Bring the best of human thought and AI automation together at your work More on stackoverflow.com
python - Random DataFrame sample with at least 1 unique value of a column - Stack Overflow
4 Pandas dataframe, select n random rows based on number of unique values · 2 In python, how to generate random sampling without replacement of a specific column? More on stackoverflow.com
Stack Overflow
stackoverflow.com › questions › 73236209 › get-sample-of-pandas-dataframe-but-keep-all-unique-values
python - Get sample of Pandas dataframe but keep all unique values - Stack Overflow
""" df = pd.concat([X_test, y_true], axis=1) sample_df = df.sample(**args).reset_index(drop=True) target = ( len(y_true.unique()) if sample_df.shape[0] > len(y_true.unique()) else sample_df.shape[0] ) while sample_df.iloc[:, -1].nunique() != target: sample_df = df.sample(**args).reset_index(drop=True) return sample_df def get_x_test_and_y_true(df): """Split dataframe. Args: df: input dataframe. Returns: X_test and y_true dataframes. """ return df.iloc[:, :-1], df.iloc[:, -1] And so, with this toy dataframe which y_true column has 10 unique values (0 to 9) : import random import pandas as pd df
Dataiku Community
community.dataiku.com › questions & discussions › general
sample dataset with column unique values — Dataiku Community
June 20, 2023 - import dataiku import pandas as pd, numpy as np from dataiku import pandasutils as pdu # Read recipe inputs Orders_enriched_prepared = dataiku.Dataset("<YOUR_INPUT_DATASET>") outputdistinct_df = Orders_enriched_prepared.get_dataframe() # get all unique values from the column in question subset = outputdistinct_df['YOUR_COLUMN_NAME'].unique() # gets a list of 100 numbers between 0 and the length of your subset array - 1 randomlist = random.sample(range(0, len(subset) -1), 100) final_column_list = [] # use each index to get random values from your list for index in randomlist: final_column_list.append(subset[index]) # now simply check if your column isin the list of 100 random values outputdistinct_df = outputdistinct_df[outputdistinct_df['YOUR_COLUMN_NAME'].isin(final_column_list)]
Pandas
pandas.pydata.org › docs › reference › api › pandas.Series.unique.html
pandas.Series.unique — pandas 3.0.5 documentation - PyData |
Returns the unique values as a NumPy array. In case of an extension-array backed Series, a new ExtensionArray of that type with just the unique values is returned.
IONOS
ionos.com › digital guide › websites › web development › python pandas: dataframe unique
How to filter for distinct values with pandas DataFrame[].unique()
June 26, 2025 - To use unique() in a pandas DataFrame, you need to first specify the column you want to check. In the following example, we’ll use a DataFrame that contains information about the age and city of residence of a group of individuals. import pandas as pd # Create a sample DataFrame data = { 'Name': ['Alice', 'Bob', 'Charlie', 'David', 'Edward'], 'Age': [24, 27, 22, 32, 29], 'City': ['New York', 'Los Angeles', 'New York', 'Chicago', 'Los Angeles'] } df = pd.DataFrame(data) print(df)python
Pandas
pandas.pydata.org › docs › reference › api › pandas.unique.html
pandas.unique — pandas 3.0.5 documentation
pandas.unique(values)[source]# Return unique values based on a hash table. Uniques are returned in order of appearance. This does NOT sort. Significantly faster than numpy.unique for long enough sequences. Includes NA values. Parameters: values1d array-like ·
Pandas
pandas.pydata.org › docs › reference › api › pandas.DataFrame.sample.html
pandas.DataFrame.sample — pandas 3.0.5 documentation
>>> df.sample(frac=2, replace=True, random_state=1) num_legs num_wings num_specimen_seen dog 4 0 2 fish 0 0 8 falcon 2 2 10 falcon 2 2 10 fish 0 0 8 dog 4 0 2 fish 0 0 8 dog 4 0 2 · Using a DataFrame column as weights. Rows with larger value in the num_specimen_seen column are more likely to be sampled.
Top answer 1 of 3
2
You could use groupby.sample, either to pick out a sample of the whole dataframe for further processing, or to identify rows of the dataframe to mark if that's more convenient.
import pandas as pd
percentage_to_flag = 0.5
# Toy data: 8 rows, persons A and B.
df = pd.DataFrame(data={'persons':['A']*4 + ['B']*4, 'data':range(8)})
# persons data
# 0 A 0
# 1 A 1
# 2 A 2
# 3 A 3
# 4 B 4
# 5 B 5
# 6 B 6
# 7 B 7
# Pick out random sample of dataframe.
random_state = 41 # Change to get different random values.
df_sample = df.groupby("persons").sample(frac=percentage_to_flag,
random_state=random_state)
# persons data
# 1 A 1
# 2 A 2
# 7 B 7
# 6 B 6
# Mark the random sample in the original dataframe.
df["marked"] = False
df.loc[df_sample.index, "marked"] = True
# persons data marked
# 0 A 0 False
# 1 A 1 True
# 2 A 2 True
# 3 A 3 False
# 4 B 4 False
# 5 B 5 False
# 6 B 6 True
# 7 B 7 True
If you really do not want the sub-sampled dataframe df_sample you can go straight to marking a sample of the original dataframe:
# Mark random sample in original dataframe with minimal intermediate data.
df["marked2"] = False
df.loc[df.groupby("persons")["data"].sample(frac=percentage_to_flag,
random_state=random_state).index,
"marked2"] = True
# persons data marked marked2
# 0 A 0 False False
# 1 A 1 True True
# 2 A 2 True True
# 3 A 3 False False
# 4 B 4 False False
# 5 B 5 False False
# 6 B 6 True True
# 7 B 7 True True
2 of 3
1
If I understood you correctly, you can achieve this using:
df = pd.DataFrame(data={'persons':['A']*10 + ['B']*10, 'col_1':[2]*20})
percentage_to_flag = 0.5
a = df.groupby(['persons'])['col_1'].apply(lambda x: pd.Series(x.index.isin(x.sample(frac=percentage_to_flag, random_state= 5, replace=False).index))).reset_index(drop=True)
df['flagged'] = a
Input:
persons col_1
0 A 2
1 A 2
2 A 2
3 A 2
4 A 2
5 A 2
6 A 2
7 A 2
8 A 2
9 A 2
10 B 2
11 B 2
12 B 2
13 B 2
14 B 2
15 B 2
16 B 2
17 B 2
18 B 2
19 B 2
Output with 50% flagged rows in each group:
persons col_1 flagged
0 A 2 False
1 A 2 False
2 A 2 True
3 A 2 False
4 A 2 True
5 A 2 True
6 A 2 False
7 A 2 True
8 A 2 False
9 A 2 True
10 B 2 False
11 B 2 False
12 B 2 True
13 B 2 False
14 B 2 True
15 B 2 True
16 B 2 False
17 B 2 True
18 B 2 False
19 B 2 True

