pandas sample index python

return the index using pandas series.sample()?

stackoverflow.com › questions › 45968529 › return-the-index-using-pandas-series-sample

As @user48956 commented on accepted answer, is much faster to sample over the index using numpy.random.choice

np.random.seed(42)
df = pd.DataFrame(np.random.randint(0,100,size=(10000000, 4)), columns=list('ABCD'))
%time df.sample(100000).index
print(_)
%time pd.Index(np.random.choice(df.index, 100000))

Wall time: 710 ms
Int64Index([7141956, 9256789, 1919656, 2407372, 9181191, 2474961, 2345700,
            4394530, 8864037, 6096638,
            ...
             471501, 3616956, 9397742, 6896140,  670892, 9546169, 4146996,
            3465455, 7748682, 5271367],
           dtype='int64', length=100000)
Wall time: 6.05 ms

Int64Index([7141956, 9256789, 1919656, 2407372, 9181191, 2474961, 2345700,
            4394530, 8864037, 6096638,
            ...
             471501, 3616956, 9397742, 6896140,  670892, 9546169, 4146996,
            3465455, 7748682, 5271367],
           dtype='int64', length=100000)

Answer from Ameb on Stack Overflow

Stack Overflow

stackoverflow.com › questions › 45968529 › return-the-index-using-pandas-series-sample

python - return the index using pandas series.sample()? - Stack Overflow

Top answer

1 of 2

6

As @user48956 commented on accepted answer, is much faster to sample over the index using numpy.random.choice

np.random.seed(42)
df = pd.DataFrame(np.random.randint(0,100,size=(10000000, 4)), columns=list('ABCD'))
%time df.sample(100000).index
print(_)
%time pd.Index(np.random.choice(df.index, 100000))

Wall time: 710 ms
Int64Index([7141956, 9256789, 1919656, 2407372, 9181191, 2474961, 2345700,
            4394530, 8864037, 6096638,
            ...
             471501, 3616956, 9397742, 6896140,  670892, 9546169, 4146996,
            3465455, 7748682, 5271367],
           dtype='int64', length=100000)
Wall time: 6.05 ms

Int64Index([7141956, 9256789, 1919656, 2407372, 9181191, 2474961, 2345700,
            4394530, 8864037, 6096638,
            ...
             471501, 3616956, 9397742, 6896140,  670892, 9546169, 4146996,
            3465455, 7748682, 5271367],
           dtype='int64', length=100000)

2 of 2

5

Let's try and use .index after sample to return the indexes of those samples:

sample_dict[user] = series[series==user].sample(n=sample_size, replace=False).index

python - How to return the Index when slicing Pandas Dataframe

Python Pandas - series indexing notation

python - pandas series can't get index

python - Series Index Pandas

python 3.x - Returning the index value in Pandas

More results from stackoverflow.com

Pandas

pandas.pydata.org › docs › reference › api › pandas.DataFrame.sample.html

pandas.DataFrame.sample — pandas 3.0.5 documentation

E.g. sampling 2 items without replacement with weights [100, 1, 1] would yield two last items in 1/2 of cases, instead of 1/102. This is similar to specifying n=4 without replacement on a Series with 3 elements. ... >>> df = pd.DataFrame( ... { ... "num_legs": [2, 4, 8, 0], ... "num_wings": [2, 0, 0, 0], ... "num_specimen_seen": [10, 2, 1, 8], ... }, ... index=["falcon", "dog", "spider", "fish"], ...

Videos