You could convert the dataframe to be a single column with stack (this changes the shape from 5x3 to 15x1) and then take the standard deviation:
df.stack().std() # pandas default degrees of freedom is one
Alternatively, you can use values to convert from a pandas dataframe to a numpy array before taking the standard deviation:
df.values.std(ddof=1) # numpy default degrees of freedom is zero
Unlike pandas, numpy will give the standard deviation of the entire array by default, so there is no need to reshape before taking the standard deviation.
A couple of additional notes:
The numpy approach here is a bit faster than the pandas one, which is generally true when you have the option to accomplish the same thing with either numpy or pandas. The speed difference will depend on the size of your data, but numpy was roughly 10x faster when I tested a few different sized dataframes on my laptop (numpy version 1.15.4 and pandas version 0.23.4).
The numpy and pandas approaches here will not give exactly the same answers, but will be extremely close (identical at several digits of precision). The discrepancy is due to slight differences in implementation behind the scenes that affect how the floating point values get rounded.
You could convert the dataframe to be a single column with stack (this changes the shape from 5x3 to 15x1) and then take the standard deviation:
df.stack().std() # pandas default degrees of freedom is one
Alternatively, you can use values to convert from a pandas dataframe to a numpy array before taking the standard deviation:
df.values.std(ddof=1) # numpy default degrees of freedom is zero
Unlike pandas, numpy will give the standard deviation of the entire array by default, so there is no need to reshape before taking the standard deviation.
A couple of additional notes:
The numpy approach here is a bit faster than the pandas one, which is generally true when you have the option to accomplish the same thing with either numpy or pandas. The speed difference will depend on the size of your data, but numpy was roughly 10x faster when I tested a few different sized dataframes on my laptop (numpy version 1.15.4 and pandas version 0.23.4).
The numpy and pandas approaches here will not give exactly the same answers, but will be extremely close (identical at several digits of precision). The discrepancy is due to slight differences in implementation behind the scenes that affect how the floating point values get rounded.
Use axis=None
Since pandas 2.0.0, you can use df.mean(axis=None) to compute mean over the entire dataframe. Since pandas 3.0.0, you can use df.std(axis=None) to compute standard deviation over the entire dataframe.
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
df.mean(axis=None) # 3.5
df.std(axis=None) # 1.8708286933869707
Note that DataFrame.std sets ddof=1 by default while Numpy's std sets ddof=0 by default. You can check the relationships as follows:
df.std(axis=None, ddof=0) == df.values.std() # True
df.std(axis=None) == df.values.std(ddof=1) # True
Good thing about pandas mean and std is that it ignores NaN values for you if the dataframe has any whereas with numpy, you have to explicitly filter NaNs out.
# a dataframe with a NaN value
df = pd.DataFrame({'A': [1, float("nan"), 3], 'B': [4, 5, 6]})
df.values.mean() # nan <--- numpy mean/std becomes meaningless
df.values.std() # nan
df.mean(axis=None) # 3.8 <--- pandas mean/std ignores NaNs
df.std(axis=None) # 1.9235384061671346