There are three circumstances where np.nanstd might return NaN:
If the input is empty
If all of the elements in the input are NaN
If one of the elements is either positive or negative infinity. To understand why this happens, remember that the formula for standard deviation is
Since x contains inf, the mean of x will also be inf. Therefore when computing the deviations from the mean, there is at least one element that is equal to inf - inf. If you try this at the IPython prompt, you will see that inf - inf is defined as NaN.
In the former two cases you should get a helpful warning:
RuntimeWarning: Degrees of freedom <= 0 for slice.
NumPy
numpy.org › doc › stable › reference › generated › numpy.nanstd.html
numpy.nanstd — NumPy v2.5 Manual
Means Delta Degrees of Freedom. The divisor used in calculations is N - ddof, where N represents the number of non-NaN elements.
University of Texas at Austin
het.as.utexas.edu › HET › Software › Numpy › reference › generated › numpy.nanstd.html
numpy.nanstd — NumPy v1.9 Manual
Compute the standard deviation along the specified axis, while ignoring NaNs.
TutorialsPoint
tutorialspoint.com › numpy › numpy_nanstd_function.htm
NumPy nanstd() Function
The NumPy nanstd() function computes the standard deviation of the elements in an array while ignoring NaN values. It measures the spread or dispersion of a distribution, treating NaN values as missing data.
NumPy
numpy.org › doc › 2.1 › reference › generated › numpy.nanstd.html
numpy.nanstd — NumPy v2.1 Manual
Means Delta Degrees of Freedom. The divisor used in calculations is N - ddof, where N represents the number of non-NaN elements.
GeeksforGeeks
geeksforgeeks.org › numpy-nanstd-function-python
numpy.nanstd() function – Python | GeeksforGeeks
June 11, 2020 - numpy.nanstd() function compute the standard deviation along the specified axis, while ignoring NaNs.
SciPy
docs.scipy.org › doc › scipy-0.15.1 › reference › generated › scipy.stats.nanstd.html
scipy.stats.nanstd — SciPy v0.15.1 Reference Guide
Note that numpy.nanstd has a different signature. Compute the standard deviation over the given axis, ignoring nans. Examples · >>> from scipy import stats >>> a = np.arange(10, dtype=float) >>> a[1:3] = np.nan >>> np.std(a) nan >>> stats.nanstd(a) 2.9154759474226504 >>> stats.nanstd(a.reshape(2, ...
Top answer 1 of 2
13
There are three circumstances where np.nanstd might return NaN:
If the input is empty
If all of the elements in the input are NaN
If one of the elements is either positive or negative infinity. To understand why this happens, remember that the formula for standard deviation is
Since x contains inf, the mean of x will also be inf. Therefore when computing the deviations from the mean, there is at least one element that is equal to inf - inf. If you try this at the IPython prompt, you will see that inf - inf is defined as NaN.
In the former two cases you should get a helpful warning:
RuntimeWarning: Degrees of freedom <= 0 for slice.
2 of 2
2
Another possible explanation for inf output from np.nanstd is related what Numpy data type is used to store data. See the example below:
import numpy as np
a = np.array([1239., 1485., 63., 393., 37., 1186., 13., 402., 404., 915.], dtype='float16')
print(np.nanstd(a)) # returns inf
a = a.astype('float32')
print(np.nanstd(a)) # returns 519.87177
Data Science Parichay
datascienceparichay.com › home › blog › standard deviation of numpy array with nan values
Standard Deviation of Numpy Array with NaN Values - Data Science Parichay
July 9, 2022 - # std dev of array with nan values numpy.nanstd(ar) It returns the standard deviation among all the values in the array ignoring all the NaN values.
Codegive
codegive.com › blog › numpy_std_ignore_nan.php
Numpy std ignore nan
The concept of numpy std ignore nan refers to the ability within the NumPy library to compute the standard deviation of an array while automatically excluding any NaN (Not a Number) values from the calculation.
SciPy
docs.scipy.org › doc › numpy-1.15.0 › reference › generated › numpy.nanstd.html
numpy.nanstd — NumPy v1.15 Manual
Compute the standard deviation along the specified axis, while ignoring NaNs.
GeeksforGeeks
geeksforgeeks.org › scipy-stats-nanstd-function-python
sciPy stats.nanstd() function | Python | GeeksforGeeks
February 11, 2019 - scipy.stats.nanstd(array, axis=0) function calculates the standard deviation by ignoring the Nan (not a number) values of the array elements along the specified axis of the array.
SciPy
docs.scipy.org › doc › scipy-0.14.0 › reference › generated › scipy.stats.nanstd.html
scipy.stats.nanstd — SciPy v0.14.0 Reference Guide
Compute the standard deviation over the given axis, ignoring nans. ... >>> from scipy import stats >>> a = np.arange(10, dtype=float) >>> a[1:3] = np.nan >>> np.std(a) nan >>> stats.nanstd(a) 2.9154759474226504 >>> stats.nanstd(a.reshape(2, 5), axis=1) array([ 2.0817, 1.5811]) >>> stats.nanstd(a.reshape(2, 5), axis=None) 2.9154759474226504
GitHub
github.com › scipy › scipy › issues › 3105
numpy.std and scipy.stats.nanstd treat bias differently · Issue #3105 · scipy/scipy
December 2, 2013 - The setting bias = False is equivalent to setting ddof=1 in numpy.std, leading to the calls numpy.std(x) and scipy.stats.nanstd(x) to return different values, especially for small arrays.
Author: scipy
JAX Documentation
docs.jax.dev › en › latest › _autosummary › jax.numpy.nanstd.html
jax.numpy.nanstd — JAX documentation
jax.numpy.nanstd(a, axis=None, dtype=None, out=None, ddof=0, keepdims=False, where=None, mean=None)[source]# Compute the standard deviation along a given axis, ignoring NaNs.
NumPy
numpy.org › devdocs › reference › generated › numpy.nanstd.html
numpy.nanstd — NumPy v2.6.dev0 Manual
Means Delta Degrees of Freedom. The divisor used in calculations is N - ddof, where N represents the number of non-NaN elements.
MathWorks
mathworks.com › statistics and machine learning toolbox › descriptive statistics and visualization › descriptive statistics
nanstd - (Not recommended) Standard deviation, ignoring NaN values - MATLAB
nanstd is not recommended. Use the MATLAB® function std instead. With the std function, you can specify whether to include or omit NaN values for the calculation.
SciPy
docs.scipy.org › doc › numpy-1.12.0 › reference › generated › numpy.nanstd.html
numpy.nanstd — NumPy v1.12 Manual
numpy.nanstd(a, axis=None, dtype=None, out=None, ddof=0, keepdims=<class numpy._globals._NoValue at 0x40b6a26c>)[source]¶ · Compute the standard deviation along the specified axis, while ignoring NaNs.
Astropy
docs.astropy.org › en › stable › api › astropy.stats.mad_std.html
mad_std — Astropy v8.0.0
The function used to compute the median. Defaults to numpy.ma.median for masked arrays, otherwise to numpy.median. ... Ignore NaN values (treat them as if they are not in the array) when computing the median.
Stack Overflow
stackoverflow.com › questions › 53962179 › how-to-ignore-runtime-errors-even-when-nans-are-populated-from-np-nanstd-pyth
pandas - How to ignore Runtime Errors even when NaNs are populated from np.nanstd? - Python - Stack Overflow
December 29, 2018 - You could wrap it in the warnings.catch_warnings context manager to ignore warnings locally. Documentation · – Niklas Mertsch Commented Dec 31, 2018 at 15:44 ... Output: [0. 0. 0. 0.] ... col1 col2 col3 col4 0 5.0 1.0 6.0 NaN 1 2.0 2.0 1.0 NaN 2 NaN NaN NaN NaN 3 3.0 4.0 NaN NaN 4 NaN NaN NaN NaN ... std = [] for row in range(len(df)): k = df.iloc[row].values bool_arr = np.isfinite(k) chk = not any(bool_arr) if chk == True: k = np.nan_to_num(k, copy=True) st = np.nanstd(k) if chk == True: st = np.nan std.append(st) data = {'std_row_wise': std} std_df = pd.DataFrame(data = data) std_df