Alternatively, you can use a MaskedArray as such:
>>> import numpy as np >>> a = np.array([1,2,np.nan,4]) >>> weights = np.array([4,3,2,1]) >>> ma = np.ma.MaskedArray(a, mask=np.isnan(a)) >>> np.ma.average(ma, weights=weights) 1.75Answer from Nicolas Barbey on Stack Overflow
Alternatively, you can use a MaskedArray as such:
>>> import numpy as np >>> a = np.array([1,2,np.nan,4]) >>> weights = np.array([4,3,2,1]) >>> ma = np.ma.MaskedArray(a, mask=np.isnan(a)) >>> np.ma.average(ma, weights=weights) 1.75
First find out indices where the items are not nan, and then pass the filtered versions of a and weights to numpy.average:
>>> import numpy as np
>>> a = np.array([1, 2, np.nan,4])
>>> weights = np.array([4, 3, 2, 1])
>>> indices = np.where(np.logical_not(np.isnan(a)))[0]
>>> np.average(a[indices], weights=weights[indices])
1.75
As suggested by @mtrw in comments, it would be cleaner to use masked array here instead of index array:
>>> indices = ~np.isnan(a)
>>> np.average(a[indices], weights=weights[indices])
1.75
try this:
>>> np.nanmean(ngma_heat_daily)
This function drops NaN values from your array before taking the mean.
Edit: the reason that average(ngma_heat_daily[ngma_heat_daily != nan]) doesn't work is because of this:
>>> np.nan == np.nan
False
according to the IEEE floating-point standard, NaN is not equal to itself! You could do this instead to implement the same idea:
>>> average(ngma_heat_daily[~np.isnan(ngma_heat_daily)])
np.isnan, np.isinf, and similar functions are very useful for this type of data masking.
Also, there is a function named nanmedian which ignores NaN values. Signature of that function is: numpy.nanmedian(a, axis=None, out=None, overwrite_input=False, keepdims=<no value>)
You can create a masked array like this:
data = np.array([[1,2,3], [4,5,np.NaN], [np.NaN,6,np.NaN], [0,0,0]])
masked_data = np.ma.masked_array(data, np.isnan(data))
# calculate your weighted average here instead
weights = [1, 1, 1]
average = np.ma.average(masked_data, axis=1, weights=weights)
# this gives you the result
result = average.filled(np.nan)
print(result)
This outputs:
[ 2. 4.5 6. 0. ]
You can simply multiply the input array with the weights and sum along the specified axis ignoring NaNs with np.nansum. Thus, for your case, assuming the weights are to be used along axis = 1 on the input array sst_filt, the summations would be -
np.nansum(sst_filt*weights,axis=1)
Accounting for the NaNs while averaging, we will end up with :
def nanaverage(A,weights,axis):
return np.nansum(A*weights,axis=axis)/((~np.isnan(A))*weights).sum(axis=axis)
Sample run -
In [200]: sst_filt # 2D array case
Out[200]:
array([[ 0., 1.],
[ nan, 3.],
[ 4., 5.]])
In [201]: weights
Out[201]: array([ 0.25, 0.75])
In [202]: nanaverage(sst_filt,weights=weights,axis=1)
Out[202]: array([0.75, 3. , 4.75])