The documentation for sys.float_info says that this is the way to get the smallest possible float:
>>> math.ulp(0.0)
5e-324
The value of sys.float_info.min (part of a previous answer that I deleted) gives the smallest normalized float, a much bigger value.
The documentation for sys.float_info says that this is the way to get the smallest possible float:
>>> math.ulp(0.0)
5e-324
The value of sys.float_info.min (part of a previous answer that I deleted) gives the smallest normalized float, a much bigger value.
Another way to approach this problem is to look into the details of how floats work, e.g. on Wikipedia. Acknowledging that python uses IEEE double precision floats on most platforms, and slogging through the relevant bits of https://en.wikipedia.org/wiki/Double-precision_floating-point_format#Double-precision_examples:
+2−1022 × 2−52 = 2−1074 ≈ 4.9406564584124654 × 10−324 (Min. subnormal positive double)
The smallest positive usable number
Smallest positive number machine can store
4.9406564584124654e-324 ---------- This apparently is the smallest usable positive number in Python. ------ Is it the same in R ?
> .Machine$double.xmin [1] 2.225074e-308
on my computer. YMMV. You do know that different computers can have different floating point?
Often you can work around precision problems by thinking. If you do calculation with no thought -- just pretending that the computer's "real" numbers are real real numbers -- then no amount of precision is enough to always get correct answers.
Edit: BTW "smallest usable positive number" doesn't define anything. That may account for the disagreement in numbers here. Look up denormalized numbers, which most but not all computers have.
More on reddit.comIncrement a Python floating point value by the smallest possible amount - Stack Overflow
https://stackoverflow.com/questions/38477908/smallest-positive-float64-number
4.9406564584124654e-324
This apparently is the smallest usable positive number in Python.
-
How does this compare with R, C and other languages ?
-
Don't people doing simulations, (numerical analysis?) etc. want more digits (more precision) ?
> .Machine$double.xmin
[1] 2.225074e-308
on my computer. YMMV. You do know that different computers can have different floating point?
Often you can work around precision problems by thinking. If you do calculation with no thought -- just pretending that the computer's "real" numbers are real real numbers -- then no amount of precision is enough to always get correct answers.
Edit: BTW "smallest usable positive number" doesn't define anything. That may account for the disagreement in numbers here. Look up denormalized numbers, which most but not all computers have.
Maybe you need to review what precision is, because nothing you have mentioned is related to precision. Machine$double.eps is about precision.
There are packages for handling extended precision in various languages including the ones you reference, but there is a heavy performance cost that is only rarely worth the effort of using them. As mentioned in another response, IEEE754 is the most common digital floating point representation definition, so it gets used for cheap fast numerical analysis most of the time by default.
Since Python 3.9 there is math.nextafter in the stdlib. Read on for alternatives in older Python versions.
Increment a python floating point value by the smallest possible amount
The nextafter(x,y) functions return the next discretely different representable floating-point value following x in the direction of y. The nextafter() functions are guaranteed to work on the platform or to return a sensible value to indicate that the next value is not possible.
The nextafter() functions are part of POSIX and ISO C99 standards and is _nextafter() in Visual C. C99 compliant standard math libraries, Visual C, C++, Boost and Java all implement the IEEE recommended nextafter() functions or methods. (I do not honestly know if .NET has nextafter(). Microsoft does not care much about C99 or POSIX.)
None of the bit twiddling functions here fully or correctly deal with the edge cases, such as values going though 0.0, negative 0.0, subnormals, infinities, negative values, over or underflows, etc. Here is a reference implementation of nextafter() in C to give an idea of how to do the correct bit twiddling if that is your direction.
There are two solid work arounds to get nextafter() or other excluded POSIX math functions in Python < 3.9:
Use Numpy:
>>> import numpy
>>> numpy.nextafter(0,1)
4.9406564584124654e-324
>>> numpy.nextafter(.1, 1)
0.10000000000000002
>>> numpy.nextafter(1e6, -1)
999999.99999999988
>>> numpy.nextafter(-.1, 1)
-0.099999999999999992
Link directly to the system math DLL:
import ctypes
import sys
from sys import platform as _platform
if _platform == "linux" or _platform == "linux2":
_libm = ctypes.cdll.LoadLibrary('libm.so.6')
_funcname = 'nextafter'
elif _platform == "darwin":
_libm = ctypes.cdll.LoadLibrary('libSystem.dylib')
_funcname = 'nextafter'
elif _platform == "win32":
_libm = ctypes.cdll.LoadLibrary('msvcrt.dll')
_funcname = '_nextafter'
else:
# these are the ones I have access to...
# fill in library and function name for your system math dll
print("Platform", repr(_platform), "is not supported")
sys.exit(0)
_nextafter = getattr(_libm, _funcname)
_nextafter.restype = ctypes.c_double
_nextafter.argtypes = [ctypes.c_double, ctypes.c_double]
def nextafter(x, y):
"Returns the next floating-point number after x in the direction of y."
return _nextafter(x, y)
assert nextafter(0, 1) - nextafter(0, 1) == 0
assert 0.0 + nextafter(0, 1) > 0.0
And if you really really want a pure Python solution:
# handles edge cases correctly on MY computer
# not extensively QA'd...
import math
# 'double' means IEEE 754 double precision -- c 'double'
epsilon = math.ldexp(1.0, -53) # smallest double that 0.5+epsilon != 0.5
maxDouble = float(2**1024 - 2**971) # From the IEEE 754 standard
minDouble = math.ldexp(1.0, -1022) # min positive normalized double
smallEpsilon = math.ldexp(1.0, -1074) # smallest increment for doubles < minFloat
infinity = math.ldexp(1.0, 1023) * 2
def nextafter(x,y):
"""returns the next IEEE double after x in the direction of y if possible"""
if y==x:
return y #if x==y, no increment
# handle NaN
if x!=x or y!=y:
return x + y
if x >= infinity:
return infinity
if x <= -infinity:
return -infinity
if -minDouble < x < minDouble:
if y > x:
return x + smallEpsilon
else:
return x - smallEpsilon
m, e = math.frexp(x)
if y > x:
m += epsilon
else:
m -= epsilon
return math.ldexp(m,e)
Or, use Mark Dickinson's excellent solution
Obviously the Numpy solution is the easiest.
Python 3.9 and above
Starting with Python 3.9, released 2020-10-05, you can use the math.nextafter function:
math.nextafter(x, y)Return the next floating-point value after x towards y.
If x is equal to y, return y.
Examples:
math.nextafter(x, math.inf)goes up: towards positive infinity.
math.nextafter(x, -math.inf)goes down: towards minus infinity.
math.nextafter(x, 0.0)goes towards zero.
math.nextafter(x, math.copysign(math.inf, x))goes away from zero.See also
math.ulp().
A simpler alternative to math.copysign(math.inf, x) is to simply substitute 2*x.
>>> import sys
>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308,
min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15,
mant_dig=53, epsilon=2.2204460492503131e-16, radix=2, rounds=1)
The smallest is sys.float_info.min (2.2250738585072014e-308) and the biggest is sys.float_info.max (1.7976931348623157e+308). See documentation for other properties.
sys.float_info.min is the normalized min. You can usually get the denormalized min as sys.float_info.min * sys.float_info.epsilon. Note that such numbers are represented with a loss of precision. As expected, the denormalized min is less than the normalized min.
See this post.
Relevant parts of the post:
In [2]: import kinds In [3]: kinds.default_float_kind.M kinds.default_float_kind.MAX kinds.default_float_kind.MIN kinds.default_float_kind.MAX_10_EXP kinds.default_float_kind.MIN_10_EXP kinds.default_float_kind.MAX_EXP kinds.default_float_kind.MIN_EXP In [3]: kinds.default_float_kind.MIN Out[3]: 2.2250738585072014e-308
Of course not, and epsilon has little to do with it. For example,
>>> x = 1e-200
>>> x
1e-200
is far from epsilon, but
>>> x * x
0.0
underflows to 0. If we actually used epsilon instead, then, e.g., multiplying it by 0.25 would underflow to 0 too.
Provided your platform C compiler and hardware support the 754 standard, though, the sign of the zero would match the sign of the multiplicand:
>>> x * -x
-0.0
No.
In [1]: import numpy
In [2]: x = numpy.nextafter(0, 1)
In [3]: x
Out[3]: 4.9406564584124654e-324
In [4]: x*x
Out[4]: 0.0
When the exact result is between 0 and the smallest positive float, it has to round to one of those options, and in this case, 0 is closer.
If for some reason you want to customize this behavior, NumPy lets you customize the behavior of underflow and other IEEE 754 floating-point exceptions with numpy.seterr, although it won't affect operations on ordinary Python objects:
In [5]: numpy.seterr(under='raise')
Out[5]: {'divide': 'warn', 'invalid': 'warn', 'over': 'warn', 'under': 'ignore'}
In [6]: x # NumPy float, not regular float, despite its looks
Out[6]: 4.9406564584124654e-324
In [7]: x*x
---------------------------------------------------------------------------
FloatingPointError Traceback (most recent call last)
<ipython-input-7-a3ff2a28c75d> in <module>()
----> 1 x*x
FloatingPointError: underflow encountered in double_scalars
In [8]: (4.9406564584124654e-324)**2 # regular float
Out[8]: 0.0
There's no way to change the rounding mode.
epsilon is the difference between 1 and the next representable float. That's not the same as the smallest float, which would be the closest number to 0, not 1.
There are two smallest floats, depending on your criteria. min is the smallest normalized float. The smallest subnormal float is min * epsilon.
>>> sys.float_info.min
2.2250738585072014e-308
>>> sys.float_info.min * sys.float_info.epsilon
5e-324
Note the distinction between normalized and subnormal floats: min is not actually the smallest float, it's just the smallest one with full precision. Subnormal numbers cover the range between 0 and min, but they lose a lot of precision. Notice that 5e-324 has only one significant digit. Subnormals are also much slower to work with, up to 100x slower than normalized floats.
>>> (sys.float_info.min * sys.float_info.epsilon) / 2
0.0
>>> 4e-324
5e-324
>>> 5e-325
0.0
These tests confirm that 5e-324 truly is the smallest float. Dividing by two underflows to 0.
See also: What is the range of values a float can have in Python?
You actually want sys.float_info.min ("minimum positive normalized float"), which on machine gives me .2250738585072014e-308.
epsilon is:
difference between 1 and the least value greater than 1 that is representable as a float
See the docs for more info on the fields of sys.float_info.
Check out sys.float_info
>>> import sys
>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)
From the docs
min DBL_MIN minimum positive normalized float min_exp DBL_MIN_EXP minimum integer e such that radix**(e-1) is a normalized float
On my system it is mentioned as min=2.2250738585072014e-308
Check sys.float_info.min. Note that this is the minimum normalized value; the minimum denormalized value may be smaller but numerical accuracy will suffer.
With usual 64-bit floating-point numbers, the normalized min is approximately 2.2e-308. The denormalized min can be much less, down to 4.94e-324:
>>> 5e-324
4.9406564584124654e-324
It is also worth pointing out that the decimal module's Decimal type can represent arbitrarily small numbers, limited only by available memory.