Since your question is mostly about performance, this article presents you with some specific calculations (keep in mind though that this article is specific to neural networks, and your calculations may be completely different to what they're doing in the article): http://web.archive.org/web/20150310213841/http://www.heatonresearch.com/content/choosing-between-java%E2%80%99s-float-and-double
Some of the relevant material from the link is reproduced here:
Both double and float can support relatively large numbers. The upper and lower range are really not a consideration for neural networks. Float can handle numbers between 1.40129846432481707e-45 to 3.40282346638528860e+38...Basically, float can handle about 7 decimal places. A double can handle about 16 decimal places.
Matrix multiplication is one of the most common mathematical operations for neural network programming. By no means is it the only operation, but it will provide a good benchmark. The following program will be used to benchmark a double.
Skipping all the code, the table on the website shows that for a 100x100 matrix multiplication, they have a gain in performance of around 10% if they use doubles. For a 500x100 matrix multiplication, the performance loss because of using doubles is around 7%. And for a 1000x1000 matrix multiplication, that loss is around 17%.
Answer from Nico on Stack OverflowFor the small 100x100 matrix switching to float may actually decrease performance. As the size of the matrix increases, the percent gain increases. With a very large matrix the performance gain increases to 17%. 17% is worth considering.
Since your question is mostly about performance, this article presents you with some specific calculations (keep in mind though that this article is specific to neural networks, and your calculations may be completely different to what they're doing in the article): http://web.archive.org/web/20150310213841/http://www.heatonresearch.com/content/choosing-between-java%E2%80%99s-float-and-double
Some of the relevant material from the link is reproduced here:
Both double and float can support relatively large numbers. The upper and lower range are really not a consideration for neural networks. Float can handle numbers between 1.40129846432481707e-45 to 3.40282346638528860e+38...Basically, float can handle about 7 decimal places. A double can handle about 16 decimal places.
Matrix multiplication is one of the most common mathematical operations for neural network programming. By no means is it the only operation, but it will provide a good benchmark. The following program will be used to benchmark a double.
Skipping all the code, the table on the website shows that for a 100x100 matrix multiplication, they have a gain in performance of around 10% if they use doubles. For a 500x100 matrix multiplication, the performance loss because of using doubles is around 7%. And for a 1000x1000 matrix multiplication, that loss is around 17%.
For the small 100x100 matrix switching to float may actually decrease performance. As the size of the matrix increases, the percent gain increases. With a very large matrix the performance gain increases to 17%. 17% is worth considering.
Normally, I would use a double, because float doesn't have sufficient accuracy for many numerical use cases, and the performance difference is small enough not to matter.
As always, performance is implementation dependent so you will need to benchmark on your particular case in order to determine if it "matters" or not.
In general I have found:
- The performance difference for individual operations is pretty small, especially on 64-bit machines. Both a
floatand alongwill fit in a 64-bit machine word. Often there is zero difference. floats have a slight advantage in that they consume less memory, and this can help with reducing CPU cache pressure. I've found floats to be 30-50% faster when doing simple operations over large arrays.
I am learning Java, and am confuse on when to choose double or float for my real numbers or int. It feels like, it doesn’t matter because from my limited experience (with Java) both of them deliver the same results, but I don’t want to go further down the learning curve with Java and have a bad habit of using either messing up my code, and not having a clue as to why. So, when should you use float and double?
The Wikipedia page on it is a good place to start.
To sum up:
floatis represented in 32 bits, with 1 sign bit, 8 bits of exponent, and 23 bits of the significand (or what follows from a scientific-notation number: 2.33728*1012; 33728 is the significand).doubleis represented in 64 bits, with 1 sign bit, 11 bits of exponent, and 52 bits of significand.
By default, Java uses double to represent its floating-point numerals (so a literal 3.14 is typed double). It's also the data type that will give you a much larger number range, so I would strongly encourage its use over float.
There may be certain libraries that actually force your usage of float, but in general - unless you can guarantee that your result will be small enough to fit in float's prescribed range, then it's best to opt with double.
If you require accuracy - for instance, you can't have a decimal value that is inaccurate (like 1/10 + 2/10), or you're doing anything with currency (for example, representing $10.33 in the system), then use a BigDecimal, which can support an arbitrary amount of precision and handle situations like that elegantly.
A float gives you approx. 6-7 decimal digits precision while a double gives you approx. 15-16. Also the range of numbers is larger for double.
A double needs 8 bytes of storage space while a float needs just 4 bytes.
LibGDX is a framework mostly used for game development.
In game development you usually have to do a whole lot of number crunching in real-time and any performance you can get matters. That's why game developers usually use float whenever float precision is good enough.
The size of the FPU registers in the CPU is not the only thing you need to consider in this case. In fact most of the heavy number crunching in game development is done by the GPU, and GPUs are usually optimized for floats, not doubles.
And then there is also:
- memory bus bandwidth (how fast you can shovel data between RAM, CPU and GPU)
- CPU cache (which makes the previous less necessary)
- RAM
- VRAM
which are all precious resources of which you get twice as much when you use 32bit float instead of 64bit double.
Floats use half as much memory as doubles.
They may have less precision than doubles, but many applications don't require precision. They have a larger range than any similarly-sized fixed point format. Therefore, they fill a niche that needs wide ranges of numbers but does not need high precision, and where memory usage is important. I've used them for large neural network systems in the past, for example.
Moving outside of Java, they're also widely used in 3D graphics, because many GPUs use them as their primary format - outside of very expensive NVIDIA Tesla / AMD FirePro devices, double-precision floating point is very slow on GPUs.