Dude, FP16, really? Why not FP32? Ask Voltaire.

With the rise of AI, including hardware acceleration of AI, comes a renewed interested in efficient data types.  

In this spirit, we raise a toast to FP16 or float16, also known as half-precision floating point format, which can be a more appropriate format in some circumstances and some algorithms than what is known as single precision floating point which occupies 32 bits (and hence also called FP32 or float32).

But why use a less precise data type at all, when more precision options are available?

Half precision values are useful in applications where perfect precision is not required, such applications include image processing and neural networks.

FP16 is not to be confused with bfloat16 (Brain float16) which is a different format developed for Google Brain (now Google AI) with the explicit intent of accelerating machine learning and is used a variety of AI processors (including Google Cloud TPUs).