Sunday, 27 April 2025

Dude, FP16, really? Why not FP32? Ask Voltaire.

With the rise of AI, including hardware acceleration of AI, comes a renewed interested in efficient data types.  

In this spirit, we raise a toast to FP16 or float16, also known as half-precision floating point format, which can be a more appropriate format in some circumstances and some algorithms than what is known as single precision floating point which occupies 32 bits (and hence also called FP32 or float32).

But why use a less precise data type at all, when more precision options are available?

Half precision values are useful in applications where perfect precision is not required, such applications include image processing and neural networks.

FP16 is not to be confused with bfloat16 (Brain float16) which is a different format developed for Google Brain (now Google AI) with the explicit intent of accelerating machine learning and is used a variety of AI processors (including Google Cloud TPUs).

Friday, 25 April 2025

What the GAUSS is GNA?

Intel GNA is the Intel "Gaussian and Neural Accelerator" which is a low-power neural co-processor for "continuous inference at the edge" (in other words, hardware-supported real-time neural networks).  Its primary use case presently is AI-driven audio applications.

The "Gaussian" part refers to Gaussian Mixture Models, where data is modelled as coming from various Gaussian distributions.

Markdown documentation is available here.  

It is part of OpenVINO (TM) which is aimed at fast AI on Intel hardware. 

OpenVINO uses a proprietary model format (OpenVINO IR) for which convertors exist from TensorFlow, PyTorch, ONNX and other frameworks.

Dude, what are On-Board Diagnostics (ODB, ODB-II)?

ODB-2 is a self-diagnostic system for vehicles (it stands for On-Board Diagnostics).  Mechanics used ODB-2 readers to communicate with vehicles (usually by connecting the reader to a 16 pin connector near the steering wheel, called the ODB2 diagnostic port). ODB requests are then answered with ODB responses (e.g. speed, fuel level, DTCs).

Some common ODB-related acronyms:

DTC - diagnostic trouble code, ECU - engine control unit, MIL - malfunction indicator light, VCM - vehicle control module (in-car computer responsible for engine management, transmission management and in general all things engine-related except for emissions).