mixed precision fp16

What is Mixed Precision? (ELI5) Imagine you have two ways to measure something: one super-precise and another that’s a bit faster but not as exact. In computing, these two “measuring sticks” are called 32-bit precision (FP32) and 16-bit precision (FP16). Normally, computers use the super-precise 32-bit to handle all the calculations. But here’s the trick: … Continue reading mixed precision fp16