haxxiy on 17 August 2023
Good old analogue. People have been trying to bring it back for this sort of computing task since the 90s. Even Intel tried but it didn't work.
In theory, it would make sense for inference since it's so low precision but integrated digital circuits have become just too good. Especially now that one might come to rely on CPUs as well if sparse FP8/INT8 becomes an industry standard.
Some even envision a future for FP2/INT2, and neural networks that would work basically on binary logic, but it's doubtful any meaningful data could be retrieved at such low precision.
