Learn
GPU guide
How to read an accelerator specification when your interest is AI rather than graphics.
Memory capacity
Decides which models fit. As a rough guide, roughly two gigabytes per billion parameters at 16-bit precision, before overheads.
Memory bandwidth
Usually determines how quickly a language model produces text.
Form factor
Server modules are passively cooled and rely on chassis airflow. They will not work in an ordinary desktop.
Power
Add every accelerator together, then check the power supply, the circuit and the room.
Precision support
Lower-precision formats increase throughput and reduce memory use, at some accuracy cost.
Interconnect
Matters as soon as a workload spans more than one accelerator.