NB AI • Systems & Infrastructure Engineering Lab
In-depth engineering analysis: Running 70B parameter models on workstation: memory bandwidth bottleneck. Practical diagnostics, logs and production configuration.
#inference
Engineering writeups and incident post-mortems for #inference.
Articles: 1