home > domains > aphoh.us
Description: Will's blog of fun stuff
Llama 405b is a large model which requires lots of memory.
I have access to a 4-node SLURM cluster each with 4 A100 80GB GPUs each.
So how do we get these to work together?