I'd also consider dual A6000-48GB (96GB total) if you have a budget of $8000 or ...

dheera on April 18, 2024 | parent | context | favorite | on: Meta Llama 3

I'd also consider dual A6000-48GB (96GB total) if you have a budget of $8000 or dual V100-32GB (64GB) if you have a budget of $4000.

V100 is old and slower, but for AI applications, RAM is king and there are lots of enterprise V100's coming off racks and being sold on eBay for cheap.