I’ve been closely following the recent developments from NVIDIA, and their latest announcement has really caught my attention: the H200 with the new GH200 chip. This beast is said to pack a staggering 141 GB of RAM and offers a blazing 4.8 TB/s speed. The premiere of the H200 is slated for the second quarter of 2024, and I can’t help but ponder its potential impact.

The most exciting aspect for me, and probably for many of you, is its capability to run LLAMA2 70B at twice the speed of the current H100. That’s a significant leap in performance!

So here’s the big question for the community: are any of you planning to upgrade to the H200, or are you planning to stick with the H100 for a while longer?

I’m currently using the 8xH100 rig and it’s been a workhorse, but the prospect of doubling my LLAMA2 70B performance is very tempting. However, I’m also weighing the cost versus the benefits. The H200 seems like a substantial investment, and I’m wondering if the performance gain justifies the upgrade, especially considering the still-capable H100.

I’d love to hear your thoughts, experiences, and plans.