• 1 Post
  • 4 Comments
Joined 1 year ago
cake
Cake day: November 1st, 2023

help-circle

  • If Nvidia isn’t upgrading GPU’s past 24GB for the RTX 50 series then that will probably factor into the open source community keeping models below 40b parameters. I don’t know the exact cutoff point. A lot of people with 12gb VRAM can run 13b models but you could also run 7b 8-bit with 16k context size. It will get increasingly difficult to run larger contexts with larger models.

    Some larger open models are being released but there won’t be much community there to train on a bunch of datasets to the huge models to nail the ideal finetune.