Wish we could just solder more VRAM to the cards. Such a silly thing to keep holding us back.

Open ai does marketing better than they make their models.

I use airoboros 7b to give me plots and write the beginnings of stories. It’s very useful for that.

Imo running any AI on anything other than VRAM makes it so slow it’s unusable. So, you can play around with 13b quantized models.

Too bad I think that Windows support for it was lacking (at least, last time I checked it). It needs a separate thing to make it work properly, and this thing was only for Linux.