From what I’ve read mac somehow uses system ram and windows uses the gpu? It doesn’t make any sense to me. Any help appreciated.

  • DarthNebo@alien.topBEnglish
    1·
    1 year ago

    7B models will work just fine with FP4 or INT4. Similarly 13B as well with little offloading if needed.