minus-squareDavidSJ@alien.topBtoLocalLLaMA@poweruser.forum•Macs with 32GB of memory can run 70B models with the GPU.linkfedilinkEnglisharrow-up1·10 months ago There will hopefully be more optimizations to speed this up. Speculative, Jacobi, or lookahead decoding could speed things up quite a bit. linkfedilink
Speculative, Jacobi, or lookahead decoding could speed things up quite a bit.