I want to buy a laptop or build a machine powerful enough that I can run these LLMs locally. I am ok with either investing in desktop or MBP though MBP was lucrative with a capability to run these models on laptop itself. Any pointers would be helpful. I tried researching but so much of info is there that I got spooked. Any initial pointers would really help. Thank you!
Apple Silicon Macs are great options for running LLMs, especially so if you want to run a large LLM on a laptop. With that said, there aren’t big performance differences between the M1 Max & M3 Max, at least not for text generation, prompt processing does show generational improvements… Maybe this will change in future versions of MacOS if there are optimizations to unlock better Metal Performance Shader performance on later GPU generations, but for right now, they are pretty similar.
Apple Silicon Macs aren’t currently a great option for training/fine tuning models. There isn’t a lot of software support for GPU acceleration during training on Apple Silicon.