Following the release of Dimensity 9300 and S8G3 phones, I am expecting growth in popularity of LLMs running on mobile phones, as quantized 3B or 7B models can already run on high-end phones from five years ago or later. But despite it being possible, there are a few concerns, including power consumption and storage size. I’ve seen posts about successfully running LLMs on mobile devices, but seldom see people discussing about future trends. What are your thoughts?
i am running tinyllama and deepseek 1.3B on a almost 3 year old cheap Poco X3 (snapdragon 732G) and its great. Will post the video soon. So the new phones, and high-end ones, well i am sure some people can run mistral on those. But i also wish that phones gets some of its prices reduced, high-end phones are becoming more expensive the most laptops i cant afford.