Following the release of Dimensity 9300 and S8G3 phones, I am expecting growth in popularity of LLMs running on mobile phones, as quantized 3B or 7B models can already run on high-end phones from five years ago or later. But despite it being possible, there are a few concerns, including power consumption and storage size. I’ve seen posts about successfully running LLMs on mobile devices, but seldom see people discussing about future trends. What are your thoughts?

  • jamesstarjohnson@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    If llm is packaged in an app and installed on a phone with all the rights given and good function calling ability to use os api it will be able to do lots of things with voice commands even without being fully integrated like siri