I’m new in the LLM world, I just want to know if is there any super tiny LLM model that we can integrate with our existing mobile application and ship it on the app store.

If I give a simple example, I have one news application so it’s possible to integrate llm with my news application so I can perform some operations on the application to provide a better experience to users without sharing their personal information on the internet. Like: summarises the article in different types of tones (like 5, 10, 15-year-old kid, in the poem, old and Gen Z style). Track the type of articles the user likes and display only those articles in his feed) and many more.

And if this is not possible, is there any platform where we can host these types of tiny models like a Firebase ML model (these services are not changed that much as compared to other LLM hosting services)?

  • erelim@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    It will be very unfriendly for the user having to have a 1-2GB app that eats RAM and battery like a mobile game, still you won’t get good or quick results. Check on Replicate, runpod or vast.ai for cheap GPU