So far, I have experimented with the following projects:

https://github.com/huggingface/chat-ui - Amazing clean UI with very good web search, my go to currently.

https://github.com/oobabooga/text-generation-webui - Best overall, supports any model format and has many extensions

https://github.com/ParisNeo/lollms-webui/ - Has PDF, stable diffusion and web search integration

https://github.com/h2oai/h2ogpt - Has PDF, Web search, best for files ingestion (supports many file formats)

https://github.com/SillyTavern/SillyTavern - Best for custom characters and roleplay

https://github.com/NimbleBoxAI/ChainFury - Has great UI and web search (experimental)

https://github.com/nomic-ai/gpt4all - Basic UI that replicated ChatGPT

https://github.com/imartinez/privateGPT - Basic UI that replicated ChatGPT with PDF integration

LM Studio - Clean UI, focuses on GGUF format

-

Really love them and wondering if there are any other great projects,

Some of them include full web search and PDF integrations, some are more about characters, or for example oobabooga is the best at trying every single model format there is as it supports anything.

What is your favorite project to interact with your large language models ?

Share your findings and il add them!

  • SideShow_Bot@alien.topBEnglish
    1·
    10 months ago

    So, in the end which one would you recommend for someone just beginning to run LLMs locally? Windows machine (thus Sanctum is out of the question for now). I’m interested in 3 use cases, so maybe there would be a different answer for each of them:

    1. Python coding questions
    2. Linux shell questions
    3. RAG: in particular, I would like to be able to ask questions and have the model retrieve an answer online, supported by one or more working hyperlinks
  • itsuka_dev@alien.topBEnglish
    1·
    10 months ago

    What is your favorite project to interact with

    I still don’t have a favorite, tbh. I’ve tried a few of the UIs you shared, and I found them to be either too complicated or lacking in certain areas I need. Like many others, I ended up building my own.

    Share your findings

    Recently, I started collecting local UIs and this is what I’ve gathered so far: UI list.

      • XhoniShollaj@alien.topBEnglish
        1·
        10 months ago

        Actually will need 30 more days to get approved. Feel free to contribute additional projects to it at the meantime :)!

      • XhoniShollaj@alien.topBEnglish
        1·
        10 months ago

        Thank you - I submitted a pull request to add it there. Hopefully gets approved. Let me know if there are other lists you would like me to add it to.

      • itsuka_dev@alien.topBEnglish
        1·
        10 months ago

        I have two kind of lists, one for OpenAI API-powered UIs (source, last updated this July), and another for one for local UIs (I’ll update this with the list from XhoniShollaj). I feel like a better organization is needed, e.g. whether the UI is open source or not, model backend and architecture, differentiating feature, etc. Otherwise, the list is impossible to navigate (at least for me).

        • XhoniShollaj@alien.topBEnglish
          1·
          10 months ago

          Thats a very neat layout u/itsuka_dev - love your project. I think we can keep both at the meantime (I want to add my mine to other Awesome lists for more exposure) - let me know what breakdown makes more sense from your end so I can improve my repo.

          • itsuka_dev@alien.topBEnglish
            1·
            10 months ago

            Thank you for the kind words! I stopped maintaining OAI UIs because things got a bit stagnant a few months ago (there were literally no new UIs for weeks, I think). But with the new features announced at DevDay, I’m expecting to see a surge in new UIs, especially those that leverages multi-modality. As a maintainer of list projects, this is such an exciting time.

            I think we can keep both at the meantime

            Absolutely. Someone needs to maintain an awesome list for local UIs, and I don’t think my projects fall into that category. Besides, my list got a mix of native UIs in there too, which is important to me since I’m building both web and native UIs.

            let me know what breakdown makes more sense from your end

            For local UIs, sorting/grouping by model backend (e.g. llama.cpp, ollama, ExLlama) makes the most sense, IMO - and the rest of what I mentioned above is optional.

  • 9gigsofram@alien.topBEnglish
    1·
    10 months ago

    Any of these projects supply clustering multiple gpus/users, or even multiple machines.

  • uhuge@alien.topBEnglish
    1·
    10 months ago

    I’ve got mixed experiences with Bavarder, native UI, fair choice of models to grab, but offen not working reliably. They seem to improve it slowly but steadily.

  • kaloskagatos@alien.topBEnglish
    1·
    10 months ago

    Hi, is there a good UI to chat with ollama and local files (pdf, docx, whatever) and if possible multiple or even a lot of files ?

    By the way, what is the difference between ollama and llamacpp, are the API incompatible ?

  • No-Belt7582@alien.topBEnglish
    1·
    10 months ago

    I use kobold cpp for local llm deployment. It’s clean, it’s easy and allows for sliding context. Can interact with drop in replacement for OpenAI.

    • RYSKZ@alien.topBEnglish
      1·
      10 months ago

      Right now, I’m using your earlier project [1]. It’s proving to be incredibly helpful, thank you!.

      Since it’s a desktop application, it’s more convenient for me than the WebUIs, because I tend to have a lot of tabs open in my browser, which makes it pretty chaotic. I have set up an AutoHotkey script to can easily launch it using a easy to remember hotkey.

      [1] https://github.com/shinomakoi/magi_llm_gui

    • iChrist@alien.topOPBEnglish
      1·
      10 months ago

      Wow looks very good indeed, how is the web extraction plugin? can you share some screenshots?