• vlodia@alien.topBEnglish
    1·
    11 months ago

    How many tokens is its limit? And how do you change its token parameters in python?

  • fediverser@alien.topEnglish
    1·
    11 months ago

    This post is an automated archive from a submission made on /r/LocalLLaMA, powered by Fediverser software running on alien.top. Responses to this submission will not be seen by the original author until they claim ownership of their alien.top account. Please consider reaching out to them let them know about this post and help them migrate to Lemmy.

    Lemmy users: you are still very much encouraged to participate in the discussion. There are still many other subscribers on !localllama@poweruser.forum that can benefit from your contribution and join in the conversation.

    Reddit users: you can also join the fediverse right away by getting by visiting https://portal.alien.top. If you are looking for a Reddit alternative made for and by an independent community, check out Fediverser.

  • Hinged31@alien.topBEnglish
    1·
    11 months ago

    Can it summarize documents (say, around 5k words)? Anything that is adapted to that?

  • GasBond@alien.topBEnglish
    1·
    11 months ago

    TheBloke openhermes 2.5 mistral 16k 7B q5_k_m gguf. what about this? anyone using this model?

    • LoSboccacc@alien.topBEnglish
      1·
      11 months ago

      I am using it, my favorite finetune so far, however, ignore the instructions to use chatml prompt formattig and use instead vicuna, with USER: and ASSISTANT:, possibly one of the best model I’ve seen for long conversation

  • AdTotal4035@alien.topBEnglish
    1·
    11 months ago

    I’ve tried 10 coding models ranging from 7b to 13b, and open hermes is by far the best. All the other models struggled with anything not python. They couldn’t even get hello world to be typed backwards in c++. If anyone has any proven suggestions to a 13B model thats better, please do share.

  • __Maximum__@alien.topBEnglish
    1·
    11 months ago

    Can you please tell about how and with what tools you are using it? Like with as much details as you can, because I would like to setup it today and work with it.

  • head_robotics@alien.topBEnglish
    1·
    11 months ago

    Does the model have a clean opensource dataset?
    (free from OpenAI model/proprietary generated data)?

  • r3tardslayer@alien.topBEnglish
    1·
    11 months ago

    Ah is this a model based off of intels newly released model? Also I don’t like it because when I get it to help me create malware as a test , it will lobotomize itself Have not tried this one though let me know if it’s uncensored

  • LoSboccacc@alien.topBEnglish
    1·
    11 months ago

    yeah possibly one of the best for rag at this size, it sticks to facts extremely well, but it’s hard to have it do any form of creative interpretation of the context.

    • zis1785@alien.topBEnglish
      1·
      11 months ago

      Curious to know if you have built a RAG application with it ? Any specific embedding models you used ?