I’m struggling to get the 7b models to do something useful, obviously I’m doing something wrong as it appears many people strive for 7b models.

But myself I can not get them to follow instructions, they keep repeating stuff and occasionally they start to converse with themselves.

Does anyone have any pointers what I’m doing wrong?

  • DarthNebo@alien.topBEnglish
    1·
    1 year ago

    Try to use the instruct models like Mistral. Ensure your template is the correct one a well.

      • DarthNebo@alien.topBEnglish
        1·
        1 year ago

        It should be model page on HuggingFace, they also have a explicit template module which you can import automatically when interacting using model-id.

        Llama ones are forgiving for not using structure but the mistral-instruct is very bad if structure is not maintained