I’ve been using self-hosted LLM models for roleplay purposes. But these are the worst problems I face every time, no matter what model and parameter preset I use.

I’m using :

Pygmalion 13B AWQ

Mistral 7B AWQ

SynthIA 13B AWQ [Favourite]

WizardLM 7B AWQ

  1. It messes up with who’s who. Often starts to behave like the user.

  2. It writes in third person perspective or Narrative.

  3. Sometimes, generates the exact same reply (exactly same to same text) back to back even though new inputs were given.

  4. It starts to generate more of a dialogue or screenplay script instead of creating a normal conversation.

Anyone has any solutions for these?

  • sophosympatheia@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    What you highlighted as problems are the reasons why people fork out money for the compute to run 34b and 70b models. You can tweak sampler settings and prompt templates all day long but you can only squeeze so much smarts out of a 7b - 13b parameter model.

    The good news is better 7b and 13b parameter models are coming out all the time. The bad news is even with all that, you’re still not going to do better than a capable 70b parameter model if you want it to follow instructions, remember what’s going on, and stay consistent with the story.

    • involviert@alien.topB
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      No, the problems described are not representative of Mistral 7B quality at all. That’s almost certainly just incorrect prompting, format wise.