I’m trying to use a LLM to help me flesh out some filler for my stories. I find that a lot of people that do this put a lot of emphasis and importance in the quality of the writing it produces, where as I’m looking more for something that is capable of advanced reasoning and understanding. I plan on going through and rewriting everything to fit my personal prose, but I like to use ChatGPT to kind of get the ball rolling. The problem is that it’s censorship is a bit much. I don’t usually write NSFW stuff, but even things like violence and bloodshed get censored pretty heavily.

Is there a model that excels at understanding more than others that can be used on a 4090? I don’t care about speed, just decent results.

  • Ravenpest@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    10 months ago

    Speed+quality = Nous-Capybara 34b. Offload 13 layers to system and get a Q5_K_M. If you have enough system RAM and a decent CPU you wont even feel it. Just quality, Euryale 1.3 70b. It will be slow - up to 200 seconds for a single message at Q5_K_M - but it will deliver.