I use in both cases q4_K_M

  • meetrais@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Same experience here. I got excellent results from quantized models of Intel-Neural-7B and Mistral-7B but bad results with quantized model of Yi-34B.

    • Inevitable_Host_1446@alien.topB
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      I’m not sure what the point of Neural-7B is, given that it’s super censored corporate safety bot. If that’s what people want they might as well just use ChatGPT, which is faster and better otherwise.

      • Nixellion@alien.topB
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Privacy and cost Also no, 7B is as fast or faster than ChatGPT depending on ChatGPT load.

      • grigio@alien.topOPB
        link
        fedilink
        English
        arrow-up
        1
        ·
        11 months ago

        neural-chat from Intel is not censored! Just use a good system prompt