I want to begin by saying my specs are an rtx 4080 with 16gb VRAM + 32GB regular ram.
I’ve managed to run chronoboros 33b model pretty smoothly, even though a tad slow.
Yet I’ve ran into hardware issues (I think) trying to run TheBloke/Capybara-Tess-Yi-34B-200K-GPTQ and Panchovix/WizardLM-33B-V1.0-Uncensored-SuperHOT-8k (Tried both AWQ and GPTQ), is there a reason models with a pretty similar amount of parameters won’t run?