I upgraded my system a year ago. Amongst these was a GPU 1070 -> 3090 upgrade. My old card rusts now in the cellar. I heard people around here have mentioned they run desktop on their old card to completely free VRAM in their workhorse card.
is this worth doing in the context of loading LLMs and playing around with them, no training yet? The downside is a worse thermal situation if I cram them together.
I just use Linux. Stop the X server or don’t install it altogether. (If you have it, just Ctrl+Alt+F1, log in, sudo systemctl stop lightdm.) Enjoy the entirety of your VRAM minus 1 MB available for you, so from there you run oobabooga or an API server or whatever and connect to it from your laptop.
As an added benefit, I can leave the GPU and all the noisy fans and their heat somewhere else in my home. And you don’t need to connect a display to the Linux box either, just setup openssh-server, keys and work remotely.
Clever. I often run SD and textUI in --listen mode than use it on my IPad.