ijeff@lemdro.id to Technology@lemmy.worldEnglish · 1 year agoLarge Language Models up to 4x Faster on RTX With TensorRT-LLM for Windowsblogs.nvidia.comexternal-linkmessage-square1fedilinkarrow-up154arrow-down14file-textcross-posted to: technews@radiation.party
arrow-up150arrow-down1external-linkLarge Language Models up to 4x Faster on RTX With TensorRT-LLM for Windowsblogs.nvidia.comijeff@lemdro.id to Technology@lemmy.worldEnglish · 1 year agomessage-square1fedilinkfile-textcross-posted to: technews@radiation.party
minus-squarekorewa@reddthat.comlinkfedilinkEnglisharrow-up3·1 year agoDang I need to try these for now only the stable diffusion extension for automatic 1111 is available. I wonder if it will accelerate 30b models that doesn’t fit all in the gpu vram. If it only accelerates 13b then it was already fast enough
Dang I need to try these for now only the stable diffusion extension for automatic 1111 is available.
I wonder if it will accelerate 30b models that doesn’t fit all in the gpu vram.
If it only accelerates 13b then it was already fast enough