minus-squarematkley12@alien.topBtoLocalLLaMA@poweruser.forum•on-demand inference or batch inference?linkfedilinkEnglisharrow-up1·1 year agoDoes llama.cpp support batch inference on CPU ? linkfedilink
Does llama.cpp support batch inference on CPU ?