minus-squareRight-Structure-1619@alien.topBtoLocalLLaMA@poweruser.forum•TabbyAPI released! A pure LLM API for exllama v2.linkfedilinkEnglisharrow-up1·1 year agoDoes anyone know if they expose all the good stuff that Guidance uses for their guided generation and speedup? This plus guidance (kv cache, grammar control, etc) would be fast fast! linkfedilink
Does anyone know if they expose all the good stuff that Guidance uses for their guided generation and speedup? This plus guidance (kv cache, grammar control, etc) would be fast fast!