minus-squaredodo13333@alien.topBtoLocalLLaMA@poweruser.forum•Spread LLM to multiple hostslinkfedilinkEnglisharrow-up1·1 year agoIf i didnt missunderstood your question, answer is Petals: https://research.yandex.com/blog/petals-decentralized-inference-and-finetuning-of-large-language-models linkfedilink
minus-squaredodo13333@alien.topBtoLocalLLaMA@poweruser.forum•What kind of performance should we expect?linkfedilinkEnglisharrow-up1·1 year agoWhat is TGI? linkfedilink
If i didnt missunderstood your question, answer is Petals:
https://research.yandex.com/blog/petals-decentralized-inference-and-finetuning-of-large-language-models