Most AI training code is just a big for loop with each line calling highly performant c/cpp libraries underneath. There is no value that go or rust can add here (most libs used in python are not in python only their stubs are in python).

some people want to train on procedural generators (eg game engine) which would be in C++. being able to have the whole codebase in one language would smooth this out. (in my case I have a rust 3d engine code base that i’d like to use to drive AI )

ggml is a great idea IMO.

I was thinking of distributed MoEs as well.Question I have is how do you route queries? I don’t know how to do that if all the Es are in the same cluster let alone distrivuted.

yeah its a work in progress. Its not trivial to setup . it’s easy to imagine a way it could be done , but it all has to be built, tested, refined.

llama cpp is out there, I am a c++ person but I dont have deep experience with LLMs (how to fine tune etc) generally and have other projects in progress. but if you look around in the usual places with some search terms you’ll find the attempts in progress, and they probably could use volunteers.

my aspirations are more toward the vision side, I’m a graphics person and need to get on with producing synthetic data or something

I’ve seen projects along these lines getting going, it’s coming.

EDIT I hah didn’t know Petals existed either.

there’s going to be multiple attempts