Hi I am a newbie c# dev, I am trying to create a home project and until recently I was using Llamasharp. There is little support for it and since the last updates I’ve been unable to get it to work at all with the recent updates.
i’m trying to build a little chat wpf application which can either load AWQ or GGUF LLM files. Are there any simple and easy to use libraries out there which I can facilitate in c#?
I have a GTX 3060 and I’d preferably like to use my GPU RAM if it’s faster than using DDR4 RAM. I admit I am under a few misconceptions. Ideally I’d like to be able to load the Mistral models in c#.
I don’t know an alternative, but I did some experimenting with it. I kinda rewrote large parts of it, and I also used a custom build of llama.cpp dll’s. I’m pretty sure it’ll still work with the newest llama.cpp build, you might need to update some native calls if they’ve been expanded or renamed.
My changes are at https://github.com/TheTerrasque/LLamaSharp/tree/feature/clblast - I haven’t really documented it much, but maybe the git history will help