I’ve been tinkering a lot with Ooba webui and Silly Tavern, and now have ideas for a hobby project I’d like to build. I’ve used ST to serve remotely, but the idea of rebuilding even a portion of ST is more than I want to attempt right now. In general though, I’m wondering what tools or libraries I should consider if building my own LLM from scratch with the intent of serving as a host for remote connections.

And, in terms of front end, I want to intercept and do more with the chat content to add some gamey elements to it. I’m a frontend dev in real life and feel good about the frontend part, and I’m interested in exploring my backend libraries around Flask/Django.