Thank you for the detail and references. Yeah I know, I have used sentence transformers and before them BERT/T5 embeddings for a long time (e.g. Kaggle competitions, few hackathons around the issue…), but I am just wondering what motivated you to create an embeddings server as opposed to running the embeddings in place in the code with the SBERT models or calling an API as you mention with those alternatives? Is the python code you write in the get started part much faster than just using the SentenceTransformer module with batch arrays?

Because I have found, such as when competing in the Learning Agency competitions, you can build the indexes locally or use open source tools like LlamaIndex equivalents with SBERT, rather then need to set up a server. Am I missing something to do with speed or do new models take longer to embed? What’s the problem you and others are facing to use a server for embeddings rather than do it in the code?

I am aware OpenAI made RAG more accessible by chunking and embedding files that users upload, but haven’t many use cases of LLMs long used RAG?

Has there been a recent breakthrough or quality upstep in embeddings I am unaware of? What makes this different to a library like Sentence-Transformers may I ask please?
I’ve found those models can embed thousands of paragraphs a minute on local hardware which is enough for most local use cases.

With the upvotes and comments I’m sure you’ve built something really useful I’d just love to know how you envision it being used.