Absolutely (I’m a datascientist too), I own a fresh company in ML/DL/AI and it works pretty well.

There is so much room for creation even with current models (RAG pipeline on private data, prototyping locally and train or finetune on instances, real world application mixing others branch like CV, cinematography with SD, implementing liquid NN with LLM and so on)

You can find the method here :

https://medium.aiplanet.com/advanced-rag-providing-broader-context-to-llms-using-parentdocumentretriever-cc627762305a

There is ton of fine tuned models and maybe 6-7 quantisized models per model and fine tuned models, open source, business usable, uncensored, for RAG, for photo description, for TTS, for CV, with updates of checkpoints and so on.

At the contrary fortunately there is people and enough diversity to adapt with hardware and objectives without pay fortunes to train, finetune models.

ie: If your needs are commercial, with a model speaking fluently spanish, small enough to inference fast for many clients and with censor, 100% on your local server, treating with confidential data there is almost no choice

Some updates:

• I changed to Jina-small-en-v2, base model crash due to lack of RAM under WSL2
• Make a parents retriever (chunk 2000) and input it child retriever (chunk 400), 0 overlap (will share the method)
• Still use sciphy model but this time using the right template (From the indication from The bloke) by adding a template prompt rather than Alpaca prompt and it resolves the problem of hallucination
• Put text oobabooga on instruct by default, loader exllamav2hf

I got a strong 90% of success with the PDF, will send the code when this will be cleaned and optimized, thank you all for the help 😊