Hi everyone. I am trying to create a lab report analyzer which takes in lab report data of a patient(blood work, cholesterol, sugar level, etc) and gives a detailed analysis of it.
To do this, I want to use llama2 or mistral as the base model. I’m looking for datasets right now. So I have 2 questions: 1)Which base model will be the best for this considering this will require some logical analysis on the model’s part. 2)when I get the dataset, should I use RAG or do a fine tune? Which is likely to give me better results? 3)😅well… do you guys know of any such datasets? It would help a ton.
Thanks!
You must log in or register to comment.
I would be very wary of such an application. There is no current model which does not hallucinates at times. And certainly if you are asking for a factual analysis.
I am using an llm to extract data from some texts, but for every answer it gives I do a simple search through the input text to first see if the text exists in the input text. Because if it does not then it cannot be true. If it exists it doesn’t mean it is correct or anything like that. And that simple check goes wrong on a finetuned model about 1 in a 100 answers.
Or look at the hg leaderboards, if it says 98% on a test then it basically says even after special training on known data it still has 2% percent wrong and now you want to throw unknown data with an unknown question at it.
Sometimes it will return rubbish which you can filter out but sometimes it will just output 23 instead of 22 which was in your input text ( or there was 23 for a different fact in your input text) and these are very hard to filter out and they don’t matter with most applications. But if you want to produce analyzes or facts than these are simply wrong