My main usecase for LLMs is literally as an auto-complete, mainly via coding, so I was wondering whether anyone has played with/had any luck using the base model for use cases that are close to simply auto completing? I could imagine the instruction tuning adding a sycophancy bias in those areas
You must log in or register to comment.
Get a base model of your choice, finetune it with plain text book of a style you want it to talk. Done.
Fine tune as in gradient updates or as in ICL?
I’m the local “examples/completion is better than chat/instruction” nut
I advise developers to learn how to use few-shot examples and completion instead of writing programs that beg chatbots to do a task. Chat/instruction imposes severe limitations, while examples/completion can peform virtually any task you can think of without need for fine-tuning
Here are some examples: classification, rewrite sentence copying style, classify, basic Q&A example, fact check yes/no, rewrite copying style and sentiment, extract list of musicians, classify user intent, tool choice, rewrite copying style again, flag/filter objectionable content, detect subject changes, classify profession, extract customer feedback into json, write using specified words, few-shot cheese information, answer questions from context, classify sentiment w/ probabilities, summarize, replace X in conversation
Just to be clear, you aren’t doing fine tuning here as in gradient updates, you are using the base model + ICL?
Yep, basically like taking a few samples from a dataset and turning them into a short text “document” with an obvious pattern so the LLM will complete it
Few-shot vs fine-tuning comparison:
Pros:
- converge behavior with much fewer examples
- dynamic. changes to “dataset” applied without modifying model weights
- no worry about whether important information is lost
- can do things like average logits of single-token classification problems from multiple inferences (work around context length limitations)
Cons:
- needs context length, so can’t provide too many examples or too large
- sometimes need “adversarial” examples to discourage repetition of text from other examples
- models that are too small have worse ICL