Icaruswept@alien.topBtoLocalLLaMA@poweruser.forum•New Microsoft codediffusion paper suggests GPT-3.5 Turbo is only 20B, good news for open source models?English
1·
1 year agoYour reminder that OpenAi also has access to an enormous amount of hand-annotated and human-generated data for training on: https://www.theverge.com/features/23764584/ai-artificial-intelligence-data-notation-labor-scale-surge-remotasks-openai-chatbots
We’ve seen multiple times that data quality matters a lot. Not surprising if they can fine-tune a 20b model into a high-quality chatbot.
Seriously impressive work, well done!