Look at this, apart Llama1, all the other “base” models will likely answer “language” after “As an AI”. That means Meta, Mistral AI and 01-ai (the company that made Yi) likely trained the “base” models with GPT instruct datasets to inflate the benchmark scores and make it look like the “base” models had a lot of potential, we got duped hard on that one.
“As an AI language model” is pretty much a meme at this point.
A base model catching on to it is disappointing but not completely unexpected.
In fact there have been several sources in the past highlighting the upcoming issue of ChatGPT creeping into future datasets and here we are with proof of what we were warned about 6+ months ago having now happened.