It’s because of the way an LLM works, they’re completely blind to things like what a word starts with. Ask it something like “List 10 words that start and end with the same letter but are not palindromes.” and it completely shits the bed, because it can only process words as unified tokens, it can’t look inside the words to see how they’re structured.
It’s because of the way an LLM works, they’re completely blind to things like what a word starts with. Ask it something like “List 10 words that start and end with the same letter but are not palindromes.” and it completely shits the bed, because it can only process words as unified tokens, it can’t look inside the words to see how they’re structured.
GPT-4, prompt: “List 10 words that start and end with the same letter but are not palindromes.”
Even without the palindrome condition, it got some of these and a few palindromes.
It missed 7/10. That’s about the same as random I’d say.
According to your logic, 30% of words would satisfy this property.