
Tamara
Google Research published a study effectively saying:
- Even when LLMs have all the facts they need to answer a question, pretty often they still get it wrong
- It's not a knowledge problem, it's a recall problem
- LLMs are way better at subject-object facts, and when you flip the question to ask about the object first (like on Jeopardy) they fail more
> Our results suggest that recall is tightly coupled to the conditions under which a fact was learned. When the query diverges from the training-time context, phrasing, or ordering in which the fact was encountered, recall becomes harder.
> the fact appears to be encoded, and even recognizable, but difficult to recall when the query direction departs from how the fact was encountered during training. The reversal curse is a recall problem.
*Clarity, directness → Creativity, variety*
There are no official AEO implications drawn in this research but it seems smart to stick to basic "What is A?" "A is B" type phrasing for in web content to make it easier for machines to get it right.