Can large language models (LLMs) reason by analogy? Some outputs suggest that they can, but it has been argued that these results reflect mimicry of the results of analogical reasoning in the models’ training data. To test this claim, LLM’s have been asked to solve counterfactual problems that are unlikely to be similar to problems in training data sets. Here is an example:
Let’s solve a puzzle problem involving the following fictional alphabet:
[x y l k w b f z t n j r q a h v g m u o p d i c s e]
Here is the problem:
[x y l k] [x y l w]
[j r q a] [ ? ]
What four letters solve the puzzle? The correct answer would be “j r q h,” as h is one letter beyond a in the fictional alphabet, just as w is one letter beyond k in the fictional alphabet. However, many models have been unable to solve similar problems. Taylor W. Webb and colleagues propose that the failure to solve these counterfactual problems has more to do with LLM’s well known difficulty in counting, since the problems require basic counting, in order to establish the position of each letter in the sequence. The authors evaluated a recent version of GPT-4 that can write and execute code, which allowed the model to create a code to count items. This LLM was able to solve these counterfactual letter-string analogies at a roughly human level of performance, and gave coherent and accurate explanations of why the correct solution was correct. According to the authors, GPT-4 can use analogies to reason, a capacity that may be supported by a set of structured operations and emergent relational representations.
PNAS Nexus
Evidence from counterfactual tasks supports emergent analogical reasoning in large language models
27-May-2025
Taylor Webb is a postdoctoral researcher at Microsoft Research.