Large Language Models (LLMs ), such as GPT-4, can fail or make mistakes in answering a question that are not easily detected. Such scenarios often involve nuanced contextual understanding, the interpretation of ambiguous statements, or the generation of inherently biased or misleading content. For example, LLMs can struggle with the task of reliably distinguishing between valid and invalid reasoning or may generate content based on spurious correlations in the training data. They might also make factual errors as they generate information based on patterns learned during training, not by accessing real-time or post-training-cutoff data. These cases highlight the limitations of LLMs and underscore the need for better understanding and further improvement of these models. Finding questions where LLM fails can be useful to generate more complex questions or check for student misbehavior in answering the questions.

Giulia Cademartori

University of Genoa