Temporal Reliability Decay in Large Language Models: A Domain-Stratified Empirical Study

Abstract

Large Language Models (LLMs) are commonly evaluated against a fixed knowledge cutoff, but this view does not explain what happens to factual accuracy as information gradually becomes more distant from the model’s training period. In practice, not all facts age in the same way. A country capital or a scientific term may remain stable for years, while an office holder, sports result, or political appointment may change within a short time. This paper examines this problem by introducing Temporal Reliability Decay (TRD), an empirical framework for measuring the change in factual accuracy as a function of temporal distance from the model knowledge cutoff. The study evaluates GPT-3.5-turbo, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct using 2,400 questions selected from TempLAMA and TimeQA. The questions were grouped into Science, Geography, Sports, and Politics and evaluated across six temporal bins. The results show a clear difference between relatively stable and fast-changing domains. Science and Geography show slow decline, while Sports and Politics lose accuracy more rapidly as temporal distance increases. Based on these patterns, the paper proposes the Temporal Confidence Score (TCS), a domain-aware estimate that can help indicate when an LLM response should be checked, supported with retrieval, or reviewed by a human. The findings are limited to English question answering, four domains, three evaluated models, and zero-shot prompting.