Decoupling reasoning and facts in language models

Large language models (LLMs) have demonstrated very good capabilities in knowledge retrieval and text generation tasks. Our investigation in this post suggests that performance on knowledge-based tasks may not transfer to complex reasoning scenarios. This blog look into whether small language models (SLMs) (<30B parameters) exhibit a dissociation between knowledge retrieval and multi-step conditional reasoning.

Questions

Experiment design

We employed an 11 × 2 factorial design (all 11 model configurations were each tested on both tasks), evaluating 11 model configurations (9 distinct models) across 2 tasks:

Task 1: Quantum entanglement (knowledge retrieval)

Task 2: Modified trolley problem (reasoning)

Model Selection (zero-shot)

GPT-OSS-20B was evaluated with four reasoning effort configurations (none, low, medium, high) to assess inference-time computational effects on reasoning capability. Reasoning models feature explicit chain-of-thought mechanisms.

Hardware and software

Hardware: AMD Ryzen Threadripper PRO 9965WX (24-core, 48-thread), 2× NVIDIA RTX 5090 (32 GB VRAM each), 256 GB RAM, Ubuntu 24.04 LTS.

Inference: llama.cpp server with OpenAI-compatible API. Models quantized to Q4_K_M or Q8_0 formats. Identical hardware configuration across all evaluations

Results

Overall performance

Task-specific analysis

Quantum entanglement (knowledge retrieval)

Mean scores: Reasoning models (M = 9.50, SD = 0.00), Standard LLMs (M = 8.00, SD = 3.05). Mann-Whitney U = 10.50, p = 0.787 (not significant). Both model types successfully retrieved and explained quantum mechanics concepts, with 8/9 models correctly stating that entanglement does not enable FTL communication.

Exception: Qwen3-0.6B (600M parameters) incorrectly claimed FTL communication was possible, attributing this to "breaking time dilation" - a fundamental physics error suggesting inadequate training corpus or insufficient model capacity.

Trolley problem (reasoning)

Mean scores: Reasoning configurations (M = 8.75, SD = 1.09), Standard/low-effort configurations (M = 1.43, SD = 0.98). Mann-Whitney U = 28.00, p = 0.0049 (highly significant).

Inference-time reasoning effort impact: GPT-OSS-20B demonstrated that reasoning capability can be activated through computational allocation:

With low or no reasoning effort, GPT-OSS-20B made the same error as other standard LLMs: incorrectly claiming pulling the lever saves both the 5 criminals AND preserves the doctor's 100 future saves. With medium reasoning effort, the model correctly identified that pulling the lever kills the doctor and eliminates 100 future saves. With high reasoning effort, it produced a formal decision tree with perfect accuracy. This represents a significant improvement from low to medium reasoning effort, demonstrating that inference-time computation allocation is a critical hyperparameter.

Error analysis

Standard and low-effort configurations exhibited systematic errors on the trolley problem:

Example Error (GPT-OSS-20B with low reasoning effort)

Speed-accuracy trade-off

Efficiency analysis

Findings

This experiment demonstrates a robust dissociation between knowledge retrieval and multi-step reasoning capabilities in small language models. While model size predicts knowledge task performance (r = 0.68, p = 0.044), it does not predict reasoning task performance (r = -0.12, p = 0.759). Instead, two complementary factors constitute the primary determinants of reasoning capability:

Our evaluation of GPT-OSS-20B with varying reasoning effort levels reveals that standard LLMs possess latent reasoning capability that can be activated through increased inference-time computation. This suggests a continuous spectrum rather than binary architectural divide, with reasoning capability emerging from both architectural design and computational allocation.

Interpretation

The observed dissociation suggests two distinct cognitive processes:

Bias effects

The systematic errors on the trolley problem reveal strong prior biases from the classic formulation. Models exhibited:

These errors persisted despite explicit numerical calculations, suggesting that decision-making pathways may bypass computational results in standard architectures without sufficient reasoning effort.

Increased reasoning effort likely allocates more computational steps before producing output, enabling: (1) multi-step lookahead and backtracking, (2) explicit consideration of counterfactuals, and (3) verification of intermediate calculations. The medium→high transition in GPT-OSS-20B yields diminishing returns (+11% score improvement), suggesting medium effort represents the optimal balance for most applications.

Implications on model selection

For Production systems requiring reasoning:

Option 1: Reasoning architecture (recommended)

Option 2: Standard LLM + high reasoning effort

For Knowledge-based applications:

Limitations

Takeaways

Future work should investigate: (1) whether reasoning effort benefits generalize across model families, (2) the computational cost-benefit relationship at different scales, (3) whether hybrid approaches (small reasoning architecture + medium effort) outperform either approach alone, and (4) research existing benchmarks that control for inference-time parameters.