Volume 2 - Issue 3, May - June 2026

ConceptSteer injects concept-specific correction vectors directly into the residual stream of a Large Language Model (LLM) to self-correct multi-step mathematical reasoning without external tools. The core theoretical contribution formalizes reasoning errors as deviations from linear concept subspaces in the activation space of transformer layers. Under mild assumptions, the optimal steering direction—minimizing the forward KL divergence between the corrected and target correct distributions—is the difference of expected activations of correct and incorrect steps. A sparse autoencoder refinement isolates monosemantic concept features, and a lightweight probe triggers steering only when an error is detected. Experiments on GSM8K, MATH, and the newly introduced ConceptMath benchmark show that ConceptSteer improves final-answer accuracy by +8.2 points over chain-of-thought and cuts steplevel errors by 31%, with under 0.5% added computation per step. ConceptSteer connects mechanistic interpretability with controllable generation for real-time, self-contained error correction.

Karan Gupta, Krishna Dwivedi, Tanishk Kulshrestha, Dr.S.K Sharma,"Concept-Level Activation Steering for Self-Correcting Mathematical Reasoning in Large Language Models: A Theoretical and Empirical Study" International Journal of Advanced Multidisciplinary Research and Educational Development, V2(3): Page(928-931) May-June 2026. ISSN: 3107-6513. www.ijamred.com. Published by Scientific and Academic Research Publishing.