Volume 2 - Issue 3, May - June 2026

Offline reinforcement learning (RL) is limited by the coverage of the static training dataset: regions of the state-action space absent from the data lead to conservative, brittle policies. Data augmentation addresses this by adding synthetic transitions, but existing methods either apply random perturbations that ignore dynamics or generate model-based rollouts that invite model exploitation— neither accounts for the causal structure of the environment. We propose Counterfactual Diffusion Augmentation (CDA), which uses conditional diffusion models to generate counterfactual next states for actions not present in the offline dataset. A dual plausibility filter—combining forward dynamics uncertainty and inverse dynamics consistency—discards unrealistic synthetic transitions before they reach the training buffer, blocking model exploitation. The filtered counterfactuals augment any standard offline RL algorithm without modifying its training objective. On D4RL MuJoCo and Adroit tasks, CDA raises the returns of CQL, IQL, and TD3+BC by up to 25%, with the largest gains in low-data regimes. The same diffusion model also supports offline-toonline fine-tuning without catastrophic forgetting. CDA shows that counterfactual generation with diffusion models is a practical augmentation strategy for offline RL.

Rishi Ojha, Shivam Rajput, Shivam Dhakad, Vikash Narveriya, Dr. Shiv Kumar Sharma,"Counterfactual Data Augmentation for Offline Reinforcement Learning via Conditional Diffusion Models" International Journal of Advanced Multidisciplinary Research and Educational Development, V2(3): Page(1254-1257) May-June 2026. ISSN: 3107-6513. www.ijamred.com. Published by Scientific and Academic Research Publishing.