Comparing Stochastic Training Environments for Deep Reinforcement Learning with Deep Q-network: Learning Stability and Transfer to Real Data
Journal of Mathematical Models and Methods in Engineering · 2026
DOI: 10.66901/q1d2dq55 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Mathematical Models and Methods in Engineering, 2026. Verfasst von Yuriy Yurchenko und 1 weiteren Autor:innen.
Abstract (Originalsprache)
The paper investigates how the choice of a generative stochastic model of the training environment affects the numerical stability of the learning and the transfer of the Double DQN policy to real time series. Four environments — GBM, GARCH(1,1)-t, Minimal Neural SDE, and Neural Jump-Diffusion — are compared, created on chronologically previous data of three financial time series: NVDA, SPY, and KO. For each combination of asset and environment, ten training runs are performed, a total of 120, with the architecture, state and action spaces, reward functions, and hyperparameters unchanged. The transfer was evaluated on a 250-day chronologically separated real window. Differences were found between the environments in all groups of indicators studied. Training in the GARCH environment was accompanied by the largest temporal-difference error and the largest gradient norms, but at the same time the smallest discrepancy of state space ranges and the lowest turnover on all three assets. In the Double DQN configuration used with Huber loss function and gradient clipping, no numerical divergence of estimates was observed in 120 runs; at the same time, the positive bias of Q-scores was preserved in all twelve asset–environment combinations. . In 14 of the 120 runs, the trained agent outperformed the corresponding fixed-allocation baseline selected on synthetic validation data. Agents generally held a larger share of capital in the asset when trained in environments with higher drift, although this pattern was not consistent across all assets. The obtained results characterize the engineering interaction between the generative environment and the DQN agent, but do not indicate the superiority of agents over passive deterministic control bases.
Zitieren
APA
Yurchenko, Y., & Zakovorotnyi, O. (2026). Comparing Stochastic Training Environments for Deep Reinforcement Learning with Deep Q-network: Learning Stability and Transfer to Real Data. Journal of Mathematical Models and Methods in Engineering. https://doi.org/10.66901/q1d2dq55
Vancouver
Yurchenko Y, Zakovorotnyi O. Comparing Stochastic Training Environments for Deep Reinforcement Learning with Deep Q-network: Learning Stability and Transfer to Real Data. Journal of Mathematical Models and Methods in Engineering. 2026. doi:10.66901/q1d2dq55
BibTeX
@article{Yurchenko2026,
author = {Yurchenko, Yuriy and Zakovorotnyi, Olexandr},
title = {Comparing Stochastic Training Environments for Deep Reinforcement Learning with Deep Q-network: Learning Stability and Transfer to Real Data},
journal = {Journal of Mathematical Models and Methods in Engineering},
year = {2026},
doi = {10.66901/q1d2dq55}
}
EUVIMED