This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.
Efficient Variance-Aware Reinforcement Learning in Time-Inhomogeneous Linear MDPs
arXiv:2611.23596v1 Announce Type: new Abstract: We study variance-dependent reinforcement learning in episodic time-inhomogeneous linear Markov decision processes (MDPs) with feature dimension $d$, horizon $H$, and $K$ episodes. While recent works have established variance-dependent regret with $\tilde{O}(1)$ deterministic-regime overhead for linear mixture MDPs (Zhao et al., 2023) and worst-case nearly minimax optimal regret $\tilde{O}(d\sqrt{H^3 K} + d^7 H^8)$ for linear MDPs (He et al., 2023),, existing variance-aware analyses for linear MDPs (Li and Sun, 2023; Huang et al., 2023) rely on a coarse concentration step whose resulting regret guarantees retain a polynomial $\tilde{O}(K^{1/4})$ additive term even in deterministic environments. We propose VA-LSVI-UCB++, a computationally efficient algorithm that achieves the variance-dependent regret bound $\tilde{O}(d\sqrt{H\min\{\mathrm{Var}_K^,\mathrm{Var}_K^\pi\}}+d^5H^4)$, where $\mathrm{Var}_K^$ and $\mathrm{Var}_K^\pi$ denote the…