Robust Decision Making for Autonomous Vehicles at Highway On-Ramps: A
Constrained Adversarial Reinforcement Learning Approach
Abstract
Reinforcement learning has demonstrated its potential in a series of
challenging domains.
However, many real-world decision making tasks involve unpredictable
environmental changes or unavoidable perception errors that are often
enough to mislead an agent into making suboptimal decisions and even
cause catastrophic failures.
In light of these potential risks, reinforcement learning with
application in safety-critical autonomous driving domain remains tricky
without ensuring robustness against environmental uncertainties (e.g.,
road adhesion changes or measurement noises).
Therefore, this paper proposes a novel constrained adversarial
reinforcement learning approach for robust decision making of autonomous
vehicles at highway on-ramps.
Environmental disturbance is modelled as an adversarial agent that can
learn an optimal adversarial policy to thwart the autonomous driving
agent.
Meanwhile, observation perturbation is approximated to maximize the
variation of the perturbed policy through a white-box adversarial attack
technique.
Furthermore, a constrained adversarial actor-critic algorithm is
presented to optimize an on-ramp merging policy while keeping the
variations of the attacked driving policy and action-value function
within bounds.
Finally, the proposed robust highway on-ramp merging decision making
method of autonomous vehicles is evaluated in three stochastic mixed
traffic flows with different densities, and its effectiveness is
demonstrated in comparison with the competitive baselines.