This paper fuses ideas from Reinforcement Learning (RL), Learning from Demonstration (LfD), and Ensemble Learning into a single paradigm. Knowledge from a mixture of control algorithms (experts) are used to constrain the action space of the agent, enabling faster RL refining of a control policy, by avoiding unnecessary explorative actions. Domain-specific knowledge of each expert is exploited. However, the resulting policy is robust against errors of individual experts, since it is refined by a RL reward function without copying any particular demonstration. Our method has the potential to supplement existing RLfD methods when multiple algorithmic approaches are available to function as experts. We illustrate our method in the context of a Visual Servoing (VS) task, in which a 7-dof robot arm is controlled to maintain a desired pose relative to a target object. We explore four methods for bounding the actions of the RL agent during training. These methods include using a hypercube and convex hull with modified loss functions, ignoring actions outside the convex hull, and projecting actions onto the convex hull. We compare the training progress of each method with and without using the expert demonstrators. Our experiments show that using the convex hull with a modified loss function significantly improves training progress. Furthermore, we demonstrate faster VS error convergence while maintaining higher manipulability of the arm, compared to classical image-based VS, position-based VS, and hybrid-decoupled VS.

This paper is accepted for the IEEE SMC 2023. Adjusting the level of autonomy in human-machine systems (e.g., human-robot systems) holds great potential for achieving high system performance while maintaining operator involvement. To support operators with the task of setting the proper level of autonomy, we present a novel approach to realise a Model Predictive Controller that determines the optimal LoA for each tessellation in the robot’s path plan based on the estimated performance degradation due environmental adversities. We also report on an experimental evaluation of a mixed-initiative system where both the operator and the Model Predictive Controller are in charge of dynamically adjusting the level of autonomy cooperatively while performing a challenging navigational task with a mobile ground robot in a high-fidelity simulation. To this end, we conducted a user study with 15 participants comparing the performance and user experience of the model predictive system with a state-of-the-art system. The results show significant benefits of the model predictive system in terms of a reduction of conflicts for control and an improved user experience. Additionally, there are indications of benefits in terms of robot health and, consequently, performance for the model predictive system.

This paper is accepted for the IEEE SMC 2022 conference and addresses the problem of transfer of control authority between a robot’s AI and a remote human operator, when controlling a Mixed-Initiative (MI) robotic system. We propose a negotiation-theoretic method that enables the robot’s AI and the human operator to cooperatively and dynamically determine (i. e. negotiate) the transfer of control authority between these two agents. An experimental study is presented in which a state-of-the-art Expert-guided Mixed-Initiative Control Switcher (EMICS) method is compared with our proposed Negotiation-Enabled Mixed-Initiative Control Switcher (NEMICS) algorithm. Results suggest that the NEMICS framework is able to successfully avoid conflicts for control, which is a fundamental challenge encountered with previous MI control methods. Comparing NEMICS with the EMICS, we provide evidence of improved navigational safety (i. e. fewer collisions). Additionally, our usability study suggests that human operators perceived their interactions with NEMICS as less intrusive than with EMICS. The accompanying video demonstrates the differences between NEMICS and EMICS.