Magnetic soft robots offer unique advantages in biomedical applications, particularly in drug delivery and minimally invasive procedures, due to their untethered and fast actuation, and physical adaptability. However, real-time control of magnetic soft robots is challenging because of non-linear dynamics and complex interactions with external magnetic fields. We thus propose a novel, model-free approach to controlling magnetic soft robots based on Proximal Policy Optimization (PPO) combined with Random Network Distillation (RND), integrated with a Convolutional Encoder-Decoder Network (CEDN). The control system autonomously generates action-state pairs to train the CEDN, which predicts control inputs for desired state transitions by abstracting low-level dynamic behaviors. This combination resulted in a real-time control of the robot without necessitating prior knowledge of the robot's geometric design, magnetic or dynamic profile. Experimental validation proves the control system's effectiveness in complex tasks, including successful multi-target navigation and obstacle avoidance. The control system was evaluated using an H-shaped magnetic soft robot. This method addresses the limitations of traditional control methods for magnetic soft robots, offering a practical and rapid solution for controlling magnetic soft robots in real-world biomedical applications potentially enabling practical applications of these robots. This work has significant implications for healthcare industry such as targeted drug delivery, minimally invasive procedures, and on-chip tissue manipulation.