In recent times, there has been an increasing deployment of autonomous underwater vehicles (AUVs) for tactical acoustic communications. This necessitates the establishment of secure and reliable AUV-links to safeguard sensitive information. However, existing methods such as cryptography and channel coding introduce extra overheads and computational complexity. This is primarily due to the inherent challenges posed by acoustic communications, such as limited bandwidth and low energy efficiency. To overcome these challenges, intelligent reflecting surfaces (IRSs) in conjunction with reinforcement learning (RL) techniques is proposed. This to facilitate simultaneous secure and reliable communications between AUVs and buoyed nodes, resulting in what is termed as RL-based Buoyed-IRS-AUV (RL-BIA) links. The RL-BIA link is engineered to dynamically adjust its beam width and IRS's depth in response to seawater turbulence induced by wind and tide speeds. In this paper, we introduce a comprehensive link model that includes; pointing errors, path loss, interference, and noise. Furthermore, an RL-based system model that integrates the RL technology into the BIA link is proposed. A Max-Min optimization problem is also formulated in this work which integrates the channel secrecy and outage probability, and solves it iteratively using the Q-learning and SARSA algorithms. Our numerical results exhibit the efficacy of the proposed approach, with the RL-BIA link achieving an impressive 500% increase in channel secrecy compared to an RL-based buoyed-AUV (RL-BA) link.