Numerous studies have demonstrated that numerous animal species are capable of goal-directed navigation using environmental information for dead reckoning. The stable magnetic field of the earth provides important information for the migration of animals over long distances. Inspired by the goal-directed navigation of animals, a novel Geomagnetic Navigation with Temporal Attention-based Data-Driven Dead Reckoning (Attention-DR) method for Autonomous Underwater Vehicles (AUV) is presented in this article, which only utilizes the inclination angle (I) and the declination angle (D) of the Geomagnetic Field (GF) for underwater navigation without any prior knowledge of the geographical location or geographic map. This article proposes a Temporal Attention-based Long Short-Term Memory (TA-LSTM) neural network by combining history, GF information, and location in time series to achieve the optimization heading angle of underwater dead reckoning. Due to its ability to adjust the utilization weights of local and global temporal information in the path, TA-LSTM possesses the capability to continually update neural network model parameters through a data-driven process during the navigation process. This enables the Attention-DR to achieve efficient navigation in regions without prior magnetic maps and overcome navigation failures caused by magnetic field anomalies. The simulation outcomes confirm the efficiency, precision, and practicability of the proposed algorithm.