1. Animal movement studies are conducted to monitor ecosystem health, understand ecological dynamics and address management and conservation questions. In marine environments, traditional sampling and monitoring methods to measure animal movement are invasive, labour intensive, costly, and measuring movement of many individuals is challenging. Automated detection and tracking of small-scale movements of many animals through cameras are possible. However, automated techniques are largely untested in field conditions, and this is hampering applications to ecological questions. 2. Here, we aimed to test the ability of computer vision algorithms to track small-scale movement of many individuals in videos. We apply the method to track fish movement in the field and characterize movement behaviour. First, we automated the detection of a common fisheries species (yellowfin bream, Acanthopagrus australis) from underwater videos of individuals swimming along a known movement corridor. We then tracked fish movement with three types of tracking algorithms (MOSSE, Seq-NMS and SiamMask), and evaluated their accuracy at characterizing movement. 3. We successfully detected yellowfin bream in a multi-species assemblage (F1 score = 91%). At least 120 of the 169 individual bream present in videos were correctly identified and tracked. The accuracies among the three tracking architectures varied, with MOSSE and SiamMask achieving an accuracy of 78%, and Seq-NMS 84%. 4. By employing these emerging computer vision technologies, we demonstrated a non-invasive and reliable approach to studying fish behaviour by tracking their movement under field conditions. These cost-effective technologies potentially will allow future studies to scale-up analysis of movement across many underwater visual monitoring systems.