Self-supervised animal pose estimation and tracking

Scholarship Available - Develop a computer-vision pipeline for self-supervised pose-tracking of larval zebrafish from high-speed recordings of hunting behaviour.

SHOLARSHIP AVAILABLE: The successful applicant for this project will be appointed as a Jubilee Joint Scholar. Applicants who are fourth year students at the School of Computing with an outstanding academic record may also be eligible for a scholarship of $8000 (conditions apply - contact Dr McCullough for details).

This project is funded by the Talo Computational Biology Accelerator Program. The successful applicant will have priority access to high-performance GPU compute resources to complete this project.


Develop a computer-vision pipeline for self-supervised pose-tracking of larval zebrafish from high-speed recordings of hunting behaviour.


Observations of behaviour provide critical insight into the function and health of the brain and nervous system. Measurements of animal behaviour in laboratory experiments are used to investigate the neural circuitry that controls behaviour, understand how neurodevelopmental disorders alter behavioural patterns and choices, and investigate potential drug therapies for an extensive range of conditions. The larval zebrafish is a powerful and widely used vertebrate model for such investigations in developmental biology and systems neuroscience. The brains of larval zebrafish have high physiological and anatomical similarity to the mammalian brain and can be imaged to record whole-brain activity with single-neuron resolution using non-invasive optical methods. Importantly, larval zebrafish display a diverse repertoire of complex behaviours from an early age, including hunting. However, existing approaches for tracking zebrafish behaviour only capture a limited subset of pose-information.

Self-supervised learning has seen great success in a range of domains from natural language processing (e.g., ChatGPT) to image analysis (e.g., DINO) and segmentation (e.g., SegmentAnything), but has yet to make a significant impact on the analysis of animal behaviour, where supervised methods are the de facto standard. The expense of annotations in this domain, alongside the need for generalisation beyond a single lab setup, make self-supervision worth investigating.

The primary aim of this project is to create a computer vision pipeline for pose-tracking of larval zebrafish from high-speed recordings of hunting behaviour using self-supervised learning via conditional autoencoders. The project may also involve the use of motion segmentation via deep optical flow estimation.


The project requires a strong interest and background in computer vision and machine learning. Experience with HPC environments is highly desirable. Suggested courses: COMP4528, COMP8536, COMP8536.


Jakab, T., Gupta, A., Bilen, H. & Vedaldi, A. Unsupervised Learning of Object Landmarks through Conditional Image Generation. in Advances in Neural Information Processing Systems vol. 31 (Curran Associates, Inc., 2018).

Teed, Z. & Deng, J. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. in Computer Vision – ECCV 2020 (eds. Vedaldi, A., Bischof, H., Brox, T. & Frahm, J.-M.) 402–419 (Springer International Publishing, 2020).

McCullough, M. H. & Goodhill, G. J. Unsupervised quantification of naturalistic animal behaviors for gaining insight into the brain. Curr. Opin. Neurobiol. 70, 89–100 (2021).

Datta, S. R., Anderson, D. J., Branson, K., Perona, P. & Leifer, A. Computational Neuroethology: A Call to Action. Neuron 104, 11–24 (2019).

Zhu, S. I. & Goodhill, G. J. From perception to behavior: The neural circuits underlying prey hunting in larval zebrafish. Front. Neural Circuits (2023).


Dr Michael McCullough (cc: Dr Dylan Campbell)

You are on Aboriginal land.

The Australian National University acknowledges, celebrates and pays our respects to the Ngunnawal and Ngambri people of the Canberra region and to all First Nations Australians on whose traditional lands we meet and work, and whose cultures are among the oldest continuing cultures in human history.

arrow-left bars search times arrow-up