SHOLARSHIP AVAILABLE: The successful applicant for this project will be appointed as a Jubilee Joint Scholar. Applicants who are fourth year students at the School of Computing with an outstanding academic record may also be eligible for a scholarship of $8000 (conditions apply - contact Dr McCullough for details).
This project is funded by the Talo Computational Biology Accelerator Program. The successful applicant will have priority access to high-performance GPU compute resources to complete this project.
Aim
Develop a computer-vision pipeline for self-supervised pose-tracking of larval zebrafish from high-speed recordings of hunting behaviour.
Background
Observations of behaviour provide critical insight into the function and health of the brain and nervous system. Measurements of animal behaviour in laboratory experiments are used to investigate the neural circuitry that controls behaviour, understand how neurodevelopmental disorders alter behavioural patterns and choices, and investigate potential drug therapies for an extensive range of conditions. The larval zebrafish is a powerful and widely used vertebrate model for such investigations in developmental biology and systems neuroscience. The brains of larval zebrafish have high physiological and anatomical similarity to the mammalian brain and can be imaged to record whole-brain activity with single-neuron resolution using non-invasive optical methods. Importantly, larval zebrafish display a diverse repertoire of complex behaviours from an early age, including hunting. However, existing approaches for tracking zebrafish behaviour only capture a limited subset of pose-information.
Self-supervised learning has seen great success in a range of domains from natural language processing (e.g., ChatGPT) to image analysis (e.g., DINO) and segmentation (e.g., SegmentAnything), but has yet to make a significant impact on the analysis of animal behaviour, where supervised methods are the de facto standard. The expense of annotations in this domain, alongside the need for generalisation beyond a single lab setup, make self-supervision worth investigating.
The primary aim of this project is to create a computer vision pipeline for pose-tracking of larval zebrafish from high-speed recordings of hunting behaviour using self-supervised learning via conditional autoencoders. The project may also involve the use of motion segmentation via deep optical flow estimation.
Requirements
The project requires a strong interest and background in computer vision and machine learning. Experience with HPC environments is highly desirable. Suggested courses: COMP4528, COMP8536, COMP8536.
Reading
Jakab, T., Gupta, A., Bilen, H. & Vedaldi, A. Unsupervised Learning of Object Landmarks through Conditional Image Generation. in Advances in Neural Information Processing Systems vol. 31 (Curran Associates, Inc., 2018).
Teed, Z. & Deng, J. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. in Computer Vision – ECCV 2020 (eds. Vedaldi, A., Bischof, H., Brox, T. & Frahm, J.-M.) 402–419 (Springer International Publishing, 2020).
McCullough, M. H. & Goodhill, G. J. Unsupervised quantification of naturalistic animal behaviors for gaining insight into the brain. Curr. Opin. Neurobiol. 70, 89–100 (2021).
Datta, S. R., Anderson, D. J., Branson, K., Perona, P. & Leifer, A. Computational Neuroethology: A Call to Action. Neuron 104, 11–24 (2019).
Zhu, S. I. & Goodhill, G. J. From perception to behavior: The neural circuits underlying prey hunting in larval zebrafish. Front. Neural Circuits (2023).
Contact
Dr Michael McCullough (cc: Dr Dylan Campbell)