This project is only available for domestic PhD students. Contact Penny Kyburz (penny.kyburz@anu.edu.au) with a short (<1 page) proposal outlining how you would approach this project and your skills/interest in this project by 31 October 2025.
This project is part of the Next Generation AI Graduates Program on Human-AI Interaction in the Metaverse: https://www.csiro.au/en/work-with-us/funding-programs/funding/Next-Generation-Graduates-Programs/Awarded-programs/Human-AI-Interaction-Metaverse
You will work in a multi-disciplinary team, in collaboration with industry partners and supervisors. The project includes a $43,333 stipend and a placement with our partner organisation, Gradient Institute, as well as training provided by CSIRO and a cost of living allowance of $5,000, training allowance of $5,000 per year (to cover courses, workshops, conferences, networking, collaboration) and travel allowance of $5,000 (in addition to ANU-provided funding). See funding details here: https://www.csiro.au/en/work-with-us/funding-programs/funding/Next-Generation-Graduates-Programs/NextGen-scholarship-information
Project Description
Large Language Models are no longer just the engines behind the most advanced chatbots like ChatGPT. They are becoming the cognitive core of autonomous agents capable of executing complex, multi-step tasks with minimal human supervision. The frontier of this research is now scaling to multi-agent systems, where agents interact and coordinate in diverse configurations – from specialised teams collaborating on shared objectives to autonomous agents pursuing individual goals while coexisting in a shared environment. From automating intricate business workflows to deploying department-specific agents that each manage their own domain while coordinating on cross-functional issues, the promise of multi-agent coordination is immense, offering a step-change in efficiency and capability.
However, this exciting frontier brings a fundamental challenge: system-wide reliability. Recent research has established that the interactions between agents can amplify single-agent risks as well as create entirely new risks: evidence shows agents reinforcing each other’s errors, falling into coordination traps, and exhibiting correlated vulnerabilities that create systemic blind spots. A system of reliable agents is not necessarily a reliable system.
The scientific understanding as well as the engineering tools and know-how required to govern these systems well are still in their infancy. This PhD project, conducted in collaboration with researchers from Gradient Institute, will tackle this gap by pioneering new techniques aiming at advancing the reliable governance of multi-agent systems. Gradient Institute works closely with industry and this research will be grounded on concrete, practical realities arising from real-world needs. We will explore questions that are both of a fundamental nature and directly relevant to practice. Examples of potential questions are: How can we design robust coordination protocols, allowing agents to reliably negotiate complex agreements between organisations? How can we model and predict the risk of cascading failures in safety-critical systems? How do we design an automated system to effectively oversee the collective behaviour of a multi-agent system? How do we jointly train a group of agents within an organisation to achieve a desired collective outcome? In the context of a specific use-case, can we build a reliable multi-agent system even when its agents are individually unreliable? How can the understanding of collective human behaviour inform the design of more robust multi-agent systems?
Answers to these questions will be helpful not only to advance fundamental understanding. Progress in this area will benefit a wide range of decision-makers – from industry leaders looking to safely deploy multi-agent systems for increased productivity to policymakers tasked with regulating how organisations will be allowed to use these technologies. By developing the science and tools for trustworthy multi-agent systems, this research will contribute to a future where society can harness the immense potential of autonomous AI agents while controlling their considerable downside risks, ensuring these systems operate safely and reliably in critical sectors of our economy.