This position is offered through the ANU Computing Internship ([COMP4820] / [COMP8830])
Company
Penten (Applied AI) https://www.penten.com/
Penten leverages AI-driven solutions to enhance cyber defence. Active Cyber Defence protects critical infrastructure, Electronic Deception disrupts adversaries, and Cyber Training prepares organisations with realistic simulations, all ensuring robust protection and strategic advantage against digital threats.
Project
Improved Memory for LLMs
How can we best implement an efficient, realistic and robust concept of “memory” that most accurately reflects human behaviour, when interacting with an LLM?
Retrieval Augmented Generation (RAG) is one solution, but the implementation is not mature or complex and will not scale particularly well. It also doesn’t reflect true human memory. What methods have been explored experimentally? Are there any novel approaches we could test out? What recommendations can be made specifically to suit the Penten product offering?
This would involve a variety of Natural Language Processing techniques as well as some scientific research into behaviour and memory.
Research considerations:
- The capabilities and trade-offs of potential improvements (eg. realism, performance, response time, engineering complexity, and cost).
- The local processing resources (e,g CPU or GPU) and storage requirements.
- The prompt engineering required to generate realistic text and images.
Required/Preferred technical skills
Programming experience (preferably Python), use of LLMs, Linux
Required/preferred professional and other skills
Able to collaborate, work independently towards a goal.
Special Requirements
• Intern requires Australian citizenship
Delivery Mode
• In-Person Internship
Type of internship
Paid internship. The intern/s will be engaged as a casual employee.
How to apply
Applications are invited from students who have already passed the eligibility checks for the Computing Internship courses COMP4820 or COMP8830. Further information about the Computing Internships can be found on the Computing Internship page.
You can nominate multiple preferred Internship projects and host organisations through the one application form.
Student applications will be open from the 13th to 29th Sep, 2024.