This position is offered through the ANU Computing Internship ([COMP4820] / [COMP8830])
Company
Trellis Data Group https://www.trellisdata.com
Business unit/Division: ML Research Labs
We leverage AI to streamline processes at the enterprise level to solve complex problems that save time and money.
Project
Democratise Deep Learning Through Algorithmic Optimisation
Recently, deep learning models have grown in performance and in size. For example, Llama-3 has 408 billion parameters, while Google’s PaLM has over 500 billion parameters. They can no longer be trained or deployed on regular computers. For example, the electricity used to train GPT-3 is equivalent of an entire suburb’s electricity consumption over a month.
That means most people won’t have the resources to train or run large-scale deep learning models. In order to let deep learning models to run on regular computers again, we need to use the computation resources more efficiently.
You will focus on:
- explore an LLM inference framework called SGL (https://github.com/sgl-project/sglang)
- understand how it works(go deep into its research paper)
- setup it and document the steps.
- write a script to do inference with it. Benchmark the speed of it and document the result.
Required/Preferred Technical Skills
- Software Engineering, Python, Pytorch
Required/Preferred Professional/Other Skills
- Professional Written and Oral Communication
Special Requirements
The intern requires a Police Check.
Delivery Mode
In-person.
Type of internship
Unpaid placement.
How to apply
Applications are invited from students who have already passed the eligibility checks for the Computing Internship courses COMP4820 or COMP8830. Further information about the Computing Internships can be found on the Computing Internship page.
You can nominate multiple preferred Internship projects and host organisations through the one application form.