Professor Jing Jiang.
Teaching a computer to understand language has been one of the core problems of machine learning. This field, known as natural language processing (NLP), forms the basis for large language models like Claude. Through neural networks, LLMs, trained on vast text datasets, learn to associate words with meaning and represent the relationships between words. Predicting the next word allows these models to hold conversations with users.
As one of the pillars of modern AI, the field of NLP is moving beyond language to new frontiers, including integrating with computer vision to combine language with videos and photos.
Academics like ANU School of Computing Professor Jing Jiang are helping redefine the field for a new era. Recently selected as VP-elect of the globally influential Association for Computational Linguistics (ACL), she will lead NLP’s future identity as a research area.
“It is an exciting time for all of us in the field as these large language models are making great breakthroughs,” Jiang said. “But the identity of NLP is now not that clear as we go beyond language understanding into foundation models that can handle different modalities of data. I believe we should keep an open mind rather than just thinking of us as working on language only. NLP is no longer an isolated field; it’s embedded into a wide range of applications like coding and science discovery.”
The ACL is the premiere society for NLP researchers, with chapters around the world. Jiang’s election will eventually transition to the presidency and recognizes her wide-ranging impact on the field as a researcher, conference chair and journal editor. In addition to shaping the future of the field within AI, she will help manage new challenges, such as the growing number of low-quality papers created with AI tools.
“The peer review process is being affected, with people not doing a proper job and sometimes relying on AI for review,” she said. “It’s a big challenge for conferences to manage this change and make sure we are still doing meaningful research.”
Unexpected breakthroughs
Jiang discovered her passion for NLP through initial research into databases as an undergraduate student. While applying for her PhD at the University of Illinois at Urbana-Champaign, she joined a group focused on databases that gradually shifted to working on textual data handling, storage, and retrieval, which became NLP.
“I realized it’s really fun to apply machine learning techniques to language,” she said. “Mathematical models have always been my interest, using precise ways to understand, process, and generate text.”
Prior to LLMs, neural networks saw success in the 2010s with computer vision models working on large datasets of images for uses like facial recognition. Through a breakthrough technique called a transformer model, researchers combined the vast amount of textual data online with the increasingly powerful hardware necessary to train and run these models.
“I don’t think people anticipated the models suddenly becoming so powerful,” Jiang said. “In hindsight, there is good reason for that: language contains knowledge of the entire human civilization. An understanding of human society is already encoded in the large body of text these machines have access to. Compared to learning from image and media data, learning from text is more efficient.”
For example, an illustration in a picture book takes upwards of ten thousand bytes to express, while the text for the entire story takes a few hundred bytes.
New places bring new opportunities
Jiang completed her PhD in 2008 at the University of Illinois at Urbana-Champaign and joined the faculty of Singapore Management University. After living in Singapore for sixteen years, she and her family decided to explore a different place to live, closer to nature, and landed on the ANU, which also has strategic advantages in impact and policymaking.
Growing up in a small city in China, Jiang’s journey as a researcher stems from the choice to go to new places for new opportunities. At fifteen, she had the opportunity to go to Shanghai for senior high school. Surrounded by people interested in science and technology, she broadened her interests and decided to go the United States for her bachelor’s degree. During her summer research internship and encouraged by her role model Jennifer Widom, she decided to pursue a PhD rather than join industry.
Her advice for future researchers is to follow their passions by making a connection between those interests and the current state of the field.
“Look for connections so that you feel happy about what you are researching, but can also make sure your work is recognised,” Jiang said. “If your topic is too isolated, it might struggle to find an audience.”
Jiang’s current research focuses on how to steer and change LLM behaviour without heavy computation and expensive training. Since joining the ANU, she has been excited about the work her colleagues are doing. She sees opportunities to join forces and build foundational models that integrate multiple types of data.
With the University located in the nation’s capital, there are many projects to safely explore and use AI in the public sector. Jiang recently worked with an agency that oversees healthcare data, providing input on how to leverage agentic AI to improve data access.
“We got a chance to work on that project partially because we are here at ANU,” she said, referencing the ANU’s unique position and influence as Australia’s only national university. “I would encourage students across Australia and overseas to consider pursuing research at the ANU.”