Professor Yaoliang Yu has been awarded $100,000 by the Ministry of Colleges and Universities Early Researcher Awards program to develop deep generative machine learning models. The Ministry’s amount is matched by an additional $50,000 from the University of Waterloo, bringing the total funding to $150,000 to support one PhD, two master’s, and a number of undergraduate students over five years.
Professor Yu is one of five researchers at Waterloo to receive a 2024 Early Researcher Award, a provincial program that recognizes exceptional young faculty by helping them expand their research team.
Professor Yaoliang Yu is an expert in machine learning and optimization. He is a faculty member at the Vector Institute, a Canada CIFAR AI Chair, and was a Cheriton Faculty Fellow from 2020 to 2023.
He has won best paper awards at ACM SIGSOFT as well as at multiple workshops, and has published more than 70 top journal and conference papers in machine learning. His research has been cited more than 4,000 times with an h-index of 31 as of March 2024 according to Google Scholar. Over the past decade, he has supervised and co-supervised nine undergraduate, six master’s and ten doctoral students.
“We are immensely grateful to the Ontario government for their investment in the University of Waterloo’s researchers through the Early Researcher Awards,” says Charmaine Dean, vice-president, Research and International. “Waterloo is advancing innovation in many impactful areas to develop new technologies and boost economic development.”
Background
After years of foundational research, we are witnessing a truly explosive growth of artificial intelligence. Almost monthly another significant breakthrough is heralded, from AlphaGo, the computer program that beat the best human players at Go, to AI agents that have set new records in determining a protein’s 3D shape based on the sequence of its constituent amino acids, to restoring and determining provenance of ancient texts, to solving open problems in math and computer science, among many more developments.
Among the factors that have contributed to AI’s success is its ability to process huge amounts of data using unsupervised learning, a type of machine learning that does not require manual labelling of data or reinforcement by humans. Perhaps the simplest way to achieve unsupervised learning is by using generative modelling. The well-founded assumption is that if the models can automatically generate data they must have gained significant knowledge and are ready to generalize.
But as impressive as they are, deep generative models are difficult and expensive to train because they require much data and computing power. In fact, training has become so expensive that often it is not possible to fix a bug discovered after a model has been trained. Moreover, as training involves more and more user data and computing power, containing costs and striking a balance between model utility, robustness against malicious use, and privacy all become increasingly important.
About Professor Yu’s proposed research
With support from the Early Researcher Awards program, Professor Yu and his students will address these issues by creating lightweight generative models that require significantly less data, storage and computing power, by developing distributed systems that enable training on low-cost smart devices, and by accounting explicitly for concerns about trustworthiness.
Generative models are an indispensable part of unsupervised learning and artificial intelligence. In short, a generative model learns to generate new data similar to data it has been given. A new emerging theory formulates generative models as learning a sequence of deterministic mappings that turn input data — e.g., a photograph of a dog — gradually into noise. By reverting this process new data can be generated from pure noise.
The reverse map — termed a push-forward in probability theory — can be parameterized by deep neural networks whose weights are then learned through stochastic gradient algorithms. This new method has been extremely fruitful and has revealed surprising connections between different algorithms and fields. Professor Yu’s proposed research aims to develop this idea of push-forward substantially and to broaden the horizon of generative modelling significantly.
