Around 30% of Canadians rely on AI for personal and work use, from proofreading work emails to planning trips— but do we truly understand how AI works?
In today’s world, AI may dictate our everyday outcomes and choices more often than man-made decisions. Even the most essential sectors are adopting AI. Self-driving cars, a game-changer in transportation, use AI to sense its surroundings and control its movements. Some clinics are employing object detection and recognition models, a form of AI, to detect cancerous tumours from X-ray scans— and do it at a much faster and more accurate rate than human doctors.
Yet, most of the public is unaware of how AI makes decisions, leading to possible misuse or mistrust. One promising solution is Explainable AI (XAI) visualization, which illustrates the inner workings and performance of AI models through visualizations. Unfortunately, XAI visualizations are geared towards experienced AI users, such as machine learning engineers or model developers.
What if we could explain AI through video games? That is the vision of Yuzhe You (MMath '23), a second-year PhD student at the University of Waterloo’s David R. Cheriton School of Computer Science. Inspired by research on the positive learning outcomes of gamification, Yuzhe is leveraging interactive visualizations to make XAI more meaningful and accessible to non-technical users.
“When people hear the word ‘visualization,’ they tend to imagine simple, static bar charts or scatter plots. In reality, visualization can incorporate design elements, such as interaction and animation, to transform tedious information into something more engaging and easier to understand,” explains Yuzhe.
“Sometimes, I see XAI tools that are claimed to be designed for non-experts, yet they are extremely overwhelming, even for someone like me,” says Yuzhe. “People deserve to have full transparency on how AI models work. Why do they make certain mistakes or decisions? What is the technology behind these models? I want to help everyday people make informed decisions when they interact with AI systems.”
To support her transformative plan, Yuzhe was awarded with $145,000 in various scholarships, including the NSERC Canada Graduate Scholarship, the Ontario Graduate Scholarship and the University of Waterloo President’s Graduate Scholarship. With this incredible funding, she will conduct user studies to investigate people’s experiences with XAI visualization, including their understanding, attention, and cognitive load. Through these insights, she can develop prototypes that incorporate gaming concepts.
For example, Yuzhe designed a data interface, where a user could ask a non-playable character (NPC) about the data: “What does this datapoint represent?” or “What does the projection view panel mean?” If the user notices any issues in the dataset, like misclassified images, they can probe the chatbot, stripping away any misunderstandings.
Yuzhe infused humour and playfulness into the interface. She designed eight different NPCs, each with their own unique personality, from a wisecracking sidekick to a grumpy teenager who sounds like Squidward.
A demo of Yuzhe’s prototype that uses NPCs to guide users throughout their data analysis. The NPCs embody common video game archetypes, such as a humorous sidekick to a moody teenager.
Recently, Yuzhe and her supervisor, Professor Jian Zhao, conducted a comparative study, where ten non-technical users explored the interface with and without the chatbot. The participants were given only 40 minutes to complete 12 data encoding and interpretation tasks. To demonstrate their understanding of data analysis, they had to complete a pre-quiz and post-quiz relating to the system’s interface, XAI concepts, and visualization interpretation.
Notably, the group with the chatbot performed the best on the post-quiz, boasting a higher task completion and accuracy rate. They found the gamified approach interactive and relatable. In particular, the NPCs created a support network where they could ask AI-related questions without any judgment. Ultimately, this innovative prototype could lower barriers for novice users.
However, some users felt the NPCs’ constant chattering disrupted their workflow. So, Yuzhe wants to explore “personalized gamification,” where the interface is tailored to the user’s level of expertise.
“If the NPCs sense that the user has a low level of visualization literacy, then they could modify the data points and dialogue to make the visualizations more meaningful.”
Like other Cheriton researchers, Yuzhe shows that we can solve the world’s most pressing problems with strokes of creativity. As a teenager, Yuzhe dreamt of being an artist and even attended Rhode Island School of Design’s pre-college program. During her senior year of high school, she took a programming class and realized that she could “use computer science to bring my art to life.”
“When I was younger, I had this misconception that art and computer science are different sides of the spectrum. But when I developed my first-ever app, I was both the programmer and the artist,” says Yuzhe on why she pursued computer science for her undergraduate. “I realized that programming can be used to invent different kinds of applications that serve a purpose. But when you combine it with art, you can create something more powerful.”
Her PhD research comes full circle with her programming journey.
“As a programmer who came from an art background, I understand how nerve-racking it is to pick up something that’s totally out of your domain. That’s why I’m so interested in gamification to explain AI techniques and processes to people with less technical background. If I had these tools when I first started, then it would have been a more comfortable and easier learning experience.”