Meet Matt Brehmer, a human-computer interaction and data visualization researcher interested in ubietous information experiences

Matt Brehmer joined the Cheriton School of Computer Science as a tenure-track Assistant Professor in August 2024.

Previously, he was a lead researcher at Tableau Research in Seattle, where he specialized in new experiences for communication and collaboration around data. Before joining Tableau, he was a postdoctoral researcher with the Extended Perception, Interaction & Cognition (EPIC) and Human-Computer Interaction groups at Microsoft Research. He completed his PhD and an MSc at the University of British Columbia, where he was a member of the InfoVis group and the Multimodal User Experience lab. He also holds an undergraduate degree in cognitive science from Queen’s University.

In 2022, Professor Brehmer was elected to the Visualization Conference (VIS) Executive Committee, which oversees the planning and success of the IEEE VIS conference, and he was appointed to the IEEE Visualization and Computer Graphics Technical Community’s executive committee. He is also dedicated to connecting research with practice: he currently serves as an advisor to the Maryland Institute College of Art (MICA)’s Master of Professional Studies in Data Analytics and Visualization, has served on the executive nomination committee of the Data Visualization Society, has co-organized the VisInPractice event at IEEE VIS, and has spoken at practitioner events including OpenVisConf, the Microsoft Data Insights Summit, and the Tableau Conference.

As of February 2025, Professor Brehmer’s research has been cited more than 2,900 times with an h-index of 24 according to Google Scholar.

What follows is a lightly edited transcript of a conversation with Professor Brehmer, where he discusses his research, his advice for aspiring computer scientists, and what excites him about joining the Cheriton School of Computer Science.

My research specializes in information visualization. It’s a blend of human-computer interaction (HCI), graphics and databases, but I come at it from the HCI side. Since my PhD, I’ve been focusing on visualization in the context of storytelling and visualization beyond the desktop, or beyond mouse and keyboard interaction: from tangible interaction on mobile devices to gestural interaction in augmented reality environments.

Currently, I’m focusing on what I call “ubietous information experiences.” “Ubiety” is a rare but useful word with a specific meaning: it means being sensitive to place. There’s a related term that people may be more familiar with, “ubiquity,” which is about being “everywhere, all at once,” with an implication of pervasiveness. When I say “ubietous information experiences,” it is information designed for a specific place and time: using the right information in the right presentation format at the right time. I’m interested in interactive and dynamic information visualization that fulfill this criteria, particularly for places where people collaborate and communicate, either in person or virtually.

It depends on who you talk to. I’ve had one foot in the academic world and another in the practitioner / industry world.

Among the academic visualization community, many are likely familiar with some of my PhD dissertation. It was a structured way of describing why people visualize data. It is an approach to classifying tasks that is both hierarchical and sequential. It helps visualization designers  and researchers articulate tasks in terms of sequences of actions, allowing them to determine what visualization or interaction design works for a particular task or activity.

Researchers found the related paper from my dissertation to be a useful way of thinking about visualization tasks. Whether that was in the context of designing experiments, justifying design choices, or doing fieldwork to observe how people use visualization tools and techniques. Before, visualization design was more of a process of “I have this data, …so which techniques could I use to visualize it? Instead, our paper can link someone’s goals (why) and their data (what) with techniques (how). One benefit of this approach is that it can lead to cross-pollination of ideas across application domains with common task and data abstractions. 

This paper, co-authored with my PhD advisor Tamara Munzner, was the centrepiece of my dissertation, and I used it throughout the other chapters, including design or evaluation projects. It continues to be widely cited. I’ve seen people using it to explain what tasks they chose for a particular user study or an evaluation. And at the 2023 IEEE Visualization Conference (VIS) conference, we received a Test of Time award for the paper. It was great to see that this work still resonated 10 years after it was published. 

Meanwhile, I’ve been in industry for the better part of the last decade. The visualization practitioner community may know me as an expert on time-oriented data visualization and timelines. I’ve spoken about this at industry conferences and visualization practitioner events like OpenVisConf. I’m also a member of the Timeline Consortium, which has proposed a standard for timeline data. Most recently, I was working with Tableau in Seattle, which is devoted to helping people see and understand data, building data visualization and business intelligence software. The Tableau “DataFam” might know me as someone who has worked on bringing Tableau to new modalities and platforms, including spatial computing with Tableau for visionOS. Another project, Tableau Gestures, focused on compositing interactive data visualization over video in live teleconference calls: you could use gestures and body language to point at data and direct your audience’s attention without using screen-sharing.

The journalist and writer Ellen Ullman has written about how technology has brought about a “disintermediation” between people since the dot.com era and through the rise of social media and AI. There is a mentality that the individual customer or user always knows which data, product or service best suits them. But there remain many situations in our daily lives and professional environments where we benefit greatly from intermediaries. So, you could say that I’m broadly interested in “reintermediation.” Another way of putting it is that I want to augment and mediate rather than replace or automate human-to-human communication. Particularly in cases where there are asymmetries in knowledge levels, like between teachers and students or journalists and the general public. Information visualization can help mediate between people with different levels of expertise, especially during real-time communication. How can you have an engaging and informative experience or interaction with another person that’s augmented with information? Or can data-rich hybrid or remote meetings be something that’s qualitatively different to in-person meetings while still being similarly rich and informative? Some of my research is driven by the belief that there’s great potential in augmented reality technology and using multiple modalities to present information in these contexts. 

There’s also a growing body of research about AI assistants, and how they might help you perform a task in real-time. I’m interested in applying that concept to multiple people having a meeting that is grounded in data. How can AI assistants help people reach a consensus or agreement? Could they raise everyone’s understanding to a baseline level during the meeting or presentation? Can they help with negotiation or conflict resolution? 

Finally, it’s not clear how we evaluate these interventions that we build. They’re inherently collaborative. They’re often distributed, especially if you’re talking about supporting remote or hybrid meeting scenarios. Often you don’t see the effects of these interventions after one meeting. It’s really challenging to determine how to validate the efficacy of some of the work that we do. What methodologies should we consider? Even beyond HCI, do we look to the humanities and educational psychology? How do we test these interventions that we build, both longitudinally and in these distributed formations of people, where multiple people are involved in a classroom, discussion or decision-making activity?

Even in systems like Zoom, there’s an AI assistant that’s built-in. It’ll give a distilled summary of the meeting, such as who spoke and what were the major topics and action items. But these summaries are retrospective. A potential intervention would be used as the meeting unfolds, where we could show representations of data that are relevant or complementary to the conversation. Hopefully, this intervention allows people to reach a common understanding or provide some participants with the missing context to help them catch up. 

HCI and visualization researchers and practitioners come from many different disciplines. So, having a multidisciplinary background is a tremendous asset if you want to pursue research in these areas. Students can seek out opportunities to work with someone in a particular domain that they don’t have previous knowledge in. These experiences could be rich sources for informing new techniques or applications that could potentially be used across other domains.

It can also be useful to read broadly, so you can see how others relay complex data-rich concepts to a lay audience. I like to read popular science, science history, science communication, and long-form journalism. These genres or mediums have the same goal as my research, just with a different modality.

Finally, it’s helpful to have side projects where you can collect and visualize data in different ways. Maybe it can be your personal data like your daily exercise or step count, or things you’re interested in like video games or manga. Even if your project doesn’t lead to a novel technique, it’s a way of practicing the craft and building your portfolio. Put your work out there and receive feedback. I recommend getting in touch with online communities, local meet-up groups or professional societies like the Data Visualization Society, which started in 2019: it is really welcoming and has a number of programs dedicated to early-career professionals.

Yes, it’s a big reason for what drew me here! The HCI group within the Department of Computer Science and the broader HCI community at the University of Waterloo are well-known, respected, and varied in terms of their interests. I was already collaborating with Professor Jian Zhao and one of his students before joining, and I look forward to more collaborations like this in the months and years to come.

Starting this term, I’ve started collaborating with some of our computer science undergraduate students through the Undergraduate Research Assistantships (URA) program. I knew that Waterloo’s undergraduate students are strong, and I’ve been impressed so far with their fresh perspectives and initiative.

Beyond the HCI community at Waterloo, there are also some exciting research opportunities that could be fun to collaborate on with other department members. Explainable AI is an emerging topic in the visualization community: using interactive visualization to explain how these algorithms or processes work. More broadly, CS education present many opportunities for using data visualization in explorable explanations. Finally, I’m also keen to work on visualization in the context of the ethics of computing, such as with visualization for platform or algorithm audits to illustrate how data flows through these sociotechnical systems.

I’ve returned to academia after spending nearly a decade in industry. There, I would have a PhD intern for about 12 weeks for a very focused project. I had many ideas, but I could only hire one intern per year. So, having this ability to form a group around my research ideas is very exciting. This term, I’ve started working with undergraduate students through the Undergraduate Research Fellowship and URA programs. These programs make it easier to work with students at different levels of their careers, and I’m eager to explore other similar programs. Another aspect of working in industry research was that  I had to align my projects with a productization mindset and with top-down corporate goals like increasing productivity or monetizing new product features. Here, I can have a broader range of projects and outcomes, and I feel more free to work on curiosity-driven ideas.

My PhD advisor Tamara Munzner certainly influenced my early career work. Since then, another frequent collaborator has been Bongshin Lee, who’s now at Yonsei University. I’ve had so many inspiring discussions with both of them.

Beyond my collaborators, Bill Buxton is also an inspiring figure. I first met him when I was undergrad, when he received an honorary doctorate at Queen’s University. After my PhD, we ended up being on the same team for a while at Microsoft Research. He was remote, though, so we didn’t work on projects together. But what I admired about Bill is the breadth of knowledge he brought to his visits. It was him that introduced me to this concept of “ubiety,” or designing something that appears at the right place at the right time. It’s been years since I learned about that term from him, but it’s still stuck with me. I find it a useful framing concept for what I’m thinking about now.

There are also historical figures that I find inspiring, especially in the field of visualization and information design.  One such figure is Otto Neurath, a member of the Vienna Circle, from around 100 years ago.  Along with Gerd Arntz, he pioneered this system of pictographic representation called Isotype for communicating statistical information to the working class, so that they might better understand their working conditions and the inequalities they faced.

There are also contemporary visualization and information design practitioners that I greatly respect like Alberto Cairo from the University of Miami, a mediator between the visualization and journalism communities. Then there’s Nigel Holmes, who was at Time Magazine for the longest time, who created information graphics were really fun and whimsical. Like Neurath, he uses pictographic elements to draw people in and help them understand the story. Finally, Giorgia Lupi has designed detailed information graphics that would be the equivalent of a long-read piece of journalism that you would sit down and read on a Sunday afternoon. Lupi’s work is beautifully done, incorporating visual metaphors and pictographic forms to convey multi-dimensional information to a general readership. As you can tell, I’m generally inspired by people who have this shared goal of conveying complex topics or statistics to a general audience.

I have an active toddler, so there’s not a lot of spare time between researching and being in “dad mode.” But when I need to unwind, I like to read for leisure, both fiction and non-fiction. With respect to the former, my favourite literary genre is hysterical realism. As for the latter, I typically like books covering topics at the intersection of technology, culture, and history.

I’ve also been collecting vinyl records for my entire adult life. I like to unwind by putting on an LP, sitting back and just listening to it with a good pair of headphones. I listen to a lot of genres and eras, but in general, I like music with a lot of dynamic range, varied instrumentation, and a fair bit of reverb. Recently, I discovered that the Waterloo Public Library has a vinyl record library, which I’ve been enjoying.

When the weather’s good, I like hiking. I miss the mountains of the West Coast, but it turns out that Southwestern Ontario isn’t as flat as I thought it might be. I also enjoy being out on the water. I started paddleboarding during the pandemic. Now, it’s something my dog and I do in the summertime. When we were in Seattle, we enjoyed paddling in Puget Sound.

Before my dad era, I would spend my winter weekends and evenings on music production. Once my toddler is older, I hope that we can jam and produce something together.