A former postdoctoral researcher at the Cheriton School of Computer Science is leading the charge in artificial intelligence in the entertainment and film industry with her company, Kroop AI.
Jyoti Joshi, Kroop AI’s cofounder and CEO, worked as a Waterloo researcher in the specialized fields of computer vision and affective computing in 2016. She collaborated with Professor Jesse Hoey, whose research in emotional artificial intelligence and affective computing complimented her interests.
“I was looking at possible postdoctoral positions and had previously met Professor Hoey at a conference,” Jyoti says. “Our work had so many synergies. He knew my research and made it so easy to say yes, and I benefited immensely from our work together.”
Dr. Jyoti Joshi is the Founder and CEO of Kroop AI. She has a PhD from the University of Canberra in Australia. After completing her doctorate, she was a postdoctoral researcher at the Cheriton School of Computer Science, collaborating with Professor Jesse Hoey on AI assistive technologies for people with dementia.
The main research project Jyoti worked on at Waterloo focused on creating virtual avatars to assist people with dementia in their day-to-day activities. The personalizing influence of affective AI set out to make these virtual avatars responsive to the individualized needs of users.
“My time at Waterloo was an amazing experience,” Jyoti continues. “Now, what my company does is build out some of that postdoctoral work. We create programs that generate virtual avatars. It can cut short studio recording times and saves costs.”
Kroop AI, launched in 2021, offers two main service lines.
The first service, called ‘The Artiste AI Studio,’ allows creatives in film or in other artistic or communications fields to rapidly and seamlessly create a virtual character who will speak any lines of text, even in different languages. Such AI-generated characters could be edited into a movie, for example, and allow for reduced production costs and better transfer to different language markets. The product can be integrated in an existing system using an API or the easy-to-use cloud-based editor.
Kroop AI’s second service line builds on the first, allowing users to accurately detect when similar technologies are used to create fake videos, such as so-called deepfakes. These types of videos are becoming more common in online forms of misinformation and disinformation, so having a reliable way to detect deepfakes and present evidence of them is a highly valuable tool for organizations of all kinds. Content creators can now validate if original content has been modified without permission.
Because of the way the company sets out to produce such technology for cinematic purposes, and because of the way it allows users to uncover deepfakes, Kroop AI brands itself as an ethically oriented company, aiming to both educate and entertain.
Jyoti recently attended the Cannes Film Festival, promoting the new product and showing its versatility for producers, editors and for any savvy amateurs.
“What we are doing at Cannes is showing production houses how they can save cost and time, and also how they can get their films in front of larger audiences,” Jyoti says. “In this sector, there is a real need to produce content in different languages and to be able to use voice-overs.”
For example, the Kroop AI software enables creators to automatically dub a film into a different language, but then also have the lips of the characters sync to the new language, which is a breakthrough in an industry that has for so long dubbed films but had the characters lips out of sync.
“A lot of this work with dubbing into a different language happens on the post-production side of things,” Jyoti says. “But the same technology can enable the best rendering of 3D animation in the main part of production, and we are opening new avenues there as well.”
“We’re excited to be bringing this technology to the industry and the response so far has been amazing.”
Learn more about Kroop AI and view the promotional video below.
The Artiste AI Studio for content generation can be accessed at https://studio.theartiste.ai/.