Last week, Nvidia released a new version of Nvidia broadcast (opens in new tab) — Deep learning and AI-powered software for noise suppression, background removal/replacement, camera framing, and now eye contact. The last one is currently in beta and should probably remain in beta.
AI and deep learning have been in the news a lot lately, and for good reason. Dall-E, Midjourney, Stable Diffusion and others have created art from text, often with pretty impressive results. Of course, in some cases, you end up with his creature, a miserable mutant with two and a half heads and too many limbs. In terms of text, ChatGPT has produced a ton of English essays and easy-to-read sentences that many fear would mean the death knell of journalism (no, I didn’t write this news post. Hmm).
The idea behind eye contact is simple. When attending a webcast or meeting, we often look away from the camera.In fact, there is a real chance that you are everytime Look away from the camera because it is at the top of the screen and what you want to see is on the screen. But what if there was a way to make it look like you’re looking at the camera without looking at it?
What if you could train an AI model on your face and teach it to fix images where someone isn’t looking straight into the lens? Take millions of properly tagged images and feed them into the network. And then you see a nice tool, don’t you?
The implementation is not that simple. Nvidia has been talking about its Eye Contact feature for over a year now, and it’s now in public (beta) release. The myriad facial differences around the world is a difficult problem to “solve” and even now the results are… imperfect (that’s a good thing to say).
Anyway, I tested it on a system with an RTX 3090 Ti.
One thing I noticed during testing was that the live video feed oscillated between me looking at the camera and me looking elsewhere, even though my focus stayed in the same spot. is common. It’s possible that this is intentional, as having someone staring directly at the camera throughout the entire video chat can get a bit creepy, but if that’s the case, the timing needs to be adjusted.
The more difficult question is whether this kind of effect is beneficial at all. If you want it to look like you’re looking at a camera, you probably need to learn to look at it.Resolving human error with AI can only encourage bad habits. What if you end up with a video feed that doesn’t correct eye contact?
Either way, RTX owners can now test Nvidia Broadcast with Eye Contact. I tested with his RTX 3090 Ti, but Nvidia lists his RTX 2060 as the entry point (which, as far as I know, should include the mobile RTX 3050 GPU). Longer term, I think Nvidia will someday make some AI models that are more complex than the RTX 2060 and require faster hardware than the RTX 2060 — just like DLSS 3’s frame generation capabilities are on par with the RTX 40 series. Just like you’d need a graphics card — but for now every RTX GPU manufactured in the last four years is capable of stepping up this feature.
Do you like the effect, hate it, find it creepy, or is it something else? Let us know in the comments, along with other effects you’d like to see. I look forward to the time when virtual cartoon avatars like Jensen speak in place of real humans, perhaps read articles written by AI, and both videos and articles are consumed by AI.
It’s been a bot ever since!
