Leading facial performance capture studio Dimensional Imagining 4D (DI4D) recently announced the launch of PURE4D 2.0, the latest in hyper-realistic facial capture technology. DI4D’s work can be seen in titles like theCall of Duty: Modern Warfareseries,theF1games,The Quarry, Venom: Let There Be Carnage,Marvel’sShe-Hulk,and much more. The company specializes in accurate facial capture that eloquently communicates the subtlety and nuance of actors' every emotion to create a breathtakingly realistic “digital double” of the performer. Now, for the updated version of the company’s proprietary 4D facial capture system, DI4D has partnered with award-winning actor, performance capture artist, and capture and voice director Neil Newbon to highlight exactly what PURE4D 2.0 can do.
Game Rant had a chance to speak with Newbon about all things PURE4D 2.0. Bursting with excitement, Newbon discussed what it was like getting to use the new technology, his reaction to “meeting” his own “digital double,” and briefly his upcoming short film - appropriately titledDouble -which features PURE4D 2.0 in action. Newbon had a few stories to share about his role as snarky vampire Astarion inBaldur’s Gate 3.He also shares his thoughts on his acting journey until now and how technology like PURE4D 2.0 will center human actors in an increasingly AI-focused world.The following transcript has been edited for clarity and brevity.
Neil Newbon On Working With DI4D
Q: What can you tell us about DI4D’s new PURE 4D 2.0 technology? You’re one of the first to get to work with it, right?
Newbon:It was awesome! I mean, they’d used it a bit, forCall of Dutyand things like that, but I’m one of the first actors who worked with them on the new update. The updated version of the technology, which I was working with, is new. For me, as an actor, I’m very used to HMC (head-mounted camera) work, so te experience of that hasn’t changed in terms of what it’s like to be an actor in an HMC. I think the changes are from their side, in terms of the fidelity and the nuance. The detail of the skin is really like - “Wow, my pores were not good that day.” I probably should have done a face mask or some clay or something. It was very “Woah.” “Cool.” Very interesting to see.
Q: With AI unfortunately seeing increased use in game development, do you think more realistic capture technology like PURE4D 2.0 will help combat that?
Newbon:I really like DI4D’s ethos about performers. Because, like myself, they’re staunchly againstreplacement with AI. Because ultimately, AI is not there, and it’s not going to be there in the way that people are worried about for a long time. It might be there at some point, and I think what we’re trying to do is future-proof performance. As cool as AI can be in so many things, like talking robots and such, it’s a slippery slope to lose the best parts of our humanity. One of them is art. I think, if it’s being replaced by something beyond human, how can we be convinced that they are not human?
Now, it’s okay to have an art form that’s by a robot, but it has to be very tightly highlighted: This is not a human thing. I think that would then rob people of such wonderful experiences: opportunities, jobs, also connections to other humans, which I think is really important. I mean, I’m a big fan of Iain M. Banks.Banks’Culturenovelsare really cool, and the machines and humans sort of integrated as a culture, which is cool. I’m not entirely sure that’s real. I don’t think that we could get there, you know, but that’s a cool idea. I just don’t know if it’s going to work out like that.
So, for me, it’s important that companies like DI4D are championing actors. I think what this does, the “digital double” tech that they have, is really putting performance first. Get your actors in, and it has some amazing software that can scan them - if you choose to scan them. You don’t have to. You can take their face off or put them on, it’s up to you. Either way, you’re going to have this incredibly nuanced detailed performance. It’s so subtle - you can have a little wince of the eye and that will be completely read. It’ll look beautiful, if that’s the tonality that you’re going for. I like their attitude. I think it’s cool that they want to represent the nuance of each individual actor in the game, through their hardware and software.
I think that’s important because I want to work with humans. I don’t want to be given a lot ofAI stuffto then have to try and make work. I think it’s incredibly boring. You become like a programmer - all due respect to programmers. I mean, there’s creativity in programming, for sure, but for me as a creative, the idea of trying to work with AI would take 1000 times longer than saying to an actor: “Just take the heat off that a little bit. Raise the stakes on that line. Just give me a beat and really, really stay with that” or something like that. Or an actor showing you something that you couldn’t possibly have predicted! Their take on a character. Actually, that’s kind of what I did as Astarion.
Performance Capture Work: From Detroit to Faerun and More
Q: Yourperformance as Astarionis iconic - it’s hard to believe there was ever doubt about it.
Newbon:Initially, Swen Vincke was not sure about me and Jason Latino was definitely in my corner on that. Because the choices I was making, I think, were quite big. And a risk, I would say - which I felt was the right thing to do. I know not everybody was sure about it initially, until they got what I was trying to do, andStephen Rooneyis such a beautiful writer. His writing is so good, as well as the contributions from the other writers, and it obviously worked super well.
Perhaps a little bit too well, Darling! Just a little soupçon too much![In Astarion’s voice]
It’s completely legitimate, totally fair enough. Ultimately, it’s their vision. I think, as an actor - well, I ended up as a director on it as well - but, as an actor coming into a role, you have the idea of your character. Ultimately, you’re there to serve the vision of the story. I think, sometimes, that’s when actors and directors can sometimes be in conflict - not in a bad way, in a professional, courteous way. But sometimes you have those moments of “I don’t really know what you’re trying to go for, but I’m not sure about it. It doesn’t feel right, whatever it is.” It’s never a guaranteed thing, I think.
Q: What would it have been like doing performance capture for any of your previous roles using the new DI4D tech?
Newbon:Well, withAstarionwe used video referencing, and then we had an amazing team of animators that did all the face stuff based on our video references. There are definitely things that I did facially for the performance that they liked a lot, and kept, and added in there. Also, sometimes they had to manage it themselves to match up the audio and stuff like that, to make it really good and keep up the fidelity because we weren’t using head-mounted cameras - it was too much data and would take too long. Everything else, the movement, was all us - but the face was used as a reference.
Laezel’s Devora Wilde is brilliant. There’s this head movement that Lae’zel does, and they would have seen it and gone: “That’s really cool. That’s what we’re going to do,” and make sure that it lines up to the face. I don’t know enough about the technology to be able to talk further about it, but that’s also a really valid way of working when you’re dealing with so much information.
Then, for things likeDetroit: Become Human,we used face markers, which is super-old-fashioned.Planet of the Apes:Last Frontierwas the first time I used markerless face, which is very cool. Up until about six years ago, beforeKingsglaive: Final Fantasy 15,is the first time I experienced it-it was a much bigger performance.
Before about 2015, we typically would enter with characters and we’d have to be a lot bigger with our performance choices, more theater than film. With the advent of early technology of HMCs, we’re allowed, suddenly, subtlety. Having something like the DI4D 2.0, if we had that, that would have just accelerated. The state where narrative is now, which is meeting gameplay - I saw that being there. 15 years ago, I knew, because of the way that computers have evolved, games have evolved, and games machines have evolved, I knew that was going to come at some point. I just wasn’t sure when. The idea of nuance, subtlety, stillness, etc. reading as well as animated action. I think that was a big game changer.
When HMC started developing very quickly, suddenly you could start getting subtlety into the performance. There’s a bit inKingsglaive-I did the performance capture for Nyx, as well as the voice and performance capture for Petra. There’s this moment where I sort of wince a bit, and they caught it on the HMC. They kept it in the final because it was actually caught on data, so that’s a really good example of how suddenly you’re able to start getting more nuance and subtlety into performances. I think it was also important to then start that process of allowing narrative to come to the forefront, to be as good as gameplay. People’s tastes developed as well. You saw more integration between TV, film, and games together. Things changed a lot.
Newbon Stars In Upcoming Short Film “Double”
Q: You’re making a film with DI4D using the new capture technology - what can you tell us about that?
Newbon:I can’t tell you much about it. I can tell you that it’s calledDouble,and that it will be out in the summertime. It’s about the idea of the digital double, of nuance, of subtlety and performance. I’m not going to say more than that, because they’ll kill me.[Laughs]
Q: What was it like doing performance capture and playing yourself, rather than a character?
Newbon:The thing about performance is, it’snotme. I am quite a private person, in many ways. I don’t really talk at great length about my personal private life, for a very good reason. I don’t think it’s interesting. I don’t make money from it. It’s not something I want to monetize. I’d like to protect the people in my family as well from any situations that would be difficult, and I want the character to be front and center.
So you know, I’m playing myself in a way for that, but at the same time, I’m not. It’s me, but not me. It’s a performance. I think it’s worth noting that it’s still not everything about me. It’s just like - here’s a version of this that I’m performing. Like spoken word, for instance. I used to do spoken word in Covent Garden. Poetry - bad poetry, although I didn’t realize it - but I used to do it for the experience. That was cool and that is an extension of you, like acting, but the difference is there’s no character. You’re doing an extension, a performance of your creativity, so treat it more like that than like me saying “Hi, how are you doing, just going to have a coffee now.” It’s not like that.
Q: So was it weird, seeing such an accurate digital double of yourself?
Newbon:Yeah. I mean, it’s always uncomfortable. I’m not going to lie. I like the fact thatDoublehas this script that’s kind of like a celebration of acting. For me, as an actor, obviously, that’s my passion. I never got used to seeing myself on screen. In film and television, I absolutely detest it. I hate it. I do because I have a show reel I have to look at, but I tend not to watch the stuff I’ve done or read a lot of comments and things. I try not to look at lots of videos and things like that. It’s unhealthy for me. For me, personally, I never liked it. I always felt like it gave me the creeps, actually, to see myself playing a role because I’ve had the experience on set or on stage or in the volume - so then to see myself doing it back is a bit trippy. It messes with my sense of “I’ve already done that.” It’s a different version of how I remember it to be.
Seeing my face animated is better because I can still tell it’s in animation. I think the difference is it looks startlingly like me, so that’s just like: “Wow. Okay. That really, really, reallylooks like me.” But I’ll try - I’ll watch it for sure because I want to see the work and how it turned out. I’m definitely better than I used to be, but it is very close so I probably won’t be watching it many times. Let’s put it that way. I like taking my face off the characters. I also like using my face. I like both, so I would never say that one should only use the likeness of an actor. I think it’s completely legitimate to take your face off and play a different character - as long as it’s appropriate to your ethnic background and casting type. However, I think it’s cool to see yourself in the game as well if you’re right for the role, if your face looks right for the role. You know, I do like the fact that, in games, character actors can become leading performers, and leading performers can become character actors. I think that’s a very important thing to say.
Q: Astarion, on the other hand, would love looking at his digital double because it would be a chance to see himself again - wait, do you think a vampire could see his digital double?
Newbon:I have no idea. It would depend on if you can see a vampire on camera. If you could see him on camera, then maybe? Probably? It depends on your point of view about vampires. It’s difficult to work out.
Newbon Offers Advice To Actors Working With DI4D’s Tech
Q: What advice do you have for actors who will be using the DI4D PURE4D 2.0 technology in future projects?
Newbon:Keep deepening your craft. Keep developing character. Find the subtleties, break your habits, learn the habits of the character - because now, more than ever, all of thisperformance can be seen and captured in animation, which is amazing. Again, I started 15 years ago when it was quite blocky, you know. The faces were kind of cel-shaded, in some ways, or[Makes a face impersonating the awkward, stiff facial animation of the time].
It wasn’t subtle and nuanced so much. Whereas now, you can be, so make sure that your acting craft meets the technology.
Be studying more, throughout your life. I don’t think “two or three years at drama school, then you’re done” is applicable to the craft. I think that’s a foundation, a start, but you’re going to learn from everything through life. Everything through your career will inform you further and deepen the craft further, so ensure your craft work is good and consistent because then you get the opportunity to use it in its entirety.
WithAstarion, everything I ever learned was used in that role. Literally everything. Every tool set, every experience from acting, all went into Astarion, which is why this character andBaldur’s Gate 3mean so much to me. I used everything I ever learned, and it’s just awesome and very cool. That’s my advice to people using this kind of technology. Make sure your craft is consistent, and do as much as you can to create a character. Don’t get lazy. It’s a craft, you know. That’s the point of it. It’s a never-ending search for perfection that you’ll never attain, and it’s not about perfection. It’s not about getting it right. It’s about the experience, the endeavor, of getting it to something magical, you know. You don’t know that until after you’ve done it. There’s no target, in a way. It’s just the process of it. It’s very zen. The process of the craft is the point; whether you hit one target or another is immaterial. You just go through the process, do your work - and whatever happens, that’s where it lands. It’s very cool, I like it a lot.
Q: And you’re constantly improving your craft with new work - you voiced Vecna in a recentD&Dtrailer, is that correct?
Newbon:Yes! I’mVecna! RISE![Evil laughter]
It was really cool. They very sweetly asked me to do a voice for Vecna, and being a D&D nerd, playing Vecna in any capacity is very cool. So yeah, I had a good time. It was great.
Q: Anything else you’d like to tell Game Rant’s readers?
[END]
