Virtual Production Series: The Future of Content

With thanks to: 

Jo Coombes, Project Director at AdGreen

Rachel Stones, Business Development at Epic Games

Geoffrey Goodwin, Senior Director, Studios + Virtual Production at RYOT | Verizon Media

This is a guest post by Anton Christodoulou, Group CTO for award-winning experience design company, Imagination.

Watch the full panel discussion video here.  

The Future of Content

Like many ‘new’ technologies, virtual production is not entirely new, rather a more commercially viable and useful update of older techniques. The game-changing addition we see now comes from the use of video game engines that can generate realtime photorealistic 3D graphics tracked from the camera’s perspective, combined with LED screens displaying the environment, rather than traditional ‘green screens’. Now you can film, and act, in a visible environment – in realtime. Virtual production is accelerating the creative process exponentially, improving not only the visual spectacle but also the ease with which content is made and delivered to the desired audience. Furthermore, free and easily-accessible tools, such as Epic Games’ Unreal Engine and Blender, are democratising access to this groundbreaking technology.

Working in realtime

This whole space can initially appear to be overwhelming, but it shouldn’t be. As with any new technology, there are different starting points to jump into this. The most productised, and what most people know and understand virtual production to mean right now, is something called in-camera VFX, often referred to as Mixed Reality Shooting.

For filmmakers, working in a ‘real’ virtual environment rather than a green screen creates a much more natural space for actors, creates natural reflections on physical objects in the space (such as the sky or a cityscape, on a vehicle or sunglasses) and takes a lot of guesswork out of the filmmaking process. It enables visual effects to be delivered live on set, significantly reducing the amount of time required on post-production. However, you need a lot more pre-production planning, which means creating new pipelines and processes. Furthermore, new techniques and capabilities are becoming more and more accessible and usable all the time, such as photogrammetry, volumetric capture and motion capture. The cost and knowledge gap is closing.

When you look back to how this used to be done, this is already having an enormous impact on the filmmaking industry, and long-form episodic content, like The Mandalorian.

Watch this film by Imagination in collaboration with Epic Games to see exactly how this process works, and the creative possibilities.

For marketers and brands, telling stories across the ever-evolving consumer ecosystem is a challenge. From TV ads, to TikTok or Decentraland, there have never been more channels on which your brand needs to have a voice. An asset created in a realtime engine negates the need to design bespoke content for each of these channels as it can be used across all digital, and physical, media. While reusing digital assets is not impossible today, it’s not easy, and most organisations have several digital versions of one asset. By building a library of assets that can be used anywhere, brands can be free to expand their storytelling, pushing it into new channels and transforming the way they engage with their audiences.

Democratising content creation

The access to telling stories through new technologies has never been greater. The cost of producing content continues to go down, and the quality of tools has skyrocketed. This leads to much more accessible ways to create really high-quality broadcast content, making it more accessible outside of the film industry.

On top of this, the consumer landscape has also drastically altered and the power to create quality content is literally in the hands of billions of people around the world, they, therefore, expect a much higher and more creative output from the ‘professionals’.

When discussing new technology, Geoffrey Goodwin, Senior Creative Director & Head of Studios at Yahoo Inc. says “It’s not just virtual production, I look at a TikTok and I fall off my chair half the time because they’ve launched a new filter or a new post feature that would have cost a hundred thousand pounds to do a few years ago!”

This democratisation of technology is empowering. Used prolifically by game developers for years, Unreal Engine is now being used across all industries, from automotive, architecture and fashion, due to its ability to render high fidelity 3D in realtime for media and entertainment. For example, creating a CG human in the past was the preserve of specialist VFX studios in the film and video game industry, costing hundreds of thousands. Now, using MetaHuman Creator, anyone can create a photorealistic 3D human that’s fully rigged, ready to go, and can be configured any which way you want, straight out of the box. This is game-changing for content creators.

Despite this leap forward in capability, there is still a job to do when it comes to upskilling professionals. Rachel Stones, Business Developer at Epic Games, says “One of the things that they get asked a lot by brands and agencies is, where do I start? To say to anyone, just clear the table, start again, is way too overwhelming. We tend to point them in the direction of a studio who can help them learn as they collaborate on a project together”.

Sustainability and efficiency

Planning as much as you can upfront is also really important for sustainability, if you want to know what your production footprint is going to look like.

Jo Coombes, Project Director at AdGreen, who works with the advertising industry to measure and reduce their emissions, launched a Carbon Calculator in September. Jo explained that one of the key areas they look at is the split of emissions between an agency, a production company and a service company, and what people think of as part of the shoot. Everything that comes together in a studio on the shoot day is usually the smallest part, if there’s a lot of travel involved. Reducing travel has a massive impact on the sector’s overall footprint. According to AdGreen, based on industry data, including BAFTA and Albert, travel accounts for forty percent of the average footprint.

Simply adopting virtual production as a technique is not enough on its own. Jo went on to say “Content creators also need to think about the spaces they’re using and how they’re powered – are they on renewable energy tariffs? What kind of work is being done to create the environment in the first place?”.

Using the carbon calculator, you can record the space element, breaking it down into different areas of production. For example, with suppliers of generators, traditionally it’s always been measured based on hours, when measuring actual fuel consumption is much more accurate, which is more difficult to extract from the supplier. Preparing suppliers to provide the right information when trying to complete the footprint data is important. Using a tool to measure and quantify this compared with if you shot this for real in X country, can make a tangible difference to your footprint.

If we flash forward, when a creative is at concept stage, with an idea in their mind, they will be able to start the journey in a realtime engine and create the narrative right there and then, ultimately taking you all the way through. We’re not there yet, but we will get there.

Now we’re starting to talk about the user experience of data and how that can work going forward. Instead of just looking at binary numbers and spreadsheets, having data drive the visuals in a virtual world makes it a more natural and satisfying experience. A great example of this is Lume, which uses Unity, another realtime game engine, to track and monitor particles and structures at nanoscale, for the scientific community.

Data is also fundamental to sustainability. AdGreen is looking carefully at what metrics we can get, and use. Albert, for example, uses emissions as the metric, for making one hour of broadcast TV. What it doesn’t account for, is the efficiencies you can realise if you run that programme multiple times, dividing the emissions by the amount of time to run.

Efficiency and storytelling 

Stories that live on and have a continuous narrative are very common in advertising. Geoffrey notes that “When you have a commercial that audiences love and it delivers ROI, then you’re ready to do it again”. The Coca-Cola “lorry advert” that comes out every year is a good example. There’s lots you can rework to create something new. Even if you’re only using part of what you’ve created again, you’re still being more efficient than the first time around.

This brings us to the concept of creating realtime characters who can then interact like any normal human can. A couple of years ago, Epic created an interactive version of John Lewis’ Edgar the Dragon in their London Lab using Unreal, an Xsens mocap suit, and a head-mounted iPhone to track facial expressions. Rachel believes that ultimately assets can have a life much further than the cuddly toys. “We will see brands creating 3D characters living outside of that content. In the same way that we look at celebrities and kind of transfix ourselves with what they do and what they wear to go shopping, we will have digital characters that will have lives and narratives that are generated around”.

Wouldn’t it be cool if a customer could walk into John Lewis and meet Edgar in-store?

The recent acceleration of remote working has massively opened up the talent pool and presented the opportunity to collaborate and co-create with a more diverse mix of individuals from across the globe. A remote writing room ten years ago would have been unacceptable. You would have to be based in LA or London. Now it’s no longer deemed necessary or productive to be in one location to create mind-blowing productions. This new way of working has the opportunity to unearth hidden voices and richly impact storytelling across the board.

An area we’ll be exploring further in this series is how virtual production is also driving digital transformation, with realtime technology fundamentally changing the way businesses will operate at a functional and operational level, making them more efficient, sustainable, and creative. At Imagination, instead of building physical scale models, or printing out 3D renders, we are building 3D digital models in realtime, sharing the creative on-screen, or putting people in headsets, so they can look around, walk around and even interact with the experience. This may end up as linear content, a physical/digital experience, or purely physical, such as a building structure, or installation.

What is next for virtual production?

We are still just at the beginning of what is possible with this groundbreaking technology. Things will start to get really interesting when technology is no longer a barrier to creativity. As Rachel says “Once we get past the knowledge threshold, to the point of being comfortable, then it’s all about what else can you do?”. One thing that has already been happening, is the combination of virtual production with other technological innovations. Geoffrey, for instance, is incredibly excited to “See where volumetric capture technology can play a role in driving new types of storytelling within the space”.

Besides the creative possibilities this technology offers, there is a huge moral driver we think will continue to contribute to the adoption of virtual production. With COP26 recently making headlines, sustainability is firmly on the agenda and virtual production’s sustainability credentials will only make it a more popular choice for content creators. As Jo says “It’s exciting to see people emailing us to say that they are thinking about using virtual production to reduce their carbon footprint. How do we record it? How is it better? Tell us how it works from this point of view? We can see this seeping into people’s consciousness in a way that we haven’t seen before”.

The next session in the series will look at the future of live experiences, and then finally, we will dive into how organisations will be transformed using realtime, which will both be in early 2022.