Virtual Reality Transforms Artistic Storytelling

The global virtual reality market is projected to reach USD 67.6 billion. This powerful technology is enabling artists to create deep emotional experiences and redefine storytelling. More than 171 million people worldwide are estimated to use virtual reality technology.

Credit: Geoff Coombs

Immersive director Charlotte Mikkelborg foresaw a future where presence itself could be the story when she first used a virtual reality headset a decade ago. Her latest work, "Adventure," is a five-part virtual reality exclusive series for Apple Vision Pro.

Credit: Christian Black

"Adventure" takes audiences inside the lives of extreme athletes, showcasing how technology brings their experiences to life. The series was filmed using a proprietary Apple camera in 8K stereoscopic 3D.

The camera features two lenses at 180 degrees, allowing audiences to view the scene smoothly without motion sickness. Ms. Mikkelborg stated, "We developed precisely what we needed for the show."

Each "Adventure" episode required custom-built rigs and specific designs to capture motion in diverse environments, from mountain climbs to underwater dives. Postproduction in immersive mediums is more complex than in 2D, involving a high level of shot detail and potential visual effects work.

Ms. Mikkelborg explained challenges include stereo disparity and shot choreography considerations. She noted a cut that seems loose on a monitor might be too fast in a headset, causing loss of context and impact.

Apple's Vision Pro format focuses on 180-degree filmmaking rather than 360-degree, which directs the viewer's attention to where it naturally falls. This approach provides greater creative control. Ms. Mikkelborg added, "180 focuses the super high resolution of these cameras and of the Vision Pro in the front half of your field of view which research shows us is where audiences are looking the vast majority of the time."

In 2018, Ms. Mikkelborg was commissioned by British Airways to create "Fly," an immersive location-based experience for their centenary celebrations. She designed a multisensory journey through humankind's relationship with flight.

The experience began with a young Leonardo Da Vinci and progressed 100 years into the future of flight. Ms. Mikkelborg aimed to enable audiences to fly through key narrative moments.

Working with French creative technologist Amaury La Burthe, and Neil Courbould's Academy Award-winning motion control team led by Glen Winchester, she designed a game engine-based narrative experience. This was enjoyed in a virtual reality headset atop a motion platform, with movement synced to the in-headset experience to prevent motion sickness.

Multiple multisensory elements, including scent, temperature changes, and wind, were integrated to fully suspend disbelief. This made the audience feel truly immersed in the time-travelling journey.

XR artist Estella Tse approaches technology from a complementary angle, focusing on its potential for deep emotional experiences. Ms. Tse, who has been in the industry for a decade, still believes in this potential.

She stated, "The immersive nature of VR metaphorically and literally puts the viewer into a different world. The brain feels like it is transported to another place." Ms. Tse considers this powerful for building empathy and felt experience.

In 2019, Ms. Tse collaborated with the Dutch Pavilion at CES to immerse people in Rembrandt's "The Night Watch." She painted all pieces to appear accurate from a front view, mimicking the 2D painting.

This allowed viewers to step into the artwork. Ms. Tse emphasises designing for humans first, aiming for experiences that are easy for a layperson to access.

The technical challenges involve ensuring 3D models, materials, shaders, and textures maintain quality and fidelity when transferred between 3D programmes.

Ms. Tse also highlights the ephemeral and fleeting lifetime of XR pieces, which depends on software companies maintaining and developing their software. "Its survival is truly dependent on whether the software companies decide to keep the software, continue developing on it, or decide to can it altogether," she said.

"This happens all the time," Ms. Tse noted, adding that it makes her work impermanent and subject to the whims of these companies.

The tension between art and infrastructure drives much of her practice. Ms. Tse tries to donate or lend older headsets to those without access to the technology.

She also tries to incorporate nature into her XR creations, bringing organic and humanity into what can be a tech-heavy experience.

Both Ms. Tse and Ms. Mikkelborg see significant opportunities in how XR opens doors for creators. Ms. Mikkelborg observed that women, tired of obstacles in the male-dominated film industry, saw the nascent immersive sector as a chance to forge a more egalitarian space.

As an Asian woman, Ms. Tse was often perceived as submissive. She recounted an experience at a 2015 virtual reality conference.

When Ms. Tse asked a panellist about software for artists entering virtual reality, he suggested Adobe programmes, saying "Unity will be too difficult for you." Other men laughed along, to which Ms. Tse responded, "I've been coding since I was 12."

For Ms. Tse, filmmakers like Ms. Mikkelborg exemplify how XR can merge art and cinema. She expressed appreciation for "any creativity bringing out the humanity in XR," and for how technology and art push each other to evolve.

---

• The global virtual reality market is projected to reach USD 67.6 billion, with over 171 million users worldwide.

• Immersive director Charlotte Mikkelborg's "Adventure" series for Apple Vision Pro uses 8K stereoscopic 3D 180-degree filmmaking to showcase extreme athletes.

• XR artist Estella Tse focuses on virtual reality's potential to build empathy, demonstrated in works like her collaboration on Rembrandt's "The Night Watch."

Source: FORBES