History as a Virtual Reality Experience

March 16, 2020

Warning: This article contains a minor spoiler for 1917.

Recently, I saw a fantastic performance by the Iranian-Canadian group Niyaz at University of Washington’s Meany Hall. The show was part of their groundbreaking Fourth Light Project – a collaboration between the group and designer/visual artist Jerome Delapierre. This “immersive, multi-sensory experience” combines live music and dancing with “interactive technology and advanced project/body-mapping techniques that respond to movement in real time.”[1] The audience is transported into a near out-of-body experience through mesmerizing three-dimension visuals, the hypnotic vocals of singer Azam Ali, and a whirling dervish performance by Tanya Evanson. As I watched in an almost trance-like state, I was reminded of different forms of early cinema that attempted to create immersive experiences of their own. Early films from Biograph and Kalem, for example, possess what Nanna Verhoeff describes as a “museal quality”: non-fiction, short-subject films from the nineteen naughts were seen as an excellent tool for “teaching about history, geography, and ethnography,” as the spectator’s awareness of the “pastness” of the subject creates a certain reality effect that transforms into a “museum effect” that generates excitement and satisfies curiosity.[2]

In Niyaz: The Fourth Light Project, the spectator’s awareness of the pastness of the group’s inspiration (medieval Sufi poetry) fuels, in part, a willingness to surrender and be transported into an immersive experience. One could argue that the audio-visual strategies of The Fourth Light Project were anticipated by the museal qualities of early cinema, and we can see these same qualities in recent immersive art experiences such as Alejandro Gonzalez Iñarritu and cinematographer Emmanuel “Chivo” Lubezki’s virtual reality experience Carne y Arena (or Flesh and Sand). In this six-minute virtual reality film, participants are thrust into a group of migrants traveling across the U.S./Mexican border – eventually stopped by U.S. border patrol agents. Departing from earlier virtual reality formats – in which the spectator is seated in front of a concave, half-spherical screen – participants in Carne y Arena are dressed in full-body suits and are free to move around a 360-degree environment untethered. As described by American Cinematographer, Carne y Arena uses a contemporary geo-political issue to bring the viewer back to a pre-Lumiere and Edison period of visual culture, imbued with moral seriousness and a sense of urgency.[3] This approach to turning history into an immersive experience was foreshadowed by Iñarritu and Lubezki’s previous film, The Revenant (2015), through camera technology that echoes the optical strategies and exhibition practices of 19th century panoramas. Moving panoramas, cycloramas, or dioramas in North American or European cities during the 1800s were explicitly designed to address the restrictions posed by static art forms: created to be multi-planar and often employing deep focus. Other recent historical films, such as Dunkirk (Christopher Nolan, 2017) and 1917 (Sam Mendes, 2019), also draw upon the same influence of pre-cinema panoramic forms to present the past as a quasi-virtual reality experience – a new model of conscious cinema that Steven Shaviro characterizes as offering “the spectator decentered free play.”[4]

In The Revenant, images of the vast and the sublime are punctured by gruesome reminders that history, according to Fredric Jameson, is what hurts.[5] Loosely based on the life of early 19th century frontiersman Hugh Glass, the film’s images recall the landscapes of the Hudson River School and other 19th century American landscapists. Influenced by European Romanticism, the Hudson River School envisioned the American landscape in ways that emphasized America as a “spiritual state of mind,” according the Eleanor Jones Harvey, exhibiting a “primal experience in nature” and layered with identifiable metaphors tuned into national moods and anxieties.[6] The Revenant is layered with debates about violence, revenge, and national identity, amplified through augmented sounds of nature and a score from Ryuichi Sakamoto that the composer describes as composed both “vertically and horizontally” and layered with acoustic and electronic sounds.[7]

Dunkirk, by contrast, builds on the influence that the battle panorama had on the development of the cinematic form. Exemplified notably by Paul Philoppoteaux’s Gettysburg Cyclorama, panoramas provided 19th century spectators with a proto-cinematic experience that anticipated the Imax format. This was a form that Walter Benjamin identified, along with magic lantern shows, as arts of projection that lead to the advent of cinema.[8] The evolution of the panoramic form, however, can also be read as a forerunner to virtual reality – with changes to scale, presentation, and exhibition practices attempting to compensate for the restricted vision of other art forms. War panoramas also appear to anticipate what Paul Virilio referred to as “panoramic telemetry” in pictorial form, a salient visual strategy in war cinema that is expanded on in Dunkirk through IMAX photography (about 70% of the film to be exact), a cartographic approach to narrative structure with its three overlapping environments, and a sound designed that is explicitly intended to expand the frame.[9] The film’s “quasi-virtual reality experience” is created through its exaggerated formal elements and an expansion of traditional panoramic vision of combat – combined with the film’s self-awareness of the role that panoramic vision has played in war cinema.[10]

1917 offers a slightly different “quasi-virtual reality experience” through its presentation of World War I as one shot – a stylistic approach that played a major role in marketing the film ahead of its release. When asked how he planned the blocking of his actors, director Sam Mendes responded that he drew upon his experience as a theater director: “I’m not unused to trying to establish rhythm and tempo…without cutting.” The film’s visual design as one take recalls the immersive experience offered by virtual reality and the “decentered free play” found in contemporary war video games, and yet Mendes’ approach that draws on stage directing that harkens back to the same pre-cinema visual culture that ushered in early films imbued with the museal qualities found in Niyaz: The Fourth Light Project and Carne y Arena. At the same time, the logistics of perception in the film draw on visual strategies found in 19th century panorama spectacles, in pursuit of quasi-virtual reality experience similar to what Nolan wanted to achieve with Dunkirk.

In reading the early 19th century responses to panorama paintings by art critics, one can find a recurring desire for a freedom from the constraints of subjectivity that static forms imposed on the viewer. In his Handbook of Aesthetics in 1807, J.A. Eberthardt had four criticisms of panorama paintings. The first three criticisms, as Bernard Comment observes, centered on “faults that hampered a sense of illusion.” First, was the lack of sound accompaniment. Second, the immobility of the figures; and third, the lack of variety in the landscape. These initial three constituted, according to Comment, a piercing of the illusion by the consciousness of the deception. The fourth criticism was a lack of “freedom” and an inability by the spectator to escape what Eberhardt referred to as “spurious chimera.”[11]

“Neither the knowledge that I am nowhere near the vantage point, nor the daylight, nor the contrast with my immediate environment can rouse me from this ghastly dream….One can put an end to the illusion the moment it becomes unpleasant; but the technique is not available to all spectators of the panorama.” -- J.A. Eberthardt (1807) 

What is striking about Eberhardt’s criticisms is that they seem to anticipate virtual reality as a corrective. The first corrective began, however, before Eberhardt’s handbook with the designs for spherical theaters before the panorama emerged as a form of popular spectacle. Foucault notes that circular architecture was so popular during the latter half of the 18th century as “it was the expression of a particular political utopia,” and we can see this in early designs such as Etienne-Louis Boullee’s Cenotaph to Newton from 1784 and Jeremy Bentham’s panopticon design from 1791.[12] Following the designs for the Cenotaph and the Panopticon were Charles Delanglard’s Georama, a spherical theater constructed in Paris in 1826, which the Royal Geographical Society described as an “architecture of immersion” in which the spectator sees “every country of the world in its true dimensions and form, and its correct relative position.” This immersive quality is particularly interesting when one considers that virtual reality places the spectator at the center of a virtual sphere, or at least some portion of a sphere

• Rundblick: A circular gaze along a horizonal axis.

• Überblick: Gaze from above. 

• Aufblick: Gaze from below.

The spherical designs also helped to advance two key notions of panorama optics. The first, rundblick, was a circular gaze across that “embraced the whole horizon in one, or almost one, go.” The term, literally meaning a panoramic view, was integrated into the vernacular of panorama painting through Eberthardt’s writing as well as the work of Scottish panorama painter and exhibitor Henry Ashton Barker, son of the Leister Square rotunda designer Robert Barker. The second notion, uberblick, literally implied a gaze from above, and both concepts, as Barker and others found could not be contained by a single exhibition model. 

Both notions manifested themselves in exhibition and design practices, notably with the diorama invented by Daguerre, that responded in part to Eberhardt’s critique of the panoramic form by combining the “effect of transparency, lighting, and complementary colors” to create a sense of movement and accentuating a sense of rundblick. Another innovation was the elevated viewing platform for panorama exhibitions, as seen here in this wood engraving of Jules Arsene Garnier’s Constantinople panorama in Copenhagen. This practice, also exemplified by the viewing platform at the present-day Gettysburg Cyclorama exhibition, offers the spectator with uberblick, a sense of command over the spectacle through the ability to look downward from an elevated position. Both exhibition forms, combined with the advent of moving panoramas that were sometimes accompanied by live music, placed the spectator in a spherical optical range with a greater command of rundblick and uberblick in pursuit of maximum illusion, according to Bernard Comment.[13] If we can consider VR part of the same “pursuit of maximum illusion” then we can add the concept of “aufblick” – a gaze from below, or gaze upward. Each of these optical concepts informs the visual strategies and staging in Dunkirk, The Revenant, 1917, and Carne y Arena. 

Staging and immersion in The Revenant recall the Deleuze/Guatarri concept of “camera consciousness,” in which a new mode of individuation is generated through a detachment of the gaze. This form of free-floating spectatorship exhibited in Inarritu’s film, informed by video gameplay, “explodes,” as Deleuze puts it, “the static views of the world” – in this case the static views of landscapes familiar through the Hudson River School and American landscapists of the 19th century.[14]  Rundblick, uberblick, and aufblick gazes can be felt throughout The Revenant through a variety of conscious camera movements, first notably with the dolly in and whip pan (using a wide-angle lens), shown here. This technique is used throughout the film to replace over-the-shoulder back and forth cutting – disrupting traditional continuity editing techniques and maintaining an uninterrupted sense of rundblick, uberblick and aufblick are exercised through keeping the conscious camera at a low angle during moments of action allowing for quick pans upward or downward, as seen in this tree sniper moment here or in the iconic cliff plunge sequence.

The cliff plunge, and other sequences in the film, were shot using a motorized “technocrane” with a stabilized head built by Leica Geosystems called a libra head – a very fast and highly adaptable platform. This is a device that recalls the rotoambulator, a camera crane attached to a dolly with three wheels, invented by Bell and Howard for the panoramic tracking shots in Lewis Milestone’s 1930 World War I film All Quiet on the Western Front – a film that Christopher Nolan has cited as an influence on his approach to Dunkirk’s visual strategies.[15]

The Revenant was also shot using an Alexa M camera, providing an ideal color range for shooting in natural light, a filming condition often touted in the film’s marketing campaign. The camera often used wide lica lenes, even for close and medium shots of dialogue between characters, with an anti-flare coat and was constantly kept dry with a hairdryer in between takes. This camera was cut in half for greater mobility, with the other half on the camera assistant’s back connected by a long cable. A signal is sent the to a monitor received by a small tablet computer held by Inarritu on-set – providing ideal conditions to incorporate rundblick, uberblick, and aufblick optics in the staging of sequences. This optical range was also achieved through a tail slating process, in which scenes were slated at the end with an upside down clapper board, as well as with fly swatters: 20 x 12 black canvases, were also used for action moving forward shots, with the camera shooting behind the characters towards a light source. This creates a lighting that is reminiscent of contemporary video game action.

In Dunkirk, contemporary video game action is also evoked through conscious camera movements and figure movement that contribute to the film’s diegetical rundblick construction. Wide angle pursuit of characters and a near 4th wall breaking of tracking with characters, as see here and again in a wide frame, recalls what Jonna Eagle refers to as “strenuous spectatorship,” in which the spectator is provided with a “privileged vantage point on action alongside the fantasy of assault.”[16] This extends to how uberblick and aufblick are employed to push the panoramic form. The film assumes three distinct environments identified early in the film as 1) The Mole, 2) The Sea, and 3) The Air. The film’s widescreen IMAX form serves to merge these environs at varying points, mimicking VR’s roots in spherical theater design and also incorporating into its panoramic vision Warburg’s concept of pathos formula – the aesthetic strategies to provide the spectator with the ability to navigate the intensity of the spectacle from a safe vantage point.[17] Dunkirk is a story that Christopher Nolan describes as ending in “neither surrender nor annihilation,” a quality that leaves the story with an open endedness that contributing to its presentation of history as a near virtual reality experience.[18]  

With Dunkirk, this optical range was achieved not with light and mobile cameras but rather with heavy Imax cameras that provided a constant reminder that, according to cinematographer Hoyt Van Hoytema, this is sensory cinema. The entire film was shot on 65mm film – 70% in 15 perforations per frame IMAX and the other 30 percent in 5 perforations per frame 65mm for dialogue sequences to avoid having to loop the dialogue. The film was also finished photochemically so that the 5 perforation per frame can be blown up to IMAX. This approach, according to Nolan, “allows you to hold shots longer…allowing you to cut slower and let things play out longer,” achieving a similar effect to The Revenant’s tailslating.[19] Also similar to The Revenant was the use of a motorized technocrane, creating shots used to expand the specator’s rundblick perception and also tapping into a familiarity with video game “walk-through” gameplay – again, recalling the techniques of Milestone’s All Quiet on the Western Front. Crucial to generating uberblick and aufblick are the aerial sequences. Van Hoytema developed a snorkel lens attachment that would allow the camera to be in a slightly different position, to put the lens where the pilot’s head would be to provide identifiable points of view, cutting between downward and upward wide frame views to expand the panoramic scope of the frame. This technique was furthered by the use of a Romanian plane called a Yak, similar in size and shape to a Spitfire, providing the ability to put a real actor in a real plane with a camera outside looking at the cockpit, and have the real pilot sitting behind the actor just out of shot.

Save Every Breath: The Dunkirk VR Experience, a 360/VR production developed by Practical Magic. The experience takes place in each of three environments portrayed in the film. “The viewer,” according to creator Matthew Lewis, “is underwater in the English Channel, then pilots an RAF Spitfire, and finally ends up on the beach, as one of the many trapped Allied soldiers.” The production maintained practical effects whenever possible – shooting underwater and on the actual beach in France, for example, and not using 360 degree cameras but rather on Red Epic Dragon 6K resolution cameras that divided the frame into splices, up to 24 per shot. A custom-built lightweight motion control base was built for the beach sequences, and a spatial mix for sound.[20] In looking at the impact of these audio-visual strategies, one can see that Save Every Breath deploys this technology in a form that museal, early cinema anticipated: education and the preservation of historical memory as a sensory experience. 

Concluding with 1917: there is a pivotal moment in the film when Lance Corporal Will Schofield (George MacKay) is trying to navigate his way through the abandoned and burning village of Écouste-Saint-Mein at night – avoiding German snipers and patrols. In this moment (pictured left), the traditional lighting principles of background and foreground staging are reversed, with the background fully illuminated by fire and the foreground carved in shadow. Like The Revenant, the lighting in this scene is reminiscent of contemporary video game action, as an Alexa mini LF camera (a smaller version of the Alexa used on The Revenant) on trinity stabilizer follows Schofield around through a set that had originally been planned using a 3D miniature model; miniature models played an important role in the film’s pre-production, as they were used to determine where shadows would be if lights, especially from fires and flares, were shone from a particular direction. One the one hand, this approach to the film’s cinematography represents a way forward: Roger Deakins noted that the Alexa mini LF is one of the lightest camera he has ever used and that the trinity stabilizer “might be the future.” According to Deakins, “you couldn’t have shot it with traditional cameras,” as not only could they not be powered long enough but also the weight of the camera, changing as the film strip passes from magazine chamber to the other, would shift the balance of the camera, causing the rig to not work properly. On the other hand, the film harkens back to the panoramic traditions that ushered in war film genre conventions during the silent era – the use of deep focus found in 19th century battle panoramas and the principles of rundblick, uberblick, and aufblick informing the film’s sense of point-of-view and subjectivity. This influence can be felt strongly at a moment earlier in the film where Schofield comforts his dying friend Lance Corporal Tom Blake (Dean-Charles Chapman), fatally stabbed by a German pilot who crash-landed into a barn where Schofield and Blake had paused to take shelter. Up until that moment, the two had provided the viewer with a navigation through no-man’s land – the camera following along a meticulously rehearsed track, maintaining deep focus. Like The Revenant and Dunkirk, the viewer’s sense of rundblick, uberblick, and aufblick are determined by the authorial hands of the filmmakers, in contrast to the spectator-as-participant experience found in Carne y Arena; the sounds of planes flying overhead as Blake and Schofield enter no-man’s land earlier are not immediately followed by an upward glance from the camera, despite the viewer anticipating such a move, and puddles of muddy water in craters occasionally are used to tease a partial aufblick through their reflections (something we see again when Schofield arrives at Écouste-Saint-Mein and crosses a river). Yet, as Blake dies in Schofield’s arms, the embers from barn burning behind them cross into the foreground of the frame – registering in the viewer a sense of spatial relations that are, in one part, a continuation of the 19th century panoramic vision through cinema and at the same time closely modeled on the effects of virtual reality on our sense of perception.    

Notes

[1] https://sroartists.com/artists/niyaz/

[2] Verhoeff, Nanna. “Virtual Museums.” The West in Early Cinema. Amsterdam UP, 2006. 

[3] https://ascmag.com/blog/the-film-book/carne-y-arena-vr-masterpiece-innaritu-lubezki

[4] Shaviro 1993, 42

[5] For further reading, see Jameson’s The Political Unconscious Narrative as a Socially Symbolic Act

[6] Harvey, Eleanor Jones. The Civil War and American Art. Yale University Press, 2013, pg. 171. 

[7] https://www.vanityfair.com/culture/2015/12/the-revenant-composer-ryuichi-sakamoto-interview

[8] For further reading, see Steiner, Uwe. Walter Benjamin: An Introduction to His Work and Thought. U of Chicago P, 2010.

[9] For further reading, see Virilio, Paul. War and Cinema: The Logistics of Perception. London: Verson, 1989.

[10] Part of the inspiration for this article was from a paper given by Colleen Glenn at the SCMS conference in Toronto in 2018, in which she described Dunkirk as a “quasi-virtual reality experience.” A week later, I flew to L.A. for another conference and experienced Carne y Arena at LACMA. 

[11] Comment, Bernard. The Painted Panorama. London: Reaktion Books, 1999, pg. 97.

[12] Pg. 139

[13] Pg. 141

[14] For further reading, see Pisters, Patricia. The Matrix of Visual Culture: Working With Deleuze in Film Theory. Stanford UP, 2003. 

[15] https://www.indiewire.com/gallery/dunkirk-christopher-nolan-films-inspired/all-quiet-on-the-western-frontposter1930-universali-v/

[16] Eagle, Jonna. “A Rough Ride: Strenuous Spectatorship and the Early Cinema of Assaults.” Screen. Volume 53, Issue 1, Spring 2012, Pages 18–35.

[17] For further reading, see Trafton, John. The American Civil War and the Hollywood War Film. NY: Palgrave, 2015. 

[18] https://www.npr.org/2017/07/20/538088576/-dunkirk-director-christopher-nolan-we-really-try-to-put-you-on-that-beach

[19] https://www.indiewire.com/2017/07/dunkirk-christopher-nolan-cinematography-hoyte-van-hoytema-1201851502/

[20] https://www.mettle.com/360-inspiration-dunkirk-vr-experience-practical-magic/