Tuesday, January 15, 2013

HFR and The Hobbit: There and Back Again

This entry appears on the Oxford University Press blog site: http://blog.oup.com 

     Is it the sense of experiencing reality that makes movies so compelling? Technological advances in film, such as sound, color, widescreen, 3-D, and now high frame rate (HFR), have offered ever increasing semblances of realism on the screen. In The Hobbit: An Unexpected Journey, we are introduced to the world of 48 frames per second (fps), which presents much sharper moving images than what we've seen in movies produced at the standard 24 fps. Yet many viewers, including myself, have come away with a less-than-satisfying experience as the sharp rendering of the characters portrayed is reminiscent of either old videotaped TV programs (soap operas, BBC productions) or recent CGI video games. What features of HFR create this new sensory experience and why does it appear so unsettlingly similar to the experience of watching a low budget TV program?

      One factor that can be ruled out is the potential difference in flicker rate. Moving images are of course created by the rapid succession of still frames, and thus the flicker or on-and-off rate must be fast enough so that we do not perceive any change in illumination between frames. With early silent films, the flicker rate was less than 16 fps, and a noticeable flashing or flickering was apparent (hence the term "flicks" to refer to these early movies). Since the advent of sound, the standard has been 24 fps, though the flicker rate is increased with the use of a propeller-like shutter that spins rapidly in a movie projector so that a movie running at 24 fps actually presents each frame two or three times, thereby increasing the flicker rate to 48 or 72 fps. Thus, with respect to flicker rate we have always watched movies at HFR.
         Two factors have motivated the current interest in HFR. The obvious one is that actions recorded at more rapid frame rates, such as a car chase shot at 48 fps vs 24 fps, would reduce by half the distance objects move across successive frames. With HFR we are presented shorter increments of movement, and our brains need not work as hard to extrapolate apparent motion across frames, which may result in a smoother sense of motion. I, however, do not think that it is this between-frame difference that is driving our sensory experience as we watch The Hobbit. A second, less known factor, is that the movie was shot at a faster shutter speed than movies shot at 24 fps. Filmmakers have a rule that states that the shutter speed at which each frame is shot should be half as long as the frame duration. Thus, most movies we've seen have been shot at 24 fps with a shutter speed of 1/48 sec for each frame. Those of you who have played with photography know that this shutter speed would produce rather blurry images when the camera is hand held. On a tripod, a movie filmed with this shutter speed would show fast moving objects (e.g., cars) with a noticeable blur. When movies filmed at 24 fps are shot with a faster shutter speed and less motion blur, actions appear jerky and unnatural.
         The Hobbit was filmed with a shutter speed of 1/64 sec, which produced less motion blur and thus sharper images compared to movies shot at 24 fps. At the faster frame rate, the jerkiness associated with presenting sharp images at 24 fps is largely reduced, though I did notice that on some occasions large camera movements and fast movements of actors appeared stilted and unnatural. A psychological study by Kuroki and colleagues showed that in order to perceive naturalistic movements with sharp moving images (i.e., no motion blur) it is necessary to use frame rates of 250 fps or faster. Interestingly, the shutter speed used for The Hobbit closely matches that used for old videotaped TV programs, which were filmed at 30 fps with a shutter speed of 1/60 sec. I suspect that it is this close match in shutter speed (and thus similarity in image sharpness) that creates the impression of viewing a soap opera when we watch Bilbo Baggins and company.
         In the future, after years of experiencing HFR movies, will we be able to appreciate the more realistic renderings garnered by this new technology? Will a younger generation without prior associations to videotaped TV programs be enamored by the sharper images? Time will tell, though I'm skeptical. HFR does offer a more realistic rendering than what we've previously encountered at the movies, and further advances may help to refine its use. Yet do we really want to have an entirely realistic portrayal? In most cases that would mean having the experience of sitting next to the director watching actors on a sound stage with artificial lighting, which is exactly the impression I had while watching Bilbo backlit by what was supposed to be moonlight. Instead, we may end up preferring a softer image which maintains the illusion of being engaged in an adventure with our favorite fictional characters and partaking in a wonderfully unexpected journey.

 Watch this video for details on motion blur and shutter speed:

Tuesday, January 1, 2013

Experiencing Movies: Why We Watch Them and Why We Like Them

This entry marks the official opening of the blog, which is devoted to the psychological (and biological) underpinnings of our movie experience or what I call psychocinematics. What follows is an intro to the blog partially excerpted from my chapter in Psychocinematics: Exploring Cognition at the Movies:

 In the opening scene of The Graduate, Benjamin Braddock (Dustin Hoffman) exits a plane and enters the terminal at the LA International Airport. When he steps onto a moving walkway, the camera begins to track alongside at the same pace, which keeps Benjamin's position fixed and isolated to the far right of an otherwise empty screen. The opening credits begin to fill the space as we listen to The Sounds of Silence, Simon and Garfunkel's anthem to social alienation. This beginning anticipates the entire movie which tells a story of a newly minted college graduate entering adulthood without any sense of purpose or direction. Benjamin is moving though he doesn’t appear to be going anywhere. 

 Through acting, staging, sound, camera movements, and editing, movies develop a narrative plot that fully engages us. How do filmmakers maintain our attention so that the drama keeps us riveted to the screen? How do editing techniques link events in an often seamless manner? How do movies drive our emotions, instilling suspense, laughter, horror, sadness and surprise along the way? I believe that filmmakers are like magicians who through sleight of mind can manipulate our attention and emotions and create the illusion of being enveloped in a scene. Over the decades, filmmakers have developed techniques that enhance this movie magic, though we have very little understanding of how these techniques actually work. 

 When it comes to the aesthetics of movies, my first response is: "It's the story, stupid!" For me, if I don't find the plot interesting or identify with the characters portrayed, my interest soon diminishes. Of course, how a filmmaker engages us into the plot is determined by a variety of factors: the acting may be superb, the visuals stunning, the editing seamless or interestingly quirky, the drama riveting, or the suspense overwhelming. As a scientist, I try to analyze these rather subjective features by developing a framework for a research program. Toward this end, I've come up with a simple scheme that captures the psychological features of our movie experience, which I call the I-SKE model. The name is an acronym for what I believe are four essential components of our aesthetic response to movies: the intention (I) of the filmmaker and three psychological components of the viewer, sensation, knowledge, and emotion (SKE). 

 I initially developed the I-SKE model to describe our aesthetic response to visual art (e.g., paintings, photography). In my upcoming book, Experiencing Art: In the Brain of the BeholderI cover the I-SKE model in detail. With respect to movies, filmmakers can play on our audiovisual experience (sensations), make us think about the world in new ways (knowledge), and of course instill a variety of feelings and empathetic responses (emotion). According to the I-SKE model, when these three components are at their heights we experience that "wow" feeling or say as the movie credits roll, "that was fantastic!" These components may also interact and enhance each other. 

 My goal for this blog is to consider the ways in which movies impinge on our sensations, knowledge, and emotions, and as a result drive our interest and enjoyment. I will discuss scientific findings that help us understand how movies work. Although we readily consider sensations and emotions as part of our aesthetic response to an artwork, we don't always think that knowledge plays an important role. Yet our personal and cultural background is always in play, and I hope to pay special attention to the influence of prior knowledge on our movie experience. For example, if you have seen The Hobbit: An Unexpected Journey in HFR (high frame rate), you may have, like me, been reminded of watching a cheesy soap opera or old BBC TV program. Why does the sensory experience of HFR give this impression of the past? 

See the next blog entry...