Friday, March 15, 2013

Your Brain on Movies

     How do our brains respond to movies? With the advent of functional magnetic resonance imaging (fMRI), it is now possible to have individuals watch a movie in a scanner and record the brain regions that are active during the experience (something that was considered science fiction fantasy only 20 years ago). Brain circuits have been identified that pertain to various mental processes, such as vision, memory, language, emotion, and decision making. Of course, the whole brain must work together to give us the ability to perform complex operations, such interpreting the plot of a feature film. Mostly likely, when we watch movies we borrow many of the brain mechanisms we use in everyday experiences. Indeed, we often are sucked into a dramatic film almost as if we are in the scenes themselves. Recently, a few brain scientists have explored our movie experience—from the ways movies drive visual processes to ways they evoke brain responses associated with feelings. Neuroscientist Uri Hasson has advocated neurocinematics, which has the specific goal of understanding the neural underpinnings of our movie experience.

     Hasson uses a statistical method called inter-subject correlations (ISC), in which he calculates the degree to which a movie evokes the same or coherent brain responses in a group of subjects. Interestingly, not all movies show the same level of coherent brain activity. In one analysis, a clip from Bang! You're Dead, a TV episodic from Alfred Hitchcock Presents, evoked coherent brain activity in over 65% of the cortex (green region in figure). A clip from Sergio Leone's The Good, Bad, and Ugly elicited coherent activity in 45% of the cortex (blue region). Less mutual activity was observed in a TV episode of Curb Your Enthusiasm (red region) and a single-shot video clip of people walking around Washington Square Park near New York University (orange region)—these two clips offered minimal or no real plotline (Curb Your Enthusiasm is often shot without a script). These findings suggest that some movies engage comparable brain processes in all subjects, whereas others are less successful in doing so.

     Not only can neurocienematic studies offer intriguing findings about our movie experience, they also can help elucidate the neural underpinnings of natural, everyday viewing. Jack Gallant and colleagues showed two hours of movie clips to subjects in a fMRI scanner and created a mathematical model—a kind of dictionary—that translated brain responses in terms of the textures, edges, and motion depicted at any given moment. Once this dictionary was established, the experimenters obtained brain responses to other clips that had not been seen before. They then used the dictionary of prior scans to predict or "decode" what subjects were seeing in these new clips. The image on the left is a frame from one of the test clips (Steve Martin in the 2006 remake of The Pink Panther). The image on the right is a reconstruction of what the model "thinks" the subject is seeing based on the brain activity to the clip. Remarkably, the model does an amazing job of decoding the image purely on the basis of a brain scan!
     Other neuroimaging studies have ventured into conceptual and emotional processes while viewing movies. Gallant and colleagues have also developed models from fMRI data that define conceptual "maps" that can predict what objects people are viewing. The cognitive neuroscientist Jeff Zacks has studied the way we segment events as we watch movies. I and others have looked at brain responses while viewing emotionally laden film clips (see Shimamura et al., 2013). I contend, however, that merely recording brain activity while watching movies is not enough, as it is important to consider the psychological processes that are defined by such neural activity. We cannot fall into a modern-day version of phrenology where bumps on the head are replaced by bright spots on a brain scan. We need to go further and develop theories that describe the functional dynamics of neural activity and how brain regions interact to enable us to see, think, and feel. This is why I prefer the term "psychocinematics" rather than the more restrictive "neurocinematics" to describe the scientific inquiry into our movie experience. In the end, we'll need minds, brains, and even more—sociology, history, anthropology, and other disciplines to gain a thorough understanding of the magic of movies.

Friday, March 1, 2013

Story Arcs and The Rhythm of Movies

     What makes a good story? Aristotle in The Poetics suggested that a well-formed plot should be structured as a complete tale with a beginning, middle, and end. Characters are to be developed through recognition or realization, such as Luke Skywalker realizing that Darth Vader is his father. The screenwriting guru Syd Field articulated a three-act story arc for movies: Act I, the Set-up, lasts about a quarter of a movie and introduces the characters offering insights into their nature, goals, and predicament. Act II, Confrontations, is the longest segment, encompassing the middle half of a movie, which sets the protagonist off ("the game's afoot!") and builds tension through a series of challenges and conflicts. Act III, Resolution, leads to the climax with the last scene serving as a final release of tension. Acts are separated by plot points, which move the action and "spins it around in another direction" (Fields, 2005, pg 26). Since antiquity this storytelling scheme has been successful in capturing our attention and driving our emotions. In Hollywood movies, it has been followed so rigorously that many consider it excessively formulaic (can a Hollywood action movie end without a final chase scene?). 

     The film theorist Kirsten Thompson analyzed the story arc of over a hundred movies and identified a turning point at the midpoint of many movies, which splits Fields' middle act into two segments: "The Complicating Action" and "Development." This turning point is marked by a dramatic change in the direction of the main character's goals and disposition. In Casablanca, Thompson defined the end of her Act II (The Complicating Action) as the scene where Ilsa tries to explain to a drunken Rick why she didn't meet him at the train station in Paris. Rick ends the scene dejected after cruelly dismissing her story ("Was it Lazlo or were there others in between…or aren't you the kind that tells"). The scene ends with a fade to black at halftime (51 min into the 102 min movie). In Act III (Development), Rick must turn his former bitterness into saving Ilsa from a desolate lifetime in Casablanca. With Thompson's four-act structure, each section is of equal length (about 20-30 min) and marked by a prominent turning point. These large-scale descriptions are of course only guidelines, and Thompson has shown that movies may conform to a 3-, 4- or even 5-act structure, often depending on the length of the movie.

     The narrative structure of a good movie is much like a rollercoaster ride which begins with anticipation, creates waves of tension and release, and when we reach the end we are satisfied by a thrilling experience. A movie's rhythmic pacing involves the varying of tempo with stops and starts, questions and answers, and successes and failures. Buck Henry, the screenwriter for The Graduate, said: "If it's just a series of climaxes, you can go crazy. You have to find some way to moderate the tempo so that it's not all one crescendo, or one diminuendo. There have to be changes of pace to give the audience time to stop and start again" (Froug, 1999, pg 209). In this way, the filmmaker flirts with the viewer by concealing and revealing knowledge as the story progresses. There may be a hierarchy of tensions and releases with small-scale predicaments woven into the fabric of larger-scale conflicts and goals.

     Cinemetrics, which pertains to statistical analyses of film features, has been used to capture the rhythmic pacing of films. Analyses by Barry Salt and James Cutting have shown both large-scale and small-scale rhythms or waves defined by clusters of short and long shot lengths. Shown in the graph below is an analysis by Barry Salt  of the shot lengths across the movie Ride Lonesome. Shot lengths varied from 2 to over 20 secs, though it is clear that the ordering of these edits is not random as there are moments of rapid cuts and clusters of shots with long durations. The red line shows a larger-scale averaging of the shot durations which seems to conform to a 3-act segmentation with each act starting with long shot durations (low points) and followed by a cluster (hump) of rapid shots. As the climax approaches, we see a distinct series of very rapid shots with the final shots increasing in duration. James Cutting and colleagues analyzed 150 feature films and showed that "…the sequence of shot lengths in films has a waxing and waning structure…that occurs simultaneously at different scales—that is, fluctuations on the orders of tens of seconds, minutes, tens of minutes, and longer (Cutting et al., 2013, pg 4).

     When we watch a movie, we are not generally conscious of its rhythmic pacing as we tend to get sucked into the story and pay attention to the drama. Yet filmmakers are well aware of the story arc and its rhythmic scheme, though of course not all movies are successful, as much has to "work" with respect to the acting, visuals, editing, and most importantly the storyline. A worthy challenge for psychocinematics is to consider how alterations in the event structure or shot length rhythm influence our attention and emotional engagement.