Friday, March 15, 2013

Your Brain on Movies

     How do our brains respond to movies? With the advent of functional magnetic resonance imaging (fMRI), it is now possible to have individuals watch a movie in a scanner and record the brain regions that are active during the experience (something that was considered science fiction fantasy only 20 years ago). Brain circuits have been identified that pertain to various mental processes, such as vision, memory, language, emotion, and decision making. Of course, the whole brain must work together to give us the ability to perform complex operations, such interpreting the plot of a feature film. Mostly likely, when we watch movies we borrow many of the brain mechanisms we use in everyday experiences. Indeed, we often are sucked into a dramatic film almost as if we are in the scenes themselves. Recently, a few brain scientists have explored our movie experience—from the ways movies drive visual processes to ways they evoke brain responses associated with feelings. Neuroscientist Uri Hasson has advocated neurocinematics, which has the specific goal of understanding the neural underpinnings of our movie experience.

     Hasson uses a statistical method called inter-subject correlations (ISC), in which he calculates the degree to which a movie evokes the same or coherent brain responses in a group of subjects. Interestingly, not all movies show the same level of coherent brain activity. In one analysis, a clip from Bang! You're Dead, a TV episodic from Alfred Hitchcock Presents, evoked coherent brain activity in over 65% of the cortex (green region in figure). A clip from Sergio Leone's The Good, Bad, and Ugly elicited coherent activity in 45% of the cortex (blue region). Less mutual activity was observed in a TV episode of Curb Your Enthusiasm (red region) and a single-shot video clip of people walking around Washington Square Park near New York University (orange region)—these two clips offered minimal or no real plotline (Curb Your Enthusiasm is often shot without a script). These findings suggest that some movies engage comparable brain processes in all subjects, whereas others are less successful in doing so.

     Not only can neurocienematic studies offer intriguing findings about our movie experience, they also can help elucidate the neural underpinnings of natural, everyday viewing. Jack Gallant and colleagues showed two hours of movie clips to subjects in a fMRI scanner and created a mathematical model—a kind of dictionary—that translated brain responses in terms of the textures, edges, and motion depicted at any given moment. Once this dictionary was established, the experimenters obtained brain responses to other clips that had not been seen before. They then used the dictionary of prior scans to predict or "decode" what subjects were seeing in these new clips. The image on the left is a frame from one of the test clips (Steve Martin in the 2006 remake of The Pink Panther). The image on the right is a reconstruction of what the model "thinks" the subject is seeing based on the brain activity to the clip. Remarkably, the model does an amazing job of decoding the image purely on the basis of a brain scan!
     Other neuroimaging studies have ventured into conceptual and emotional processes while viewing movies. Gallant and colleagues have also developed models from fMRI data that define conceptual "maps" that can predict what objects people are viewing. The cognitive neuroscientist Jeff Zacks has studied the way we segment events as we watch movies. I and others have looked at brain responses while viewing emotionally laden film clips (see Shimamura et al., 2013). I contend, however, that merely recording brain activity while watching movies is not enough, as it is important to consider the psychological processes that are defined by such neural activity. We cannot fall into a modern-day version of phrenology where bumps on the head are replaced by bright spots on a brain scan. We need to go further and develop theories that describe the functional dynamics of neural activity and how brain regions interact to enable us to see, think, and feel. This is why I prefer the term "psychocinematics" rather than the more restrictive "neurocinematics" to describe the scientific inquiry into our movie experience. In the end, we'll need minds, brains, and even more—sociology, history, anthropology, and other disciplines to gain a thorough understanding of the magic of movies.