How
do our brains respond to movies? With the advent of functional magnetic resonance imaging (fMRI), it is now possible to have individuals watch a movie in a
scanner and record the brain regions that are active during the experience
(something that was considered science fiction fantasy only 20 years ago). Brain
circuits have been identified that pertain to various mental processes, such as
vision, memory, language, emotion, and decision making. Of course,
the whole brain must work together to give us the ability to perform complex
operations, such interpreting the plot of a feature film. Mostly likely, when
we watch movies we borrow many of the brain mechanisms we use in
everyday experiences. Indeed, we often are sucked into a dramatic film
almost as if we are in the scenes themselves. Recently, a few brain scientists
have explored our movie experience—from the ways movies drive visual processes
to ways they evoke brain responses associated with feelings. Neuroscientist Uri Hasson has advocated neurocinematics, which has the specific goal of understanding the
neural underpinnings of our movie experience.
Hasson
uses a statistical method called inter-subject correlations (ISC), in which he
calculates the degree to which a movie evokes the same or coherent brain
responses in a group of subjects. Interestingly, not all movies show the same level
of coherent brain activity. In one analysis, a clip from Bang! You're Dead, a TV episodic from Alfred Hitchcock Presents, evoked
coherent brain activity in over 65% of the cortex (green region in figure). A
clip from Sergio Leone's The Good, Bad,
and Ugly elicited coherent activity in 45% of the cortex (blue region).
Less mutual activity was observed in a TV episode of Curb Your Enthusiasm (red region) and a single-shot video clip of
people walking around Washington Square Park near New York University (orange
region)—these two clips offered minimal or no real plotline (Curb Your Enthusiasm is often shot
without a script). These findings suggest that some movies engage comparable brain
processes in all subjects, whereas others are less successful in doing so.
Not
only can neurocienematic studies offer intriguing findings about our movie experience,
they also can help elucidate the neural underpinnings of natural, everyday
viewing. Jack Gallant and colleagues showed
two hours of movie clips to subjects in a fMRI scanner and created a
mathematical model—a kind of dictionary—that translated brain responses in
terms of the textures, edges, and motion depicted at any given moment. Once
this dictionary was established, the experimenters obtained brain responses to other clips that had not been seen before. They then used the dictionary of prior scans to predict or "decode" what subjects were seeing in these new clips. The image on the left is a frame from
one of the test clips (Steve Martin in the 2006 remake of The Pink Panther). The image on the right is a reconstruction of
what the model "thinks" the subject is seeing based on the brain
activity to the clip. Remarkably, the model does an amazing job of decoding
the image purely on the basis of a brain scan!
Other
neuroimaging studies have ventured into conceptual and emotional processes
while viewing movies. Gallant
and colleagues have also developed models from fMRI data that define conceptual
"maps" that can predict what objects people are viewing. The
cognitive neuroscientist Jeff
Zacks has studied the way we segment events as we watch movies. I and
others have looked at brain responses while viewing emotionally laden film
clips (see Shimamura
et al., 2013). I contend, however, that merely recording brain activity while
watching movies is not enough, as it is important to consider the psychological
processes that are defined by such neural activity. We cannot fall into a
modern-day version of phrenology where bumps on the head are replaced by bright
spots on a brain scan. We need to go further and develop theories that describe the
functional dynamics of neural activity and how brain regions interact to enable
us to see, think, and feel. This is why I prefer the term
"psychocinematics" rather than the more restrictive
"neurocinematics" to describe the scientific inquiry into our movie
experience. In the end, we'll need minds, brains, and even more—sociology,
history, anthropology, and other disciplines to gain a thorough understanding of
the magic of movies.