Monday, April 15, 2013

Kuleshov Revisted: Emotional Reactions Across Time

Among aficionados of early silent films, Lev Kuleshov, the Soviet film director, is known for his use of editing to evoke different emotions. He took shots of an actor posing with a neutral expression and placed them after various scenes, such as one of a bowl of soup, a child's coffin, or a seductively dressed woman. Depending on the preceding shot, the actor's "neutral" expression seemed to change—from one of hunger to sadness and then to lust (see YouTube video).

Facial expressions offer potent displays of emotions, though the "Kuleshov effect" shows that expressions can be ambiguous and thus depend upon the context or situation. Following a close-up shot of a person crying a zoom out could show a funeral scene, though it could just as likely reveal a wedding procession. Even in the same setting, identical expressions may be perceived differently. Check out the clever drawing by psychologist Roger Shepard which shows a perceptual illusion--the two monsters are physically the same size though in the context of the tunnel, linear perspective makes our brains interpret the background monster to appear much larger than the one in the foreground. There is also an emotional illusion as the facial expressions are identical, yet the monster in the foreground appears scared, whereas the one in the background appears angry. 

In a recent set of psychological experiments, Diane Marian and I played on a kind of Kuleshov effect by presenting short film clips of facial expressions that changed over a few seconds. Let's say you are at a gathering and you look up from a conversation and see an unfamiliar face turn and smile at you. Your emotions would be buoyed. Now consider that expression in reverse—you look up and see someone smiling at you, but then the person turns and the expression fades to a blank stare. In our experiment, we showed a variety of short clips in which expressions moved from neutral to happy, happy to neutral, neutral to angry, and angry to neutral. A neutral face that had just been smiling at you actually looks rather grumpy (see video).

Conversely, when an angry face moves to a neutral expression, the person at the end appears to have a rather pleasant disposition.

These effects have the appearance of a perceptual illusion as the very same "blank" expression actually looks different (grumpy or pleasant) depending on whether it started as a happy or angry expression. In fact, we never interpret facial expressions as static, momentary images. In everyday experiences, facial expressions move within a rich contextual environment as they track and signal social interchanges. Of course, movies play on a character's facial expressions as the plot thickens. Jack Nicholson's evolution into madness in The Shining comes immediately to mind. These influences reinforce the idea that much of our emotional involvement at the movies is an unfolding or dynamic experience that occurs as much in the brain as on the screen.