Kuleshov Effect + We Feel Fine = Lev's all right?

September 10, 2010

Kuleshov Effect + We Feel Fine = Lev’s all right?

Was discussing in the Web Made Movies IRC today with David Humphrey the need to create a new, and vastly different than our last, demo for the upcoming Open Video Conference.

We wanted to create something that presented a dynamically generated video based on who was watching, and that created different meaning from different juxtapositions of shots.

This reminded me of the Kuleshov effect - the famous experiment by Russian film theorist Lev Kuleshov, in which he proved that meaning was created in cinema through montage.

Kuleshov took a shot of a man staring at the camera, which was devoid of emotion. He then made 3 sequences where he spliced a bowl of soup, an attractive woman, and a child in a coffin, followed by a shot of the same man.

When showing the 3 sequences to audiences, he found that despite the expressionless face of the man, they perceived him to be hungry, lustful or sad, depending on which shot was placed in the middle. Thus, he declared, the meaning of individual shots and sequences was changed by whatever preceded or followed them. It was an extremely influential finding.

Seeing as Web Made Movies is about seeing what happens when we mix the traditions of cinema (time) and the traditions of the web (hypertext), Dave and I wanted to experiment with how this experiment would be different in the context of the web.

I though it would be fun to mash this experiment with the ethos of We Feel Fine, a human emotion experiment by Jonathan Harris and Sep Kamvar that presents web visitors with visualizations of the worlds’ emotions by gathering data from searches and blogs.

So I mocked up a wireframe that you can see below, and I’m looking for some feedback on how it could more meaningful riff of these two experiments. In the example below, the user is asked for their twitter account, as well as their location.

The demo would then build a unique sequence of 3 shots that would be tailored to the users emotions. The first image would be a shot of a face, staring similarly to Kuleshov’s actor, which would be either sad, happy or angry. Which emotion appeared on the actors face would be determined by how often the twitter user mentioned those emotions in their twitter stream of the last month (or so). The second image would be pulled from flickr - from a search of an emotion (you can see that interesting results, for instance, can be found be a flickr search of “sad”.

The third image would be the same as the first.

The music would be chosen from a sampling of 10 songs (or so) which would be categorized based on emotion. The chosen song would correspond to whatever the “world” was feeling, which would be transmitted from We Feel Fine’s API.

The background of the page playing the video would be influenced by the weather in the users location - if it was rainy, it would be dark, sunny would be bright, and so on. Potentially we could also throw in some weather effects via something like processing.js.

Finally, the user would be asked to categorize the emotion of the video. This would be added to a third page, which would provide a grouping of the emotions similar to the 9 elements demo (only works in FF 3.6, not FF4). Users can then preview the different videos that were generated for others.

So it doesn’t feel quite right yet, but we’re thinking we might start and see where it gets us. But could really use some feedback on how this experiment could feel more true and more, well, meaningful. The wireframe draft is below. Comments here most appreciated.