WHEN YOUR EYES AND EARS ARE OUT OF SYNC
By: Jim Kloet
The Hobbit was one of my favorite books as a kid, so I was really excited last year when the new movie version was released. I was a bit surprised when I went to buy tickets, though: there were six different versions of the film playing at the theater in Evanston! I could see it in standard high definition or two different kinds of 3D; there were also two different frame rates for each type of picture. The standard frame rate for movies is 24 frames per second (fps), which means that we see the equivalent of 24 still images for every second of movie. The Hobbit was filmed with special cameras that also allowed a 48 fps version to be created, which means our brains get double the amount of visual information packed into every second of the film.
I saw the 2D version at 48 fps, and it was so weird. Everything looked out of sync to me – if I didn’t focus intently on the movements of the characters’ mouths when they spoke, I kept getting the impression that the sound was just slightly slower than what we saw. Frankly, it ruined a lot of the movie for me, and I left the theater feeling like I just wasted three hours of my life. It was a bummer.
Recently, a case study was published about a man who always feels like what he hears is out of sync with what he sees. Specifically, the man, who is referred to by his initials PH, hears what people say slightly before he sees their lips move. This happens to him when he’s with people face to face, as well as when he’s watching television. PH actually first noticed this was happening when he was watching TV at his daughter’s house – he thought her television was malfunctioning, then noticed the same thing on a TV in another room, and then he finally experienced the same out-of-sync feeling when he was talking to his daughter.
Doctors suspect that PH’s deficit, which is called acquired desynchronization, was caused by complications from heart surgery that left him with two lesions in deep parts of his brain that are responsible for processing visual and auditory inputs (i.e. things we see and hear). Only one other case of someone acquiring problems with audio-visual synchronization after a brain injury has ever been documented. Patient AWF one day experienced numbness on the left side of his body, and when the numbness went away, he had trouble understanding things if he saw someone speaking, but could understand just fine if he was just listening without looking. AWF never experienced the same kind of “voice leading the lips” desynchronization that PH reported, however.
Cases like PH and AWF are great examples of how much work our brains have to do to make sense of all the information constantly bombarding our senses. If one little piece of the brain system involved in sorting and synchronizing that information isn’t working properly, we can experience all kinds of peculiar symptoms. I haven’t seen The Hobbit again since my first experience, but I suspect I would get a lot more enjoyment out of the standard 24 fps version than I did from the sped-up 48 fps. And I definitely know what version I’ll see when the second part comes out this winter!
Photo credit: NASA Goddard Photo and Video/FLICKR