Pierre Baldi and Laurent Itti describe a way to go about quantifying surprise. Their approach is to evaluate the Kullback-Liebler (KL) distance between the Bayesian prior and posterior probability distribution of an event. This seems to work very well.
Using this framework we measure the extent to which humans direct their gaze towards surprising items while watching television and video games. We find that subjects are strongly attracted towards surprising locations, with 72 percent of all human gaze shifts directed towards locations more surprising than the average, a figure which rises to 84 percent when considering only gaze targets simultaneously selected by all subjects
I especially like their "Wow" units.
If you recall, in a previous entry, I mentionned the work of Kevin Pelphrey on gaze following deficiency in autism where he mentions:
On congruent trials, subjects watched as a virtual actor looked towards a checkerboard that appeared in her visual field, confirming the subject's expectation regarding what the actor ‘ought to do’ in this context. On incongruent trials, she looked towards empty space, violating the subject's expectation. Consistent with a prior report from our laboratory that used this task in neurologically normal subjects, ‘errors’ (incongruent trials) evoked more activity in the STS and other brain regions linked to social cognition, indicating a strong effect of intention in typically developing subjects (n = 9). The same brain regions were activated during observation of gaze shifts in subjects with autism (n = 10), but did not differentiate congruent and incongruent trials, indicating that activity in these regions was not modulated by the context of the perceived gaze shift.
So somehow, the autistic subjects are not surprised by the incongruent test. Either the STS is not doing the processing of determining what a surprise is or it is paying too much attention and overwhelmed by details.
Indeed, as shown by Edward Vogel, an assistant professor of cognitive neuroscience at the University of Oregon, it looks like awareness, or "visual working memory," depends on one's ability to filter out irrelevant information.
Until now, it's been assumed that people with high capacity visual working memory had greater storage but actually, it's about the bouncer – a neural mechanism that controls what information gets into awareness," Vogel said.
....
Working with two of his graduate students, Andrew McCollough and Maro Machizawa, Vogel recorded brain activity as people performed computer tasks asking them to remember arrays of colored squares or rectangles. In one experiment, researchers told subjects to hold in mind two red rectangles and ignore two blue ones. Without exception, high-capacity individuals excelled at dismissing blue, but low-capacity individuals held all of the rectangles in mind.
I've previously heard of information content used as a measure of surprise, but this is better.
ReplyDeleteRegarding the eye following study, Itti and Baldi's surprise metric specifically discounts being overwhelmed by details. The Bayesian explanation would be that the autistic subjects did not form an apriori expectation that the person would look in a particular direction, and thus were not surprised when they did not.
Paul:
ReplyDeleteYou are absolutely right.
Igor.