Elsevier

NeuroImage

Volume 55, Issue 2, 15 March 2011, Pages 705-712
NeuroImage

fMRI item analysis in a theory of mind task

https://doi.org/10.1016/j.neuroimage.2010.12.040Get rights and content

Abstract

Conventional analyses of functional magnetic resonance imaging (fMRI) data compare the brain's response to stimulus categories (e.g., pictures of faces, stories about beliefs) across participants. In order to infer that effects observed with the specific items (a particular set of pictures or stories) are generalizable to the entire population (all faces, or all stories about beliefs), it is necessary to perform an “item analysis.” Item analyses may also reveal relationships between secondary (non-hypothesized) features of the items and functional activity. Here, we perform an item analysis on a set of stories commonly used for localizing brain regions putatively involved in Theory of Mind (ToM): right and left temporo-parietal junction (RTPJ/LTPJ), precuneus (PC), superior temporal sulcus (STS) and medial prefrontal cortex (MPFC). We address the following questions: Do brain regions that comprise the ToM network respond reliably across items (i.e. different stories about beliefs)? Do these brain regions demonstrate reliable preferences for items within the category? Can we predict any region's response to individual items, by using other features of the stimuli? We find that the ToM network responds reliably to stories about beliefs, generalizing across items as well as subjects. In addition, several regions in the ToM network have reliable preferences for individual items. Linguistic features of the stimuli did not predict these item preferences.

Research Highlights

► Item analysis shows that the ToM network generalizes across subjects and items. ► Several ToM brain regions have reliable preferences for items within condition. ► Linguistic features of the stimuli do not predict item preferences.

Introduction

Consider the following scenario: a researcher wishes to investigate brain regions recruited for Theory of Mind (ToM), i.e. the ability to attribute and reason about the mental states of other individuals. To that end, she creates two sets of stories, one set describing beliefs held by different protagonists and a set of control stories not including beliefs. Using a standard analysis strategy, each brain region's response to belief versus control stories is evaluated for significance, by comparing the average effect size (belief > control) to the variability of the effect across subjects. The researcher concludes that the resulting brain network is recruited more for processing/representing the category of stories about mental states than the category of control stories. This conclusion, however, goes beyond what was explicitly tested. From a standard analysis, she can only conclude that a contrast between these exact stimuli will on average reveal the same brain regions in a different group of subjects. She cannot conclude that these brain regions will reliably be recruited for other (or all) stories about beliefs.

This example illustrates the “fixed-effects fallacy” (Clark, 1973) or the unfounded inference that conclusions about items sampled from a population generalize to the entire item population. Up until the late 1990s, neuroimagers fell victim to this same problem with subject-wise analyses by treating subjects as “fixed” variables. In treating subjects as a “fixed” variable, the variability of an effect across subjects is not taken into account, which is especially problematic if there is substantial variability in the effect size across participants. Consider an instance in which only one out of five subjects shows a very large effect for some condition, and the other subjects show no effect. Averaging the effect across subjects, the entire group will appear to exhibit a medium-sized effect. Therefore, in modern neuroimaging analyses, random-effects analysis is used, which compares an effect to its variability across subjects. This strategy allows researchers to test whether their findings will generalize to the population from which the subjects are sampled (Friston et al., 1999, Holmes and Friston, 1998). In the example above, treating subjects as a random variable would reveal that the apparent medium-sized average effect is not reliable across subjects.

This same issue exists at the item level. Perhaps only one or two of the stories about beliefs recruit a brain region very strongly (due to some theoretically irrelevant feature), while the remaining stories have no effect. Averaging across items, the group of belief stories will appear to recruit this brain region to a moderate degree. In order to make (theoretically more important) generalizations about the category to which an item belongs, one needs to evaluate the effect size relative to the variation of the effect across items, in an item-wise random-effects analysis. Only upon doing so can one validly conclude that the given brain network is reliably recruited for the item category.

Item analysis has yet to become common practice for neuroimagers despite the fact that it has the potential to reveal theoretically relevant distinctions and is known to be feasible. For example, Bedny et al. (2007) tested the hypothesis that distinct brain regions in the frontal and temporal cortex process nouns and verbs (grammatical-class effect). Using subject-wise analysis, they found a differential brain response to verbs versus nouns in the left posterior temporal lobe and left inferior frontal lobe. Conversely, item analysis revealed that only the effect in the temporal lobe was reliable across items, whereas the effect in the frontal lobe was not. This finding suggests that the grammatical-class effect in the frontal region was specific to the nouns and verbs used in the experiment; the effect would not necessarily generalize to other items from the same categories. Therefore, it would be erroneous to conclude that a region within the frontal lobe was specialized for processing verbs versus nouns, despite the significant subject-wise effect. In that case, the item analysis reconciled conflicting findings regarding the role of the prefrontal cortex in processing specific aspects of language (Bedny and Thompson-Schill, 2006, Davis et al., 2004, Shapiro et al., 2006, Tyler et al., 2001, Tyler et al., 2004).

The first goal of the current study was therefore to apply item-wise random effects analyses to an experimental paradigm frequently used to identify brain regions involved in ToM. Following in the tradition of developmental investigations of ToM (Wimmer and Perner, 1983), neuroimaging studies have often used “false belief” stories to test belief reasoning. In these stories, a protagonist performs an action based on a belief that is false (e.g., Maxi believes his chocolate is in the green drawer, but his mother moved it to the blue drawer). Participants reading these stories are thus required to represent the outdated belief of the protagonist in order to understand their actions (e.g., looking in the green drawer even though the chocolate is actually in the blue drawer). These stories are contrasted with “false photograph” stories, which also require the representation of false or outdated content (e.g., an old photograph that no longer accurately depicts the landscape of a burgeoning city). False belief and false photograph stories are therefore matched in their general difficulty, logical complexity, and inhibitory demands, but differ in the need to think about someone's thoughts. Accordingly, a set of stories about false beliefs and false photographs (Saxe and Kanwisher, 2003) is commonly employed across a range of studies to identify brain regions in the so-called “ToM network”: right and left temporo-parietal junction (RTPJ/LTPJ), superior temporal sulcus (STS), precuneus (PC) and medial prefrontal cortex (MPFC) (Kliemann et al., 2008, Mitchell, 2008, Saxe and Powell, 2006, Saxe et al., 2006, Saxe and Wexler, 2005, Scholz et al., 2009, Young et al., 2007, Young et al., 2010a). It is therefore theoretically important to establish that these regions' recruitment generalizes beyond the specific commonly used stimuli. Here, we used an item analysis to formally test whether the brain response to these specific stories about false beliefs can be generalized to the category of such stories.

Item analyses also have a second advantage. An item analysis produces an estimate of the response in each brain region, to each specific stimulus. If a region has a reliable preference for specific items within a category, this preference may provide a clue about the region's function. Every item in an fMRI experiment can be characterized on multiple different dimensions or features (e.g., for stories about beliefs, the number of people or mental states mentioned, the degree of syntactic complexity, the specific context of the story, etc.). It may be possible to determine which item dimensions or features best predict each region's response. These dimensions or features may be confounds, which explain away previous categorical effects, or they may confirm and expand prior results, by allowing a higher-resolution picture of the region's processing, within stimulus categories. One specific concern is that activity in the ToM network is best accounted for by linguistic features of the stories that are concomitant with the presence of belief information. The extent to which these factors account for activity in ToM brain regions can be evaluated within a single paradigm by analyzing data at the item level.

In the current paper, we used item analyses to investigate the ToM network. We ask the following questions: (1) Does item-wise analysis replicate subject-wise analysis? That is, does the response in ToM brain regions generalize across items within each stimulus category (false beliefs and false photographs)? (2) Do ToM brain regions demonstrate a reliable preference for items within each category? And, (3) what features of the items account for activity/stable preference for items in these regions? We characterized the stimuli used by Saxe and colleagues (e.g., Saxe and Powell, 2006, Saxe and Wexler, 2005, Young et al., 2007, Young et al., 2010b) on several dimensions (words per story, number of people per story, Flesch reading ease level, visualizability and several other linguistic aspects), and asked whether any of these features could predict differences between items in the response of ToM brain regions.

Section snippets

Participants

Sixty-two right-handed naïve adults (M age = 22 ± 4 years, range = 18–35, 35 females) participated in the experiment for payment. All participants were native English speakers, had normal or corrected-to-normal vision, and gave written informed consent in accordance with the requirements of the internal review board at MIT. For a portion of the analyses, the participants were split into two independent groups. The two groups did not differ in age or gender (Group 1: n = 32, M age = 22 ± 3, 17 females; Group

Subject- and item-wise whole brain random effects analysis

We first asked whether brain regions that reliably respond to belief information across subjects also respond reliably across items. To that end, we performed whole brain subject- and item-wise analysis on the same data set. Fig. 1 depicts the brain regions significantly more active (FDR corrected, p < .01, k > 10) for the BELIEF versus PHOTO stories across subjects, and across items, in the whole brain (see Table 2 for list of brain regions). All of the brain regions thought to comprise the ToM

Discussion

Inferences about the cognitive function of brain regions based on fMRI data can be significantly strengthened by item analyses, to show that effects of specific items generalize to the entire item category. However, item analyses remain rare, and have not previously been used in social cognitive neuroscience. Our first objective was to determine whether activity in the ToM network generalizes from a commonly used set of false beliefs stories to the category of such stories about false beliefs.

Conclusion

In summary, item analysis provides two important features beyond those of conventional fMRI analyses: They allow the generalization of effects from items employed in a specific experiment to entire categories of items and can provide insight into the more subtle relationships between functional activity and cognitive processes that would normally be obscured by analysis at the category level. In consideration of these ideas, we employed item analysis on a false belief task for ToM brain

Acknowledgments

The authors would like to thank Jacqueline Pigeon, Adrianna Jenkins, Hyowon Gweon and Emile Bruneau for help with data collection, Steven T. Piantadosi for advice on data analysis, and Elizabeth Redcay for comments on the manuscript. This work was supported by a John Merck Scholars Grant, the Ellison Medical Foundation and the Office of Naval Research. Data were collected at the Athinoula A. Martinos Imaging Center at the McGovern Institute for Brain Research, MIT.

References (42)

Cited by (0)

View full text