Apolipoprotein E (ApoE) ε4 genotype is a strong risk factor for developing Alzheimer's disease (AD). Conversely, the presence of the ε2 allele has been shown to mitigate cognitive decline. Tensor-based morphometry (TBM), a novel computational approach for visualizing longitudinal progression of brain atrophy, was used to determine whether cognitively intact elderly participants with the ε4 allele demonstrate greater volume reduction than those with the ε2 allele. Healthy "younger elderly" volunteers, aged 55-75, were recruited from the community and hospital staff. They were evaluated with a baseline and follow-up MRI scan (mean scan interval = 4.72 years, s.d. = 0.55) and completed ApoE genotyping. Twenty-seven participants were included in the study of which 16 had the ε4 allele (all heterozygous ε3ε4 genotype) and 11 had the ε2ε3 genotype. The two groups did not differ significantly on any demographic characteristics and all subjects were cognitively "normal" at both baseline and follow-up time points. TBM was used to create 3D maps of local brain tissue atrophy rates for individual participants; these spatially detailed 3D maps were compared between the two ApoE groups. Regional analyses were performed and the ε4 group demonstrated significantly greater annual atrophy rates in the temporal lobes (p = 0.048) and hippocampus (p = 0.016); greater volume loss was observed in the right hippocampus than the left. TBM appears to be useful in tracking longitudinal progression of brain atrophy in cognitively asymptomatic adults. Possession of the ε4 allele is associated with greater temporal and hippocampal volume reduction well before the onset of cognitive deficits.