The discovery of spontaneous neuronal replacement in the adult vertebrate brain has changed the way in which we think about the biology of memory. This is because neuronal replacement is likely to have an impact on what a brain remembers and what it learns. Neuronal replacement has also changed the way in which we go about exploring new strategies for brain repair. Our new outlook on both these matters is all the more remarkable because of the pervasiveness of the earlier dogma, which for warm-blooded vertebrates relegated neurogenesis to embryonic development and, for a few neuronal classes, early postnatal life. The discovery of constant neuronal replacement in the adult brain was remarkable, too, in that it was not required by what we thought to be the logic of nervous system function. Moreover, no previous facts prepared us for it. Much of the modern theory of learning embraced the view of modifiable synapses as the key players in learning and as the repositories of memory. But if this were so, what would be the point of neuronal replacement in healthy brain tissue? In what follows, I will briefly review the work of Joseph Altman, because he was the first one to challenge the notion that new neurons were not produced in adulthood. I will then review what we know about neuronal replacement in the song system of birds, which my laboratory has studied for many years. In closing, I will offer a general theory of long-term memory that, if true, might explain why adult nervous systems constantly replace some of their neurons.