The DIADEM and Beyond

Among the many challenges in understanding the functions of healthy and diseased brain is our limited ability to thoroughly analyze its structure and activity at neuronal, circuit and network levels. Despite the advent of computer technology, neuronal reconstructions are still largely performed by hand. Reconstructing a single cell may take months, presenting a bottleneck to progress. To meet this challenge, the DIADEM (Digital Reconstruction of Axonal and Dendritic Morphology) Challenge provided incentives for the development of computer algorithms to support accurate high-speed, automated three-dimensional (3D) reconstruction of neuronal projections. Together with the sponsors of the DIADEM Challenge,¹ several Institutes of the National Institutes of Health (NIH)² co-sponsored the DIADEM Scientific Conference in conjunction with the final phase of the DIADEM Challenge.

The conference brought together computational scientists and experimental neuroscientists to evaluate the advances made by the finalist teams, to summarize state-of-the-art technologies, and to discuss the remaining challenges and future research directions. Over sixty scientists attended the conference, including many postdoctoral fellows and graduate students. The conference opened with remarks by the sponsors,³ which was followed by an overview of “The Future of Connectomics” by keynote speaker Dr. Jeff Lichtman of Harvard University.

The original articles, mini-reviews, and commentaries published in this special issue of Neuroinformatics address central themes of the DIADEM Challenge. This editorial provides a brief summary of the Challenge and the Conference. Additional DIADEM legacy material can be found at the DIADEM Challenge website coordinated and maintained by Dr. Giorgio Ascoli’s laboratory (http://diademchallenge.org).⁴

Historical Perspectives

The 12th century French philosopher Bernard of Chartres noted the “modern dwarfs” of his era could see more than the Greek and Roman scholars because they stood on the shoulders of those “earlier giants.” Several authors in this issue provide historical perspectives of the development of modern neuromorphology, which is one of the oldest branches of neurosciences. Svoboda stresses the revolutionary principles of the “Neuron Doctrine” and the “Dynamic Polarization” from Cajal and his peers.⁵ Senft traces the technical development from as early as the 16th century, reminding us of the numerous contributions made by early pioneers who have been largely forgotten.⁶ Martin fast-forwards from the “Cajal era” and summarizes both the technical and conceptual advances made to date. He emphasizes the conceptual frameworks from “early giants”, such as Sherrington, Hodgkin and Huxley, and Rall, which are still valid in the 21st century.⁷

The 2010 DIADEM

Six research groups contributed data for the DIADEM challenge, which was curated by Ascoli and colleagues. In their mini-review, the data providers describe the rich and diverse data sets, which represent different labeling methods (fluorescent or non-fluorescent, conventional or molecular genetics), imaging technologies (bright-field, confocal and two–photon microscopy) and histological preparations, as well as many different tissue origins (in vivo, in vitro, neuronal subtypes, brain regions and animal species).⁸ The Ascoli team also developed the DIADEM Metrics, which enables quantitative comparison between different reconstructions of the same data set.⁹ Using these metrics, three criteria were applied when evaluating the auto-reconstruction: 1) the accuracy (in comparison with the Gold Standard—manual reconstruction conducted on the same data sets); 2) the speed (20 times faster than manual reconstruction); and 3) the broad applicability of the reconstruction algorithms to the diverse data sets.

A team consisting of the data providers and several outside experts judged the performance of each computational solution for 3D auto-reconstruction, developed independently by the five finalist teams. Although there were commonalities among the computational approaches, each team used different strategies to meet the DIADEM Challenge.

• The Roysam team discussed the computational toolkit they developed to manage and integrate the three critical steps in auto-reconstruction: enhancing image quality by pre-processing algorithms, generalizing neuronal tracing by an “open-curve snake” approach, and correcting automated tracing errors by trace-editing software.^{10 , 11 , 12}

• The Stepanyants team constructed a “Neural Circuit Tracer” software package to reconstruct sparsely labeled neurons. They used a segmentation-based, multiple-step global strategy to tackle problems such as connecting “broken” branches and separating “fused” fibers.¹³

• The Myers team demonstrated how to improve performance by using a combination of a local tracing approach—modified with the “shortest path” strategy, and a model-based tracing method—a tube model designed for global skeletonization.¹⁴

• The González team established a novel probabilistic machine learning approach, which optimized the global objective function by taking into account the geometric properties of the image data. While fitting candidate points along the neurites into a set of optimized tree structure, they tried to minimize false positive errors.¹⁵

• The Erdogmus team adopted a “recursive tracing” method that is built upon the statistical machine learning “principal curve” concept. They deployed a smooth interpolation technique for density estimation, as well as a distributed spectral-screening algorithm to trace recursively the ridge of the density.¹⁶

Beyond the 2010 DIADEM

The final session of the DIADEM Conference discussed remaining challenges in meeting the original competition objectives and future directions for automated structural analysis.

• 1. The Remaining Challenges and Potential Solutions

The majority of auto-tracing algorithms developed by the finalist teams performed reasonably well against the Gold Standard of accuracy. However, the need for time-consuming human intervention to correct errors compromised the speed gained by auto-tracing. In their commentary, Peng and colleagues point out that automatic proof-editing is essential for achieving lower error-rate, less human intervention and higher speed.¹⁷ Kozloski, in his mini-review, further clarifies the problem—auto-tracing errors due to uncertainty accumulate nonlinearly, which reduces both speed and accuracy.¹⁸ Both authors made suggestions for better tracing and proof-editing tools. Another obvious way to enhance accuracy and speed of reconstruction is to increase imaging data quality either by improving current imaging methods or by applying newly developed “super-resolution” technologies.¹⁹ Improvement of computational preprocessing will also enhance signal-to-noise ratio, thereby “sharpening” the images.²⁰ (see footnote 11)

Regarding the applicability to diverse data sets, some teams’ algorithms handled most of the data sets reasonably, but none were able to manage all data sets adequately. Many conference participants believe the distinctive characteristics of each of the diverse data sets may not allow a “one-size-fits-all” computational solution. One tactic is to use a global approach to merge neurites from sequential segmentations, and only merge branches into tree structures when the confidence of merging is high (see footnote 13). Another strategy is to improve computational toolkits that manage and integrate a suite of flexible algorithms tailored to meet diverse needs (see footnote 11). Other potential solutions to reduce variability include expanding “imaging pipelines” that optimize and standardize staining methods and imaging protocols (e.g., similar to those developed at Janelia Farm) to the neuroscience research community.

• 2. Future Directions

As outlined by the other two co-organizers,²¹ “the intent is for DIADEM to function as a starting point rather than a final result” (see footnote 8), and the DIADEM is “ringing a new era in the study of neuronal structure and neural circuits” (see footnote 5). Many participants at the conference made recommendations for future research directions beyond DIADEM, which are reflected in this issue.

3D to 4D

Time-lapse 3 dimensional imaging (that is, 4D imaging) in animal models provides a powerful tool to study development, plasticity, degeneration and regeneration of the nervous system. More importantly, it can be used for therapeutic development to test the effects of potential treatments. Although technical obstacles remain (e.g., image alignment over long imaging intervals, identification of new structures), new advances in the field will make the 4D semi-auto reconstruction possible.²² One of the DIADEM “products”—the DIADEM Metric, was designed to compare reconstructions of the same data sets, which with modification, can serve as a measurement for 4D reconstruction (see footnote 9).

Cellular to Subcellular

Several participants recommended tracking subcellular structures, including synaptic sites, synaptic proteins and organelles, such as mitochondria²³ (see footnote 6). This would require the development of specific markers for subcellular structures in combination with the application of the nascent “super-resolution” optical imaging techniques.

Sparse to Dense

All the DIADEM data sets, except one, are sparsely stained neuronal projections, which can be traced by using 3D rendering strategies. For densely-packed neurites, such as motor neuron fibers labeled by the innovative Brainbow technique, the tracing is much more technically demanding. Lu and his colleagues introduced a “2.5D” semi-auto tracing framework to tackle this challenge (see footnote 20). In his mini-review, Lu also provided a brief comparison of different tracing strategies designed for either sparse or dense neural processes (see footnote 19).

Axonal to Dendritic, Short-Distance to Long-Distance

In the current DIADEM, the tested in vivo data are all axonal projections within a relatively small brain region. De Paola and his colleague suggested in their commentary to extend the reconstruction to whole-cell level to include soma and dendritic trees in deeper brain layers (see footnote 23). This poses profound experimental and computational challenges.

Optical Microscopy and Electron Microscopy

A 2007 Janelia Farm Conference on Neural Circuit Reconstruction discussed the challenges facing the reconstruction of complete circuit diagrams, including topics on both optical and electron microscopic approaches. The concept of DIADEM to first focus on optical imaging reconstruction was originated at that conference.

As pointed out by several authors, while there are distinct differences in experimental methods to acquire image data between the two modalities, it would be beneficial for both fields to share and adapt computational strategies, such as machine learning and segmentation tools (see footnotes 6 and 19).

Neuronal to Neural Circuit to Connectome

The current DIADEM Challenge has focused on reconstruction of optical imaging at the neuronal level. However, one of the ultimate goals of neuroscience is to understand the structural and functional connectivity of the nervous system, which has been defined by some neuroscientists as the “connectome.” In order to achieve this goal, it will require investigations at all levels, from genetic, molecular, synaptic, neuronal levels all the way up to the whole brain and behavioral levels. Recently, large efforts to tackle the connectivity challenge at different scales of the organization have been made in laboratories and institutions around the world. For example, the goal of the Human Connectome Project supported by the NIH is to map the neural pathways underlie human brain function. (http://www.humanconnectome.org/consortia/)

In the 1980’s, Martin and colleagues proposed the “canonical cortical circuit” concept.²⁴ They and many other neuroscientists believe that microcircuits in the brain are the key structural and functional units, which can serve to link studies from micro to macro levels (see footnote 7). He and Kozloski, in their mini-reviews, made suggestions for microcircuitry analysis, such as linking structural knowledge with functional data (e.g., paired physiological recordings) and combining statistical approaches with analyses of the rules that organize the circuits to generate predictive computational models of connectivity (see footnotes 7 and 18). Kozloski further discussed how the coupling between high-throughput reconstruction of structural connections and large-scale simulation of functional connections could constrain parameters in predictive modeling and advance our knowledge of brain connectivity (see footnote 18).

Future Collaborations

The major conclusion derived from the DIADEM is that collaboration is the key for progress on all fronts. The DIADEM itself was an innovative collaboration among academic and non-profit research organizations, and government agencies. The result of successful auto-reconstruction research demonstrated at the conference was clearly dependent on close collaborations between experimental neuroscientists and computational scientists.

Recommendations for future close collaborations include those among:

• Neuromorphologists to improve imaging quality and data sharing methods, such as standardized data curation

• Algorithm developers to improve and integrate computational tools and share them with the experimental research community

• Experimental neuroscientists and computational scientists to cross-fertilize the current finding in neuroscience, the strategies of computation, and more data sharing for further development and testing of automation tools

• All scientists who are involved in the field, such as neuromorphologists, neurophysiologists, computational scientists and theorists to develop conceptual frameworks

At the conference, it was proposed to form a DIADEM Consortium, and a mailing list of the DIADEM participants has been created (see footnote 8). Anyone who is interested in this area of research is welcome to join. (http://listserv.janelia.org/mailman/listinfo/diadem_challenge)

• 3. How Can We “See” More and Better?

One major question facing all modern sciences is: Can technology alone enable us to see more and better than the “ancient giants”? As Martin reminded us, new technologies demand new concepts. In modern neuroscience, with overwhelming quantity and complexity of data generated by advanced technologies, the real Grand Challenge lies in the development of 21st century Cajal-equivalent conceptual frameworks (see footnote 7).

A few years ago, during Gordon Shepherd’s tribute to Wilfred Rall’s contribution to computational neuroscience, he shared with us a fragile page from their 1960’s research notebook. On the page was the sketch of their thoughts of the predictive model of dendro-dendritic synaptic connection. When asked how they developed such a powerful model based on just a few subtle and puzzling electrophysiological observations, their modest answer was that “since computers were slow in those days, we had time to think!” ²⁵