Intra-and Inter-Cellular Awareness for 3D Neuron Tracking and Segmentation in Large-Scale Connectomics

Hao Zhai, Jing Liu, Bei Hong, Jiazheng Liu, Qiwei Xie, Hua Han

State Key Laboratory of Multimodal Artificial Intelligence Systems, Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences
School of Future Technology, University of Chinese Academy of Sciences
Changping Laboratory
Research Base of Beijing Modern Manufacturing Development, Beijing University of Technology

Abstract

Currently, most state-of-the-art pipelines for 3D micro-connectomic reconstruction deal with neuron over-segmentation, agglomeration and subcellular compartment (nuclei, mitochondria, synapses, etc.) detection separately. Inspired by the proofreading consensus of experts, we established a paradigm to acquire priori knowledge of cellular characteristics and ultrastructures, as well as determine the connectivity of neural circuits simultaneously. Following this novel paradigm, we were keen to bring the Intra- and Inter-Cellular Awareness back when Tracking and Segmenting neurons in connectomics. Our proposed method (II-CATS) utilizes few-shot learning techniques to encode the internal neurite representation and its learnable components, which could significantly impact neuron tracings. We further go beyond the original expected run length (ERL) metric by focusing on biological constraints (bERL) or spanning from the nucleus to spines (nERL). With the evaluation of these metrics, we perform typical experiments on multiple electron microscopy datasets on diverse animals and scales. In particular, our proposed method is naturally suitable for tracking neurons that have been identified by staining.

Citation

Zhai H, Liu J, Hong B, Liu J, Xie Q, Han H (2024). Intra-and Inter-Cellular Awareness for 3D Neuron Tracking and Segmentation in Large-Scale Connectomics. In Proceedings of Machine Learning Research, 227, 1691–1712. https://proceedings.mlr.press/v227/zhai24a.html.