Skip to main content

Advertisement

Log in

Fast and Precise Hippocampus Segmentation Through Deep Convolutional Neural Network Ensembles and Transfer Learning

  • Original Article
  • Published:
Neuroinformatics Aims and scope Submit manuscript

Abstract

Automatic segmentation of the hippocampus from 3D magnetic resonance imaging mostly relied on multi-atlas registration methods. In this work, we exploit recent advances in deep learning to design and implement a fully automatic segmentation method, offering both superior accuracy and fast result. The proposed method is based on deep Convolutional Neural Networks (CNNs) and incorporates distinct segmentation and error correction steps. Segmentation masks are produced by an ensemble of three independent models, operating with orthogonal slices of the input volume, while erroneous labels are subsequently corrected by a combination of Replace and Refine networks. We explore different training approaches and demonstrate how, in CNN-based segmentation, multiple datasets can be effectively combined through transfer learning techniques, allowing for improved segmentation quality. The proposed method was evaluated using two different public datasets and compared favorably to existing methodologies. In the EADC-ADNI HarP dataset, the correspondence between the method’s output and the available ground truth manual tracings yielded a mean Dice value of 0.9015, while the required segmentation time for an entire MRI volume was 14.8 seconds. In the MICCAI dataset, the mean Dice value increased to 0.8835 through transfer learning from the larger EADC-ADNI HarP dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  • Ahdidan, J., Raji, C.A., DeYoe, E.A., Mathis, J., Noe, K., Rimestad, J., Kjeldsen, T.K., Mosegaard, J., Becker, J.T., Lopez, O. (2016). Quantitative neuroimaging software for clinical assessment of hippocampal volumes on MR imaging. Journal of Alzheimer’s Disease, 49(3), 723–732.

    Article  PubMed  Google Scholar 

  • Aljabar, P., Heckemann, R.A., Hammers, A., Hajnal, J.V., Rueckert, D. (2009). Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. NeuroImage, 46(3), 726–738.

    Article  CAS  PubMed  Google Scholar 

  • Bateman, R.J., Xiong, C., Benzinger, T.L., Fagan, A.M., Goate, A., Fox, N.C., Marcus, D.S., Cairns, N.J., Xie, X., Blazey, T.M., et al. (2012). Clinical and biomarker changes in dominantly inherited Alzheimer’s disease. New England Journal of Medicine, 367(9), 795–804.

    Article  CAS  Google Scholar 

  • Bernasconi, N., Bernasconi, A., Caramanos, Z., Antel, S., Andermann, F., Arnold, D. (2003). Mesial temporal damage in temporal lobe epilepsy: a volumetric MRI study of the hippocampus, amygdala and parahippocampal region. Brain, 126(2), 462–469.

    Article  CAS  PubMed  Google Scholar 

  • Boccardi, M., Bocchetta, M., Morency, F.C., Collins, D.L., Nishikawa, M., Ganzola, R., Grothe, M.J., Wolf, D., Redolfi, A., Pievani, M., et al. (2015). Training labels for hippocampal segmentation based on the EADC-ADNI harmonized hippocampal protocol. Alzheimer’s & Dementia, 11(2), 175–183.

    Article  Google Scholar 

  • de Brébisson, A., & Montana, G. (2015). Deep neural networks for anatomical brain segmentation. arXiv:150202445.

  • Bremner, J.D., Narayan, M., Anderson, E.R., Staib, L.H., Miller, H.L., Charney, D.S. (2000). Hippocampal volume reduction in major depression. American Journal of Psychiatry, 157(1), 115– 118.

    Article  CAS  Google Scholar 

  • Brosch, T., Tang, L.Y., Yoo, Y., Li, D.K., Traboulsee, A., Tam, R. (2016). Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Transactions on Medical Imaging, 35(5), 1229–1239.

    Article  PubMed  Google Scholar 

  • Carreira, J., Agrawal, P., Fragkiadaki, K., Malik, J. (2016). Human pose estimation with iterative error feedback. In: Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, pp. 4733–4742.

  • Chen, H., Dou, Q., Yu, L., Heng, P.A. (2016). Voxresnet: Deep voxelwise residual networks for volumetric brain segmentation. arXiv:160805895.

  • Chen, Y., Shi, B., Wang, Z., Sun, T., Smith, C.D., Liu, J. (2017). Accurate and consistent hippocampus segmentation through convolutional LSTM and view ensemble. In: International workshop on machine learning in medical imaging, Springer, pp. 88–96.

  • Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B., Shelhamer, E. (2014). cuDNN: Efficient primitives for deep learning. arXiv:14100759.

  • Chincarini, A., Sensi, F., Rei, L., Gemme, G., Squarcia, S., Longo, R., Brun, F., Tangaro, S., Bellotti, R., Amoroso, N., et al. (2016). Integrating longitudinal information in hippocampal volume measurements for the early detection of Alzheimer’s disease. NeuroImage, 125, 834–847.

    Article  PubMed  Google Scholar 

  • Choi, H., & Jin, K.H. (2016). Fast and robust segmentation of the striatum using deep convolutional neural networks. Journal of Neuroscience Methods, 274, 146–153.

    Article  PubMed  Google Scholar 

  • Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O. (2016). 3d u-net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp. 424–432.

  • Collins, D.L., & Pruessner, J.C. (2010). Towards accurate, automatic segmentation of the hippocampus and amygdala from MRI by augmenting ANIMAL with a template library and label fusion. NeuroImage, 52(4), 1355–1366.

    Article  PubMed  Google Scholar 

  • Collobert, R., Kavukcuoglu, K., Farabet, C. (2011). Torch7: A matlab-like environment for machine learning. In: BigLearn, NIPS workshop, EPFL-CONF-192376.

  • Coupé, P, Manjón, JV, Fonov, V., Pruessner, J., Robles, M., Collins, D.L. (2010). Nonlocal patch-based label fusion for hippocampus segmentation. In: International conference on medical image computing and computer assisted intervention, Springer, pp. 129–136.

  • Dolz, J., Desrosiers, C., Ayed, I.B. (2017). 3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study. NeuroImage.

  • Du, A., Schuff, N., Amend, D., Laakso, M., Hsu, Y., Jagust, W., Yaffe, K., Kramer, J., Reed, B., Norman, D., et al. (2001). Magnetic resonance imaging of the entorhinal cortex and hippocampus in mild cognitive impairment and Alzheimer’s disease. Journal of Neurology. Neurosurgery & Psychiatry, 71(4), 441–447.

    Article  CAS  Google Scholar 

  • Fischl, B., Salat, D.H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., Van Der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., et al. (2002). Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron, 33(3), 341–355.

    Article  CAS  PubMed  Google Scholar 

  • Gidaris, S., & Komodakis, N. (2017). Detect, replace, refine: Deep structured prediction for pixel wise labeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5248–5257.

  • Giraud, R., Ta, V.T., Papadakis, N., Manjón, J V, Collins, D.L., Coupé, P, Alzheimer’s Disease Neuroimaging Initiative, et al. (2016). An optimized patchmatch for multi-scale and multi-feature label fusion. NeuroImage, 124, 770–782.

    Article  PubMed  Google Scholar 

  • Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th international conference on artificial intelligence and statistics, pp. 249–256.

  • Harrison, P.J. (2004). The hippocampus in schizophrenia: a review of the neuropathological evidence and its pathophysiological implications. Psychopharmacology, 174(1), 151–162.

    Article  CAS  PubMed  Google Scholar 

  • Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H. (2017). Brain tumor segmentation with deep neural networks. Medical Image Analysis, 35, 18–31.

    Article  PubMed  Google Scholar 

  • He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.

  • Inglese, P., Amoroso, N., Boccardi, M., Bocchetta, M., Bruno, S., Chincarini, A., Errico, R., Frisoni, G., Maglietta, R., Redolfi, A., et al. (2015). Multiple RF classifier for the hippocampus segmentation: Method and validation on EADC-ADNI harmonized hippocampal protocol. Physica Medica, 31(8), 1085–1091.

    Article  CAS  PubMed  Google Scholar 

  • Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:150203167.

  • Jack, C.R., Bernstein, M.A., Fox, N.C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P.J., L Whitwell, J., Ward, C., et al. (2008). The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. Journal of Magnetic Resonance Imaging, 27(4), 685–691.

    Article  PubMed  Google Scholar 

  • Jack, C.R., Barkhof, F., Bernstein, M.A., Cantillon, M., Cole, P.E., DeCarli, C., Dubois, B., Duchesne, S., Fox, N.C., Frisoni, G.B., et al. (2011). Steps to standardization and validation of hippocampal volumetry as a biomarker in clinical trials and diagnostic criterion for Alzheimer’s disease. Alzheimer’s & Dementia, 7 (4), 474– 485.

    Article  Google Scholar 

  • Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B. (2017). Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis, 36, 61–78.

    Article  PubMed  Google Scholar 

  • Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:14126980.

  • Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-Hein, K., Bendszus, M., Biller, A. (2016). Deep MRI brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage, 129, 460–469.

    Article  PubMed  Google Scholar 

  • Krizhevsky, A., Sutskever, I., Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105.

  • Kushibar, K., Valverde, S., Gonzalez-Villa, S., Bernal, J., Cabezas, M., Oliver, A., Llado, X. (2017). Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features. arXiv:170909075.

  • Landman, B., & Warfield, S. (2012). MICCAI 2012 Workshop on Multi-Atlas Labeling. CreateSpace Independent Publishing Platform, ISBN: 1479126187.

  • Langerak, T.R., van der Heide, U.A., Kotte, A.N., Viergever, M.A., Van Vulpen, M., Pluim, J.P. (2010). Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (SIMPLE). IEEE Transactions on Medical Imaging, 29(12), 2000–2008.

    Article  PubMed  Google Scholar 

  • LeCun, Y., Bottou, L., Bengio, Y., Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Article  Google Scholar 

  • Ledig, C., Heckemann, R.A., Hammers, A., Lopez, J.C., Newcombe, V.F., Makropoulos, A., Lötjönen, J, Menon, D.K., Rueckert, D. (2015). Robust whole-brain segmentation: application to traumatic brain injury. Medical Image Analysis, 21(1), 40–58.

    Article  PubMed  Google Scholar 

  • Leung, K.K., Barnes, J., Ridgway, G.R., Bartlett, J.W., Clarkson, M.J., Macdonald, K., Schuff, N., Fox, N.C., Ourselin, S., Alzheimer’s Disease Neuroimaging Initiative, et al. (2010). Automated cross-sectional and longitudinal hippocampal volume measurement in mild cognitive impairment and Alzheimer’s disease. NeuroImage, 51(4), 1345–1359.

    Article  PubMed  Google Scholar 

  • Li, K., Hariharan, B., Malik, J. (2016). Iterative instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3659–3667.

  • Long, J., Shelhamer, E., Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440.

  • Maglietta, R., Amoroso, N., Boccardi, M., Bruno, S., Chincarini, A., Frisoni, G.B., Inglese, P., Redolfi, A., Tangaro, S., Tateo, A., et al. (2016). Automated hippocampal segmentation in 3D MRI using random undersampling with boosting algorithm. Pattern Analysis and Applications, 19(2), 579–591.

    Article  PubMed  Google Scholar 

  • Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L. (2007). Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. Journal of Cognitive Neuroscience, 19(9), 1498–1507.

    Article  PubMed  Google Scholar 

  • Mehta, R., Majumdar, A., Sivaswamy, J. (2017). BrainSegNet: a convolutional neural network architecture for automated segmentation of human brain structures. Journal of Medical Imaging, 4(2), 024003.

    Article  PubMed  PubMed Central  Google Scholar 

  • Milletari, F., Ahmadi, S.A., Kroll, C., Plate, A., Rozanski, V., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Bötzel, K, et al. (2017). Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound. Computer Vision and Image Understanding, 164, 92–102.

    Article  Google Scholar 

  • Moeskops, P., Viergever, M.A., Mendrik, A.M. (2016). de Vries LS, Benders MJ, Iṡgum I. Automatic segmentation of MR brain images with a convolutional neural network. IEEE Transactions on Medical Imaging, 35(5), 1252–1261.

    PubMed  Google Scholar 

  • Morra, J.H., Tu, Z., Apostolova, L.G., Green, A.E., Toga, A.W., Thompson, P.M. (2010). Comparison of AdaBoost and support vector machines for detecting Alzheimer’s disease through automated hippocampal segmentation. IEEE Transactions on Medical Imaging, 29(1), 30.

    Article  PubMed  Google Scholar 

  • Nair, V., & Hinton, G.E. (2010). Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807–814.

  • Patenaude, B., Smith, S.M., Kennedy, D.N., Jenkinson, M. (2011). A bayesian model of shape and appearance for subcortical brain segmentation. Neuroimage, 56(3), 907–922.

    Article  PubMed  Google Scholar 

  • Pereira, S., Pinto, A., Alves, V., Silva, C.A. (2016). Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35(5), 1240–1251.

    Article  PubMed  Google Scholar 

  • Platero, C., & Tobar, M.C. (2017). Combining a patch-based approach with a non-rigid registration-based label fusion method for the hippocampal segmentation in Alzheimer’s disease. Neuroinformatics, 15(2), 165–183.

    Article  PubMed  Google Scholar 

  • Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S. (2014). CNN features off-the-shelf: an astounding baseline for recognition (Vol. 2014, pp. 512–519). IEEE Conference on: IEEE.

    Google Scholar 

  • Ronneberger, O., Fischer, P., Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp. 234–241.

  • Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211–252.

    Article  Google Scholar 

  • Scoville WB, & Milner B. (2000). Loss of recent memory after bilateral hippocampal lesions. The Journal of Neuropsychiatry and Clinical Neurosciences, 12(1), 103–a.

    Article  PubMed  Google Scholar 

  • Sdika, M. (2010). Combining atlas based segmentation and intensity classification with nearest neighbor transform and accuracy weighted vote. Medical Image Analysis, 14(2), 219–226.

    Article  PubMed  Google Scholar 

  • Shakeri, M., Tsogkas, S., Ferrante, E., Lippe, S., Kadoury, S., Paragios, N., Kokkinos, I. (2016). Sub-cortical brain structure segmentation using F-CNN’s. In: 2016 IEEE 13th international symposium on biomedical imaging (ISBI), IEEE, pp. 269–272.

  • Shen, D., Moffat, S., Resnick, S.M., Davatzikos, C. (2002). Measuring size and shape of the hippocampus in MR images using a deformable shape model. NeuroImage, 15(2), 422–434.

    Article  PubMed  Google Scholar 

  • Sled, J.G., Zijdenbos, A.P., Evans, A.C. (1998). A nonparametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Transactions on Medical Imaging, 17(1), 87–97.

    Article  CAS  PubMed  Google Scholar 

  • Smith, S.M. (2002). Fast robust automated brain extraction. Human Brain Mapping, 17(3), 143–155.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sun, C., Shrivastava, A., Singh, S., Gupta, A. (2017). Revisiting unreasonable effectiveness of data in deep learning era. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, pp. 843–852.

  • Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A. (2017). Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: AAAI conference on artificial intelligence.

  • Tong, T., Wolz, R., Coupé, P, Hajnal, J.V., Rueckert, D., Initiative, A.D.N., et al. (2013). Segmentation of MR images via discriminative dictionary learning and sparse coding: application to hippocampus labeling. NeuroImage, 76, 11–23.

    Article  PubMed  Google Scholar 

  • Wachinger, C., Reuter, M., Klein T. (2017). DeepNAT, Deep convolutional neural network for segmenting neuroanatomy. NeuroImage.

  • Wang, H., & Yushkevich, P.A. (2013). Multi-atlas segmentation with joint label fusion and corrective learning an open source implementation. Frontiers in Neuroinformatics 7.

  • Wang, H., Das, S.R., Suh, J.W., Altinay, M., Pluta, J., Craige, C., Avants, B., Yushkevich, P.A., Alzheimer’s Disease Neuroimaging Initiative, et al. (2011). A learning-based wrapper method to correct systematic errors in automatic image segmentation: consistently improved performance in hippocampus, cortex and brain segmentation. NeuroImage, 55(3), 968–985.

    Article  PubMed  Google Scholar 

  • Wang, H., Suh, J.W., Das, S.R., Pluta, J.B., Craige, C., Yushkevich, P.A. (2013). Multi-atlas segmentation with joint label fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(3), 611–623.

    Article  PubMed  Google Scholar 

  • Wang, L., Wang, L., Lu, H., Zhang, P., Ruan, X. (2016). Saliency detection with recurrent fully convolutional networks. In: European conference on computer vision, Springer, pp. 825–841.

  • Wang, Y., Ma, G., Wu, X., Zhou, J. (2018). Patch-based label fusion with structured discriminant embedding for hippocampus segmentation. Neuroinformatics, 1–13.

  • Yang, J., Staib, L.H., Duncan, J.S. (2004). Neighbor-constrained segmentation with level set based 3-D deformable models. IEEE Transactions on Medical Imaging, 23(8), 940–948.

    Article  PubMed  PubMed Central  Google Scholar 

  • Yosinski, J., Clune, J., Bengio, Y., Lipson, H. (2014). How transferable are features in deep neural networks? In: Advances in neural information processing systems, pp. 3320–3328.

  • Zarpalas, D., Gkontra, P., Daras, P., Maglaveras, N. (2014a). Accurate and fully automatic hippocampus segmentation using subject-specific 3D optimal local maps into a hybrid active contour model. IEEE Journal of Translational Engineering in Health and Medicine, 2, 1–16.

    Article  Google Scholar 

  • Zarpalas, D., Gkontra, P., Daras, P., Maglaveras, N. (2014b). Gradient-based reliability maps for ACM-based segmentation of hippocampus. IEEE Transactions on Biomedical Engineering, 61(4), 1015–1026.

    Article  PubMed  Google Scholar 

  • Zhang, W., Li, R., Deng, H., Wang, L., Lin, W., Ji, S., Shen, D. (2015). Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage, 108, 214– 224.

    Article  PubMed  Google Scholar 

  • Zhu, H., Cheng, H., Yang, X., Fan, Y., Initiative, A.D.N., et al. (2017). Metric learning for multi-atlas based segmentation of hippocampus. Neuroinformatics, 15(1), 41–50.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

The authors gratefully acknowledge the support of NVIDIA Corporation with the donation of the GPU used for this research.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dimitrios Ataloglou.

Ethics declarations

Conflict of interests

The authors declare no conflicts of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Information Sharing Statement

Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (RRID:SCR_003007, adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ataloglou, D., Dimou, A., Zarpalas, D. et al. Fast and Precise Hippocampus Segmentation Through Deep Convolutional Neural Network Ensembles and Transfer Learning. Neuroinform 17, 563–582 (2019). https://doi.org/10.1007/s12021-019-09417-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-019-09417-y

Keywords

Navigation