Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: Classification and segmentation

https://doi.org/10.1016/j.compbiomed.2020.104037Get rights and content

Highlights

  • Multitask deep learning based model can be used to detect COVID-19 lesions on CT scans.

  • The proposed model can improve state of the art U-NET by leveraging useful information contained in multiple related tasks.

  • Obtained a dice coefficient of 88% for image segmentation and an accuracy of 94.67 for multiclass classification.

  • The proposed model can be used as a support tool to assist physicians.

Abstract

This paper presents an automatic classification segmentation tool for helping screening COVID-19 pneumonia using chest CT imaging. The segmented lesions can help to assess the severity of pneumonia and follow-up the patients. In this work, we propose a new multitask deep learning model to jointly identify COVID-19 patient and segment COVID-19 lesion from chest CT images. Three learning tasks: segmentation, classification and reconstruction are jointly performed with different datasets. Our motivation is on the one hand to leverage useful information contained in multiple related tasks to improve both segmentation and classification performances, and on the other hand to deal with the problems of small data because each task can have a relatively small dataset. Our architecture is composed of a common encoder for disentangled feature representation with three tasks, and two decoders and a multi-layer perceptron for reconstruction, segmentation and classification respectively. The proposed model is evaluated and compared with other image segmentation techniques using a dataset of 1369 patients including 449 patients with COVID-19, 425 normal ones, 98 with lung cancer and 397 of different kinds of pathology. The obtained results show very encouraging performance of our method with a dice coefficient higher than 0.88 for the segmentation and an area under the ROC curve higher than 97% for the classification.

Keywords

Deep learning
Multitask learning
Image classification
Image segmentation
Coronavirus (COVID-19)
Computed tomography images

Cited by (0)

Amine Amyar, Publications:

1- Amyar, A. et al. “

3-D RPET-NET: development of a 3-D pet imaging convolutional neural network for radiomics analysis and outcome prediction.” IEEE Transactions on Radiation and Plasma Medical Sciences 3.2 (2019): 225-231.

2-Amyar, Amine, et al. “Radiomics-net: Convolutional neural networks on FDG PET images for predicting cancer treatment response.” Journal of Nuclear Medicine 59.supplement 1 (2018): 324-324

3- Amyar, Amine, et al. “Weakly supervised pet tumor detection usingclass response.” arXiv preprint arXiv:2003.08337 (2020).

Romain Modzelewski, Publications:

1- Belharbi, Soufiane, et al. “Spotting L3 slice in CT scans using deep convolutional network and transfer learning.” Computers in biology and medicine 87 (2017): 95-103.

2- Paul, Desbordes, et al. “Feature selection for outcome prediction in oesophageal cancer using genetic algorithm and random forest classifier.” Computerized Medical Imaging and Graphics 60 (2017): 42-49.

3- Lanic, Hélène, et al. “Sarcopenia is an independent prognostic factor in elderly patients with diffuse large B-cell lymphoma treated with immunochemotherapy.” Leukemia & lymphoma 55.4 (2014): 817–823.

Hua Li, Publications:

1- Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132. KK Brock, S Mutic, TR McNutt, H Li, ML Kessler Medical physics 44 (7), e43-e76

2- Vessels as 4-D curves: Global minimal 4-D paths to extract 3-D tubular surfaces and centerlines H Li, A Yezzi IEEE transactions on medical imaging 26 (9), 1213-1223

3- Current status of Radiomics for cancer management: Challenges versus opportunities for clinical practice H Li, I El Naqa, Y Rong Journal of applied clinical medical physics 21 (7), 7

Su Ruan, Publications:

1- Nie, Dong, et al. “Medical image synthesis with context-aware generative adversarial networks.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2017.

2- Joint tumor segmentation in PET-CT images using co-clustering and fusion based on belief functions C Lian, S Ruan, T Denoeux, H Li, P Vera IEEE Transactions on Image Processing 28 (2), 755-766

3- Segmenting multi-source images using hidden markov fields with copula-based multivariate statistical distributions J Lapuyade-Lahorgue, JH Xue, S Ruan IEEE Transactions on Image Processing 26 (7), 3187-3195

View Abstract