Utilizing convolutional neural networks for discriminating cancer and stromal cells in three-dimensional cell culture images with nuclei counterstain

Huu Tuan Nguyen, Nicholas Pietraszek, Sarah E. Shelton, Kwabena Arthur, Roger D. Kamm

Link to Paper:

Abstract: Significance: Accurate cell segmentation and classification in 3 dimensional (3D) images are vital for studying live cell behavior and drug responses in 3D tissue culture. Evaluating diverse cell populations in 3D cell culture over time necessitates non-toxic staining methods, as specific fluorescence tags may not be suitable and immunofluorescence staining can be cytotoxic for prolonged live cell cultures. Aim: Our goal is to perform machine learning-based cell classification within a live heterogeneous cell culture population grown in a 3D tissue culture relying only on reflectance, transmittance, and nuclei counterstaining images obtained by a confocal microscope. Approach: In this study, we employed a supervised Convolutional Neural Network (CNN) to classify tumor cells and fibroblast within 3D-grown spheroids. These cells are first segmented by the Marker-Controlled Watershed image processing method. Training data included nuclei counterstaining, reflectance, and transmitted light images, with stained fibroblast and tumor cells as ground-truth labels. Results: Our results demonstrate the successful Marker-Controlled Watershed segmentation of 84% of spheroid cells into single cells. We achieved a median accuracy of 67% (95% confidence interval of the median is 65-71%) in identifying cell types. We also recapitulate the original 3D images using the CNN-classified cells to visualize the original 3D-stained image’s cell distribution. Conclusion: This study introduces a non-invasive, toxicity-free approach to 3D cell culture evaluation, combining machine learning with confocal microscopy, opening avenues for advanced cell studies.

Link to all images: https://omero.mit.edu/webclient/?show=project-559