Self-Supervised Learning (SSL) for Radiology

by Qinmei Xu, MD, PhD

AI systems can learn effectively from massive amounts of carefully labeled data. However, the scarcity of high-quality annotated medical imaging datasets is a major problem that limits the advancement of applications in medical imaging analysis.

Self-supervised learning (SSL) is an emerging training paradigm that enables learning robust representations without the need for human annotation. Thus, SSL can be considered as an effective solution to address the scarcity of annotated medical data.

SSL is designed to learn semantically useful features for a certain task by generating supervisory signals from a pool of unlabeled data without the need for human annotation. These feature representations are then used for subsequent tasks where the amount of labeled data is limited or even unavailable (Liu et al.).

The workflow of SSL includes the pretext task and downstream task. In the pretext task, the self-supervised learning actually develops a model in a supervised fashion using the unlabeled data by creating pseudo-labels from the data, enabling the model to learn the intrinsic useful representation from the data. In the downstream task, the learned representations from the pretext task are transferred as initial weights to the downstream task to accomplish its intended goal (Holmberg et al.).

SSL can be appealing to analyze medical images, where the amount of available annotated data is relatively small. Representative studies have demonstrated the effectiveness of the self-supervised learning approach for medical image analysis tasks such as classification, localization, and segmentation tasks. The Awesome Self-Supervised Learning in Medical Imaging GitHub repository serves as a key scientific resource in the evolving field.

If AI systems can glean a deeper, more nuanced understanding of reality beyond what’s specified in a training data set, they potentially will be more useful and ultimately will bring AI closer to real-world medical applications. Self-supervised learning is much more scalable than traditional supervised learning because class label annotation is not required.

Qinmei Xu, MD, PhD is a member of this journal’s Trainee Editorial Board.
She is a radiology resident at Nanjing Jinling Hospital and an incoming postdoctoral fellow at the Stanford Center for Biomedical Informatics
Research. @Mayxu02147222