Synthetic X-Ray Image Generation

This project was motivated by the ongoing COVID pandemic with the aim of experimenting with computer vision techniques that could help in diagnosing COVID. The X-ray dataset that was obtained from Kaggle had a limited number of COVID X-rays, hence our project focused on a technique to augment this dataset with the aim of improving the classification accuracy of a COVID detection model.

The project consists of two components: Generative Adversarial Network (GAN) to generate new images to augment the existing COVID X-ray dataset, and Convolutional Neural Network (CNN) Model to help classify whether a patient X-ray has COVID or not.

Generative Adversial Network:

We wanted to learn the distribution of the input Chest X-Ray training data which would in turn help us to generate new covid/normal X-Ray images. We designed a Conditional GAN for the purpose of image generation. The Conditional GAN takes in labels and noise as inputs and outputs an image based on the input label. The architecture of the GAN is included below:

GAN training:

We had a total of 7232 images for training (3616 covid images and 3616 normal images). We trained the discriminator 1 step for every K (=10) steps that we trained the generator. This helps to prevent the discriminator from learning too fast which can cause the generator to diverge and create noisy results. The discriminator uses the Binary Cross Entropy Loss function to discriminate between real and fake images. Following are the results from running the GAN for 10 epochs:

Convolutional Neural Network:

We experimented with three pretrained (on Imagenet) models: InceptionV3, Resnet50, VGG16 as the backbone. We removed the original fully-connected layers and augmented with custom fully-connected layers for classification task at hand. We froze the layers of the pretrained models. The data was split in a 0.6-0.2-0.2 train,validation and test split. The data has perfect class balance and each set has equal number of covid and normal examples.

The architecture of our CNN model is included below:

The performance of our model on the original Kaggle dataset:

The performance of our model on the augmented dataset consisting of images generated by the GAN:

Learnings:

• We found that training the Conditional GAN was a difficult task, because of the multitude of hyperparameters involved which influence whether the generator and discriminator are able to learn from each other or not.

• Although the GAN did not produce great quality (visually) images, it did seem that it was able to encode some meaningful data because we observed improved performance on two out of three models after augmenting them with the GAN generated data.

• Our best model is the GAN data augmented VGG16 which produced a test set accuracy of 0.9675.