Facial skin disease prediction using StarGAN v2 and transfer learning

Abstract

Deep learning algorithms have become the most prominent methods for medical image analysis over the past years, leading to enhanced performances in various medical applications. In this paper, we focus on applying intelligent skin disease detection to face images, where the crucial challenge is the low availability of training data. To achieve high disease detection and classification success rates, we adapt the state-of-the-art StarGAN v2 network to augment images of faces and combine it with a transfer learning approach. The experimental results show that the classification accuracies of transfer learning models are in the range of 77.46–99.80% when trained on datasets that are extended with StarGAN v2 augmented data.

1. Introduction

Lack of available training data is one of the crucial challenges in designing deep learning-based methods for classification and segmentation. This paper addresses the classification of skin diseases in faces from image data. Even though some public datasets are available [1, 2, 3], the amount of data that they contain is incomparable to the amounts of data that are available for other standard computer vision problems, and thus, the application of models and their performances for this problem are restricted. Additionally, larger training datasets, especially in the case of deep learning methods, lead to better performance [4, 5].

Generative adversarial networks (GANs) are a popular approach for generative modeling using deep learning and have been demonstrated to be effective in several tasks, such as image synthesis [6], image translation [7] and video generation [8]. They enable the generation of new output data.

Figure 1.

The proposed approach for improving the classification accuracy in facial skin disease classification, for which only small datasets are available. The classifier performance is enhanced by enlarging the input dataset by generating images with StarGAN v2.

Data generation, which is an unsupervised learning prob-lem, is posed within the process of GAN training using two submodels: a generator model, which is trained to generate new output data that are similar to the input, and a discriminator model, which classifies outputs of the generator model as fake or real. In this way, training is performed in a supervised manner.

The main advantage of GANs is their ability to generate realistic new examples based on input data such that humans are unable to distinguish whether they are fake or real [9]. Therefore, they are mostly used for knowledge translation across domains and to create high-resolution output images from input data. Since they provide outstanding performance in data generation and the capability to capture the underlying data structure, they have been shown to be a great choice for improving the accuracy of medical image segmentation [10]. There is a plethora of work that addresses the problems of data augmentation, image segmentation, detection and classification of medical images [2, 10, 4, 5] with GANs and their extensions.

Their ability to generate images that are realistic to the human eye additionally promises to overcome the challenges of data labeling in the medical field [5]. GANs are commonly used for skin-lesion analysis, segmentation, and classification [2, 11, 12]. Similarly, this paper tackles the problem of augmenting data to improve transfer learning classifiers for skin disease detection and recognition (Fig. 1).

Skin melanoma assessment [13] and skin disease classification are one of the most commonly addressed issues in medical image processing [2]. Developing automatized approaches for skin issue detection and recognition would reduce the time needed to obtain diagnoses and decrease the amount of resources that must be expended by both patients and health care institutions. Transfer learning is a common approach for addressing skin disease classification problems [11]. Transfer learning enables knowledge transfer from large datasets to a specified target task for which only a smaller dataset is available. This is done by first training a CNN model on a large dataset and then retraining this network to address the target problem.

One of the main challenges is obtaining adequate datasets for achieving high accuracy in skin disease detection and classification. Therefore, this paper’s primary goal is to generate high-quality augmented data to improve the classification accuracy for facial skin diseases.

The work presented in this paper extends StarGAN v2 [14], which is a powerful framework that can synthesize rich style images, to medical-domain image generation. We test augmented data for training eight different classification models in the Transfer Learning Suite in PyTorch. 1

https://pytorch.org/vision/stable/models.html.

The experiments are conducted with the Kaggle dataset 20+ Skin Disease Directories with Face Images. 2

https://www.kaggle.com/syedaun/20-skin-disease-directories-with-face-images/code.

We augment images and achieve high accuracy in the of classification of skin diseases. We find that transfer learning models have validation accuracies of 77.46–99.801% when trained on a dataset extended with the StarGAN v2 augmented data.

The main contributions of this article are:

•

We propose a GAN model for facial skin image generation based on the StarGAN v2 architecture. The experimental results show great efficiency in the generation of a rich diversity of skin images.

•

We examine and verify transfer learning-based classifiers for skin disease detection and classification with newly generated data in combination with the original data.

•

A framework for medical image analysis, generation and classification is created with the possibility of automatic diagnosis (Fig. 2).

The VGG19 model showed the highest overall validation accuracy of 99.85%.

Figure 2.

Block diagram of the proposed approach for facial skin disease prediction.

Figure 3.

Sample images from the five diagnostic categories of facial skin diseases, namely, Acne and Rosacea, Eczema, Systemic Disease, Urticaria Hives and Vascular Tumors, used in this paper.

2. Dataset overview

This paper examines the problem of classifying the fol-lowing skin diseases: Acne and Rosacea, Eczema, Systemic Disease, Urticaria Hives and Vascular Tumors. Examples from the “20+ Skin Disease Directories with Face Images” dataset are shown in Fig. 3.

Acne and rosacea are common facial skin conditions that cause tiny pimples, redness, and evident blood vessels.

Eczema, or atopic dermatitis, is a skin disease characterized by symptoms such as inflamed, itchy and cracked skin patches. In some cases, blisters can also appear. The symptoms of ezcema can vary depending on a person’s age.

A systemic disease is an inflammatory skin condition that can affect the scalp, central chest, face, usually eybrows, ears, or, in some cases, whole body. The skin is covered by thin erythematous plaques and greasy scales.

Urticaria hives are red, itchy welts, which are also known as nettle rash, that result from a skin reaction and appear for more than six weeks. The skin welts vary in size and appear and fade repeatedly. Usually, symptoms occur frequently over months or years.

A vascular tumor is a soft tissue growth. It is usually caused by excess growth of blood vessels. Vascular tumors can be malignant or benign.

3. Literature review

Transfer learning [15] and generative adversarial networks are a prevalent deep learning method in industry and academia. They have also received attention in the field of medical image processing [10, 2, 5].

Kazeminia et al. [5] analyzed applications of GANs in the medical field and provided a comprehensive overview of available approaches that use GANs. The main identified benefits of GANs are in enlarging datasets and exploitation in semi- and unsupervised settings. The main disadvantages based on their analysis are the trustworthiness of the generated data and the unstable training of standard GANs. Similarly, Xun et al. [10] identified instability, low repeatability and low interpretability as the main drawbacks based on an analysis of 120 GAN-based architectures for medical image segmentation.

Bissoto et al. [2] conducted a detailed study of GAN-based image synthesis for skin lesion detection and classification. Their work showed that data augmentation with GANs provides favorable results only on test sets with anomalous inputs, and they advise careful application for medical image processing.

Qin et al. [11] constructed a framework for the analysis of medical images and skin lesion classification. They also used GAN-based data synthesis. They proposed a deep learning-based classifier, which improved the overall classification results. However, their method encountered mode monotony for some categories in generated images. Similarly, we also address improving the performance of transfer learning models with augmented data.

Lei et al. [12] proposed GANs with dual discriminators for addressing the problem of data synthesis in the skin lesion segmentation task.

Their novel GAN demonstrated superior performance on the skin lesion dataset from the ISIC 2017 challenge compared to state-of-the-art methods. Their approach’s main challenge was representing images with exceedingly obscure boundaries, low contrast, and noise. Due to their remarkable performance, GANs have attracted much attention in the scientific community [8, 16, 17, 18, 19, 9, 9].

Recent advances in GANs have overcome well-known problems such as training instability and mode collapse [8, 19]. The versatility and robustness of GANs have also been improved [18].

Armandpour et al. [19] introduced partition-guided GANs, which divide the data space into smaller regions to overcome the mode collapse problem. These partitions have simpler distributions and alleviate the mode collapse problem for parts of the data with nonexistent density. There have also been advances toward utilizing the latent space interpretation of GANs for semantic editing and useful direction determination [20].

Additionally, for image generation, GANs spontaneously learn multiple interpretable features in the latent space, e.g., the gender of a face or the lighting conditions in the case of scene synthesis.

Recent methods are capable of identifying versatile semantics from GAN networks of different types in an unsupervised learning setup [16]. Tritrong et al. [17] showed that GANs can be used not only for synthesis but also for few-shot semantic part segmentation.

They set up novel GANs for unsupervised representation learning for problems involving scene semantics, object part reasoning, and, alternatively, prediction using pixel-wise already-discriminative representations obtained from GAN processes of synthesis.

Lin et al. [9] introduced a scalable training process for training unconditional GANs. The main benefit of their approach is that it adapts to different latency requirements coming from hardware or a user.

4. Proposed approach

The proposed approach is presented in Fig. 2. In this section, we describe image synthesis with StarGAN2 in detail and present an overview of the classifier construction.

Generate With StarGAN v2 Input: Set of images $X$ , Set of domains $Y$ ,Output: Set of generated images $X_{G}$ [1] number of epochs $\mathbf{x}\in X$ $s\leftarrow E_{y}(\mathbf{x})$ { Style encoder $E$ extracts the style of the input $\mathbf{x}$ } randomly sample a latent code $\mathbf{z}\in Z$ randomly sample the target domain $\tilde{y}\in Y$ generate the style code $\mathbf{s}\leftarrow F_{\tilde{y}}(\mathbf{z})$ { Mapping network $F$ generates the style} Generator $G$ generates the image $G(\mathbf{x},\mathbf{\tilde{s}})$ Discriminator $D$ creates outputs $D(\mathbf{x})$ and $D(\mathbf{G(\mathbf{x},\mathbf{\tilde{s}})}$ update parameters of G, D, F and E $\leftarrow\underset{G,F,E}{\text{min}}\,\underset{D}{\text{max}}\,(\mathcal{L}% _{adv}+\lambda_{sty}\mathcal{L}_{sty}-\lambda_{ds}\mathcal{L}_{s}+\lambda_{cyc% }\mathcal{L}_{cyc}$ ) {maxmin of full objective} $\mathbf{x}\in X$ $X_{G}\leftarrow X_{G}\cup G(\mathbf{x},\mathbf{\tilde{s}})$

4.1 Image synthesis

This section briefly introduces the concepts of StarGAN v2. Full details can be found in the paper introducing StarGAN v2 by Choi et al. [14].

A general GAN consists of two neural networks: a generator ${C}$ and a discriminator ${D}$ . The generator network ${C}$ maps a random vector $z$ from a prior distribution to images. The goal of the discriminator network ${D}$ is to differentiate the actual data ${x}$ from the data synthesized by the generator C. A minimax operation with the generator $C$ and the discriminator D is used to optimize the GAN. In StarGAN v2, the generator ${C}$ translates an input ${X}\in{X}$ into an output $C(X,s)$ , where $s$ represents the style for a domain $y\in Y$ , which enables C to generate images for all domains. Adaptive instance normalization (AdaIN) [21] is used to implant $\mathbf{s}$ into generator $C$ thereby avoiding the necessity of directly inputting $y$ into the generator network $C$ . To enable this, a mapping network and a style encoder are introduced.

[h] Face Skin Disease Prediction Input: Set of images for training $X_{\textit{train}}$ , Set of images for testing $X_{\textit{test}}$ , Set of corresponding classes $Y$ , Set of pre-trained classifier models $\mathcal{C}$ ,Output: High accuracy classifier for face skin diseases $C^{*}$ [1] $X_{\textit{generated}}\leftarrow\textit{generateWithStarGANv2}(X_{\textit{% train}},Y)$ $X_{\textit{augmented}}\leftarrow\textit{classicalImageAugmentation}(X_{\textit% {train}}\cup X_{\textit{generated}},Y)$ $\textit{Accuracy}_{C*}\leftarrow 0$ {Initialization} $\textit{Accuracy}_{C*}\leftarrow 0$ {Initialization} $C_{i}\in\mathcal{C}$ $C_{i}^{r}\leftarrow\textit{retrain}(C_{i},X_{\textit{train}}\cup X_{\textit{% generated}}\cup X_{\textit{augmented}},Y)$ {Obtain classifiers with Transfer Learning} $\textit{Accuracy}_{ci}\leftarrow\textit{test}(C_{i}^{r},X_{\textit{test}})$ $\textit{Accuracy}_{ci}>\textit{Accuracy}_{c*}$ $\textit{Accuracy}_{c*}\leftarrow\textit{Accuracy}_{ci}$ $C^{*}\leftarrow C_{i}^{r}$ ${F}$ is the mapping network that creates a style code ${S}$ $=F_{y}(z)$ for a corresponding domain y from a given latent code ${Z}$ . The network ${F}$ , which consists of a multiple-layer perceptron with output branches for each domain, creates various style codes. This is done by randomly sampling the domain $y\in Y$ and the latent vector $z\in Z$ . The encoder $E_{y}$ corresponding to the domain $y$ is used to extract the style $s=E_{y}(x)$ With different reference images, $E$ produces diverse style codes that enable ${C}$ to generate an output that reflects the style of a reference image.

The StarGAN v2 discriminator is a multitask discriminator where each output branch $D_{y}$ is a binary classifier deciding if an image is from the original dataset or it is an image produced by $G(\mathbf{x},\mathbf{s})$ . The training objective is a minmax operator where objective function $V(G,D)$ is defined with

$\displaystyle\underset{G,F,E}{\text{min}}\,\underset{D}{\text{max}}\,\mathcal{% L}_{adv}+\lambda_{sty}\mathcal{L}_{sty}-\lambda_{ds}\mathcal{L}_{s}+\lambda_{% cyc}\mathcal{L}_{cyc}$ (1)

where $\lambda_{sty},\lambda_{ds},\lambda_{cyc}$ are hyperparameters and corresponding terms are defined as follows.

An adversarial objective $\mathcal{L}_{adv}$ enables that the mapping network creates a style $\mathbf{\tilde{s}}=F_{\tilde{y}(\mathbf{z})}$ and that the generator $G$ learns to use $\mathbf{\tilde{s}}$ to synthesise an image which is similar to original images of the domain $\tilde{y}$ . It is defined as $\mathcal{L}_{adv}=\mathbb{E}_{\mathbf{x},y}[\text{log}D_{y}(\mathbf{x})]+% \mathbb{E}_{\mathbf{x},\tilde{y},\mathbf{z}}[\text{log}(1-D_{\tilde{y}}(G(% \mathbf{x},\tilde{\mathbf{s}})))].$ The style reconstruction loss $\mathcal{L}_{sty}$ enforces the usage of the style code $\tilde{\mathbf{s}}$ while synthesising of the $G(\mathbf{x},\tilde{\mathbf{s}})$ and is defined as $\mathcal{L}_{sty}=\mathbb{E}_{\mathbf{x},\tilde{y},\mathbf{z}}[||\tilde{% \mathbf{s}}-E_{\tilde{y}}(G(\mathbf{x},\tilde{\mathbf{s}}))||_{1}].$ The cycle consistency loss $\mathcal{L}_{cyc}$ ensures preserving domain invariant characteristics of its input and is defined as $\mathcal{L}_{cyc}=\mathbb{E}_{\mathbf{x},y,\tilde{y},\mathbf{z}}[||{\mathbf{x}% }-G(G(\mathbf{x},\tilde{\mathbf{s}}),\hat{\mathbf{s}})||_{1}].$ . For the input $\mathbf{x}$ and the original domain $y$ $\hat{\mathbf{s}}=E_{y}(\mathbf{x})$ is the estimation of style code.

Table 1

Overview of the transfer learning models

Method	Parameters	Depth
VGG19	143,667,240	26
VGG16[23]	138,357,544	23
ResNet50	25,636,712	168
DenseNet121	8,062,504	121
DenseNet201	20,242,984	201
InceptionV3	23,851,784	159
MNASNet	5,326,716	NA
NASNetLarge	88,949,818	NA

Table 2

Additional data augmentation and preprocessing steps

Operation	Parameters values
Random Horizontal Flip	Probability $p=$ 0.20
Color Jitter	Brightness $=$ 0.25
Gaussian Blur	Kernel size: 3
Normalize ( $\mu,\sigma$ )	[0.485, 0.456, 0.406],
	[0.229, 0.224, 0.225]

4.2 Classifier construction

We built classifiers using the transfer learning approach, which enables the utilization of knowledge from a previously trained source model to train a new model more rapidly and with better performance. This is especially beneficial when we have several domains of interest that potentially have different distributions and might lie in different feature spaces and we do not have sufficient training data in some of these domains. In these cases, knowledge transfer improves the classification performance. In our scenario of skin disease classification, this knowledge transfer was performed by a pretrained ImageNet [22] model. We examined several classification models. An overview of these models is presented in Table 1. To train classifiers on the newly obtained dataset consisting of the original data and synthesized data, which was still relatively small, additional classical computer vision data augmentation was necessary to increase the dataset size. This included the following data augmentation and preprocessing steps:

•
Flipping is a type of image data augmentation in which each image from the training set is flipped horizontally with a given probability. This rearranges the original image pixels $I(x,y)$ while protecting the image features such that $x^{\prime}=$ $-x,y^{\prime}=y$ , where $I\left(x^{\prime},y^{\prime}\right)$ is a transformed augmented image.

Table 3
StarGAN v2 network architecture details

Generator Latent dimension 16

Style dimension 64

Number style 5

Mapping net Hidden dimension 512

Discriminator Spectral normalization False

Dataset number 1060

Parameters Generator 33,892,995

Discriminator 20,853,829

Training Batch size 8

Maximum iteration 25,000

Ds iteration 100,000

Figure 4.
An example of reference-guided image generation results. In the first row are given the source images and in the left column are given the reference images.

•
Color Jitter is a data augmentation method in which the brightness, contrast and saturation of an image are randomly changed with a specified probability by changing the RGB values.

4.3 StarGan v2 training details

The model was trained for 25,000 epochs with a batch size of eight. It was implemented in PyTorch with a learning rate of $1{e}-4$ using the Adam optimizer. For running experiments, the execution platform Google Colab Pro+ was used, where two Tesla V100 GPUs were utilized. The execution time for 25,000 epochs of StarGAN V2 was 23 hours and 17 minutes. The weights described in Eq. (1) were set as suggested by the original implementation of StarGAN v2 [14]. The network architecture is summarized in Table 3.

5. Experimental results

This section describes the evaluation setup and conducted experiments.

Table 4
Transfer learning models trained with StarGAN v2

TL models	Accuracy with StarGAN v2
ResNet50	99.715%
VGG16	98.01%
VGG19	99.85%
Inception V3	83.90%
NasNetLarge	86.26%
DenseNet201	82.00%
DenseNet121	83.33%
MNasNet	77.46%

Figure 5.

Accuracy and loss over 25 epochs for the transfer learning classification model using STARGAN v2 augmented images.

Transfer Learning Training Details Generated data and the original data were used for transfer learning models. The train-test split was 67%–33%. The models were trained for 25 epochs. The average execution time for 25 epochs of 8 transfer learning models was 20 minutes and 19 seconds with implementation in PyTorch using the same execution platform as for StarGAN v2. The additional data augmentation and preprocessing steps are summarized in detail in Table 2.

Figure 6.

Overall results of the StarGAN v2 generation of latent representations. In each row, the source image is on the left, while the remaining images are synthesized images.

Figure 7.

Overall results of StarGAN v2 in reference-guided image generation. In the first row, the source and reference images are presented, and in the first column, the original images are presented, while the remaining images are synthesized images.

Evaluation MetricsEvaluation of GAN-based models is challenging since no general and accurate approach is available for it [11]. For evaluation, we adopted a common method using the Frechét inception distance (FID) [14] which is defined as $\text{FID}=||\mu-\mu_{w}||^{2}_{2}+tr(\Sigma+\Sigma_{w}-2(\Sigma^{1/2}\Sigma_{% w}\Sigma^{1/2})^{1/2}$ where $\mathcal{N}(\mu,\Sigma)$ where is the distribution of NN features of the images synthesized by the GAN and NN( $\left(\mu{w},{E}_{{w}}\right)$ is the distribution of the same NN features from the real images. The validation loss and accuracy were used to evaluate the transfer learning models.

Image synthesis evaluation was conducted from two perspectives: latent-guided synthesis, where the method produces multiple outputs using random noise, and reference-guided synthesis, where style code is obtained from a reference image. Final results of reference-guide image synthesis are shown in Fig. 4, while Figs 6 and 7 give an overview of output changes over the increase of epochs.

Classification evaluation was done by obtaining the classification accuracy on the dataset expanded with augmented data from StarGAN v2 (training set size: 5917 images, validation set size: 350 images). We compare the accuracy results of classification outputs from 8 different transfer learning models in Table 4.

6. Conclusion

This article proposes a framework for medical image synthesis and classification. Face skin images are generated via the StarGAN v2 data augmentation technique, and a skin condition classifier is created based on the transfer learning approach. The generated images are used together with the original data to improve the classifier’s performance. Changes in the classification network’s accuracy and loss of training and validation are visualized in Fig. 5.

The proposed approach using StarGAN v2 successfully renders distinctive styles. The style codes of the references are captured from face skin images representing a specific disease, and new realistic images are synthesized, as shown in the experimental results. High-level semantics, such as skin condition features, are inferred from the reference images, and the identity and pose are obtained from the source images. The VGG19 model showed the highest overall validation accuracy of 99.85%.

Future work will integrate trained models into a robotic system (Misty 2) for online skin disease detection and recognition. We will apply our attribution-based confidence metric [24] for detecting adversarial attacks on skin cancer detection systems.

Footnotes

Acknowledgments

This material is based upon work supported by the Henry Luce Foundation – Clare Boothe Luce Fund. The authors acknowledge the support provided by Dave Reed, Carol Zuegner, Catherine Baker, Joi Katskee, and all the faculty members from the Department of Computer Science, Design, and Journalism, Creighton University.

References

Ballerini

Fisher

Aldridge

Rees

. Non-melanoma skin lesion classification using colour image data in a hierarchical K-NN classifier. Proceedings – International Symposium on Biomedical Imaging. 2012; 5; 358-361.

Bissoto

Valle

Avila

. GAN-Based Data Augmentation and Anonymization for Skin-Lesion Analysis: A Critical Review. ISIC Skin Image Analysis Workshop at CVPR 2021; 2021.

Karras

Laine

Aittala

Hellsten

Lehtinen

Aila

. Analyzing and Improving the Image Quality of StyleGAN: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2020. pp. 8107-8116.

Menegola

Fornaciali

Pires

Bittencourt

Avila

Valle

. Knowledge transfer for melanoma screening with deep learning. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017); 2017. pp. 297-300.

Kazeminia

Baur

Kuijper

van Ginneken

Navab

Albarqouni

, et al. GANs for medical image analysis. Artificial Intelligence in Medicine. 2020; 109: 10193 8. Available from: https//www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0933365719311510.

Karras

Laine

Aila

. A Style-Based Generator Architecture for Generative Adversarial Networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Chang

Wang

Peng

Chiu

. All About Structure: Adapting Structural Information Across Domains for Boosting Semantic Segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20 2019. Computer Vision Foundation/IEEE; 2019. pp. 1900-1909. Available from: shttp//openaccess.thecvf.com/content_CVPR_2019/html/Chang_All_About_Structure_Adapting_Structural_Information_Across_Domains_for_Boosting_CVPR_2019_paper.html.

Hyun

Kim

Heo

. Self-Supervised Video GANs: Learning for Appearance Consistency and Motion Coherency. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, 2021. pp. 10821-10830. Available from: doi: 10.1109/CVPR46437.2021.01068.

Lin

Zhang

Ganz

Han

Zhu

. Anycost GANs for Interactive Image Synthesis and Editing. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2021. pp. 14981-14991.

10.

Xun

Zhu

Chen

Wang

, et al. Generative adversarial networks in medical image segmentation: A review. Computers in biology and medicine. 2021 November; 140: 105063. doi: 10.1016/j.compbiomed.2021.105063.

11.

Qin

Liu

Zhu

Xue

. A GAN-based image synthesis method for skin lesion classification. Computer Methods and Programs in Biomedicine. 2020; 195: 105568.

12.

Lei

Xia

Jiang

, et al. Skin lesion segmentation via generative adversarial networks with dual discriminators. Medical Image Analysis. 2020; 64: 101716. Available from: https//www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S1361841520300803.

13.

Rajinikanth

Satapathy

Dey

Fernandes

Manic

. Skin melanoma assessment using Kapurâ€™s entropy and level set – a study with bat algorithm. In: Smart intelligent computing and applications. Springer; 2019. pp. 193-202.

14.

Choi

Yoo

. StarGAN v2: Diverse Image Synthesis for Multiple Domains. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020; pp. 8185-8194.

15.

Hosny

Kassem

Foaud

. Skin Cancer Classification using Deep Learning and Transfer Learning. In: 2018 9th Cairo International Biomedical Engineering Conference (CIBEC); 2018. pp. 90-93.

16.

Shen

Zhou

. Closed-Form Factorization of Latent Semantics in GANs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021; pp. 1532-1540.

17.

Tritrong

Rewatbowornwong

Suwajanakorn

. Repurposing GANs for One-Shot Semantic Part Segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2021. pp. 4475-4485.

18.

Wang

Chen

Zhou

Loy

. Positional Encoding As Spatial Inductive Bias in GANs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2021. pp. 13569-13578.

19.

Armandpour

Sadeghian

Zhou

. Partition-Guided GANs. In: CVPR 2021; 2021. Available from: https//www.microsoft.com/en-us/research/publication/partition-guided-gans/.

20.

Yang

Chai

Wen

Zhao

Sun

. Discovering Interpretable Latent Space Directions of GANs Beyond Binary Attributes. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society; 2021. pp. 12172-12180. doi: 10.1109/CVPR46437.2021.01200.

21.

Huang

Belongie

. Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. 2017 IEEE International Conference on Computer Vision (ICCV). 2017. pp. 1510-1519.

22.

Deng

Dong

Socher

Fei-Fei

. Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee; 2009. pp. 248-255.

23.

Simonyan

Zisserman

. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR.2014abs/ 1409.1556. Available from: http//arxiv.org/abs/1409.1556.

24.

Jha

Raj

Fernandes

Jha

Jalaian

, et al. Attribution-based confidence metric for deep neural networks. Advances in Neural Information Processing Systems. 2019; 32.

Generator	Latent dimension	16
	Style dimension	64
	Number style	5
Mapping net	Hidden dimension	512
Discriminator	Spectral normalization	False
	Dataset number	1060
Parameters	Generator	33,892,995
	Discriminator	20,853,829
Training	Batch size	8
	Maximum iteration	25,000
	Ds iteration	100,000

Facial skin disease prediction using StarGAN v2 and transfer learning

Abstract

1. Introduction

3. Literature review

4. Proposed approach

4.1 Image synthesis

5. Experimental results

Table 4 Transfer learning models trained with StarGAN v2

Footnotes

Acknowledgments

References

Table 4
Transfer learning models trained with StarGAN v2