Multi-variants vision transformer-based malware image classification model using multi-criteria decision-making

Abstract

Visualization-based malware detection gets more and more attention for detecting sophisticated malware that traditional antivirus software may miss. The approach involves creating a visual representation of the memory or portable executable files (PEs). However, most current visualization-based malware classification models focus on convolution neural networks instead of Vision transformers (ViT) even though ViT has a higher performance and captures the spatial representation of malware. Therefore, more research should be performed on malware classification using vision transformers. This paper proposes a multi-variants vision transformer-based malware image classification model using multi-criteria decision-making. The proposed method employs Multi-variants transformer encoders to show different visual representation embeddings sets of one malware image. The proposed architecture contains five steps: (1) patch extraction and embeddings, (2) positional encoding, (3) multi-variants transformer encoders, (4) classification, and (5) decision-making. The variants of transformer encoders are transfer learning-based models i.e., it was originally trained on ImageNet dataset. Moreover, the proposed malware classifier employs MEREC-VIKOR, a hybrid standard evaluation approach, which combines multi-inconsistent performance metrics. The performance of the transformer encoder variants is assessed both on individual malware families and across the entire set of malware families within two datasets i.e., MalImg and Microsoft BIG datasets achieving overall accuracy 97.64 and 98.92 respectively. Although the proposed method achieves high performance, the metrics exhibit inconsistency across some malware families. The results of standard evaluation metrics i.e., Q, R, and U show that TE3 outperform the TE1, TE2, and TE4 variants achieving minimal values equal to 0. Finally, the proposed architecture demonstrates a comparable performance to the state-of-the-art that use CNNs.

Keywords

Vision transformer MCDM VIKOR MEREC image malware classifier

1 Introduction

Malware “malicious software”, refers to any software that has been intentionally created to inflict damage upon computer systems, networks, cloud etc. Malware can manifest in various forms, such as adware, spyware, ransomware, trojans, viruses, and worms [18]. Recent instances of malware include Trickbot, Emotet, Ryuk ransomware, Conti ransomware, LokiBot, Azorult, etc. Malware detection methods are used to identify and prevent malicious software from infecting computers, networks, and other digital devices. Signature-based and anomaly-based approaches are two categories of techniques used to detect malware. Signature-based techniques compare the digital signatures of known malware with the code on the system to find out a match whereas anomaly-based techniques monitor the behaviour of a program using machine learning to detect anomalies that may indicate malicious activity [3]. However, signature-based techniques have a significant limitation in that they are unable to detect new or unknown malware threats that do not match any existing signatures in the database. This means that these techniques can be easily bypassed by attackers using new or modified malware code that is not yet recognized by security experts [3]. Static analysis and dynamic analysis are two techniques for malware analysis. Static analysis is used as a first step to detect suspicious activity, it examines the code and structure of the malware without executing it. This can be done using various tools, such as disassemblers and decompilers, to reverse engineer the code and understand its behaviour. The drawbacks of static analysis are the difficulty in identifying obfuscated code, and the Inability to analyze encrypted or polymorphic malware. On the other hand, dynamic analysis is used as a second step to detect suspicious activity, it runs the malware in a controlled environment and observes its behaviour. This can be done using virtual machines or sandbox environments that isolate the malware from the rest of the system. By monitoring the malware’s behaviour, analysts can identify its malicious activities, such as attempts to steal data or launch attacks on other systems [17]. As dynamic malware analysis relies on ML-based techniques, improper feature engineering and false positives could adversely affect the accuracy of malware classification leading to security breaches or unnecessary alerts [17, 25].

1.1 Motivation

In recent studies, visualization-based malware detection gets more and more attention for detecting sophisticated malware that traditional antivirus software may miss [17]. The approach involves creating a visual representation of the memory or portable executable files (PEs) and then, identifying fundamental characteristics of malware code that can be captured in a visual representation. As per these characteristics, the classification of code as either benign or malicious can be accomplished using DL models. visualization-based malware detection is a dynamic technique that can detect unknown and obfuscated malware variants without a need for feature engineering [25]. Convolutional neural networks (CNNs) are the primary dynamic technique used in most recent studies for malware detection based on visualisation [18 , 34]. Convolutional neural networks (CNNs) examine the visual representation of malware code and detect malicious patterns by extracting features from the input image. Nevertheless, CNNs have certain limitations in detecting minute distinctions among different malware versions [34]. Consequently, they may fail to detect slight modifications in malicious code that could have an impact on the malware’s behaviour [34].

Though ViT [12] has a higher performance than CNNs in image classification [7, 26] and maintains the spatial representation of malware [28], a few researches are performed on malware classification using a vision transformer [4].

1.2 Research Objectives

This paper proposes a multi-variants vision transformer-based malware image classification model using multi-criteria decision-making. The key contributions of the proposed method can be outlined as follows.

The proposed architecture is a visualization-based method to detect obfuscated and packed malware that may be hidden within legitimate software. This approach eliminates the need for feature engineering or domain experts.

The proposed method employs Multi-variants transformer encoders to show different visual representation embeddings sets of one malware image. These transformer encoders are based on transfer learning, as they were initially trained on the large-scale ImageNet dataset.

The proposed malware classification model adopts a hybrid standard evaluation approach, known as MEREC-VIKOR, which combines various evaluation criteria to provide a comprehensive and reliable assessment of the classifier’s performance.

The performance of the transformer encoder variants is assessed both on individual malware families and across the entire set of malware families within two datasets i.e., MalImg and Microsoft BIG datasets. Furthermore, a comparison has been made between the proposed classifier and other state-of-the-art malware classification approaches.

1.3 Organization

The subsequent sections of the paper are structured as follows: Section 2 presents a literature review on malware classification models. The architecture and the phases of the proposed visualization-based malware classification method are discussed in section 3. Section 4 discusses the method’s performance on individual malware families and across the entire set of malware families within two datasets i.e., MalImg and Microsoft BIG datasets besides, a comparative analysis with the other state-of-the-art approaches. Section 5 concludes the paper and shows the limitations and future scopes.

2 Related work

Static methods for detecting malware are employed to identify malicious software without running it. It analyzes the malware code and other static characteristics such as signatures, file headers, code structure and syntax. Liangboonprakong et al. [20] use N-grams sequential pattern features to classify malware by disassembling the binary executable into a hexadecimal string and looking for frequently occurring sequences. Avdiienko et al. [2] use data flow analysis to detect Android malicious apps by mining the data flow for benign Android apps using FlowDroid. Narouei et al. [24] propose a technique that can detect malware accurately and resist code obfuscation or injection into legitimate software. Naik et al. [22] involve using fuzzy and import hashing techniques to improve the accuracy of malware detection. Although static malware detection techniques are useful for performance analysis, they have limitations such as being vulnerable to code obfuscation, encrypted or polymorphic malware.

Dynamic methods for detecting malware are employed to detect malicious code while it is running. These methods analyze the code’s behaviour and collect some behavioural features such as API calls, system calls, and resources’ status which help to identify the code whether malicious or benign [17]. Li et al. [1] developed a DL-based framework for detecting malware. First, the software behaviour is represented through embedding several APIs then, the semantic features of API calls are represented. Finally, Bi-LSTM is used to extract the relationship among APIs. Li et al. [19] introduced the DMalNet approach for dynamic malware detection, which extracts semantic features from API arguments and names using a hybrid encoder and then represents the relationship between API calls in a graph structure. While detection approaches using API calls have shown high accuracy in detecting malware, their complexity makes them difficult to implement in real-time environments. As a result, recent studies have proposed detection approaches using hardware to enhance the performance of detection models or frameworks [33]. Chen et al. [9] proposed a DL-based malware detector based on traces of the target software’s control flow. processors that support Intel Processor Trace are employed to minimize the overhead of execution tracing. Tian et al. [33] introduced a runtime detection technique namely, MDCHD which examines the Processor traces to collect the target software’s control flows. Unfortunately, dynamic malware detection techniques are not scalable as they require significant resources and are specific to the environment in which they are executed [17]. Also, improper feature engineering and false positives could adversely affect these models’ performance [17].

Recent research has shown that visualization-based techniques are gaining popularity in detecting sophisticated malware that traditional techniques fail to detect [17]. These methods are cost-effective, accurate, and require fewer human resources as they eliminate the need for feature extraction [25]. Falana et al. [13] presented a virtualization-based method for malware detection that uses an ensemble method for feature extraction comprising Deep-GAN and Deep-CNN. The features are extracted from RGB images that are obtained from malware binary files. [34] introduced a malware classification approach, IMCFN, that generates the RGB visual representation of malware binary sequences, and then uses a fine-tuned CNN model to classify them. Kumar et al. [17] suggested a malware classifier (DTMIC) based on pre-trained CNN. DTMIC show the grayscale visual representation of PEs which are then classified into their respective malware families. Vasan et al. Makandar et al. [21] create a texture features vector using the multi-resolution wavelets based on the discrete wavelet transform GIST and the Gabor wavelet, etc. Then, multi-class SVM is used to classify the malware. Landman et al. [18] developed a framework named Deep-Hook that applied for unknown malware detection in a Linux-based cloud. Deep-Hook shows the visual representation of the memory dump sequences of a VM that are captured while it is operating. Narayanan et al. [23] produce the visual representation of malware and viruses, and PCA is used for feature extraction. Then, the malware is classified based on a hybrid model. Tian et al. [33] introduced a visualization approach called MDCHD for detecting malware in VM environments. MDCHD generates the RGB visual representation of the target software’s control flow which is captured using IPT method at runtime. Then CNN model is used to detect them.

According to recent studies, Convolutional Neural Network (CNN) is a powerful tool frequently utilized in malware detection. However, CNNs may have difficulty distinguishing small differences and subtle changes between malware types, which leads to misclassified malware [34]. Many recent studies have employed attention mechanisms in conjunction with CNN models to improve their performance. Malbert, an approach for malware detection, was proposed by Xu et al. [36] in which a pre-trained DL model consists of 12 transformers, each comprising an attention layer followed by a DNN. Zhang et al. [37] introduced a static analysis method for ransomware detection using self-attention-based CNN and N-gram opcodes. Dosovitskiy et al. [12] introduced a vision transformer (ViT) approach, which employs an attention mechanism to classify images. ViT has been successfully employed in various domains, demonstrating superior performance compared to other methods. Xu et al. [35] used ViT to construct a framework for multi-task classification, which analyzes MR images and simultaneously predicts four glioma molecular expressions. Haurum et al. [15] classified sewer defects using a multi-scale hybrid ViT approach. Okolo et al. [26] proposed an improved ViT architecture called IEViT to classify chest X-ray images.

Though ViT [12] has a higher performance than CNNs in image classification [7, 26] and maintains the spatial representation of malware [28, 31] a few research is performed on malware classification using a vision transformer. SHERLOCK [29] is a method for Android malware detection presented by SENEVIRATNE et al. A Self-Supervised ViT that has been trained on a sizable corpus of Android application (APK) files is used in the procedure. SHERLOCK determines whether or not an APK file contains malware by capturing the representations of APK files. An improved ViT [27] was proposed by Park et al. for classifying the malware. Multiple patch encodings are used in this method to capture positional information of both global and local characteristics. Belal et al. [4] proposed butterfly vision transformer which relies on global-local attention-based for Visualization-based malware classification/detection. In this paper proposes and analyzes the performance of a multi-variant vision transformer-based malware image classification model using multi-criteria decision-making.

3 Methodology

3.1 Generating grayscale malware images

In our approach, the binary sequences associated with Portable Executable (PE) files are transformed into grayscale malware images without the need for expert knowledge or feature engineering. These grayscale images serve as inputs for the proposed model, enabling malware classification. The malware visualization process is outlined in Algorithm 1. Initially, the binary PE import sequence is divided into s octets, each representing a pixel in the gray malware image. Following this, the binary pixel values are converted into 8-bit unsigned integers, and normalized within the range [0, 1]. After normalizing the pixel values, they are structured into a vector then, this vector is reshaped into a 2D gray malware image with dimensions (h×w) and a single channel.

Algorithm(m). alware visualization process

1: Input: M_B ← {0, 1} ^k: Malware binary stream

2: output: M_2D ∈ R^h×w×c.

3: N ← k/8

4: M_1D ← []

5: For j : 0 → (N - 1)

6: R_j ← M_B [j * 8 +1] , . . . . , M_B [j * 8 +8]

7: $\bar{R}$ _j ← uint (R) $\bar{R}$ _j ∈ [0, 255]

8: $\bar{\bar{R}}$ _j ← Norm ( $\bar{R}$ _j) $\bar{\bar{R}}$ ∈ [0, 1]

9: END For

10: M_1D ← [ $\bar{\bar{R}}$ ₀, $\bar{\bar{R}}$ ₁, . . . . , $\bar{\bar{R}}$ _N-1]

11: [M_2D] _h×w ← [M_1D] _N×1 x ∈ R^h×w, M_1D ∈ R^N×1

3.2 The proposed multi-variant vision transformer-based malware image classification model

This manuscript proposes a multi-variant vision transformer-based malware image classification model using multi-criteria decision-making. The Multi-variants transformer encoders are employed to show different visual representation embeddings sets of one malware image. Moreover, the proposed malware classifier adopts a hybrid standard evaluation approach, known as MEREC-VIKOR, which combines various evaluation criteria to provide a comprehensive and reliable assessment of the classifier’s performance. The block diagram of the multi-variant vision transformer-based malware image classification model is illustrated in Fig. 1. The proposed architecture takes a grayscale image of malware as its input, and its output is the classification of the malware image. It contains five steps: (1) patches extraction and embeddings, (2) positional encoding, (3) multi-variants transformer encoders, (4) classification, and (5) decision-making. In the sections below, we’ll go through each phase’s details.

Fig. 1

Multi-variants vision transformer-based malware image classification model.

3.2.1 Patches extraction and embeddings

This is the first step in the proposed malware classifier which takes a grayscale image of malware M ∈ R^{D₁×D₂×C} as an input with (D1 × D2) dimensions; C is no. of channels. Then, the malware image is divided into K smaller sections, known as patches M⁽ⁱ⁾ ∈ R^{D₃×D₃×C} ; i = 1, . . . , N using a sliding window approach; $K = \frac{D_{1} \times D_{2}}{D_{3} \times D_{3}}$ as shown in Equation 1. Each patch is square in shape (D3 × D3) defined by a fixed size and non-overlapped with adjacent patches. After that, patch embeddings PE⁽ⁱ⁾ ; i = 1, . . . , N are computed for each patch. A patch embedding is a vector representation that encodes the visual features of the patch. As shown in Equation 2, to compute patch embeddings, the patches are flattened into a 1D vector ∈ R^{D₃.D₃.C×1}. Then, these vectors are mapped into D-dimension vector space R^D×1 using trainable linear projection.

$\begin{matrix} M \to [M^{(1)}, M^{(2)}, . . . . ., M^{(K)}]; M \in R^{D_{1} \times D_{2} \times C} \\ M^{(i)} \in R^{D_{3} \times D_{3} \times C} andi = 1, . . ., N \end{matrix}$ (1)

${PE}^{(i)} = Θ (Ψ (M^{(i)})); i = 1, . . ., N$ (2) where Θ is a Dense layer with D units which represents the linear projection function and Ψ is the flatten function.

As shown in Fig. 1, the malware image is partitioned using two different patch sizes (D3 × D3) = (16 × 16) and (32 × 32), where the no. of patches are equal to k1 and k2 respectively. Then, the (16 × 16) patch embeddings are computed by mapping the flattened patches into 768 and 1024 dimension vector space using Θ function. Similarly, the (32 × 32) patch embeddings are computed by mapping the flattened patches into 768 and 1024 dimension vector space using Θ function.

3.2.2 Positional encoding

Since the malware image patches are unordered and do not contain explicit positional information, it is necessary to incorporate positional encoding into the patches’ embeddings to provide spatial information to the transformer encoders and improve their ability to recognize complex visual patterns. Positional encoding is achieved by adding the patches’ spatial position embeddings pos⁽ⁱ⁾ . E to the patches’ embeddings PE⁽ⁱ⁾ as shown in Equation 3. The patch positions pos⁽ⁱ⁾ are mapped into D-dimension vector space (same size as patch embeddings) to generate the position embeddings pos⁽ⁱ⁾ . E.

$\begin{matrix} Z_{i} = {PE}^{(i)} + {pos}^{(i)} . E; pos . E \\ \in R^{K \times D}, i = 1, . . . ., N \end{matrix}$ (3)

Then, the class embedding $M_{class}^{(0)}$ is mapped into D-dimension vector space and added to the zero position embedding pos⁽⁰⁾ and pre-appended to the malware image patches embeddings as shown in Equation 4.

$\begin{matrix} Z = [Θ (M_{class}^{(0)}) + {pos}^{(0)} . \\ E | | Z^{(1)} | | . . . . | | Z^{(k)}]; Z \in R^{K \times D} \end{matrix}$ (4) where Θ is a Dense layer with D units which represents the linear projection function.

As shown in figure 1, there are four groups of patches’ embeddings, each group encodes the visual features of the same malware image with different variants. Positional encoding for one group is achieved by adding the patches’ spatial position embeddings to the patches’ embeddings of the respective group.

3.2.3 Multi-variants transformer encoders

The transformer encoder is a key block of ViT architecture, which is a deep-learning model used for image classification tasks. The transformer encoder block is stacked into l layers. The architecture of the transformer encoder is shown in Fig. 2. It consists of multi-head self-attention and a feedforward neural network, both are followed by layer normalization aids in stabilizing the training process and enhancing the overall performance of the model. The self-attention component uses multi-head H, which enables the model to capture different representations of the patches’ embeddings in parallel. Then, the final attended vector is obtained by concatenating and linearly projection the output vectors of heads. The feedforward neural network with two hidden layers, applies a non-linear transformation on the final attended vector.

Fig. 2

Transformer encoder architecture.

As shown in figure 1, the proposed architecture utilizes four transformer encoders with four different variants. The variants of transformer encoders are shown in Table 1.

Table 1

The variants of transformer encoders

	Patch size	D	L	H	MLP size
Transformer encoder 1	(16×16)	768	12	12	(3072×768)
Transformer encoder 2	(32×32)	768	24	12	(3072×768)
Transformer encoder 3	(16×16)	1024	12	16	(4096×1024)
Transformer encoder 4	(32×32)	1024	24	16	(4096×1024)

The proposed architecture has two transformer encoders, each with 12 attention heads and an MLP size of (3072× 768). The input data comprises a partitioned malware image with two different sizes, one of which is partitioned into 16x16 patches, while the other is partitioned into 32x32 patches. The patches are then represented as embeddings and mapped to a 768-dimensional space. The 12-layer transformer encoder addresses the 16x16 patches, while the 24-layer transformer encoder addresses the 32x32 patches. Given the figure presented in 2, the parameters for the Multi-Layer Perceptron (MLP) are determined based on the multi-head self-attention mechanism. Specifically, the MLP parameters in TE1 and TE2 are selected as follows:- 12 (no of heads) * 256 (vector size of each head) and 768 is the dimensional space of patch embedding. Moreover, there are two transformer encoders, each with 16 attention heads and an MLP size of (4096×1024). The input data comprises a partitioned malware image with two different sizes, one of which is partitioned into 16x16 patches, while the other is partitioned into 32x32 patches. The patches are then represented as embeddings and mapped to a 1024-dimensional space. The 12-layer transformer encoder addresses the 16x16 patches, while the 24-layer transformer encoder addresses the 32x32 patches. Given the figure presented in 2, the parameters for the Multi-Layer Perceptron (MLP) are determined based on the multi-head self-attention mechanism. Specifically, the MLP parameters in TE3 and TE4 are selected as follows:- 16 (no of heads) * 256 (vector size of each head) and 1024 is the dimensional space of patch embedding.

The transformer encoders show four different visual representation embeddings sets of one malware image, these four embeddings sets are used for malware classification.

3.2.4 Training and classification

In this proposed architecture, the transformer encoders are used as transfer learning-based models, meaning that they were previously trained on a large-scale dataset, in this case, the ImageNet dataset. The parameters learned during this pre-training phase are used to initialize the parameters of four transformer encoder variants. Then, the MLP head classifier is fine-tuned to adapt the model to the specific task i.e., malware classification. The fine-tuning process updates the parameters of the MLP head classifier while keeping the parameters of the transformer encoders fixed. This approach saves considerable computational resources and time as the model does not have to be trained from scratch. Additionally, it does not necessitate an extensive quantity of training samples and overcomes the overfittingissue.

In the proposed architecture, there are four MLP head classifiers which address the four different visual representation embeddings of one malware image. Each MLP head classifier is trained on malware image training samples. Besides, each one is evaluated on malware image testing samples in terms of accuracy, F-score, recall, and precision. Two malware datasets i.e., MalImg, and Microsoft BIG are used to train and evaluate the proposed architecture. Each MLP classifier has two hidden layers: (1024, 512) units and an output layer: x units where x denotes the no of classes in dataset, x = 25 and 9 for MalImg, and Microsoft BIGrespectively.

During training, the proposed malware classifier is optimized using the following hyperparameters: the batch size is 16, and the Adam optimizer is chosen, with a learning rate of 10^-4. This specific set of hyperparameters has been identified experimentally as the optimal performance for the malware classifier is achieved. The loss function used in the proposed architecture is the cross-entropy loss, which is given by Equation 5:

$L_{CE} (p_{k}, t_{k}) = - \sum_{k = 1}^{x} p_{k} \log (t_{k})$ (5) where x denotes the no of classes in dataset; x = 25 and 9 for MalImg, and Microsoft BIG respectively. For a given input sample, t_k is the true label for the k-th class, and p_k is the obtained label for the k-th class.

Data augmentation methods are often employed in the training phase to address the problem of overfitting in transformers-based image classification models. These techniques involve various methods such as scaling, shearing, rotating, shifting, flipping, and zooming, etc. By applying these methods, the number of training samples can be increased, which can help improve the model’s performance and generalization capability. This study employed a set of data augmentation techniques to generate additional training samples from the original malware samples. Table 2 presents a summary of the augmentation steps used in this paper. Notably, the testing images were not subjected to any augmentation to ensure a realistic evaluation of the model’s performance on the original data. The MalImg and Microsoft BIG datasets, used to train the proposed architecture, are imbalanced, with some malware families having notably larger or smaller image numbers than others, as indicated in Figs. 3 and 4. However, by applying data augmentation techniques to increase the no. of malware images in certain classes, the impact of unbalanced data can be reduced. The augmentation details for Malimg and Microsoft BIG are in Tables 3 and 4. However, there exists an imbalance in the number of original training samples (before augmentation) across various malware families. Several augmentation techniques (Table 2) have been implemented on these samples to address this imbalance. The objective is to augment the dataset and increase the number of training samples for each malware family. The ultimate goal is to achieve a balanced dataset with equal samples in each malware family.

Fig. 3

Malware families in MalImg Dataset.

Fig. 4

Malware families in Microsoft BIG Dataset.

Table 2

The augmentation techniques used on Malware Images

Augmentation Step	Factor/Range
Random Brightness	[–0.1 0.1]
Random Contrast	[0.8 1.2]
Random Saturation	[0.7 1.3]
Transposing	> 0.75
Horizontal flipping	True
Vertical flipping	True
Rescaling	1/255
Rotation	{0, 90, 180, 275}

Table 3

Augmentation details of Malimg

Malware Family	Training Samples (before augmentation)	Training Samples (after augmentation)	Testing Samples
Agent.FYI	93	3000	23
Adialer.C	98	3000	24
Allaple.A	2360	3000	589
Allaple.L	1273	3000	318
Autorun.K	85	3000	21
Alueron.gen!J	159	3000	39
C2LOP.P	117	3000	29
C2LOP.gen!g	160	3000	40
Dialplatform.B	142	3000	35
Fakerean	305	3000	76
Dontovo.A	130	3000	32
Instantaccess	345	3000	86
Lolyda.AA3	99	3000	24
Lolyda.AA2	148	3000	36
Lolyda.AA1	171	3000	42
Malex.gen!J	109	3000	27
Skintrim.N	64	3000	16
Lolyda.AT	128	3000	31
Obfuscator.AD	114	3000	28
Rbot!gen	127	3000	31
Swizzor.gen!E	103	3000	25
Yuner.A	640	3000	160
VB.AT	327	3000	81
Swizzor.gen!I	106	3000	26
Wintrim.BX	78	3000	19

Table 4

Augmentation details of Microsoft BIG

Malware Family	Training Samples (before augmentation)	Training Samples (after augmentation)	Testing Samples
Vundo	380	3000	95
Lollipop	1983	3000	495
Obfuscator.ACY	983	3000	245
Tracur	601	3000	150
Kelihos_ver3	2354	3000	588
Ramnit	1233	3000	308
Kelihos_ver1	319	3000	79
Simda	34	3000	8
Gatak	811	3000	202

3.2.5 Decision making

The objective of this stage in the proposed architecture is to determine the most optimal transformer encoder variant out of the four available options that is best suited for performing malware classification. The transformer-based malware classifiers generate multiple performance metrics such as precision, F1-score, accuracy, and recall. Evaluating the model based on a single metric is insufficient to satisfy all the criteria. Therefore, a standard evaluation approach should be there. Multiple Criteria Decision Making (MCDM) is an approach utilized to evaluate and rank various techniques, such as multi-variant transformers-based malware classifiers, based on multiple criteria to determine the most optimal one.

Firstly, the decision matrix A is constructed by the crossover between the multi-variant transformers-based malware classifiers and performance criteria as shown in Equation 6. The decision matrix A consists of four rows that represent the multi-variant transformers-based malware classifiers mentioned in Table 1. Each row corresponds to a specific transformer variant. The columns of A correspond to the performance metrics of the classifiers, namely precision, F1-score, accuracy, and recall. There are a total of four columns in A, each representing one metric for the transformers.

$A = {\begin{matrix} {Acc}^{(1)} & {Pre}^{(1)} & {Re}^{(1)} & F^{(1)} \\ {Acc}^{(2)} & {Pre}^{(2)} & {Re}^{(2)} & F^{(2)} \\ {Acc}^{(3)} & {Pre}^{(3)} & {Re}^{(3)} & F^{(3)} \\ {Acc}^{(4)} & {Pre}^{(4)} & {Re}^{(4)} & F^{(4)} \end{matrix}}$ (6) where A_ij denotes the value of j-th criterion for the i-th transformer variant. Acc⁽ⁱ⁾, Pre⁽ⁱ⁾, Re⁽ⁱ⁾, F⁽ⁱ⁾ are performance criteria of i-th transformer variant; i= 1,..., 4.

secondly, In order to determine the weights of the criteria, an objective weighting strategy called MEREC is employed [5]. Each weight assigned to a criterion represents its impact on the overall performance. The weights are assigned in such a way that criteria with a greater impact on performance are given higher weights, indicating their importance in the evaluation process. By following this procedure, the MEREC method determines the weights of the criteria:

1- The performance criteria values in the decision matrix are normalized using Equation 7

$A_{ij}^{'} = {\begin{matrix} \frac{\min_{u} (A_{uj})}{A_{ij}}; u = 1, . . ., 4 ifj \in B \\ \frac{A_{ij}}{\max_{u} (A_{uj})}; u = 1, . . ., 4 ifj \in \bar{B} \end{matrix}$ (7) where $A_{ij}^{'}$ is the j-th normalized metric for the i-th transformer variant. B, and $\bar{B}$ are sets of beneficial criteria and non-beneficial criteria respectively. Beneficial criteria exhibit high performance when their corresponding values are high, indicating a positive correlation. Conversely, non-beneficial criteria demonstrate high performance when their values are low, signifying an inverse relationship.

2- The performance of each transformer-based malware classifier variant is calculated using Equation 8.

${TE}_{i} = \ln (1 + \frac{\sum_{j = 1}^{4} | \ln (A_{ij}^{'}) |}{4}); i = 1, . . ., 4$ (8) where TE_i denotes the performance of i-th transformer-based malware classifier variant.

3- The performance of transformer-based malware classifier variants is calculated by considering the impact of eliminating each criterion on their performance using Equation 9.

$\begin{matrix} {TE}_{ij}^{'} = \ln (1 + \frac{\sum_{u = 1 | u \neq j}^{4} | \ln (A_{iu}^{'}) |}{4}); \\ i = 1, . . ., 4 andj = 1, . . ., 4 \end{matrix}$ (9) where ${TE}_{ij}^{'}$ the performance of i-th transformer-based malware classifier variant by considering the impact of eliminating j-th criterion.

4- The sum of absolute deviations is calculated using Equation 10.

$Φ_{j} = \sum_{i = 1}^{4} | {TE}_{ij}^{'} - {TE}_{i} |; j = 1, . . ., 4$ (10) where Φ_j denotes the impact of eliminating the jth criterion.

5- The weights of criteria are determined using Equation 11.

$W_{j} = \frac{Φ_{j}}{\sum_{u = 1}^{4} Φ_{u}}; j = 1, . . ., 4$ (11)

Where W_j denotes the j-th criterion weight.

Thirdly, in order to rank the transformer-based malware classifier variants, an MCDM-based ranking method called VIKOR is employed [5]. The following steps demonstrate the VIKOR method’s procedure:

1- The best $A_{j}^{best}$ and worst $A_{j}^{worst}$ values of each criterion are determined using Equation 12.

${\begin{matrix} A_{j}^{(best)} = \max_{i} (A_{ij}) \\ A_{j}^{(worst)} = \min_{i} (A_{ij}) \end{matrix} i = 0, . . ., 4 andj = 1, . . ., 4$ (12) where $A_{j}^{(best)}, A_{j}^{(worst)}$ is the best and worst value of j-th criterion among all transformer-based malware classifier variants.

2- The weighted decision matrix is computed using Equation 13.

$A_{ij}^{W} = W_{j} * \frac{A_{j}^{(best)} - A_{ij}}{A_{j}^{(best)} - A_{j}^{(worst)}}$ (13)

3- For each transformer-based malware classifier variant, the values U_i, R_i and Q_i are calculated as shown in Equations 14, 15, and 16 respectively.

$U_{i} = \sum_{j = 1}^{4} (W_{j} * \frac{A_{j}^{(best)} - A_{ij}}{A_{j}^{(best)} - A_{j}^{(worst)}}); i = 1, . . ., 4$ (14)

$R_{i} = \max_{j} (W_{j} * \frac{A_{j}^{(best)} - A_{ij}}{A_{j}^{(best)} - A_{j}^{(worst)}}); i = 1, . . ., 4$ (15)

$\begin{matrix} Q_{i} = \frac{α (U_{i} - U_{i}^{(\min)})}{U_{i}^{(\max)} - U_{i}^{(\min)}} + \frac{(1 - α) (R_{i} - R_{i}^{(\min)})}{R_{i}^{(\max)} - R_{i}^{(\min)}}; \\ i = 1, . . ., 4 \end{matrix}$ (16) where R_i, U_i denote the regret and utility of i-th transformer-based malware classifier variant; and Q_i denote a compromise solution that balances both utility U_i and regret R_i, while considering the relative weights or importance α assigned to each criterion.

4- Transformer-based malware classifiers are assessed using three standard evaluation lists: R_i, U_i, and Q_i. These lists are sorted in ascending order, forming a basis for ranking the classifiers. The best classifier is one that exhibits the lowest Q value, indicative of superior performance in malware classification as shown in Equation 17.

${TE}^{(best)} = Q_{i}^{(\min)}; i = 1, . . ., 4$ (17)

4 Performance analysis

The performance evaluation of the proposed architecture focused on its application in visualization-based malware classification. The performance of the transformer encoder variants is assessed both on individual malware families and across the entire set of malware families within two datasets i.e., MalImg and Microsoft BIG datasets. The performance of multi-variant (four variants) transformer encoders is evaluated in terms of multi-inconsistent metrics such as accuracy (Acc), F1-score (F1), precision (Pr), and recall (Re) as well as standard evaluation metrics such as (S, R, and Q). The multi-inconsistent metrics are defined in Equations 18, 19, 20, and 21. Whereas the standard evaluation metrics are defined in Equations 14, 15, and 16.

$Acc = \frac{\sum_{j = 1}^{k} {TP}_{j}}{N}$ (18)

$\Pr = \frac{\sum_{j = 1}^{k} {TP}_{j}}{\sum_{j = 1}^{k} {TP}_{j} + \sum_{j = 1}^{k} {FP}_{j}}$ (19)

$Re = \frac{\sum_{j = 1}^{k} {TP}_{j}}{\sum_{j = 1}^{k} {TP}_{j} + \sum_{j = 1}^{k} {FN}_{j}}$ (20)

$F 1 = \frac{2 * Re * \Pr}{Re + \Pr}$ (21) where k is the number of malware families in datasets, N is the total number of testing samples, TP_j represents the no. of true positives in the j-th malware family, FP_j is the no. of false positives in the j-th malware family, and FN_j is the no. of false negatives in the j-th malware family.

For datasets, the malimg 1 dataset comprises a collection of more than 9,000 grayscale images, organized into 25 different malware families, as detailed in Fig. 3. On the other hand, the Microsoft BIG 2 dataset comprises over 10,000 malware byte files, represented in hexadecimal format. These files are distributed among 9 different malware families, as detailed in Fig. 4. The samples in datasets are split into an 80:20 ratio, allocating 80% for training and the remaining 20% for testing during the experiments. Figure 5 shows some malware samples from MalImg and Microsoft BIG datasets.

Fig. 5

Samples of malware families in MalImg and Microsoft BIG datasets.

Furthermore, a comparison has been made between the proposed architecture and other state-of-the-art malware classification approaches.

The proposed architecture was implemented on a platform with the following capabilities: Graphics card: an NVIDIA GeForce GTX 1080 Ti with CUDA 11.2; CPU: Intel Xeon 4 GHz; operating system: Windows 10 (64-bit); RAM: 64 GB; IDE: Spyder (Python 3.9.12); Libraries: vit_keras, TensorFlow, and Keras.

4.1 Multi-variant transformer encoders performance analysis on malware family-wise

In this section, the performance of four transformer encoder variants on each malware family in two datasets i.e., Malimg and Microsoft BIG, is analyzed in terms of multi-inconsistent and standard evaluation metrics. Figure 6 shows the results of performance metrics i.e., accuracy, F1-score, precision, and recall for four transformer encoder variants on the 25 malware families individually in Malimg dataset. Figure 7 shows the results of performance metrics i.e., accuracy, F1-score, precision, and recall for four transformer encoder variants on the 9 malware families individually in Microsoft BIG dataset. As shown in two Figs. 6 and 7, the performance metrics exhibit inconsistency across some malware families. This inconsistency implies that certain variants of the transformer encoder achieve higher performance for certain metrics, while not necessarily excelling in other metrics. Consequently, determining the best variant becomes a challenging task based on these multi-inconsistent metrics. In contrast, determining the best transformer encoder variant becomes easier when a single metric is employed for evaluation. For C2LOP.gen!g malware family in MalImg, TE4 achieves the highest accuracy, while having lower precision and F1-score than TE3. For simda malware family in Microsoft, TE4 achieves the highest recall, while has lower precision and accuracy thanTE3.

Fig. 6

Multi-variant transformer encoders performance analysis on Malware family-wise in MalImg dataset.

Fig. 7

Multi-variant transformer encoders performance analysis on Malware family-wise in Microsoft BIG dataset.

To overcome this challenge, a MEREC-VIKOR technique is used to decide the best transformer encoder variant based on a standard evaluation system using multi-criteria. MEREC assigns weights to the inconsistent metrics, and VIKOR ranks transformer encoder variants to determine the optimal one. Table 5 shows the weights of the four performance metrics for the 25 malware families individually in Malimg dataset. Table 6 shows the weights of the four performance metrics for the 9 malware families individually in Microsoft BIG dataset. The weights W1, W2, W3, and W4 represent the importance assigned to accuracy, F-score, precision, and recall. The results show that the impact of metrics among malware families varies, and no single metric can be deemed preferable. In MalImg, Dontovo.A, Allaple.L, Allaple.A, Obfuscator.AD, Skintrim.N, and Wintrim.BX, prioritize accuracy as a significant metric. Adialer.C, Agent.FYI, Autorun.K, C2LOP.gen!g, Dialplatform.B, Fakerean, Instantaccess, Lolyda.AA2, Malex.gen!J, Rbot!gen, Swizzor.gen!, VB.AT, and Yuner.A prioritize precision. Alueron.gen!J, C2LOP.P, Lolyda.AA1, Lolyda.AA3, Lolyda.AT, Swizzor.gen!E, Rbot!gen, and Malex.gen!J prioritize recall. In Microsoft BIG, kelihos_ver1 and Gatak prioritize precision as a significant metric. Lollipop, Vundo, Obfuscator.ACY, Ramnit, kelihos_ver3, and simda prioritize recall. Tracur has the same impact on precision and recall.

Table 5

The metrics’ weights for each malware family in MalImg

Malware Family	W1	W2	W3	W4
Adialer.C	0.327	0.224	0.385	0.064
Agent.FYI	0.245	0.197	0.413	0.145
Allaple.A	0.271	0.243	0.229	0.257
Allaple.L	0.261	0.246	0.233	0.259
Alueron.gen!J	0.079	0.307	0.271	0.342
Autorun.K	0.264	0.245	0.266	0.225
C2LOP.gen!g	0.15	0.274	0.382	0.194
C2LOP.P	0.085	0.309	0.136	0.47
Dialplatform.B	0.199	0.228	0.482	0.091
Dontovo.A	0.334	0.222	0.171	0.273
Fakerean	0.054	0.314	0.563	0.068
Instantaccess	0.092	0.296	0.581	0.03
Lolyda.AA1	0.04	0.311	0.232	0.417
Lolyda.AA2	0.024	0.325	0.371	0.28
Lolyda.AA3	0.142	0.276	0.211	0.371
Lolyda.AT	0.154	0.284	0.127	0.435
Malex.gen!J	0.257	0.226	0.419	0.098
Obfuscator.AD	0.454	0.12	0.307	0.119
Rbot!gen	0.19	0.235	0.439	0.136
Skintrim.N	0.365	0.198	0.355	0.082
Swizzor.gen!E	0.033	0.323	0.192	0.452
Swizzor.gen!I	0.027	0.197	0.452	0.325
VB.AT	0.105	0.301	0.507	0.087
Wintrim.BX	0.312	0.19	0.29	0.208
Yuner.A	0.292	0.235	0.295	0.177

Table 6

The metrics’ weights for each malware family in Microsoft BIG

Malware Family	W1	W2	W3	W4
Lollipop	0.157	0.281	0.172	0.39
Vundo	0.152	0.283	0.122	0.444
Tracur	0.261	0.187	0.276	0.276
Obfuscator.ACY	0.214	0.221	0.133	0.431
Ramnit	0.309	0.089	0.265	0.336
kelihos_ver3	0.184	0.272	0.25	0.295
simda	0.078	0.308	0.262	0.353
kelihos_ver1	0.062	0.318	0.538	0.082
Gatak	0.137	0.221	0.479	0.163

Figure 8 shows the results of standard evaluation metrics i.e., Q, R, and U for four transformer encoder variants on the 25 malware families individually in Malimg dataset. Figure 9 shows the results of standard evaluation metrics i.e., Q, R, and U for four transformer encoder variants on the 9 malware families individually in Microsoft BIG dataset. As mentioned previously, the best classifier is one that exhibits the lowest Q value, indicative of superior performance in malware classification. As noted in Fig. 8, the TE3 and TE4 variants outperform the TE1 and TE2 variants across all malware families. This superiority is substantiated by the lower Q values observed in TE3 and TE4 compared to TE1 and TE2. Furthermore, among the TE3 and TE4 variants, TE3 demonstrates a consistently better performance across all malware families, except in the case of gent.FYI, Allaple.A, Allaple.L, C2LOP.gen!g, VB.AT, and Wintrim.BX. Similarly, as noted in Fig. 9, TE3 outperform the TE1, TE2, and TE4 variants in most malware families. This superiority is substantiated by the lower Q value observed in TE3 compared to TE1, TE2 and TE4.

Fig. 8

Multi-variant transformer encoders standard evaluation metrics on Malware family-wise in MalImg dataset.

Fig. 9

Multi-variant transformer encoders standard evaluation metrics on Malware family-wise in Microsoft BIG dataset.

4.2 Multi-variant transformer encoders performance analysis on overall malware families

In this section, the performance of four transformer encoder variants on overall malware families in two datasets i.e., Malimg and Microsoft BIG, is analyzed in terms of multi-inconsistent and standard evaluation metrics. Table 7 shows the results of performance metrics (accuracy, F1-score, precision, and recall) and standard evaluation metrics (U, R, and Q) for four transformer encoder variants across the entire set of 25 malware families in Malimg dataset. Table 8 shows the results of performance metrics (accuracy, recall, precision, and F1-score) and standard evaluation metrics (U, R, and Q) for four transformer encoder variants across the entire set of 9 malware families in Microsoft BIG dataset. As noted in Tables 7 and 8 , TE3 outperform the TE1, TE2, and TE4 variants across the entire set of malware families in two dataset i.e., MalImg and Microsoft BIG. This superiority is substantiated by the lower Q value observed in TE3 compared to TE1, TE2 and TE4. Table 9 shows the weights of the four performance metrics across the entire set of malware families in MalImg and Microsoft BIG datasets. Both datasets prioritize precision as a significant metric. Figures 10 and 11 show the relationship between the performance metrics and standard evaluation metrics for the four transformer encoders variants in MalImg and Microsoft BIG datasets respectively.

Table 7
The overall performance metrics (DM) and standard evaluation metrics of multi-variant transformer encoders for MalImg

DM (performance metrics) Evaluation Standard Metrics

TE ACC F1 Pre Re U R Q

TE1 97.24 92.05 92.89 91.38 0.843 0.278 0.867

TE2 96.56 91.87 92.58 91.32 1.0 0.312 1.0

TE3 97.64 94.49 95.45 93.63 0.0 0.0 0.0

TE4 97.55 93.91 94.75 93.16 0.203 0.076 0.223

	DM (performance metrics)	Evaluation Standard Metrics
TE1	97.24	92.05	92.89	91.38	0.843	0.278	0.867
TE2	96.56	91.87	92.58	91.32	1.0	0.312	1.0
TE3	97.64	94.49	95.45	93.63	0.0	0.0	0.0
TE4	97.55	93.91	94.75	93.16	0.203	0.076	0.223

Table 8

The overall performance metrics (DM) and standard evaluation metrics of multi-variant transformer encoders for Microsoft BIG

	DM (performance metrics)				Evaluation Standard Metrics
TE	ACC	F1	Pre	Re	U	R	Q
TE1	98.63	94.94	94.66	95.32	0.810	0.280	0.874
TE2	97.94	94.83	94.55	95.22	1.0	0.298	1.0
TE3	98.92	96.80	96.33	97.29	0.0	0.0	0.0
TE4	98.72	95.86	96.08	95.63	0.393	0.180	0.497

Table 9

The metrics’ weights for overall malware families in MalImg and Microsoft BIG

Dataset	W1	W2	W3	W4
MalImg Microsoft BIG	0.155	0.284	0.312	0.249
	0.207	0.271	0.298	0.224

Fig. 10

The relationship between the performance metrics and standard metrics for multi-variant transformer encoders in MalImg dataset. (a): TE1, (b): TE2, (c): TE3, (d): TE4.

Fig. 11

The relationship between the performance metrics and standard metrics for multi-variant transformer encoders in Microsoft BIG dataset. (a): TE1, (b): TE2, (c): TE3, (d): TE4.

Table 10 shows a comparison between the proposed architecture and recent approaches relevant to the malware classification task. The studies being compared in this context have utilized identical datasets, which have also been employed in the proposed architecture. As noted, the proposed method has employed MCDM for standard evaluation which has not been used in recent studies. However, The multi-variant ViT outperforms the other ViT-based malware classifiers in terms of malware classification performance such as accuracy, precision, recall, and F1-score. Plus, the multi-variant ViT outperforms the CNN-based classifiers in terms of computational efficiency such as training time and predictiontime.

Table 10

A comparison between the proposed architecture and other state-of-the-art approaches

			Microsoft BIG						MalImg
Ref	Method	MCDM	Accuracy	F1_score	Precision	Recall	TT(s)	PT(ms)	Accuracy	F1_score	Precision	Recall	TT(s)	PT(ms)
	Multi-variant ViT (Ours)	Yes	98.92	96.80	96.33	97.29	663	6.67	97.64	94.49	95.45	93.63	781	6.62
[27]	Enhanced ViT + Patch Encoding	NO	–	–	–	–	–	–	96.83	–	–	–	772	6.70
[29]	Self supervised ViT	NO	–	–	–	–	–	–	97.0	94.90	87.8	86.7	773	6.70
[38]	IMCLNet	NO	99.12	98.64	98.94	98.38	709	6.73	99.78	99.37	99.37	99.38	824	6.81
[32]	BHMDC	NO	99.70	99.70	99.70	99.70	761	7.37	99.90	99.90	99.90	99.90	866	7.45
[11]	MCTVD	NO	99.44	–	–	–	697	6.71	–	–	–	–	–	–
[16]	MCFT-CNN	NO	98.64	97.22	98.55	96.00	728	7.15	99.18	97.68	97.72	97.76	835	7.23
[34]	IMCFN	NO	–	–	–	–	–	–	98.82	98.82	98.85	98.81	798	6.79
[17]	DTMIC	NO	–	–	–	–	–	–	98.92	99	99	99	782	7.77
[14]	XGBoost (SSTL)	NO	99.69	99.69	99.69	99.69	792	7.42	–	–	–	–	–	–
[30]	BiLSTM + Dual Attention	NO	97.75	97.41	97.42	97.47	733	7.17	99.25	98.82	98.88	98.82	843	7.26
[8]	EfficientNetB1	NO	98	98	98	98	759	7.31	–	–	–	–	–	–
[6]	IDA + DRBA	NO	–	–	–	–	–	–	94.50	94.55	94.60	94.50	794	6.72
[10]	NSGA_II	NO	–	–	–	–	–	–	97.6	92.77	88.4	97.6	803	6.74

5 Conclusion

This paper proposes a butterfly construction-based vision transformer (B_ViT) model for visualization-based malware classification and detection. B_ViT is trained and evaluated on grayscale malware images collected from MalImg or Microsoft BIG datasets or converted from portable executable imports. B_ViT has four phases: image partitioning and patches embeddings; local attention; global attention; and training and malware classification. In local attention phase, self-attention-based local transformer encoders along with local positional encoding process the input image’s patches simultaneously to capture the local representation and features of malware image. In global attention phase, one self-attention-based global transformer encoder along with global positional encoding process the input image as one block, to capture the global representation and features of malware image. B_ViT is a transfer learning-based model that uses a pre-trained ViT model on the ImageNet dataset to initialize the training parameters of transformers, then the B_ViT is fine-tuned to fit malware classification task. All B_ViT variants i.e, BViT/B16, BViT/B32, BViT/L16, and BViT/L32 are experimented and evaluated using MalImg and Microsoft BIG datasets for image malware classification and compared with the respective variants of IEViT and ViT. The comparative analysis shows that B_ViT variants outperform the IEViT and ViT variants in visualization-based malware classification achieving accuracy equal to 98.65%, 98.28%, 99.32%, and 99.11% in MalImg; and 98.80%, 98.62%, 99.49%, and 99.26% in Microsoft BIG for BViT/B16, BViT/B32, BViT/L16, and BViT/L32 respectively. Besides, B_ViT variants are evaluated using portable executable imports for image malware detection and compared with the respective variants of IEViT and ViT. The comparative analysis shows that B_ViT variants outperform the IEViT and ViT variants in visualization-based malware detection achieving accuracy equal to 99.84%, 99.87%, 99.99%, and 99.97% for BViT/B16, BViT/B32, BViT/L16, and BViT/L32 respectively. The results show that B_ViT achieves average improvement in terms of F1-score over IEViT and ViT equal to 1.21%, and 2.48% respectively. Since B_ViT is a parallel-based architecture, a parallel analysis of B_ViT over sequential B_ViT, IEViT, and ViT is performed. The results show that B_ViT is time-effective for malware classification and detection where the average speed-up of B_ViT variants over sequential B_ViT, IEViT and ViT variants are equal to 3.84, 2.42 and 1.81 respectively. Moreover, the analysis shows the efficiency of texture-based malware detection as well as the resilience of B_ViT to polymorphic obfuscation. The proposed malware classifier/detector is visualization-based so, does not require domain experts for feature extraction, feature engineering, etc. Finally, the proposed method that uses B_ViT architecture outperforms recent visualization-based malware classification methods that use CNN architectures as well as ViT-based malware classifiers. The utilization of the B_ViT-based malware classifier/detector in practice presents certain limitations that should be acknowledged. Firstly, its implementation necessitates a high-resource platform, especially when employing a high degree of parallelism because more local transformer encoders should be run to capture the local representation of malware images. Secondly, to ensure its effectiveness, the proposed method must be thoroughly tested in an on-site environment. To overcome these limitations, future work should focus on further optimization and refinement of the approach.

Footnotes

Available online:

References

A novel deep framework for dynamic malware detection based on API sequence intrinsic features, Computers & Security 116 (2022), 102686.

Avdiienko

, Kuznetsov

, Gorla

, Zeller

, Arzt

, Rasthofer

, Bodden

, Mining apps for abnormal usage of sensitive data, In 2015 IEEE/ACM37th IEEE International Conference on Software Engineering, volume 1, pages 426–436. IEEE. (2015).

Belal

M.M.

, Sundaram

D.M.

, Comprehensive review on intelligent security defences in cloud: Taxonomy, security issues, ml/dl techniques, challenges and future trends, Journal of King Saud University –Computer and Information Sciences, (2022).

Belal

M.M.

, Sundaram

D.M.

, Global-local attention-based butterfly vision transformer for visualizationbased malware classification, IEEE Access 11 (2023a)69337–69355.

Belal

M.M.

, Sundaram

D.M.

, An intelligent protection framework for intrusion detection in cloud environment based on covariance matrix self-adaptation evolution strategy and multi-criteria decision-making, Journal of Intelligent & Fuzzy Systems (Preprint) (2023b), 1–31.

Bhodia

, Prajapati

, Di Troia

and Stamp

, Transfer learning for image-based malware classification, arXiv preprint arXiv:1903.11551. (2019).

Borhani

, Khoramdel

, Najafi

, A deep learning based approach for automated plant disease classification using vision transformer, Scientific Reports 12(1) (2022), 1–10.

Chaganti

, Ravi

, Pham

T.D.

, Image-based malware representation approach with efficientnet convolutional neural networks for effective malware classification, Journal of Information Security and Applications 69 (2022), 103306.

Chen

, Sultana

, Sahita

, Henet: A deep learning approach on intelR processor trace for effective exploit detection, In 2018 IEEE Security and Privacy Workshops (SPW), pages 109–115. IEEE. (2018).

10.

Cui

, Du

, Wang

, Cai

, Zhang

, Malicious code detection based on cnns and multi-objective algorithm, Journal of Parallel and Distributed Computing 129 (2019), 50–58.

11.

Deng

, Guo

, Shen

, Cui

, Ping

, Mctvd: A malware classification method based on three-channel visualization and deep learning, Computers & Security 126 (2023), 103084.

12.

Dosovitskiy

, Beyer

, Kolesnikov

, Weissenborn

, Zhai

, Unterthiner

, Dehghani

, Minderer

, Heigold

, Gelly

, et al. An image is worth 16×16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929, (2020).

13.

Falana

O.J.

, Sodiya

A.S.

, Onashoga

S.A.

, Badmus

B.S.

, Mal-detect: An intelligent visualization approach for malware detection, Journal of King Saud University –Computer and Information Sciences 34(5) (2022), 1968–1983.

14.

Gao

, Hu

, Shan

, Liu

, Niu

, Xie

, Malware classification for the cloud via semi-supervised transfer learning, Journal of Information Security and Applications 55 (2020), 102661.

15.

Haurum

J.B.

, Madadi

, Escalera

, Moeslund

T.B.

, Multi-scale hybrid vision transformer and sinkhorn tokenizer for sewer defect classification, Automation in Construction 144 (2022), 104614.

16.

Kumar

, et al. Mcft-cnn: Malware classification with fine-tune convolution neural networks using traditional and transfer learning in internet of things, Future Generation Computer Systems 125 (2021), 334–351.

17.

Kumar

, Janet

, Dtmic: Deep transfer learning for malware image classification, Journal of Information Security and Applications 64 (2022), 103063.

18.

Landman

, Nissim

, Deep-hook: A trusted deep learning-based framework for unknown malware detection and classification in linux cloud environments, Neural Networks 144 (2021), 648–685.

19.

, Cheng

, Zhu

, Wang

, Lv

, Wang

, Li

, Sun

, Dmalnet: Dynamic malware analysis based on api feature engineering and graph learning, Computers & Security 122 (2022), 102872.

20.

Liangboonprakong

, Sornil

, Classification of malware families based on n-grams sequential pattern features, In 2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA), pages 777–782. (2013).

21.

Makandar

, Patrot

, Malware class recognition using image processing techniques, In 2017 International Conference on Data Management, Analytics and Innovation (ICDMAI), pages 76–80. (2017).

22.

Naik

, Jenkins

, Savage

, Yang

, Boongoen

, Iam-On

, Fuzzy-import hashing: A static analysis technique for malware detection, Forensic Science International: Digital Investigation 37 (2021), 301139.

23.

Narayanan

B.N.

, Djaneye-Boundjou

, Kebede

T.M.

, Performance analysis of machine learning and pattern recognition algorithms for malware classification, In 2016 IEEE National Aerospace and Electronics Conference (NAECON) and Ohio Innovation Summit (OIS), pages 338–342. (2016).

24.

Narouei

, Ahmadi

, Giacinto

, Takabi

, Sami

, Dllminer: structural mining for malware detection, Security and Communication Networks 8(18) (2015), 3311–3322.

25.

Nataraj

, Karthikeyan

, Jacob

, Manjunath

B.S.

, Malware images: visualization and automatic classification, In Proceedings of the 8th International Symposium on Visualization for Cyber Security, pages 1–7. (2011).

26.

Okolo

G.I.

, Katsigiannis

, Ramzan

, Ievit: An enhanced vision transformer architecture for chest x-ray image classification, Computer Methods and Programs in Biomedicine 226 (2022), 107141.

27.

Park

K.-W.

, Cho

S.-B.

, A vision transformer enhanced with patch encoding for malware classification, In Intelligent Data Engineering and Automated Learning–IDEAL 2022:23rd International Conference, IDEAL 2022, Manchester, UK, November 24–26, 2022, Proceedings, pages 289–299. Springer, (2022).

28.

Raghu

, Unterthiner

, Kornblith

, Zhang

, Dosovitskiy

, Do vision transformers see like convolutional neural networks? Advances in Neural Information Processing Systems 34 (2021), 12116–12128.

29.

Seneviratne

, Shariffdeen

, Rasnayaka

, Kasthuriarachchi

, Self-supervised vision transformers for malware detection, IEEE Access 10 (2022), 103121–103135.

30.

Shen

, Chen

, Wang

, Chen

, Wang

, Feature fusion-based malicious code detection with dual attention mechanism and bilstm, Computers & Security 119 (2022), 102761.

31.

Tabbakh

, Barpanda

S.S.

, A deep features extraction model based on the transfer learning model and vision transformer” tlmvit” for plant disease classification, IEEE Access (2023).

32.

Tang

, Qi

, Jing

, Liu

, Dong

, Bhmdc: a byte and hex n-gram based malware detection and classification method, Computers & Security (2023), 103118.

33.

Tian

, Ying

, Jia

, Ma

, Hu

, Liu

, Mdchd: A novel malware detection method in cloud using hardware trace and deep learning, Computer Networks 198 (2021), 108394.

34.

Vasan

, Alazab

, Wassan

, Naeem

, Safaei

, Zheng

, Imcfn: Image-based malware classification using fine-tuned convolutional neural network architecture, Computer Networks 171 (2020), 107138.

35.

, Xu

Q.Q.

, Shi

, Dong

L.N.

, Zhu

, Xu

, A multitask classification framework based on vision transformer for predicting molecular expressions of glioma, European Journal of Radiology 157 (2022), 110560.

36.

, Fang

, Yang

, Malbert: A novel pretraining method for malware detection, Computers & Security 111 (2021), 102458.

37.

Zhang

, Xiao

, Sangaiah

A.K.

, Zhang

, Ransomware classification using patch-based cnn and self-attention network on embedded n-grams of opcodes, Future Generation Computer Systems 110 (2020), 708–720.

38.

Zou

, Cao

, Tao

, Wang

, Imclnet: A lightweight deep neural network for image-based malware classification, Journal of Information Security and Applications 70 (2022), 103313.