DDIML: Explainable detection model for drive-by-download attacks

2.1 Static analysis

The static analysis focuses on extracting static features from JavaScript codes. Likarish et al. extract 65 features, but 50 are from JavaScript keywords and symbols. The extracted feature types are not comprehensive enough [4]. Curtsinger et al. propose Zozzle, which extracts hierarchical features from the JavaScript abstract syntax tree and hooks into a browser’s JavaScript engine to solve the deobfuscation [5]. In paper [6], Canali et al. design a filter named Prophiler, extracting HTML features, JavaScript features, URL, and host-based features to construct three different feature sets. They trained three models with different feature sets to get the best effect. Also, the method has less overload than dynamic analysis methods. Nayeem et al. present an interceptor between browser and server, which extracts features and uses the wrapper method for feature selection. The interceptor leads to high accuracy [7].

Static analysis methods can quickly detect most samples for some malicious scripts, but they lack semantic information. To solve this problem, some researchers use AST to extract semantic features [8–11]. Also, some researchers leverage AST to build Graphs to extract features based graph information [12, 13]. These approaches provide better accuracy and performance than conventional approaches. Furthermore, to detect obfuscated JavaScript codes, Morishige et al. reconstruct the divided URL to improve the detection effectiveness [14]. Stokes et al. propose the ScriptNet system, which contains Pre-Informant Learning to process JavaScript files as byte sequences [15]. Guo et al. propose GAN, which can achieve high accuracy with small labeled samples [16].

2.2 Dynamic analysis

The dynamic analysis mainly relies on client honeypot technology, which simulates the communication between the browser and the target station in a virtual environment. It distinguishes normal JavaScript codes from malicious by monitoring system behaviors.

Wang et al. design the Strider HoneyMonkey, which generates an XML report containing executable files, process creation, vulnerabilities, etc. HoneyMonkey successfully detects the javaprxy.dll vulnerability [17]. Mitsuaki et al. [18] mainly extract features from exploiting phases, multiple crawler processing, tracking of malware distribution networks, and malware infection prevention. They implement a new client honeypot to detect drive-by downloads, detecting and investigating various malicious websites. Jayasinghe et al. extract opcode feature with dynamic analysis to detect drive-by download attacks [19]. The method is efficient and has low resource consumption. Xue et al. combine data dependency analysis, defense rules, and replay mechanisms to classify JavaScript codes as normal or malicious [20]. Their method is scalable and efficient but consumes more time and resources.

2.3 Semi-dynamic analysis

The semi-dynamic analysis combines static analysis and dynamic analysis. Cova et al. propose an approach named JSAND, which uses HtmlUnit emulation to execute codes and extracts ten features from redirection, deobfuscation, environment preparation, and exploitation [21]. It can detect attacks that have not occurred before and reduce the false-positive rate but low speed. Rieck et al. present a system named Cujo, which uses ADSandbox and SpiderMonkey for dynamic analysis. The System detects 95% of drive-by downloads with few false positives and an average run time of 500ms per sample [22]. JSDC et al. design the tool named JSDC, which extracts features from the text, program structures, and risky function calls to detect malicious JavaScript codes [23]. Compared with other tools, JSDC gives low false-positive and false-negative rates. He et al. [24] implement a browser plugin called MJDetector, which conducts syntax analysis and dynamic instrumentation to extract features. The plugin can detect obfuscated malicious JavaScript and has high accuracy.

Although dynamic analysis has high accuracy, it has a heavy overhead and is time-consuming. This paper extracts 49 features statically, including 26 features proposed in other papers and 23 new features. As our experiments demonstrate, these new features significantly contribute to high accuracy for detection. Also, it is faster to implement detection than dynamic approaches.

We have extracted different features from malicious and benign JavaScript files in the feature extraction. We have written a feature extractor with regular matching in python, which takes a JavaScript file as input and the feature matrix as output. Malicious JavaScript codes are generally transformed, encoded, and encrypted to conceal attackers’ purpose and intention. We manually analyze samples and extract features from five aspects as follows.

Code obfuscation. Attackers often use obfuscation techniques to make code difficult to read to hide the malicious code and avoid signature detection. Escape encryption is a commonly used obfuscation technique where encrypted data is separated by ’% ’. The encrypted data is decrypted with the unescape function and executed with the eval function. Although there are many ways to encrypt strings, they must primarily be executed by the eval function. So the eval function appears frequently in malicious codes. In many cases, obfuscating a string in malicious codes will result in a longer string than normal codes. Simultaneously, the greater the information entropy of the string, the worse the readability.

Random forest is a supervised learning algorithm composed of many decision trees, and there is no correlation between each tree. When building each decision tree, it needs to pay attention to random sampling and complete splitting. The specific process for constructing each decision tree is as follows:

N represents the number of training samples. M represents the number of features.

The Random forest can handle high-dimensional data without feature selection compared with other machine learning algorithms. Moreover, the training speed is fast. Also, it can easily measure the importance of features.

3.3 Explanation method

The purpose of DDIML is to detect whether the JavaScript code is normal or malicious. Understanding why the model predicts some JavaScript codes to be benign and some malicious. The goal is to assess whether we can trust the learned model. In order to achieve the goal, this paper mainly uses SHAP [27] to make a reasonable interpretation on the prediction of the DDIML model.

Using the feature importance API of Random forest to calculate the importance of features, we can initiate the importance ranking of features in the model. The features of the top 20 are shown in Fig. 2. Among the top 20 features, our new features account for over half, fully showing the usefulness and importance of new features to the model prediction. However, feature importance only tells which feature is essential. Hence we do not know how features affect the prediction results and how much influence they have. Simply, we can use PDP (Partial Dependency Plots), and LIME (Local Interpretable Model-agnostic Explanations) [28] to explain the effect of the features on the model prediction. PDP needs to satisfy the independence of features. If there is a correlation between the features, the desired results are unreasonable. However, LIME ignores the feature correlation. We use the Pearson correlation coefficient to calculate the correlation among features, as shown in Fig. 3. It can be seen from the figure that no feature is entirely independent of others. So we choose to use SHAP to explain the model to deal with the problem.

OPEN IN VIEWER

Fig. 2

Feature importance.

OPEN IN VIEWER

Fig. 3

Feature interaction.

SHAP (SHapely Additive exPlanation) can interpret the output of any machine learning model. Inspired by cooperative game theory, SHAP constructs an additive explanatory model in which all features are contributors. The SHAP value is the contribution degree of each feature in a sample, which can be correctly estimated even if the features are correlated. Assuming that the ith sample is x _i , the jth feature of the ith sample is x _ij , the model predicts y _i for this sample. y _base represents the mean value of the predicted results of all samples. The SHAP value obeys equation 1. y i = y base + f ( x i 1 ) + f ( x i 2 ) + ⋯ + f ( x ij )(1)

Where f (x _ij ) is the SHAP value of x _ij , and the most significant advantage of SHAP value is that SHAP can reflect the influence of each feature of the sample on the prediction result. The influence also shows positive and negative.

4.1 Experimental setup

This paper studies the detection of drive-by download attacks based on static analysis. We use regular matching to extract features and optimize regular expressions to make the features more optimal and less time-consuming. Moreover, we use scikit-learn to implement machine learning algorithms. The specific experimental configuration is described in Table 3.

Datasets. In this paper, 4107 samples are used, including 2085 normal samples and 2022 malicious samples. We crawled 20,000 normal web pages from the top 100 websites in Alexa [29] and extracted JavaScript codes from them to save as.js files. In order to balance with the malicious dataset, we randomly select 2805 JavaScript files to constitute the normal dataset. The malicious samples are from Github(All in MJDetector [30] and part in javascript-malware-collection [31]).

Experiment Design. In experiment I, setting the proportion of training set to test set is 8:2, comparing the traditional machine learning algorithms XGBoost, CART, LogisticRegression, SVM, and KNN with Random forest as a benchmark experiment. We use grid-search to determine the optimal parameters. The main parameters of our Random forest classifier are n_estimators=177, max_depth=21 and max_feature=8. In experiment II, each classifier detects samples with 10-fold cross-validation. K-fold cross-validation randomly divides the dataset into K subsets, one of which is used for the test set, and the remaining K-1 subsets are used for the training set. Cross-validation enables the model to make better use of the samples. The results of experiments I and II are displayed in Table 4. In experiment III, compared the method of this paper with Fass (2018) [8], Nayeem (2020) [26], Phung(2021) [32] and JSContana(2021) [33], the results are shown in Table 5. In experiment IV, we crawl 69,994 JS files from the top 100 websites in Github. These samples are used to evaluate the detection of our model in the natural environment, and the result is depicted in Table 6. We also design experiment V, which explains DDIML with the SHAP method to prove that it is credible.

The paper regards normal JavaScript codes as negative samples and malicious as positive samples. We adopt five evaluation metrics: accuracy, precision, recall, F1-score, and AUC to evaluate the prediction results. Accuracy represents the proportion of samples that are predicted correctly in all samples, as shown in Formula 2. Accuracy = TP + TN TP + TN + FP + FN(2)

Where TP indicates the number of correct predictions of positive samples. TN represents the number of correct predictions for negative samples. FP represents the number of negative samples predicted as positive samples. FN is the number of positive samples predicted as negative samples.

Precision is the proportion of positive samples with correct prediction in all positive samples and is defined as Formula 3. Precision = TP / ( TP + FP )(3)

We can see from Formula 4 that Recall is the proportion of the correct positive samples predicted by the classifier to all positive samples. Recall = TP / ( TP + FN )(4)

F1-score is used to measure the weighted harmonic means of Precision and Recall, as shown in Formula 5. F 1 - score = 2 × Precision × Recall Precision + Recall(5)

AUC indicates the area under the ROC curve. The ROC curve represents the relationship between FPR(x-axis) and TPR(y-axis), where FPR is the proportion of negative samples that the classifier predicts incorrectly to all negative samples, as shown in Formula 6, and TPR is equal to Recall. FPR = FP / ( FP + TN )(6)