Feature selection for blind image steganalysis using neighborhood rough sets

Abstract

Feature extraction for blind image steganalysis produces much features or high dimensional data, which bring about time consuming and even a low detection percentage. As being one of the most important phases of preprocessing, feature selection can reduce these extracted features, and improve the performance of steganalysis. Firstly, we introduce the Neighborhood Rough Sets (NRS) to the field of blind image steganalysis. Then, some concepts of feature significance and feature reduct are presented based on NRS. Furthermore, we propose a Feature Selection approach by NRS for blind image steganalysis (FSNRS). The FSNRS has the ability to delete redundant features, meanwhile maintaining the classification accuracy of a steganalysis system. The FSNRS is a filter feature selection technique for blind image steganalysis, which filtrates extracted features depending on a positive region preserving in NRS. The compact feature subset with a shortest feature dimension for blind image steganalysis is selected. Moreover, some experiments for blind steganalysis using SVM and KNN classifiers on selected feature subset are carried out. The experimental results show that our proposed approach can obtain compact features for blind image steganalysis and the performances of classifiers on those selected features are improved. Since the FSNRS is used with an adjustable neighborhood parameter, as a result, the classification performance of selected features is better than that of original whole features in most cases.

Keywords

Image steganalysis blind steganalysis feature selection neighborhood rough sets rough sets

1 Introduction

Steganography is a technique of hiding a secret information in a carrier. In history, the kings used it for sending private messages by embedding them into the body parts of messengers. Now, this technique of hiding has turned to digital carriers, such as images, texts, videos and audios. The resultant image carrying private or sensitive data is called as a stego image. In the past two decades, various steganographic algorithms [1 –5] have been developed for hiding secret messages into digital images without changing the color noticeably. However, steganography may be used by criminals or illegal persons. For example, terrorists use this technique for sending their messages to various corners of the world through internet without being discovered. Therefore, a counter technique known as steganalysis is appeared to detect such stego images.

Steganalysis is a skill of detecting any message in a digital image. With this technique, the existence of secret messages can be discovered. As a matter of fact, steganalysis is simultaneously a part of cryptanalysis and a branch of art. Steganalysis has two types, which are targeted [6, 7] and blind [8, 9]. Targeted steganalysis refers to the technique of identifying the stego image where the steganography algorithm used for hiding the message is known. In case of blind steganalysis, the steganography algorithm is unknown. Therefore, blind steganalysis becomes more difficult to research than targeted steganalysis. The significant process of blind steganalysis has an extracting of features to a classifier for training, while the trained classifier assists in recognizing hidden messages.

According to whether an image hides messages or not, images can be classified into two types: cover image with no information hidden and stego image with an information hidden. Thus, steganalysis can be considered as a classification problem to determine which class a test image belongs to. The significant problem of steganalysis just similar to a classifier is feature extraction. The extracted features from images must be sensitive to a steganography process. In recent years, several feature extraction methods [10 –13] for steganalysis have been proposed to recognize cover images or stego images. However, these methods using different feature extractors produce massive high-dimensional data, which will degrade the performances of steganalysis. And these features may include some irrelevant or redundant information affecting recognition rate. Thus, a preprocessing technique named feature selection emerges to remove the irrelevant and unimportant features from the whole feature set.

Feature selection identifies the important features and deletes redundant features from the extracted features while simultaneously maintaining the meaning of the whole data sets. The technique chooses the optimal subset of features that can represent the whole contents of the data set with the minimum error and information loss. On the one hand, feature selection methods can be classified into three categories according to different search strategies: complete [14], stochastic [15] and heuristic [16]. On the other hand, algorithms of feature selection can be classified into two categories: filter [17] and wrapper [18] methods. A filter method is a preprocessing step that selects a feature subset by a predefined evaluation function without using any learning algorithm. The chosen subset has a better generalization ability, since it is independent of a specific learning algorithm. Furthermore, the filter method usually has a lower time complexity. A wrapper method adopts the classification accuracy of a particular classifier to assess the worth of feature subsets. Thus, the wrapper method can achieve a better classification performance for a specific classifier with a cost of time complexity. However, the selected features by wrapper methods are less generalization on other classifiers. Hence, many researchers use rough set feature selection techniques [19 –21], which are filter approaches through varies positive dependent functions.

Rough set theory [22], proposed by the Polish scientist Pawlak Z. in 1982, is a valid tool in mathematics to deal with the imprecise, uncertain and inconsistent data sets. Feature selection also called attribute reduction is one of significant research topics in rough set theory. It can delete redundant features while maintaining the classification ability of a data set. The Pawlak rough set theory is mainly applicable to the decision systems with discrete data. As for a steganalysis data set, a discretization should be implemented. However, this preprocessing will result in the loss of information, reducing the classification accuracy. The notion of neighborhood rough sets was firstly introduced by Yao [23, 24], which is built on a basis of a neighbor defined by a distance function. It has an ability to granulate continuous data by a neighborhood parameter. The model of neighborhood rough sets was utilized to classification by Hu et al. [25], which can handle the knowledge classification system with not only continuous data but also discrete data. There is a wide applications on neighborhood rough sets including feature selection [20], rule extracting [26], classification [27], cluster [28], gene selection [29], image processing [30] and other fields [31]. Since feature extractions for steganalysis systems produce high-dimensional data sets, which result in time consuming and even poor forecasting accuracy on steganalysis classifiers. Considering the characteristics of steganalysis data, we propose a steganalysis frame based on neighborhood rough sets to obviously reduce extracted features. After carrying out steganography and feature extraction on images, we achieve a steganalysis data set which values are continuous. Firstly, by granulating the data set using a distance function and a neighborhood parameter, we construct some neighborhood class sets. Secondly, we define the concepts of neighborhood positive region and neighborhood dependency to evaluate the classification accuracy of a steganalysis system. Furthermore, the neighborhood dependency-based feature significance is proposed for contributing to search an optimal feature subset. Finally, we deploy SVM and KNN classifiers to test the performance of the selected feature subset. The theoretical analysis and several experiments show that the proposed feature selection method for blind image steganalysis is practical and effective.

The rest of this paper are organized as follows. An introduction to rough sets and neighborhood rough sets are presented in section 2. In section 3, the feature selection for blind steganalysis by neighborhood rough sets is introduced. Then we propose a positive-based feature significance, which is used for measuring the blind steganalysis features. Furthermore, we present a rough set frame for blind steganalysis and propose a feature selection algorithm by neighborhood rough sets, contributing to improve the performance of blind steganalysis. In section 4, some experiments are implemented to indicate the effectiveness of our proposed method. Finally, this paper is concluded with some remarks and discussions in section 5.

2 Rough sets and neighborhood rough sets

In this section, some basic concepts of rough sets and neighborhood rough sets used in a steganalysis system are presented. The further details of rough sets and neighborhood rough sets can be found in [22, 25].

2.1 Rough sets in steganalysis

The theory of rough sets begins with the concept of an information system, which is a quadruple IS = (U, A, V, f), where U is a nonempty object or sample set, and A is a family of features or attributes. V is the value domain of A and f is a mapping information function, f : U × A → V. In a particular situation, an information system is called a decision system if the feature set is A = C ∪ D. The decision system is formalized by DS = (U, C ∪ D, V, f), where C is the condition feature set, and D is the decision feature set.

In an information system, when features are extracted from gray scale images using the moments of characteristic functions, an information system of blind steganalysis is generated. The information system of blind steganalysis is simply represented as SI = (U, A), where U is a set of gray scale images, named as a sample set, and A is a family of extracted features. While the sample set includes cover and stego images, the set can be categorized into two classes. The information system of blind steganalysis is named as a decision system of blind steganalysis, which is expressed as SD = (U, C ∪ D), where C is the extracted feature set, and D is the classified feature which values label a state: hiding or no hiding. Then the decision system of blind steganalysis can be trained and tested using a machine learning classifier.

Definition 1. In an information system of blind steganalysis SI = (U, A), for any subset P ⊆ A, there is an equivalence (also called indiscernibility) relation defined by IND (P) on U:

$IND (P) = {(x, y) \in U \times U | \forall p \in P, v (x, p) = v (y, p)},$ where v (x, p) is a feature value of image x on feature p.

The IND (P) satisfies reflexivity, symmetry and transitivity. If (x, y) ∈ IND (P), then x and y are equivalent on feature subset P. The partition of U generated by IND (P) is defined as U/IND (P) = {[x] _P|x ∈ U}, where [x] _P is an equivalence class of an image x related to the equivalence relation IND (P). The partition is simply denoted as U/P, which is a set of all the equivalence classes on P in U. The elements in [x] _P are equivalent on feature subset P. Equivalence classes, also called information granules, are used to characterize any subsets of U.

Definition 2. Given an information system of blind steganalysis SI = (U, A) and a feature subset P ⊆ A, for an image subset X ⊆ U, it may be possible to describe X imprecisely. We may represent X by a pair of lower and upper approximations of X with respect to P, which are defined as follows:

$P_{*} (X) = {x \in U | [x]_{P} \subseteq X},$ (1) $P^{*} (X) = {x \in U | [x]_{P} ⋂ X \neq \emptyset} .$ (2)

The tuple <P_* (X) , P^* (X)> is called a rough set of X. The set X is definable, if the lower approximation is equal to the upper approximation. Otherwise, the set X is undefinable. The lower approximation is also called positive region, denoted as POS_P (X). And BN_P (X) = P^* (X) - P_* (X) is called a boundary set.

2.2 Neighborhood rough sets in steganalysis

The theory of classical rough sets is used to evaluate a set or a classification for categorical data in knowledge representation systems. As for a steganalysis system which values are real, a discretization should be implemented. However, this preprocessing will lead to an information loss. In the follows, a model of neighborhood rough sets [24, 25] is presented to tackle real-value data in a steganalysis information system.

A distance function is a metric on a set of images, mapping pairs of images into the non-negative real values. Suppose x and y are two images with an N-dimensional feature space A = {a₁, a₂, . . . , a_N}, v (x, a_i) and v (y, a_i) denote the values of images x and y in the ith feature a_i, a general distance metric is defined as:

$D_{A} (x, y) = (\sum_{i = 1}^{N} | v (x, a_{i}) - v (y, a_{i}) |^{p})^{1 / p},$ (3) where it is called as: (1) Manhattan distance $D_{A}^{1}$ , if p = 1; (2) Euclidean distance $D_{A}^{2}$ , if p = 2; (3) Chebychev distance $D_{A}^{\infty}$ , if p =∞.

Definition 3. Suppose SI = (U, A) is an information system of blind steganalysis and δ ∈ [0, 1] is a neighborhood parameter. For a feature subset P ⊆ A, there is a distance function on P, denoted as D_P : U × U → [0, 1], then P determines a neighborhood relation defined as:

${NR}_{δ} (P) = {(x, y) \in U \times U | D_{P} (x, y) \leq δ} .$ (4)

Obviously, the neighborhood relation satisfies reflexivity and symmetry, which is a similarity relation. A neighborhood relation can induce numerous neighborhood classes. A neighborhood class $n_{P}^{δ} (x)$ of an image x ∈ U on the feature subset P is defined as:

$n_{P}^{δ} (x) = {y | y \in U, D_{P} (x, y) \leq δ} .$ (5)

Definition 4. Suppose SI = (U, A) is an information system of blind steganalysis and δ ∈ [0, 1] is a neighborhood parameter. For any image subset X ⊆ U and any feature subset P ⊆ A, the neighborhood P-lower and P-upper approximations of X are defined as follows:

$P_{*} (X)_{δ} = {x \in U | n_{P}^{δ} (x) \subseteq X},$ (6) $P^{*} (X)_{δ} = {x \in U | n_{P}^{δ} (x) ⋂ X \neq \emptyset} .$ (7)

That is, the neighborhood lower approximation P_* (X) _δ is the union of all neighborhood classes which are contained in X, and the neighborhood upper approximation P^* (X) _δ is the union of all neighborhood classes which have a nonempty intersection with X. The tuple <P_* (X) _δ, P^* (X) _δ> is called a neighborhood rough set. Specially, a neighborhood rough set degenerates to a classical rough set when δ = 0. The neighborhood lower approximation is also called neighborhood positive region, denoted as POS_P (X) _δ. And BN_P (X) _δ = P^* (X) _δ - P_* (X) _δ is called a neighborhood boundary set.

3 Feature selection for blind steganalysis by neighborhood rough sets

A pre-analysis processing for blind steganalysis aims to separate the irrelevant and unimportant features from the relevant and important features. This process mainly includes two steps of feature extraction and feature selection. The former step is to generate a data set by extracting multidimensional features from the original images. The latter step is to simplify the extracted data by selecting an important feature subset from the whole set of candidate features. In the following subsections, we introduce a feature measure based on a dependency of neighborhood rough sets into a system of blind steganalysis. Furthermore, a feature selection algorithm by neighborhood rough sets for blind steganalysis is proposed. The framework of blind steganalysis is illustrated in Fig. 1.

Fig.1

The framework of blind steganalysis.

3.1 Feature reduction and feature significance in a system of blind steganalysis

Definition 5. Given a decision system of blind steganalysis SD = (U, C ∪ D), for a feature subset B ⊆ C, it satisfies (1)POS_B (D) _δ = POS_C (D) _δ and (2)∀b ∈ B, POS_B (D) _δ ≠ POS_B-b (D) _δ, the feature subset B is called as a reduct of C.

The first condition guarantees that there is a same classification ability for B and C since POS_B (D) _δ = POS_C (D) _δ. The second condition indicates that there is no superfluous features in the reduct. Therefore, a reduct is a minimal subset of features which has the same classification ability as the whole feature set.

Definition 6. Given a decision system of blind steganalysis SD = (U, C ∪ D), using the definition of neighborhood positive region, the dependency between C and D is defined as:

$γ_{C} (D) = | {POS}_{C} (D)_{δ} | / | U |,$ (8) where POS_C (D) _δ = ∪ _{X_i∈U/D}POS_C (X_i) _δ, X_i is the ith equivalence class induced by U/D. If γ_C (D) =1, D totally depends on C; if 0 < γ_C (D) <1, D partially (in a degree γ_C (D)) depends on C; and if γ_C (D) =0 then D does not depend on C.

Theorem 1. Given a decision system of blind steganalysis SD = (U, C ∪ D), for a feature subset B ⊆ C, it is a reduct if

(1) γ_B (D) = γ_C (D);

(2) ∀b ∈ B, γ_B (D) ≠ γ_B-b (D).

Proof. According to the Definition 6, we know γ_C (D) = |POS_C (D) _δ|/|U|. From the definition of reduct in Definition 5, we can draw the conclusion.

The reduct is not unique. It is a combinational optimization problem to find all of the reducts. There are 2^|C| combinations of feature subsets. It is does not worth reaching all of the reducts by an exhaustive method. In practice, we usually construct a greedy search algorithm to obtain a reduct, which will be used to train a classifier.

Definition 7. Given a decision system of blind steganalysis SD = (U, C ∪ D), for a feature subset B ⊆ C and a feature a ∈ C - B, the significance of the feature a is defined as:

$SGN (a, B, D) = γ_{B \cup a} (D) - γ_{B} (D) .$ (9)

By calculating the change in dependency when a feature is added into the set of candidate features, an estimate of the significance of that feature is obtained. The higher the change in dependency, the more significant the feature is. If the significance is 0, then the feature is dispensable.

Theorem 2. Given a decision system of blind steganalysis SD = (U, C ∪ D), for a feature subset B ⊆ C and a feature a ∈ C - B, B is a reduct if

(1) SGN (a, B, D) =0;

(2) ∀b ∈ B, SGN (b, B, D) >0.

Proof. Since the condition SGN (a, B, D) =0, we obtain γ_B∪a (D) - γ_B (D) =0 by the Definition 7. Consequently, for any feature a ∈ C - B, there is γ_B∪a (D) = γ_B (D). Hence, γ_C (D) = γ_B (D) is found. According to the Definition 6, we achieve POS_C (D) _δ = POS_B (D) _δ. It satisfies the first condition of Definition 5. From the condition ∀b ∈ B, SGN (b, B, D) >0, we know γ_B (D) > γ_B-b (D) according to the Definition 7, so γ_B (D) ≠ γ_B-b (D). From the Definition 6, we achieve POS_B (D) _δ ≠ POS_B-b (D) _δ. It satisfies the second condition of Definition 5. Therefore, it is proved.

This Theorem 2 presents a practical scheme to find optimal feature subsets of a steganalysis system. Hence, we design a feature selection algorithm for blind steganalysis by neighborhood rough sets in the following subsection. The Definition 7 measures the significance of a feature, which can be utilized as a heuristic information for searching a reduct.

3.2 A feature selection algorithm for blind steganalysis by neighborhood rough sets

As it is discussed in the previous section, the dependency reflects the classification ability of a feature subset. Therefore, it can be used to measure the significance of a candidate feature. The goal of feature reduction is to find a minimal subset of features which has the same classification ability as the original whole features. Therefore, the feature reduction is also called as feature selection. Although there are usually many reducts for a given steganalysis data set, it is enough to find one for training the data set. With the proposed measure of feature significance, a greedy search algorithm can be formally designed as follows.

Table 1

Algorithm: Heuristic feature selection algorithm for blind steganalysis by neighborhood rough sets (FSNRS).

Input: A decision system of blind steganalysis SD = (U, C ∪ D) and a neighborhood parameter δ.

Output: A feature reduct red.

01: Compute the equivalence classes on decision D, D = {cover, steg}.

02: Initialize: red =∅, B = C - red.

03: Loop

04: For each a_i ∈ B

05: Suppose P = red ∪ a_i.

06: For each x_j ∈ U

07: Compute the neighborhood class $n_{P}^{δ} (x_{j})$ .

08: Suppose x_j ∈ D_k.

09: Compute the neighborhood positive region POS_P (D_k) _δ.

10: End for

11: Compute the dependency γ_P (D) = | ∪ _{D_k∈U/D}POS_P (D_k) _δ|/|U|.

12: Compute significance of feature a_i: SGN (a_i, red, D) = γ_P (D) - γ_red (D).

13: End for

14: Find the feature a_max ∈ B with maximum feature significance SGN (a_max, red, D).

15: If SGN (a_max, red, D) >0

16: red = red ∪ a_max.

17: B = B - red.

18: Else

19: Goto step 22.

20: End if

21: End loop

22: Return red.

Algorithm: Heuristic feature selection algorithm for blind steganalysis by neighborhood rough sets (FSNRS).
Input: A decision system of blind steganalysis SD = (U, C ∪ D) and a neighborhood parameter δ.
Output: A feature reduct red.
01: Compute the equivalence classes on decision D, D = {cover, steg}.
02: Initialize: red =∅, B = C - red.
03: Loop
04: For each a_i ∈ B
05: Suppose P = red ∪ a_i.
06: For each x_j ∈ U
07: Compute the neighborhood class $n_{P}^{δ} (x_{j})$ .
08: Suppose x_j ∈ D_k.
09: Compute the neighborhood positive region POS_P (D_k) _δ.
10: End for
11: Compute the dependency γ_P (D) = \| ∪ _{D_k∈U/D}POS_P (D_k) _δ\|/\|U\|.
12: Compute significance of feature a_i: SGN (a_i, red, D) = γ_P (D) - γ_red (D).
13: End for
14: Find the feature a_max ∈ B with maximum feature significance SGN (a_max, red, D).
15: If SGN (a_max, red, D) >0
16: red = red ∪ a_max.
17: B = B - red.
18: Else
19: Goto step 22.
20: End if
21: End loop
22: Return red.

The algorithm FSNRS starts from an empty set, until it stops when the addition of any feature increases no change of the dependency. If there are m images and n condition features, the computational complexity for calculating a neighborhood class is O (m * n) by a hash sort [32]. So the computational complexity of dependency is O (m * n). Furthermore, the worst search time for a reduct will bring about n evaluations of the dependency. Therefore, the overall computational complexity of the proposed algorithm is about O (m * n²).

4 Experimental analyses

4.1 Preparation of data sets

In experimental studies, the breaking out steganography system (BOSSbase 1.01) has a gray scale image database, which is used for preparing the training and testing data sets. It can be downloaded from the web site: http://dde.binghamton.edu/ download/feature_extractors/. We select 3000 images from BOSSbase 1.01 and change them into jpeg format. These images are each of 512 × 512 resolution, with sizes varying from 5.64 KB to 73.1 KB. In the 3000 images, 2400 images are used for preparing a training data set and 400 images are used as a testing data set. Before a feature extracting process, a steganography need be employed in these images.

4.2 Methods of steganography

In the 3000 images, odd number images are used as clean images and the even number images are used for hiding text messages by two well known steganography algorithms. Hence, these images are classified into two categories: cover(clean) and stego. The two steganography algorithms used for embedding text in stego images are F5 [1] and UNIWARD [4]. The rates of hidden text embedding vary from 0.2 to 0.6 per pixel with step of 0.2, which are marked with payloads 0.2, 0.4 and 0.6 respectively. In the follows, a feature extracting process resulting in high dimensional data is used for evaluating experimental images.

Table 2
Simplified rates with different steganography methods, feature extractors and neighborhood parameters

Neighborhood parameters UNIWARD+SPAM UNIWARD+CHEN F5+SPAM F5+CHEN

0.2 0.4 0.6 0.2 0.4 0.6 0.2 0.4 0.6 0.2 0.4 0.6

0.05 99.85 99.85 99.85 99.79 99.79 99.79 99.42 99.42 99.27 99.18 99.18 99.38

0.10 99.85 99.85 99.85 99.79 99.79 99.79 98.98 98.83 98.98 98.77 98.77 99.18

0.15 99.85 99.85 99.85 99.79 99.79 99.79 96.65 98.11 98.40 98.15 98.15 98.77

0.20 99.85 99.85 99.85 99.79 99.79 99.79 95.63 97.67 97.81 97.33 97.53 98.35

0.25 99.85 99.71 99.71 99.79 99.59 99.59 96.06 99.13 95.92 98.56 98.35 97.74

0.30 99.71 99.71 99.71 99.59 99.59 99.59 98.83 92.42 94.46 98.56 98.97 98.56

0.35 99.71 99.71 99.71 99.59 99.59 99.59 78.57 85.13 98.25 98.77 98.56 98.56

0.40 99.71 99.71 99.71 99.59 99.59 99.59 98.98 98.98 98.98 97.74 98.97 98.97

0.45 99.71 99.71 99.71 99.59 99.59 99.59 66.18 98.83 85.71 97.74 93.21 98.15

0.50 99.71 99.71 99.71 99.59 99.59 99.56 98.83 98.69 98.40 98.15 98.35 98.56

0.55 99.71 99.71 99.56 99.59 99.59 99.59 98.40 98.25 98.11 98.15 98.35 98.56

0.60 99.71 99.56 99.56 99.38 99.38 99.18 97.67 97.67 97.67 97.74 96.30 98.15

0.65 99.56 99.56 99.56 99.38 99.38 99.38 97.67 97.38 97.81 97.53 98.15 97.94

0.70 99.42 99.42 99.42 99.38 99.38 99.38 97.23 97.08 96.79 86.01 97.74 97.12

0.75 99.42 99.42 99.42 99.38 99.38 99.18 96.50 96.36 96.79 93.42 92.39 95.06

0.80 99.42 99.42 99.42 99.38 99.38 99.18 41.40 95.48 96.06 91.15 89.09 96.09

0.85 99.27 99.27 99.27 99.18 99.18 98.77 0.15 94.46 95.04 90.95 94.86 86.42

0.90 99.27 99.27 99.27 99.18 99.18 99.18 0.15 94.75 94.75 79.84 90.74 93.21

0.95 99.13 99.13 99.13 98.97 98.15 98.97 0.15 19.24 39.80 90.95 88.89 85.60

Neighborhood parameters	UNIWARD+SPAM	UNIWARD+CHEN	F5+SPAM	F5+CHEN
0.05	99.85	99.85	99.85	99.79	99.79	99.79	99.42	99.42	99.27	99.18	99.18	99.38
0.10	99.85	99.85	99.85	99.79	99.79	99.79	98.98	98.83	98.98	98.77	98.77	99.18
0.15	99.85	99.85	99.85	99.79	99.79	99.79	96.65	98.11	98.40	98.15	98.15	98.77
0.20	99.85	99.85	99.85	99.79	99.79	99.79	95.63	97.67	97.81	97.33	97.53	98.35
0.25	99.85	99.71	99.71	99.79	99.59	99.59	96.06	99.13	95.92	98.56	98.35	97.74
0.30	99.71	99.71	99.71	99.59	99.59	99.59	98.83	92.42	94.46	98.56	98.97	98.56
0.35	99.71	99.71	99.71	99.59	99.59	99.59	78.57	85.13	98.25	98.77	98.56	98.56
0.40	99.71	99.71	99.71	99.59	99.59	99.59	98.98	98.98	98.98	97.74	98.97	98.97
0.45	99.71	99.71	99.71	99.59	99.59	99.59	66.18	98.83	85.71	97.74	93.21	98.15
0.50	99.71	99.71	99.71	99.59	99.59	99.56	98.83	98.69	98.40	98.15	98.35	98.56
0.55	99.71	99.71	99.56	99.59	99.59	99.59	98.40	98.25	98.11	98.15	98.35	98.56
0.60	99.71	99.56	99.56	99.38	99.38	99.18	97.67	97.67	97.67	97.74	96.30	98.15
0.65	99.56	99.56	99.56	99.38	99.38	99.38	97.67	97.38	97.81	97.53	98.15	97.94
0.70	99.42	99.42	99.42	99.38	99.38	99.38	97.23	97.08	96.79	86.01	97.74	97.12
0.75	99.42	99.42	99.42	99.38	99.38	99.18	96.50	96.36	96.79	93.42	92.39	95.06
0.80	99.42	99.42	99.42	99.38	99.38	99.18	41.40	95.48	96.06	91.15	89.09	96.09
0.85	99.27	99.27	99.27	99.18	99.18	98.77	0.15	94.46	95.04	90.95	94.86	86.42
0.90	99.27	99.27	99.27	99.18	99.18	99.18	0.15	94.75	94.75	79.84	90.74	93.21
0.95	99.13	99.13	99.13	98.97	98.15	98.97	0.15	19.24	39.80	90.95	88.89	85.60

4.3 Extraction and selection of features

In these experiments, two extractors are employed to achieve the features for blind steganalysis, which are the subtractive pixel adjacency model (SPAM) [11] and the Chen’s algorithm (CHEN) [12]. SPAM has 686 condition features and one decision feature, and CHEN has 486 condition features and one decision feature. Since there are massive and high dimensional data produced by the feature extraction of JPEG images, a rough set feature reduction is carried out for selecting some significant features. Then, the SVM and KNN classifiers have been used for training on these selected features from high dimensional extracted features. Also the final classification of test images, whether clean or stego, is done by the trained models. Gaussian kernel is used in SVM, while k = 5 is set in KNN. All the classification experiments are implemented by a five-fold cross-validation on selected features. The classification accuracy is the mean of five tests. The experimental scheme of blind steganalysis is illustrated in Fig. 2.

Fig.2

The experimental scheme of blind steganalysis.

4.4 Performance evaluation of the proposed method

In this subsection, the performance of the proposed feature selection method by neighborhood rough sets (FSNRS) for different steganographies and extractors is investigated. The steganographies are UNIWARD and F5 with different payloads 0.2, 0.4 and 0.6. The extractors are SPAM and CHEN methods. The values of the extracted features are normalized between 0 and 1. In the FSNRS, the Euclidean distance is used and the value of neighborhood parameter varies from 0.05 to 0.95 step by 0.05. In order to evaluate the simplification of selected features, we adopt a concept named simplified rate, which is defined as follows:

$rate = \frac{| C | - | R |}{| C |}$ , where |R| is the number of selected feature subsets and |C| is the number of original whole features. The simplified rates are showed in Table 1 with different steganography methods, feature extractors and neighborhood parameters. The numbers of selected feature subsets are illustrated in Figs. 3, 6, 9 and 12. The predictive accuracies of SVM upon the selected feature subsets are presented in Figs. 4, 7, 10 and 13, while those of KNN are illustrated in Figs. 5, 8, 11 and 14.

The Figs. 3, 6, 9 and 12 demonstrate the numbers of selected features with different neighborhood parameter values in the proposed method. The abscissa is neighborhood parameter, while the ordinate is the number of selected features. In Figs. 3 and 6, the data hiding method is UNIWARD with different payloads, while the extractors are SPAM and CHEN respectively. In Figs. 9 and 12, the data hiding method is F5 with different payloads, while the extractors are SPAM and CHEN respectively. As we know, the SPAM extractor produces 686 dimensions of features and the CHEN extractor produces 486 dimensions. From the illustrations in Table 1, Figs. 3, 6, 9 and 12, it is obvious that our feature selection method can strongly reduce these features. It will save more training time and testing time.

Fig.3

The number of selected features with UNIWARD steganography and SPAM extractor via different neighborhood parameters.

Fig.4

The SVM accuracy of selected features with UNIWARD steganography and SPAM extractor via different neighborhood parameters.

The SVM predictive accuracy of selected features with UNIWARD steganography and SPAM extractor is illustrated in Fig. 4. The ordinate is predictive accuracy of selected features, while the abscissa is the value of neighborhood parameter. But the last value in abscissa labeled as 686 represents the total features which are extracted by SPAM. It can be seen that the SVM predictive accuracies of total features are 0.9544, 0.9564 and 0.9582 under different payloads 0.2, 0.4 and 0.6. As for our selected features under payload 0.2, the SVM predictive accuracy of those is reaching the maximal value with 0.9912 at neighborhood parameter 0.8. While the payload is 0.4, the SVM predictive accuracy of selected features is reaching the maximal value with 0.999 at neighborhood parameter 0.7. All the SVM predictive accuracies of selected features under different neighborhood parameters with payloads 0.2 and 0.4 are higher than that of whole features. As for payload 0.6, the SVM predictive accuracy of selected features is reaching the maximal value with 0.996 at neighborhood parameter 0.9. Only one time, the SVM predictive accuracy of selected features at neighborhood parameter 0.05 is lower than that of whole features.

Fig.5

The KNN accuracy of selected features with UNIWARD steganography and SPAM extractor via different neighborhood parameters.

The KNN predictive accuracy of selected features with UNIWARD steganography and SPAM extractor is illustrated in Fig. 5. It can be seen that the KNN predictive accuracies of total features are 0.9544, 0.9651 and 0.9691 under different payloads 0.2, 0.4 and 0.6. As for our selected features under payload 0.2, the KNN predictive accuracy of those is reaching the maximal value with 0.9992 at neighborhood parameter 0.9. Only one time, the KNN predictive accuracy of selected features at neighborhood parameter 0.1 is lower than that of whole features. While the payload is 0.4, the KNN predictive accuracy of selected features is reaching the maximal value with 0.9929 at neighborhood parameter 0.8. Fourteen times in nineteen times, the KNN predictive accuracies of selected features are higher than that of whole features. As for payload 0.6, the KNN predictive accuracy of selected features is reaching the maximal value with 0.9997 at neighborhood parameter 0.45. Fourteen times in nineteen times, the KNN predictive accuracies of selected features are also higher than that of whole features.

From the results of Figs 3, 4, and 5, they indicate that our proposed feature selection method has an ability to find valid features, which are not only more compact but also higher predictive accuracies in SVM and KNN classifiers.

Fig.6

The number of selected features with UNIWARD steganography and CHEN extractor via different neighborhood parameters.

Fig.7

The SVM accuracy of selected features with UNIWARD steganography and CHEN extractor via different neighborhood parameters.

Fig.8

The KNN accuracy of selected features with UNIWARD steganography and CHEN extractor via different neighborhood parameters.

The Figs. 6, 7 and 8 illustrate numbers and predictive accuracies of selected features with different neighborhood parameter values and different payload hiding information. In Fig. 6, the abscissa is neighborhood parameter, while the ordinate is the number of selected features. In Figs. 7 and 8, the abscissa is neighborhood parameter, while the ordinate is the predictive accuracy of selected features. The data hiding method is UNIWARD with different payloads, while the extractor is CHEN. As we know, the CHEN extractor produces 486 dimensions of features. In Fig. 6, it can be seen that our feature selection method can simplify these features. In Fig. 7, the SVM predictive accuracies of the whole features are 0.9686, 0.9765 and 0.9844, respectively with payloads 0.2, 0.4 and 0.6. As for our selected features, the SVM predictive accuracies of those are reaching maximal values which are 0.9826, 0.9961 and 0.9909, at neighborhood parameters 0.8, 0.1 and 0.6, and with payloads 0.2, 0.4 and 0.6 respectively. In Fig. 8, the KNN predictive accuracies of the whole features are 0.9701, 0.9782 and 0.9847, respectively with payloads 0.2, 0.4 and 0.6. As for our selected features, the KNN predictive accuracies of those are reaching maximal values which are 0.9773, 0.9953 and 0.9924, at neighborhood parameters 0.55, 0.1 and 0.6, and with payloads 0.2, 0.4 and 0.6 respectively. From these results in Figs 7 and 8, the predictive accuracies of our selected features can be higher than those of whole features at suitable neighborhood parameter values.

According to the above analyses we can see that our proposed method is capable of improving the performance of a steganalysis system. Since our proposed feature selection method is used with an adjustable neighborhood parameter, the classification performance of selected features is better than that of the original whole features in most situations. As for the next Figs. 10, 11, 13, 14, we can draw the same results.

Fig.9

The number of selected features with F5 steganography and SPAM extractor via different neighborhood parameters.

Fig.10

The SVM accuracy of selected features with F5 steganography and SPAM extractor via different neighborhood parameters.

Fig.11

The KNN accuracy of selected features with F5 steganography and SPAM extractor via different neighborhood parameters.

Fig.12

The number of selected features with F5 steganography and CHEN extractor via different neighborhood parameters.

Fig.13

The SVM accuracy of selected features with F5 steganography and CHEN extractor via different neighborhood parameters.

Fig.14

The KNN accuracy of selected features with F5 steganography and CHEN extractor via different neighborhood parameters.

5 Conclusions and discussions

This paper has focused the development of information security fields, tackling the problems of dimensionality curse of high-dimensional data, as well as dealing with real values of an image steganalysis system. We have achieved this by introducing the neighborhood rough set theory and proposing a feature selection method for blind image steganalysis. Considering the continue values of steganalysis systems, we granulate these values by a neighborhood parameter and construct neighborhood granules. Consequently, a positive region of neighborhood rough sets is developed to measure the uncertainty of a steganalysis system. Based on the positive region, a dependency is proposed to evaluate the significance of extracted features in a steganalysis system. Furthermore, a feature selection algorithm by neighborhood rough sets is designed. To show the usefulness of the proposed algorithm, various experiments of feature selection and learning algorithms are carried out on extracted features which are achieved by both SPAM and CHEN extractors in an image steganalysis system. The performance of selected features by our proposed method has been compared with that of original whole features and the classification accuracy under different neighborhood parameter values has also been studied. Experimental results show that our proposed feature selection algorithm for blind image steganalysis has a good advantage over reducing dimensionality and classification performance. In the future, the proposed feature selection method will be expeditiously employed to solve the steganalysis engineering problems with high dimensions.

Nowadays, the technologies of dealing with big data are deeply influencing both present and future life. Big data usually includes data sets with large sizes beyond the ability of commonly used methods to tackle within a tolerable elapsed time. The size of a big data reflects two aspects with large samples and high dimensions. In a big data of image steganalysis system, high dimension means ten of thousands of extracted features, while large samples means massive image samples. Our work provides an optional strategy for tackling the big data with high dimensions in the future. But for the big data with massive image samples, our proposed algorithm should be improved by a parallel computing.

Compliance with ethical standards

Funding: This study was funded by the National Natural Science Foundation of China (grant numbers 61976183 and 61573297), the Natural Science Foundation of Fujian Province (grant number 2019J01850) and the Program for New Century Excellent Talents in Fujian Province University.

References

Westfeld

, F5-a steganographic algorithm, In: Proceedings of the 4th international workshop on information hiding, London, Springer-Verlag, 2001, pp. 289–302.

, Yang

, Liu

, et al., Quantitative steganalysis of JPEG additive steganography based on boundary effect, Journal of Electronic Imaging 27(2) (2018), 1–6.

Holub

and Fridrich

, Designing steganographic distortion using directional filters, In: Proceedings of the IEEE International Workshop on Information Forensics and Security, IEEE, 2012, pp. 234–239.

Holub

, Fridrich

and Denemark

, Universal distortion function for steganography in an arbitrary domain, EURASIP Journal on Information Security 1 (2014), 1–13.

Filler

and Fridrich

, Gibbs construction in steganography, IEEE Transactions on Information Forensics and Security 5(4) (2010), 705–720.

Fridrich

, Goljan

and Du

, Detecting LSB steganography in color and gray-scale images, IEEE Multimedia 8(4) (2001), 22–28.

Cai

, Agaian

S.S.

and Wang

, An effective algorithm for breaking F5, In: IEEE 7th workshop on multimedia signal processing, 2005, pp. 1–4.

Zhang

and Zhong.

S.P.

, Blind steganalysis method for bmp images based on statistical MWCF and f-score method, In: Proceedings of the 2009 international conference on wavelet analysis and pattern recognition, 2009, pp. 442–447.

Mohammadi

F.G.

and Abadeh

M.S.

, Image steganalysis using a bee colony based feature selection algorithm, Engineering Applications of Artificial Intelligence 31 (2014), 35–43.

10.

Pevny

and Fridrich

, Merging Markov and DCT features for multi-class JPEG steganalysis, In: Proceedings of the International Society for Optical Engineering, 6505 (2007), pp 1–13.

11.

Pevny

, Bas

and Fridrich

, Steganalysis by subtractive pixel adjacency matrix, IEEE Transactions on Information Forensics and Security 5(2) (2010), 215–224.

12.

Chen

and Shi

Y.Q.

, JPEG image steganalysis utilizing both intrablock and interblock correlations, In: IEEE International Symposium on Circuits and Systems, IEEE 2008, pp. 3029–3032.

13.

Kodovsky

and Fridrich

, Calibration revisited, In: Proceedings of the 11th ACM Multimedia and Security Workshop, Princeton, 2009, pp. 63–74.

14.

Narenadra

P.M.

and Fukunaga

, Abranch and bound algorithm for feature subset selection, IEEE Transactions on Computers 26 (1977), 917–922.

15.

Liu

, Liu

and Zhang

, Feature selection using mutual information: an experimental study, In: Proceeding of International Conference on Artificial Intelligence, 2008, pp. 235–246.

16.

, Miao

D.Q.

and Pedrycz

, Granular multi-label feature selection based on mutual information, Pattern Recognition 67 (2017), 410–423.

17.

and Liu

, Feature selection for high-dimensional data: A fast correlation-based filter solution, In: Proceedings of the Twentieth International Conference on Machine Learning, DBLP, 2003, pp. 856–863.

18.

Wang

, An

, Chen

, et al., Accelerating wrapper-based feature selection with Knearest-neighbor, Knowledge-Based Systems 83(1) (2015), 81–91.

19.

Chen

Y.M.

, Miao

D.Q.

and Wang

R.Z.

, A rough set approach to feature selection based on antcolony optimization, Pattern Recognition Letters 31(3) (2010), 226–233.

20.

Q.H.

, Yu

D.R.

, Liu

J.F.

and Wu

C.X.

, Neighborhood rough set based heterogeneous feature subset selection, Information Sciences 178(18) (2008), 3577–3594.

21.

Raza

M.S.

and Qamar

, An incremental dependency calculation technique for feature selection using rough sets, Information Sciences 343 (2016), 41–65.

22.

Pawlak

, Rough sets, International Journal of Computer Information Science 11 (1982), 341–356.

23.

Yao

Y.Y.

, Relational interpretations of neighborhood operators and rough set approximation operators, Information Science 111 (1998), 239–259.

24.

Yao

Y.Y.

, Granular computing using neighborhood systems, In: Proceeding of the 3rd On line World Conference on Soft Computing (WSC3), Springer-Verlag, London, 1999, pp. 539–553 .

25.

Q.H.

, Yu

D.R.

and Xie

Z.X.

, Neighborhood classifiers, Expert Systems with Applications 34(2) (2008), 866–876.

26.

, Hu

Q.H.

, Zhu

P.F.

, et al., Rule learning for classification based on neighborhood covering reduction, Information Sciences 181(24) (2011), 5457–5467.

27.

Yao

and Lu

Y.H.

, Neighborhood rough set and SVM based hybrid credit scoring classifier, Expert Systems with Applications 38(9) (2011), 11300–11304.

28.

Cao

, Liang

and Jiang

, An initialization method for the K-Means algorithm using neighborhood model, Computers & Mathematics with Applications 58(3) (2009), 474–483.

29.

Chen

Y.M.

, Zhang

Z.J.

, Zheng

J.Z.

, et al., Gene selection for tumor classification using neighborhood rough sets and entropy measures, Journal of Biomedical Informatics 67 (2017), 59–68.

30.

, Pedrycz

and Miao

D.Q.

, Neighborhood rough sets based multi-label classification for automatic image annotation, International Journal of Approximate Reasoning 54(9) (2013), 1373–1387.

31.

Chen

Y.M.

, Zeng

Z.Q.

and Lu

J.W.

, Neighborhood rough set reduction with fish swarm algorithm, Soft Computing 21(23) (2017), 6907–6918.

32.

Liu

, Huang

W.L.

, Jiang

Y.L.

and Zeng

Z.Y.

, Quick attribute reduct algorithm for neighborhood rough set model, Information Sciences 271 (2014), 65–81.