Enhanced image steganalysis through reinforcement learning and generative adversarial networks

Abstract

With the surging prominence of digital communication platforms, there has been an intensified emphasis on ensuring robust security and privacy measures. Against this backdrop, image steganalysis has emerged as a critical discipline, employing advanced methods to detect clandestine data within image files. At the core of our research is an innovative exploration into image steganalysis using an amalgamation of enhanced reinforcement learning techniques and online data augmentation. This methodology ensures the meticulous identification of concealed data within images. Our design integrates triple parallel dilated convolutions, enabling concurrent extraction of feature vectors from the input images. Once extracted, these vectors are synthesized, paving the way for subsequent classification tasks. To substantiate the efficacy of our approach, we conducted tests on a comprehensive dataset sourced from BossBase 1.01. Furthermore, to discern the influence of transfer learning on our proposed model, the BOWS dataset was employed. Notably, these datasets present a challenge due to its inherent imbalance. To counteract this, we incorporated an advanced Reinforcement Learning (RL) framework. Herein, the dataset samples are envisioned as states in a sequence of interrelated decisions, with the neural network playing the role of the decision-making agent. This agent is then incentivized or reprimanded based on its accuracy in discerning between the minority and majority classes. To bolster our classification capabilities, we innovatively employed data augmentation using images generated by a Generative Adversarial Network (GAN). Concurrently, a regularization mechanism was instituted to alleviate prevalent GAN-related challenges, such as mode collapse and unstable training dynamics. Our experimental outcomes underscore the potency of our methodology. The results highlight a remarkable capability to discern between pristine and steganographic images, registering an average accuracy rate of 85%.

Keywords

Image steganalysis reinforcement learning generative adversarial network imbalanced classification transfer learning

1. Introduction

Currently, the internet houses a virtually limitless repository of images, which serve as avenues for individuals to document their experiences, articulate their emotions, and pursue a range of personal endeavors. Unfortunately, this proliferation has been shadowed by the sophisticated evolution of image steganography software; a tool now adeptly employed by criminal syndicates to clandestinely communicate through messages embedded within standard image exchanges. The cunning integration of these concealed payloads often mimics random image noise generated by camera sensors and associated circuitry, thus rendering the naked eye virtually powerless in discerning images associated with criminal undertakings. The dire implications for public safety engendered by this technology are palpable, necessitating urgent and robust countermeasures. Given this critical backdrop, the academic and technical communities are rallying to spearhead research initiatives to pioneer advanced image steganalysis techniques. These methodologies aim not only to unveil these covert communications but also to forge a pathway towards neutralizing the looming threat they represent, thereby safeguarding public security [1].

Image steganography, a process utilized to embed data covertly within visual content, has emerged as a predominant tactic in concealed communications [2]. Central to the efficacy of image steganography is the sophistication of its underlying algorithms, which ingeniously interweave confidential communications into the complex tapestry of visual data. These algorithms are devised to ensure an impeccable integration, meticulously preventing any perceptible distortions or statistical alterations, thus preserving the confidentiality of the concealed data. By assimilating the hidden content seamlessly with the visual narrative, image steganography facilitates undercover transmissions, all the while upholding a façade of normality. This strategy is instrumental across a spectrum of sectors including secure data transfer and watermarking, where the imperative of safeguarding sensitive information is heightened, underscoring its critical role in contemporary information security frameworks.

Unlike steganography techniques, steganalysis methods [3, 4] are focused on uncovering, altering, or removing concealed information within a stego object. Steganalysis algorithms can typically be categorized into two groups: one set of methods is designed to identify whether hidden data is present or absent in digital media [5], while the other set aims to reveal the actual content of hidden data [6]. The aim of this paper is to introduce a technique for identifying whether hidden data is present or not in digital media. Regarding the processing domains, steganalysis methods can be categorized into four main areas: statistical analysis [5], transform domain analysis, blind/universal steganalysis [7], and spread-spectrum steganalysis [8]. Statistical analysis explores the statistical features of an image, such as pixel value histograms and correlations between pixels, to identify irregularities suggesting the presence of steganography. Transform domain analysis investigates an image in transformed domains, including Fourier and DCT, to pinpoint alterations made by steganographic algorithms, offering a detailed understanding of how data embedding affects image coefficients. Blind/universal steganalysis employs machine learning and statistical methods, working without knowledge of the specific steganographic techniques used, thus providing a flexible approach to detecting a wide array of steganographic tactics based on patterns learned from large datasets. Lastly, spread-spectrum steganalysis delves into the analysis of images with data hidden across a broad frequency band, utilizing sophisticated techniques to uncover subtle changes in spectral properties, despite the inherent challenges due to low signal-to-noise ratios. Each method offers a distinctive approach to unveiling concealed data in digital images, enhancing security measures in the digital communication sphere.

Over the past five years, deep learning has emerged as a groundbreaking approach, showcasing unparalleled potentials in the realm of image steganalysis [9]. Xie et al. [10] introduced an enhanced residual network (ERANet) fortified with self-attention mechanisms to overcome the constraints inherent in the existing CNN frameworks applied to image steganalysis. Though initially conceived using the BOSSbase dataset, conventional CNNs frequently fall short when applied to newer, intricate datasets such as ALASKA#2. ERANet addresses this deficiency through the integration of an advanced residual method coupled with a global self-attention strategy, enabling the extraction of more potent features and facilitating its deployment in complex environments. Moreover, the addition of the Enhanced Low-Level Feature Representation Module into other CNNs aids in cherry-picking the most illustrative features, despite incurring a marginal increase in computational demands. Similarly, Chen et al. [11] put forth a multi-scale deep neural network for steganalysis, enhancing the extraction of image region correlations. By harnessing deep residual networks, they managed to achieve a level of steganography detection precision that surpasses that of single-scale networks. Meanwhile et al. [9] demonstrated the viability of using CNN architectures to retrieve steganographic signals even in the absence of pre-established high-pass filters. However, the dwindling signal-to-noise ratio following image resizing restricts the effectiveness of a scant few state-of-the-art methods in analyzing steganographic content within images of variable dimensions, to the best of our knowledge. Tsang and Fridrich [12] proposed a method that maintains a uniform number of statistical moments extracted from feature maps preceding the entry into the fully connected segment, augmenting the analysis procedure. This collective body of work underscores the pivotal advancements in deep learning techniques, heralding a new epoch in image steganalysis characterized by heightened accuracy and refined analytical processes.

While existing steganalysis algorithms offer practicality and efficiency, they frequently presume the presence of an ideal sample distribution in the training datasets. This notion presupposes a negligible disparity in the sample size between cover and stego images. However, real-world scenarios routinely present a class imbalance problem in steganalysis, where the number of instances in different categories is unequal. Typically, the majority class, often represented by cover images (negative class), encompasses a substantially larger sample pool compared to the stego images, forming the minority or the positive class [13]. This discrepancy in sample allocation tends to favor the majority class in current models, consequently undermining the detection accuracy for the minority class, characterized by its limited and irregularly distributed information. This imbalance not only poses a formidable challenge in accurately identifying the instances of the minority class but also magnifies the repercussions of misclassification. Hence, there is a pressing need to recalibrate existing algorithms to enhance their sensitivity to the minority class, mitigating potential errors and fostering more reliable outcomes.

To date, approaches have been utilized both at the data and algorithm levels for imbalanced classification. Data-level methods encompass actions such as reducing sample size, increasing sample size, or a combination of both to alleviate the negative effects of data imbalance [14]. Algorithm-level strategies focus on increasing the importance of the minority group to tackle the imbalance [15]. The emergence of Deep Reinforcement Learning (DRL) has gained attention for its effectiveness in handling imbalanced classification tasks. However, these approaches encounter difficulties, especially in managing the delicate balance between bias and variance. However, these techniques encounter obstacles, especially when dealing with the complex balance between bias and variance. The responsiveness of DRL to hyperparameters amplifies these challenges, possibly resulting in unpredictable and varying performance across various datasets and tasks. Amidst the intricate challenges presented by these factors, Proximal Policy Optimization (PPO) stands out as a promising and encouraging solution within the realm of on-policy reinforcement learning. Its emergence offers a glimmer of hope in addressing the intricate dynamics and delicate balance required for effective reinforcement learning, particularly in scenarios where policy optimization is essential. The capacity of PPO to navigate the nuanced interplay of exploration and exploitation, coupled with its adaptability to varying environments, positions it as a potentially transformative approach for tackling complex problems in reinforcement learning.

Recently, the myriad uses of GANs [16] have experienced a notable surge in popularity, garnering attention from both academic communities and the commercial sector. GANs have shown remarkable aptitude in various undertakings such as alleviating domain discrepancies and creating unique image samples [17]. Simultaneously, there is a growing curiosity among scholars to utilize GANs for data augmentation [18]. Although conventional augmentation methods primarily rely on offline processing, which unavoidably increases dataset dimensions to enhance model performance, a shift is observed towards the adoption of online augmentation approaches. These methods are gaining traction for allowing a consistent dataset size, where each mini-batch preserves a subset of the original images, replacing the remainder with images fabricated by GANs. This agile online augmentation technique offers a resourceful way to enrich training datasets without escalating their size.

This study explores the combination of an enhanced RL training method called PPO, with a data augmentation approach driven by GANs, specifically designed for image steganalysis. To address the issue of imbalanced class distribution, our framework transforms the problem into a series of well-considered decisions. In each iteration, an agent interacts with an environmental state represented by a training sample. Following a predefined policy, the agent performs classification tasks. The results of these tasks, whether successful or not, result in corresponding positive and negative rewards. Notably, classes with lower representation in the dataset receive higher reward values, deviating from more frequently occurring classes. As the agent navigates through this decision-making process, its primary goal is to refine reward optimization, thereby improving the accuracy of sample classification. For real-time data augmentation, we leverage the potential of a deep convolutional GAN model. Recognizing the susceptibility of GANs to mode collapse and aiming to ensure training stability, we introduce a regularization term. This calculated addition helps mitigate the risk of overfitting while further enhancing the precision of the model. The effectiveness of the model is assessed by employing the BossBase 1.01 dataset. Additionally, the BOWS dataset is employed to investigate the impact of transfer learning on both the proposed model and other deep learning models.

The main findings of this research can be summarized as: 1) We present an advanced reinforcement learning method specifically designed to address the imbalanced classification issues found in image steganalysis. To avoid overly drastic policy changes, we propose using a surrogate objective function, 2) The suggested model integrates dilated convolutions, thereby amplifying its capability to extract vital features from images, leading to more accurate classification outcomes, 3) By adopting online data augmentation, our model capitalizes on a diverse set of samples. This diversity enhances its adaptability and performance on novel data, 4) TL is employed to gauge the efficacy of our model. Leveraging insights from previously trained models and associated tasks, TL allows our model to effortlessly adapt its learning to unfamiliar datasets and areas, boosting its efficiency and broad applicability.

The structure of the paper for the upcoming sections is as follows: In Section 2, we provide an overview of the existing literature. Section 3 takes an in-depth look at our proposed approach, thoroughly explaining the core methodology. Section 4 presents the results of our practical experiments and the subsequent analysis. We summarize our concluding remarks and outline potential areas for future research in Section 5.

2. Related works

Image steganography methods are designed with the objective of discreetly embedding confidential messages within a host image. Even if an external observer manages to detect the presence of steganographic content, suspicions regarding the concealed data are mitigated since the manipulated image still maintains the appearance of an ordinary one. Currently, image steganography techniques can be broadly categorized into three main classes: basic steganography [19, 20], adaptive steganography [21, 22], and deep learning-based approaches [1, 23].

Basic steganographic techniques, although straightforward and often employed for online entertainment purposes, typically result in artifacts that can be readily detected. For instance, the Least-Significant-Bit (LSB) method [19, 20] involves the alteration of pixel values to embed a message, with little regard for the magnitude of distortion introduced. These techniques are vulnerable to attacks that exploit statistical information derived from the original images [2].

Currently, adaptive steganography stands as the preferred approach due to its enhanced security measures. It involves embedding messages within more intricate regions of cover images [21, 22] and utilizes advanced steganographic codes like Syndrome Trellis Codes (STCs) [24] to minimize noticeable alterations. For instance, Pevný et al. [25] introduced a technique called HUGO, which highlighted the importance of weighted disparities in feature vectors for steganalysis, focusing on distortion analysis. Holub and Fridrich [26] developed a model that assesses the impact of altering individual pixels in their WOW technique. Subsequently, they refined this method by incorporating directional residuals obtained from a filter bank in the S-UNIWARD approach [22]. In their HILL technique, Li et al. [27] applied a high-pass filter to identify unpredictable regions and utilized two low-pass filters to create a more focused cost function. These adaptive steganography methods represent the current state-of-the-art in concealing data within images while maintaining a low risk of detection.

The realm of deep learning in steganography is continually evolving, and it can be broadly classified into four main subcategories [28]: 1) Synthesis-Driven: This approach involves the generation of images followed by the embedding of a hidden message [23, 29], 2) Probability Map Creation: For instance, in ASDL-GAN and UT-6HPF-GAN, as demonstrated by Tang et al. [30] and Yang et al. [31], respectively, a generator network is utilized to create an alteration map from the cover image. This alteration map is strategically designed to deceive discriminative networks, 3) Outsmarting CNN-Based Steganalysis: Techniques like ADV-EMB by Tang et al. [32] are designed to refine the costs associated with alterations based on gradients propagated from target networks, effectively outwitting CNN-based steganalysis methods, and 4) The Three-Player Game Strategy: Zhu et al. [33] introduced joint encoder and decoder networks known as HiDDeN networks. In this strategy, when presented with an input message and a cover image, the encoder generates a visually indistinguishable encoded image. Subsequently, the decoder can extract the original message from this encoded image.

Recent advancements in deep learning and neural network techniques have revolutionized image steganalysis, offering notable improvements in detection accuracy [9]. CNN-based steganalysis methods, in contrast to traditional approaches, negate the necessity for manual feature engineering, opting instead for the autonomous extraction of detailed feature representations via backpropagation. Qian et al. [34] pioneered with GNCNN, a custom CNN framework. With its specialized high-pass filtering kernels, feature extraction convolutional units, and classification-oriented fully-connected layers, GNCNN became the inaugural CNN-based method to match the efficiency of conventional steganalysis techniques, which often rely on meticulously handcrafted features. Vijjapu et al. [35] introduced an enhanced steganalysis technique, centering on the Yedroudj network and leveraging an exhaustive counterattack strategy. The method adeptly navigates through challenges of detecting hidden information across numerous steganographic methods, performing commendably on color images and yielding optimistic results with grayscale images, verified through testing with stego images crafted by tools like Xiao and OpenPuff. Liu et al. [36] advanced an inventive image steganalysis method, utilizing the attention mechanism and transfer learning, skillfully addressing difficulties in extracting steganographic features, especially from images with low embedding rates, thereby bolstering detection performance and surpassing established models like Xu-Net, Yedroudj-Net, and Shen-Net across various embedding rate contexts. Fu et al. [37] introduced a distinguished CNN model for spatial-domain steganalysis, emphasizing enhanced feature focus and detection accuracy. By integrating a three-module structure, channel attention mechanism, and employing convolutional pooling, the model exhibits superior detection accuracy and amplified generalization, outclassing existing models like SRNet, Zhu-Net, and GBRAS-Net. In a nuanced approach, Xu et al. [38] utilized the Tanh activation function, truncating feature map elements’ absolute values to refine statistical modeling. Their research further explored ensemble techniques, focusing on amplifying the power of similarly-trained CNN sets. Yang et al. [39], via their maxCNN methodology, underscored the importance of textured regions in steganalysis by prioritizing their features, concurrently de-emphasizing smoother regions. Meanwhile, Ye et al. [40] advanced the field with Spatial Rich Model (SRM) filters, combined with the innovative Truncated Linear Unit (TLU) activation function. Yedroudj et al. [41] introduced the Yedroudj-Net, underscoring the significance of fine-tuning the neural network architecture. Similarly, Li et al. [42] put forth ReSTNet, a distinctive approach that bolsters detection capabilities by integrating linear and nonlinear filters in a parallel-subnet CNN. Meanwhile, Boroumand et al. [9] brought to light SRNet, distinguished by its novel filter initialization strategy and profound noise residual extraction techniques.

The research presented in [56] addresses the utilization of GANs to enhance spatial domain steganalysis methods and facilitate the covert insertion of secret information within digital images. By employing Convolutional Neural Networks (CNNs) as a state-of-the-art steganalysis architecture, the study demonstrates the effectiveness of GANs in evading detection while minimizing visual alterations to the images. The proposed scheme involves a GAN coupled with a genetic algorithm to optimize the generator’s architecture for embedding hidden messages using the Least Significant Bit (LSB) steganography algorithm. Through extensive experimentation, including evaluating GAN performance, optimizing the genetic algorithm, and testing against steganalysis models, the research showcases the success of the approach in generating cover images adaptable for steganography. Furthermore, the authors in [57] introduces a novel approach to combat machine learning-based steganalysis using adversarial examples. By leveraging the assumption of similarity between noise residuals in normal image sub-regions, the proposed Siamese generator aims to learn and preserve these features to mitigate the impact of adversarial perturbations on image similarity. Using cover and stego sub-region pairs as input, the generator, trained with steganography domain knowledge, produces adversarial covers that are less susceptible to detection. The paper also presents a random embedding strategy during interactive training with steganalyzers, enhancing generalization and saving training time.

Regarding the steganalysis, [58] presents a novel convolutional neural network (CNN) architecture, termed CCNet, tailored for spatial steganalysis in digital images. Addressing the limitations of existing CNN-based models, specifically in capturing regional features with complex textures, CCNet employs a three-module design: noise extraction, noise analysis, and classification. A key innovation lies in the integration of a channel attention mechanism, facilitated by the SE (Squeeze-and-Excitation) module embedded within the residual blocks of the noise extraction and analysis modules. This mechanism enhances discrimination learning by explicitly modeling channel correlations in the CNN, amplifying useful feature channels while suppressing less influential ones, thereby improving model performance. Additionally, This authors in [59] introduces a novel method, SNRCN2, for identifying the Source Social Media Network (SSMN) of digital images, a crucial task in image forensic analysis for verifying image source, trustworthiness, and integrity. Leveraging deep Convolutional Neural Networks (CNNs) and inspired by the recognition that image content can obscure post-processing artifacts, SNRCN2 focuses on utilizing steganalysis-based noise residuals to highlight social media network-induced artifacts. The proposed method employs Spatial Rich Model (SRM) filters to extract noise residuals, which are then fed into an efficient CNN for high-level feature extraction and classification.

Figure 1.

Comprehensive architectural design of the proposed model employing a GAN-based online data augmentation strategy for enhanced CNN performance.

3. Architectural framework of the model

Figure 1 illustrates the overarching framework of the approach being proposed. This model infuses a GAN-oriented algorithm for the purpose of data augmentation.

3.1 Data augmentation

Data augmentation stands as an imperative strategy during deep neural network training sessions. Recent advancements have honed in on pinpointing the optimum augmentation strategy specifically for image categorization. However, current methodologies have underscored two pivotal issues related to data augmentation. Initially, a multitude of prevailing augmentation methodologies are executed offline, thereby severing the connection between the learning trajectory and its application. The techniques, once mastered, persist as static throughout the training and do not undergo modifications relative to the prevailing state of the training model. Secondly, these methodologies are contingent upon image processing functionalities that preserve the image class. Therefore, the deployment of these traditional offline methodologies to fresh projects might necessitate expertise to identify and implement such operations [43]. In a bid to tackle the issues previously mentioned, we present an online data augmentation strategy that harnesses a model grounded in GANs. The objective here is to curb overfitting and enhance the performance exhibited by the proposed CNN architecture. To reconstruct genuine data, we employ features derived from the penultimate layer of the discriminator, which subsequently serve as input for the second layer of the generator. Specifically, the generator leverages two inputs: initially, random noise, which adheres to the conventional GAN model to fabricate realistic samples, and secondly, the genuine data features aimed at reconstruction. The process is visually represented in Fig. 2, wherein the dashed line, depicted in a lighter black, indicates that during the real data reconstruction, the input directed towards the generator is sourced from the flattened layer of the discriminator. This indicates that while the generator reconstructs genuine data, the linear layer is omitted [44].

3.1.1 Activation function

In our proposed approach, we employ a sequence of three CNNs connected sequentially. This choice aims to enhance feature extraction capabilities from the data samples. While this configuration adds complexity to the system and increases execution time, it also augments the capacity to extract additional features. Each CNN within the sequence possesses distinct structures and hyperparameters, contributing to varied data extraction capabilities.

The proposed method comprises a generator and discriminator, interconnected to form a GAN neural structure, as depicted in Fig. 2. The activation functions utilized and their respective roles are detailed in Table 1.

Table 1
The distribution of the used activation function and its details

Activation function	Placement	Model	Role
ReLU	Gen (3^d layer)	Generator	Address the vanishing gradient
Tanh	Gen (6^th layrer)	Generator	Squashes the network’s output values to the range [ $-$ 1, 1]
ReLU	Disc (2^nd Layer)	Discriminator	Introduce non-linearity into the network and allow gradients to flow during backpropagation
ReLU	Disc (4^th Layer)	Discriminator	Apply non-linearity into the network and facilitate gradient flow during backpropagation
Sigmoid	Disc (6^th Layer)	Discriminator	Squish the network’s output values into the range [0, 1], effectively transforming them into probabilities

Figure 2.

Architectural design of the suggested GAN for image generation.

3.1.2 Loss function

GANs comprise two distinct neural networks: a generator, denoted as G, and a discriminator, symbolized as $D$ . The generator crafts synthetic samples, whereas the discriminator evaluates whether a specific sample is authentic (sourced from the actual dataset) or artificial (created by the generator).

The discriminator strives to correctly categorize genuine samples and identify synthetic ones as such. Consequently, for authentic samples, we desire $D\left(x\right)$ to approach 1, while for artificial samples $x^{\prime}$ (which are generated by $G$ from random noise $z$ ), $D\left({x^{\prime}}\right)$ should ideally approach 0.

The loss function of the discriminator, customarily a binary cross-entropy loss, can be articulated as:

$\displaystyle L_{D}=-\left({E_{x\in p_{\textit{data}}\left(x\right)}\left[{% loglogD\left(x\right)}\right]+E_{x\in p_{z}\left(z\right)}\left[{log\left({1-D% \left({G\left(z\right)}\right)}\right)}\right]}\right)$ (1)

where $E_{x\in p_{data}\left(x\right)}\left[{loglogD\left(x\right)}\right]$ represents the expected value over genuine samples of the log likelihood of the discriminator yielding 1. Moreover, $E_{x\in p_{z}\left(z\right)}\left[{log\left({1-D\left({G\left(z\right)}\right)% }\right)}\right]$ is the expected value over random noise of the log likelihood of the discriminator outputting 0 for synthetic samples.

Contrastingly, the generator endeavors to fabricate samples capable of deceiving the discriminator. Consequently, for synthetic samples $x^{\prime}$ , $D(x^{\prime})$ should approach 1, indicating the discriminator perceives them as genuine. Employing binary cross-entropy loss again, the loss of the generator can be represented as:

$\displaystyle L_{G}=-E_{x\in p_{z}\left(z\right)}\left[{logD\left({G\left(z% \right)}\right)}\right]$ (2)

where $E_{x\in p_{z}\left(z\right)}\left[{logD\left({G\left(z\right)}\right)}\right]$ denotes the expected value over random noise of the log likelihood of the discriminator identifying synthetic samples as 1.

In GAN, the discriminator can glean crucial features from the training dataset. Notably, the architecture of the generator does not deviate from other generative models approximating the likelihood distribution of the actual data. In the suggested GAN, the generator employs features extracted by the discriminator to accurately reconstruct data and integrates the reconstruction loss into the generator and discriminator loss to amplify training stability and mitigate the mode missing issue. The reconstruction loss is described as:

$\displaystyle L_{r}=E_{x\in p_{x}}\left[{\left|{\left|{G\left({D_{F}\left(x% \right)}\right)-x}\right|}\right|}\right]$ (3)

where $D_{F}$ represents the discriminator without its final layer, converting the training dataset into feature space. $G\left({D_{F}\left(x\right)}\right)$ implies that $D_{F}$ encodes the training dataset into features, and $G$ decodes the features to the training dataset. Ultimately, the loss function of the proposed GAN model is expressed as:

$\displaystyle L_{G}+\mu\ast L_{r}$ (4) $\displaystyle L_{D}+\mu\ast L_{r}$ (5)

Here, $\mu$ represents a parameter modulating the weight of reconstruction loss. Based on Eqs (4) and (5), the generator and discriminator pursue dual objectives. The discriminator differentiates between genuine and crafted data while providing informative features to minimize reconstruction loss. The generator endeavors to reconstruct samples with utmost naturalness.

3.2 Classification

3.2.1 Deep RL

DRL provides a sturdy mechanism for deep learning, enabling an agent to adaptively interact with its environment to maximize reward outcomes. The agent, through dynamic learning, negotiates a sequence of decisions even in uncertain circumstances, demonstrating utility in diverse areas such as robotics, healthcare, and finance [45]. DRL excels in managing tasks involving sequential decisions and adapting to unpredictable situations, thus showcasing its extensive practical application. One significant challenge in classification tasks arises when managing datasets that are not uniformly distributed, meaning one category significantly overshadows others. This imbalance may lead to biased learning since conventional classification methods often prioritize the dominant category, diminishing the recognition of minor categories. Under these conditions, DRL proves to be a more proficient solution for training neural networks than traditional methods. It resolves imbalances in classification by implementing a reward-based system, guiding the focus of the agent towards recognizing less frequent category instances through judicious allocation of rewards, thereby improving recognition of these less prevalent categories. This incentive-driven model assures a balanced decision-making approach, prioritizing the identification and classification of infrequently occurring or less common categories.

In the domain of deep Q-learning, the agent aspires to select actions that optimize future rewards. The accumulated rewards for future scenarios, denoted by the reward value, decrease with the discount rate $\gamma$ as time progresses, as represented in Eq. 6. In this context, $T$ represents the final time-step of an episode.

$\displaystyle R_{t}=\sum^{T}_{t^{\prime}=t}\gamma^{t^{\prime}-t}r_{t^{\prime}}$ (6)

Q-values, signifying the quality of state-action pairings, indicate the expected return of policy $\pi$ upon taking action $a$ in state $s$ , calculated as expressed in Eq. (7).

$\displaystyle Q^{\pi}\left({s,a}\right)=E[R_{t}|s_{t}=s,a_{t}=a,\pi]$ (7)

The optimal action-value function, reflecting the greatest expected reward across all strategies after observing state $s$ and taking action a, is computed as shown in Eq. (8).

$\displaystyle Q^{\ast}\left({s,a}\right)=\text{max}_{\pi}E[R_{t}|s_{t}=s,a_{t}% =a,\pi]$ (8)

This function utilizes the Bellman equation [46], which declares that the optimal expected return for a specific action is the sum of the rewards from the current action and the highest expected return from subsequent actions in the next step, illustrated in Eq. (9).

$\displaystyle Q\ast\left({s,a}\right)=E\left[r+\gamma\text{max}_{a^{\prime}}Q^% {*}(s^{\prime},a^{\prime})|s_{t}=s,a_{t}=a\right]$ (9)

Calculations for the optimal action-value function are sequentially conducted using the Bellman equation, as demonstrated in Eq. (10).

$\displaystyle Q_{i+1}\left({s,a}\right)=E\left[r+\gamma\text{max}_{a^{\prime}}% Q_{i}(s^{\prime},a^{\prime})|s_{t}=s,a_{t}=a\right]$ (10)

During training, when the network encounters state $s$ , it produces a tailored output action a. Subsequently, the environment yields a reward r and transitions to the next state $s^{\prime}$ . These variables are compiled into a tuple $\left({s,a,r,s^{\prime}}\right)$ , which is subsequently stored in memory $M$ . Tuples, named Batches $B$ , are utilized for gradient descent execution. The formula for determining loss is articulated in Eq. (11).

$\displaystyle L_{i}\left(\theta_{i}\right)=\sum{{}_{\left(s,a,r,s^{\prime}% \right)\in B}}(y-Q(s,a;\theta_{i}))^{2}$ (11)

Here, $\theta$ represents the model weights, while $y$ symbolizes the estimated target for the $Q$ function, evaluated as the sum of the reward associated with the state-action pair and the discounted maximum $Q$ value in future steps, as depicted in Eq. (12).

$\displaystyle y=r+\gamma\text{max}_{a^{\prime}}Q(s^{\prime},a^{\prime};\theta_% {k-1})$ (12)

It is imperative to observe that the $Q$ value designated to the final state is set as 0. The gradient magnitude for the loss function at iteration i can be computed using Eq. (13).

$\displaystyle\nabla\theta_{i}L\left(\theta_{i}\right)=-2\sum{{}_{\left(s,a,r,s% ^{\prime}\right)\in B}}(y-Q(s,a;\theta_{i}))\nabla_{\theta_{i}}Q(s,a;\theta_{i})$ (13)

Executing a gradient descent iteration on the loss function enables the model weights to be adjusted according to Eq. (14), seeking to minimize the discrepancy, where $\alpha$ denotes the learning rate which governs the degree of progress within the optimization process.

$\displaystyle\theta_{i+1}=\theta_{i}+\alpha\nabla_{\theta_{i}}Q\left({s,a;% \theta_{i}}\right)$ (14)

3.2.2 PPO

PPO [47] emerges as an on-policy reinforcement learning strategy, acquiring substantial recognition for its robustness and efficacy in enhancing policies across both discrete and continuous action spaces. It was devised to overcome limitations of earlier Policy Gradient methods by addressing challenges like high sample requirements and instability. Central to PPO is the concept of policy updates that circumvent drastic changes, thus reducing the likelihood of harmful adjustments that might degrade policies.

PPO ensures that policy updates remain modest and close to the initial policy by establishing a trust region through a surrogate objective function. This function seeks to gently adjust the policy while maximizing expected rewards. Depending on the type of action space, PPO employs a particular surrogate objective function. Generally, for actions, PPO utilizes the clipped surrogate objective method, constructing the objective function by selecting the minimum between two ratios. The first ratio reflects the probability of actions under the new policy relative to the old one based on collected data, while the second is bounded within a specific range, limiting the extent of policy updates [48]. PPO is notable for its proficient use of parallelization. Being an on-policy algorithm, PPO effectively employs several parallel environments to accumulate more data, leading to faster convergence and improved sample efficiency. Additionally, PPO allows for reusing previously collected data, which can stabilize learning trajectories and optimize data utilization [49].

The updating process begins with the current policy parameter $\theta_{i}$ . The core aim of PPO-clip is to infer the new policy parameter $\theta_{i+1}$ , maximizing the expected value of the surrogate objective function $L\left({s,a,\theta_{i},\theta}\right)$ across state-action pairs $\left({s,a}\right)$ drawn from the current policy $\pi\left({\theta_{i}}\right)$ . The surrogate objective function $L\left({s,a,\theta_{i},\theta}\right)$ in Eq. (15) is formed as the minimum of two terms. The first term involves the ratio of the probability of taking action a in state s under the new policy $\pi_{\theta}\left(s\right)$ to the probability of the same action under the old policy $\pi_{\theta_{i}}\left(s\right)$ . This ratio measures the divergence between the new and old policies for the specific state-action pair. The second term within Eq. (15) is a clipped version of the initial term, obtained through the use of a clip function.

$\displaystyle L_{\textit{PPO}}^{\textit{CLIP}}=E_{s\sim\rho_{\pi_{\theta_{i}},% a\sim\pi_{\theta}}}\left[{\min\left({\frac{\pi_{\theta}\left(s\right)}{\pi_{% \theta_{i}}\left(s\right)}A_{\pi_{\theta_{i}}}\left({s,a}\right),\textit{clip}% \left({\frac{\pi_{\theta}\left(s\right)}{\pi_{\theta_{i}}\left(s\right)},1-\in% ,1+\in}\right)A_{\pi_{\theta_{i}}}\left({s,a}\right)}\right)}\right]$ (15)

The employment of the clip function aims to restrain policy modifications, preventing the newly updated policy from veering too far from its precursor. A detailed definition of the clip function is presented below:

$\displaystyle\textit{clip}\left({x,a,b}\right)=\max\left({a,\min\left({b,x}% \right)}\right)$ (16)

Within the PPO-clip framework, the clip function strategically multiplies the probability ratio $\left(\frac{\pi_{\theta}\left(s\right)}{\pi_{\theta_{i}}\left(s\right)}\right)$ by the advantage estimator $A_{\pi_{\theta_{i}}}\left({s,a}\right)$ . The advantage estimator $A_{\pi_{\theta_{i}}}\left({s,a}\right)$ assesses the enhancement level action a demonstrates in state $s$ , compared to the expected value of actions in that particular state. The imposition of restrictions on policy updates through the clip function ensures the new policy securely resides within a tight band around the preceding one, mitigating any drastic policy changes that could induce instability. This facet of PPO-clip is crucial for maintaining the stability and efficiency of the algorithm, allowing it to implement gradual changes that incrementally improve the policy [48].

3.2.3 Formulation

In this work, the PPO algorithm is applied to the domain of image steganalysis, with the subsequent elucidation detailing the operation of the methodology and defining each component:

•
State $s_{t}$ : This is indicative of the image seen at time step t.
•
Action $a_{t}$ : This pertains to the classification performed on the image, representing a decision taken by the network, based on its current understanding of the goal.
•
Action $r_{t}$ : A reward is allotted for each classification, constructed to guide the network towards accurate classifications. The configuration of this reward mechanism is defined as:

$\displaystyle r_{t}\left({s_{t},a_{t},y_{t}}\right)=\{+1,a_{t}=y_{t}\ \text{% and}\ s_{t}\in D_{S}-1,a_{t}\neq y_{t}\ \text{and}\ s_{t}\in D_{S}+\lambda,a_{% t}=y_{t}\ \text{and}\ s_{t}\in D_{N}-\lambda,a_{t}\neq y_{t}\ \text{and}\ s_{t% }\in D_{N}$ (17)

where $D_{S}$ and $D_{N}$ typify the minority and majority classes, respectively. Correct/incorrect classification of a sample from the prevalent class leads to a reward/penalty of $+\lambda/-\lambda$ . The outlined strategy prompts the network to prioritize accurate classification of instances tied to the scarcer class, assigning a larger absolute reward value. Concurrently, incorporating the majority class and the adjustable reward parameter within the range $0<\lambda<1$ adds a layer of complexity to the reward framework, allowing for meticulous calibration of the focus of the network between prevalent and scarce classes.
4. Empirical evaluation

4.1 Cross validation

Cross-validation emerges as a crucial strategy for evaluating the effectiveness of deep learning models and meticulously refining their hyperparameters. While it is predominantly deployed in supervised learning scenarios, the fundamental virtue of cross-validation resides in its capacity to counteract overfitting, ensuring that the model consistently performs with reliability on previously unseen data. The method attains this by bifurcating the primary training dataset into a pair of distinct segments: a section for training and another distinctively for validation. The model subsequently undergoes training on the first segment and validation on the latter.

One approach to cross-validation that has garnered widespread adoption is the k-fold technique. In this method, the training dataset undergoes division into k partitions, each of which is roughly equivalent in size and randomized. From these, $k-1$ subsets are utilized for training purposes, with the singular remaining subset designated for validation. This cycle perpetuates k times, affording each partition its individual opportunity for validation. Following these iterations, the results are conglomerated, yielding a unified estimation.

K-fold cross-validation partitions the primary dataset into k subsets of approximately equal size, with each subset serving as both training and validation data in separate iterations. By repeatedly cycling through the dataset, each subset gets a turn for validation while the rest contribute to training, ensuring that the model is exposed to a diverse range of data points across multiple iterations. Using the $k$ -fold validation offers several advantages. The first advantage us the maximization of available data utilization, allowing for more comprehensive model training and evaluation. Secondly, it provides a more stable and reliable estimate of the model’s performance by averaging results across multiple validation sets, thereby reducing the variance in performance estimates

Table 2 elucidates crucial hyperparameters germane to the proposed models and delineates their potential spectrum of values. The pursuit of pinpointing the optimal value for each respective parameter is expedited via cross-validation. This necessitates that every conceivable combination of hyperparameter values is scrutinized with utmost diligence. This exploration entails evaluating the performance of the model utilizing predefined metrics across all folds, thereby ensuring a comprehensive assessment. The culmination of this methodology discerns the combination of hyperparameters that distinguishes itself by virtue of superior performance throughout the cross-validation stage.

Table 2
Overview of key hyperparameters, their potential values, and the optimization process employed through cross-validation for the proposed models

Hyperparameter	Values	Best
Learning rate	[0.003, 0.03, 0.3]	0.03
Epoch	[100, 200, 400, 500]	300
Batch size	[32, 64, 128, 256]	64
Number layer in MLP	[2, 4, 8, 16]	4
Activation function	[Tanh, ReLU, Sigmoid]	ReLU

4.2 Results

The experimental evaluation was conducted using the BOSSbase 1.01 [50] dataset, comprising 10,000 grayscale images with dimensions of 512 $\times$ 512 $\times$ 1. To ensure alignment with prior studies, these images were resized to 256 $\times$ 256 $\times$ 1 dimensions utilizing Matlab. For the generation of stego images, each dataset was replicated. For benchmarking purposes, the BOSSbase 1.01 dataset was partitioned into distinct subsets: 4,000 pairs were allocated for training, another 1,000 pairs were designated for validation, and the remaining 5,000 pairs were reserved for testing.

During the evaluative stage, the recommended model was subjected to a stringent comparison against five separate deep learning models: Boroumand et al. [9], You et al. [1], Vijjapu et al. [35], Liu et al. [36], and Fu et al. [37]. The objective behind this exhaustive evaluation was to furnish an in-depth comprehension of the competencies of the model when juxtaposed with prevailing methods. Moreover, to delve into different permutations of the proposed model, two alternative versions were incorporated into the analytical procedure. The initial altered version, dubbed Proposed -PPO, embraced a fundamental architecture that paralleled our model, albeit without employing GAN for the augmentation of data. Conversely, the subsequent modified version, labeled Proposed-GAN, eschewed the PPO technique for the purposes of classification. In order to gauge the efficacy of these models, conventional metrics were deployed, placing a specific emphasis on both the F-measure and geometric mean owing to their aptness for managing imbalanced data.

The outcomes, which are encapsulated in Table 3, unequivocally showcase the preeminence of the suggested model over all alternative models, including the formerly acknowledged peak performers, Fu et al. and Liu et al. In every evaluative criterion, the suggested model persistently surpassed its peers. Most notably, the suggested model realized substantial reductions in error, exceeding 18% and 10% in the F-measure and G-means metrics, correspondingly. These notable enhancements underscore the efficacy of the suggested model in navigating the obstacles presented by imbalanced data and its prowess in proffering more precise predictions.

When juxtaposing the proposed model with its modified counterparts, Proposed-PPO and Proposed-GAN, the vital importance of amalgamating data augmentation and PPO approaches becomes perceptible. Our model exhibited a striking reduction in the error rate, approximating 30%, when placed in contrast with these modified variations. This revelation accentuates the pivotal role that both data augmentation and PPO fulfill in amplifying the performance of the model, illuminating their significance in the formulation of cutting-edge deep learning models.

Table 3
Comparative analysis of the proposed model against five existing deep learning models and two modified versions

Model	Accuracy	F-measure	G-means
Boroumand et al. [9]	60.1245 $\pm$ 0.0123	60.3256 $\pm$ 0.0102	60.6574 $\pm$ 0.0098
You et al. [1]	62.5876 $\pm$ 0.0152	63.1124 $\pm$ 0.0124	63.7698 $\pm$ 0.0112
Vijjapu et al. [35]	67.4532 $\pm$ 0.0145	68.3246 $\pm$ 0.0113	68.6592 $\pm$ 0.0106
Liu et al. [36]	70.5683 $\pm$ 0.0128	71.6754 $\pm$ 0.0153	71.9874 $\pm$ 0.0142
Fu et al. [37]	74.5263 $\pm$ 0.0107	75.3256 $\pm$ 0.0115	75.8843 $\pm$ 0.0146
Proposed-PPO	78.6253 $\pm$ 0.0123	79.4321 $\pm$ 0.0134	79.7456 $\pm$ 0.0101
Proposed -GAN	82.1467 $\pm$ 0.0146	82.4322 $\pm$ 0.0112	82.9845 $\pm$ 0.0097
Proposed	85.4321 $\pm$ 0.0157	86.0000 $\pm$ 0.0123	85.6789 $\pm$ 0.0148

Figure 3.

Comparative ROC curves and AUC values illustrating the differential performances of the proposed model versus extant methodologies. Blue dashed line represents the ROC curve for a random guess.

Figure 3 proficiently showcases the Receiver Operating Characteristic (ROC) curves relative to the methodologies articulated in Table 2, employing the Area Under the Curve (AUC) as a pivotal metric to evaluate the efficacy of classifiers. An ideal AUC score of 1 embodies flawless discrimination, whereas a score lingering at 0.5 conveys a discrimination aptitude merely tantamount to random speculation. It is of quintessential note that the proposed model conspicuously outshone its counterparts, flaunting an admirable AUC of 0.74. This phenomenon not only emphasizes its amplified capability to accurately segregate between positive and negative outcomes but also fortifies the legitimacy of our strategy as a robust predictive apparatus. On the contrary, Liu et al. and Fu et al. secured only modest AUC metrics, tallying 0.64 and 0.66 respectively, thereby failing to emulate the impressive performance exhibited by our advocated method. Furthermore, Boroumand et al., You et al., and Vijjapu et al. revealed less than optimal outcomes, with AUC values vacillating between 0.54 and 0.63. Notably, VibroCNN lingered, notching up a minimal AUC of 0.54, scarcely rising above sheer random selection. The ROC analysis compellingly delineates the varying performances amongst the evaluated methods. The superb predictive capabilities of our suggested method underscore the potency of our approach and usher in possibilities for future enhancements and propitious applications in the realm of prediction.

Figure 4.

Evolution and convergence of adversarial learning: (a) Tracking the error trajectories of the generator and discriminator across epochs in GAN training, highlighting the iterative refinement and convergence of the adversarial entities; (b) Dynamic exploration of error across 500 epochs, illuminating the steady decrement and subsequent stabilization, while signifying learning progression and potential convergence in the proposed model.

Figure 4.a judiciously unveils error diagrams (loss), respectively illustrating the trajectories of both the generator and the discriminator within a GAN, navigating through a diverse array of epochs. Initially embarking with an error quantifiably measured at 0.789, the generator conspicuously faces palpable challenges, as it endeavors to generate samples that aptly mimic the genuine data distribution. Nevertheless, as the journey through the epochs unfolds, the error attributed to the generator manifests a discernible downward trajectory, hinting that it is incrementally enhancing its prowess in simulating authentic data, and thereby, adeptly encapsulating complex patterns intrinsic to the dataset. Concurrently, the discriminator, commencing its journey with an error marginally lower, precisely at 0.8, experiences its own developmental progression. Entrusted with the crucial task of discriminating between authentic and synthetically generated data, it too, over the epochs, witnesses a decrement in error, albeit not as pronouncedly as its generator counterpart. This subtlety reveals that, while the discriminator gradually hones its discriminative capabilities, the generator concurrently advances, albeit at a marginally accelerated rate, thereby synthesizing progressively more compelling samples. The pivotal interplay between these adversarial entities is fundamentally instrumental to the convergence of GAN. A perceptible attenuation in error for both components throughout the epochs intimates a harmonious and synergistic convergence amidst the GAN training process. As the generator meticulously refines its outputs, pulling them closer to realistic samples, the discriminator concurrently amplifies its evaluative capabilities, in a carefully orchestrated adversarial dance. This systematic decline in error crucially highlights not only the stability but also the perpetual progression of the GAN throughout its training journey. Explicitly, the model astutely exploits the adversarial dynamics interwoven between its constituents, fostering a performance enhancement throughout the epochs. Fundamentally, Fig. 5.a accentuates the iterative refinement embedded within GAN training, where each successive step nudges the system ever closer to synthesizing increasingly authentic synthetic data distributions.

Figure 4.b meticulously illustrates the error dynamics inherent within the proposed model across a span of 500 epochs. The error initiates at a magnitude of 10, presenting a steadfast decrement as the epochs incrementally advance. This unvarying diminution inherently signifies that the model is undergoing a learning process and perpetually enhancing its predictions as time unfolds. It is worth observing that the error attenuation is most precipitous during the initial epochs, and it begins to approach a plateau as the epoch count escalates, implying a phenomenon of diminishing returns with respect to error mitigation as training persists. Circling around the 425th epoch, the error ostensibly achieves a state of stabilization, preserving a steady value proximate to 4.2962 for the ensuing epochs. Such a plateau intimates that additional training beyond this juncture might not procure substantial enhancements in the performance of the model, and it is plausible that the model has attained its convergence. Furthermore, this stabilization may also hint at potential overfitting if there is a cessation in the enhancement of the performance of the model on validation or test datasets, providing an avenue for subsequent investigation into model generalization and robustness.

4.2.1 Exploring the influence of TL in classification framework

A plethora of real-world applications have adeptly leveraged the potential offered by TL techniques. In this section, rigorous experiments were conducted to meticulously evaluate the performance trends of diverse TL approaches when integrated into the framework of the proposed model. The chosen dataset for this assessment was sourced from the BOWS archive, comprising 5,000 testing pairs [50]. As delineated in Table 4, the advanced model consistently outperforms its counterparts, solidifying its superior stance. Notably, when juxtaposed with the model designed by Fu et al., which stands as the nearest competitor, our proposed approach achieves a remarkable 34% reduction in error on the BOWS dataset. These findings underscore the enhanced performance derived from the synergistic union of PPO and GAN in bolstering the effectiveness of TL.

Table 4
Comparative analysis of the proposed model against five existing deep learning models using TL

Model	Accuracy	F-measure	G-means
Boroumand et al. [9]	39.1234 $\pm$ 0.4056	40.2345 $\pm$ 0.4114	41.3456 $\pm$ 0.4226
You et al. [1]	42.5678 $\pm$ 0.4314	43.6789 $\pm$ 0.4456	45.7890 $\pm$ 0.4500
Vijjapu et al. [35]	48.9012 $\pm$ 0.4619	50.0123 $\pm$ 0.4796	51.1234 $\pm$ 0.4848
Liu et al. [36]	54.2345 $\pm$ 0.4947	56.3456 $\pm$ 0.5078	58.4567 $\pm$ 0.5104
Fu et al. [37]	60.5678 $\pm$ 0.5248	62.6789 $\pm$ 0.5379	64.7890 $\pm$ 0.5486
Proposed	77.5678 $\pm$ 0.6148	78.6789 $\pm$ 0.6450	79.7890 $\pm$ 0.6300

4.2.2 Exploring the influence of reward parameter

\lambda

in classification framework

Rewards apportioned to both ubiquitously common and infrequently encountered groups for accurate and erroneous classifications are represented as $+$ 1 and $\pm$ $\lambda$ , respectively. The $\lambda$ value finds its influence from the proportion of dominant to sporadically encountered instances. As this proportion escalates, a decrement in the optimal $\lambda$ value naturally follows. To delve into the ramifications of $\lambda$ , we scrutinized the proficiency of the suggested framework by utilizing a spectrum of $\lambda$ values, stretching from 0 to 1 and progressing in 0.1 increments, whilst steadfastly holding the reward for the dominant group constant. The resulting insights are graphically unfolded in Fig. 5. In instances where $\lambda$ is determined to be 0, the sway of the dominant group is rendered inconsequential. Conversely, with $\lambda$ established at 1, both the dominant and less frequent groups are rendered equivalently impactful. The revelations exhibit that the framework ascends to its pinnacle of effectiveness when $\lambda$ is firmly positioned at 0.3, an observation consistent across all evaluation metrics explored. This conveys the implication that the quintessential $\lambda$ value gracefully situates itself somewhere within the spectrum bounded by zero and one. A noteworthy point of reflection is that while manipulating $\lambda$ to diminish the influence of the dominant group is pivotal, establishing it too diminutively may jeopardize the collective efficacy of the entire framework. The information affirms that the selection of $\lambda$ profoundly shapes the triumph of the framework. The pinnacle $\lambda$ value hinges on the relative occurrences of dominant and less prevalent examples, highlighting an imperative for meticulous determination to usher forth optimal outcomes.

Figure 5.

Graphical exploration of the efficacy of various $\lambda$ values on classification reward systems.

Figure 6 meticulously maps reward pathways, offering enlightening reflections regarding the evolutionary learning achievements of the agent over temporal progression. These pathways illuminate the advancement of the agent as it entwines with the environment, progressively honing its decision-making acumen. During the infant stages of its learning expedition, the agent embarks with relatively unassuming reward values, averaging proximally to $-$ 0.397 within the preliminary episodes. In this phase, the agent delicately engages while navigating through the complex task tapestry of the environment. These foundational episodes establish a platform upon which the agent cultivates a profound comprehension of the pivotal components embedded within its allocated tasks. As it accrues additional experience and a more nuanced understanding of the environmental dynamics, a conspicuous enhancement in its decision-making materializes. This amelioration is mirrored in a consistent upward trajectory of rewards. Despite the presence of minor oscillations in the progression of the agent, it demonstrates remarkable resilience, adeptly adapting to hurdles and sustaining an overarching positive performance trajectory. Even in the face of sporadic regressions or transient plateaus, the dedication of the agent to continual learning and enhancement remains steadfast. The learning arc, as delineated by the reward pathways, accentuates the capability of the agent to navigate challenges and assimilate from its interactions, culminating in enriched decision-making capabilities. Significantly, as its learning voyage concludes, the reward trajectory climaxes at an admirable value, approximately converging at 4.33 in the concluding episode. This noteworthy accomplishment emphasizes the prowess of the agent in proficiently navigating tasks within the environment. The pronounced escalation in rewards denotes the mastery of the agent over task subtleties and its capability to perpetually forge well-informed decisions.

4.2.3 Exploring the influence of loss function in classification framework

An abundance of methodologies prevail when addressing the challenges posed by imbalances in data within machine learning model constructions, including refining data augmentation strategies and judiciously determining the most appropriate loss function. The crucial act of pinpointing the optimal loss function becomes instrumental in safeguarding the efficient learning trajectory of the model, particularly from classes that are less represented in the dataset. A thorough evaluation was conducted on the effectiveness encapsulated within five disparate loss functions: namely, weighted cross-entropy (WCE) [51], balanced cross-entropy (BCE) [52], Dice loss (DL) [53], Tversky loss (TL) [54], and Combo Loss (CL) [55]. Both BCE and WCE, being widely-adopted loss functions, administer an equivalent handling mechanism for positive and negative samples. However, in instances where datasets exhibit imbalances and a need to underscore the minority class arises, these functions might not stand out as optimal. In contrast, DL and TL manifest as being considerably suitable for datasets that are skewed, thereby providing enhanced outcomes for the minority class. Emerging conspicuously, CL presents itself as a loss function that demands attention, particularly offering advantageous outcomes in scenarios dealing with skewed data. Through judicious modulation of the weights within the loss function, CL prioritizes complex samples, giving them precedence over those that are simpler in nature. The empirical analysis conducted on these loss functions is systematically displayed within Table 5. The insights derived from the data notably reveal that CL significantly surpasses TL, culminating in a 9% diminution in the error rate pertinent to accuracy and a 32% contraction concerning the F-measure. Nonetheless, it remains imperative to underscore that CL trails by a margin of 24% when juxtaposed with the proposed model, which is meticulously tailored for tasks related to binary classification.

Table 5
Comparative analysis of diverse loss functions in addressing data imbalance

Model	Accuracy	F-measure	G-means
WCE	45.2314 $\pm$ 0.4121	46.2145 $\pm$ 0.6202	46.8792 $\pm$ 0.5410
BCE	51.3178 $\pm$ 0.5024	52.2456 $\pm$ 0.7485	52.9876 $\pm$ 0.4123
DL	57.3412 $\pm$ 0.3215	58.1296 $\pm$ 0.6001	58.6324 $\pm$ 0.6456
TL	63.4710 $\pm$ 0.5045	64.3024 $\pm$ 0.7146	64.9127 $\pm$ 0.5009
CL	71.2893 $\pm$ 0.5475	72.1742 $\pm$ 0.4203	72.8541 $\pm$ 0.3146

Figure 6.

A visualization of reward trajectories and decision-making evolution of the agent over time.

4.2.4 Exploring the influence of the number of CNNs in classification framework

In the proposed model, a consortium of CNNs is strategically employed to extract feature vectors from input images in a concurrent fashion. The quantity of CNN feature extractors is intricately intertwined with the performance of the model, asserting a pivotal role. The deployment of too few CNNs could culminate in an insufficiency in feature extraction, whereas the utilization of an excessive number might precipitate risks, such as overfitting or the extraction of superfluous information, thereby potentially compromising the efficacy of the model. To pinpoint the ideal numerical strength, an explorative performance test of the proposed model was executed, employing a spectrum ranging from one to seven CNN feature extractors. The ensuing results intimate that the model, when deploying three CNNs, achieves a zenith in performance, as visually represented in Fig. 7. A noticeable descent in performance was observed concomitant with an escalation in the number of CNNs, with configurations employing six and seven CNNs demonstrably underperforming in comparison to a singular one. The optimal number of CNNs was judiciously inferred from these elucidated performance metrics, thereby contributing to the operational potency of the model.

Figure 7.

Performance trajectory across varied CNN feature extractor configurations.

4.2.5 Exploring the feature extraction

Our feature extraction method involved exploring the ideal number of CNN feature extractors, ranging from one to seven. Results showed that employing three CNNs yielded optimal performance, as depicted in Figure 8. Conversely, increasing the number of CNNs led to diminishing returns, with configurations of six and seven CNNs notably performing worse compared to a single CNN. This informed our decision on the optimal number of CNNs, enhancing the efficiency of our model.

Figure 8.

Grad-CAM visualizations highlighting feature activation regions across diverse images for steganalysis.

To visualize feature activation regions across diverse images for steganalysis, we utilized Gradient-weighted Class Activation Mapping (Grad-CAM), as illustrated in Fig. 8. Grad-CAM provides critical insights into the decision-making process of convolutional neural networks. For instance, in Fig. 8(a), the Grad-CAM highlights predominantly focus on the sky area of a landscape scene, indicating the significance of this region in the network’s analysis. Similarly, in Fig. 8(b), the Grad-CAM activations emphasize the upper part of an urban setting image, including the sky and building roof, suggesting the model’s attention to architectural features. Lastly, in Fig. 8(c), Grad-CAM highlights are concentrated around brightly lit areas in a night street view, such as storefronts and street reflections, highlighting the model’s focus on luminous elements in the image. Figure 8(d) presents an alleyway scene where the Grad-CAM visualization highlights the right side of the image, particularly illuminating the wall and the ground. The network appears to focus on texture details and potentially shadows or light gradients along the pathway.

4.2.6 Analyzing Q-Value distribution across state

Figure 9 depicts the Q-values obtained by a trained reinforcement learning agent for ten different states, labeled as State 0 to State 9. Each subplot corresponds to a state and displays two actions, denoted as 0 and 1. The numbers within the blue boxes represent the learned Q-values, indicating the expected reward associated with selecting a specific action within that state.

Figure 9.

Learned Q-values for two actions across ten states from a trained RL agent.

In certain states, such as State 0, the agent demonstrates a preference for action 0, as evidenced by the positive Q-value (0.44), suggesting that this action is more likely to result in a favorable outcome compared to action 1, which has a negative Q-value ( $-$ 0.46). Conversely, in State 2, action 1 exhibits a substantially negative Q-value ( $-$ 0.72), indicating a strong disincentive, while action 0 has a slightly positive value (0.35), suggesting it is the preferred choice. Notably, State 8 illustrates a clear differentiation between actions, with action 1 having a significantly negative Q-value ( $-$ 0.99), indicating its undesirability, whereas action 0 has a moderately positive value (0.27), indicating its favorable nature within that state.

4.2.7 Limitations

In its architectural conception, the suggested framework represents a notable advancement in the realm of image steganalysis. Through an ingenious amalgamation of data amplification methodologies, ensemble-based learning strategies, and the finesse offered by PPO, this model showcases an admirable precision in its analytical outcomes.

The meticulous management of imbalanced datasets, especially within the confines of a RL framework, is indeed susceptible to a myriad of potential pitfalls. In cases where the minority class encompasses critical but sparsely distributed data, the effective training of an RL agent could be compromised. For instance, if one were working with a dataset aimed at identifying a rare pathology (minority class), the RL agent may not encounter enough instances during training to sufficiently calibrate its decision-making parameters towards accurately identifying said pathology. The nuanced variances that demarcate normalcy and anomaly in such instances might elude its learning trajectory, potentially resulting in inadequate diagnostic capabilities. An illustrative solution to ameliorate this limitation may involve the incorporation of SMOTE within the data preprocessing pipeline. SMOTE generates synthetic samples in the feature space, thus mitigating the scarcity of minority class instances and enabling the RL agent to derive more informed insights during its learning phase. By generating synthetic instances of the minority class, SMOTE can embellish the training data, providing a more enriched learning environment for the RL agent. Additionally, exploring alternative learning paradigms such as few-shot learning or meta-learning may also proffer viable pathways to enhance learning from limited data instances. Few-shot learning, for example, deliberately engineers’ models to make accurate predictions with extremely limited data by leveraging prior knowledge obtained from related tasks. In contrast, meta-learning models are trained across various tasks in such a way that they can quickly adapt to new tasks, even when presented with only a few examples. Incorporating these approaches might, therefore, facilitate more robust learning from minority classes, enhancing the discriminatory and predictive capacities of the model.

The task of ensuring both scalability and maintaining computational frugality, particularly within the context of a model that amalgamates numerous complex architectures and approaches, presents a pertinent challenge warranting thorough examination and solution-oriented approaches. This multifaceted model, while showcasing efficacy through the integration of triple parallel dilated convolutions, RL frameworks, and GANs for nuanced data augmentation, concurrently surfaces as a potential computational behemoth, thereby impeding its utility in resource-constrained and real-time application contexts. One plausible pathway towards ameliorating these computational and scalability challenges is to explore the deployment of lightweight neural network architectures that maintain a delicate balance between computational efficiency and predictive efficacy. MobileNet, for instance, employs depth wise separable convolutions to reduce computational load without compromising the performance of the model substantially. Embedding such lean yet powerful architectures in place of, or as a variant of, the current convolutional layers could significantly mitigate the computational demands of the model, enhancing its applicability in resource-constrained environments. Furthermore, adopting quantization and pruning strategies could also surface as beneficial in enhancing the computational efficiency of the model. Quantization, which entails reducing the numerical precision of the model weights, and pruning, which involves eliminating redundant weights, can both serve to drastically reduce the model size and, consequently, its computational and memory demands. Deploying these strategies could potentially render the model more amenable to deployment in real-time applications or environments where computational resources are scarce, without exerting an undue toll on its predictive capabilities. In parallel, leveraging hardware acceleration through the strategic use of Graphical Processing Units (GPUs) or Field-Programmable Gate Arrays (FPGAs) could be explored to expedite data processing and model inference times. Specifically, optimizing the model to leverage the parallel processing capabilities of GPUs or configuring FPGAs to create dedicated, efficient circuits for model deployment could significantly enhance its performance in real-time application contexts.

The adoption of GANs as a means to augment data does indeed present its own subset of challenges and complexities, despite their notable capability to enhance classification efforts. The notorious intricacies of GAN training, including the propensity towards mode collapse and the often-arduous journey towards achieving stable convergence, are well-documented in the realm of machine learning. Although our model incorporates a regularization mechanism aimed at ameliorating these difficulties, particular scenarios or inherent properties within the data may exacerbate the challenges tethered to GAN training, prompting a need for additional stability-inducing strategies and potential inquiries into alternative data augmentation techniques. To enhance the stability of GAN training and potentially bypass issues such as mode collapse, one viable solution may lie in the incorporation of various modified training methodologies that have demonstrated success in the literature. For instance, techniques such as gradient penalty, spectral normalization, or utilizing different learning rates for the generator and discriminator have been posited as means to stabilize GAN training and foster more robust convergence. In-depth exploration and experimentation with these varied training strategies could potentially unearth a path towards mitigating the aforementioned challenges intrinsic to GANs. Another viable avenue could revolve around exploring alternative GAN architectures that have been tailored to ensure more stable training dynamics and improved generation capabilities. Architectures such as Wasserstein GANs (WGANs) or Progressive Growing of GANs (ProGANs) might offer enhanced stability in training dynamics, and thereby serve as a potential replacement or supplement to the GAN architecture employed in the current model. Diving into these alternatives might elucidate solutions that amalgamate the robust data augmentation capabilities of GANs with more stable and reliable training dynamics.

5. Conclusion

This manuscript introduces a groundbreaking model, devised meticulously for the detection of image steganalysis. The formulated model astutely leverages data augmentation, ensemble learning, and PPO techniques to assure accuracy in its findings. The model engages a collective of CNNs to contemporaneously extract feature vectors from supplied input images. These vectors are subsequently integrated into subsequent processes, enhancing anomaly classification and amplifying the ability of the model to identify detailed patterns within the data. The efficacy of the proposed model is corroborated using an imbalanced dataset extracted from BossBase 1.01. The skewed nature of datasets can present considerable challenges during classifier training since the prevailing learning process may be biased by the overrepresented class, hence compromising the performance of the underrepresented class. To offset this challenge, we implemented a PPO-based algorithm, shaping a training procedure manifested as a chain of interconnected decisions. Within this framework, dataset samples served as states, the model acted as the agent, and the latter was conferred rewards or penalties for accurate or inaccurate classifications, respectively. This approach effectively reoriented the attention of the model towards the underrepresented class, achieving improved classification results. To further sharpen classification performance, we introduced an innovative data augmentation approach, utilizing a GAN. Images, generated by the GAN, were woven into the training process, thereby enabling the model to absorb learning from an enlarged and diversified dataset. This method enriched the ability of the model to generalize and identify anomalies that might be inadequately represented in the original dataset. However, GAN training can encounter issues such as mode collapse and unstable training dynamics. To circumvent these pitfalls, we proposed a regularization strategy that facilitated more stable and efficacious GAN training, addressing the mode omission issue and enhancing the quality of the generated images. The results underscore an exceptional ability to differentiate between clean and steganographic images, achieving an average accuracy rate of 85%.

In the future work, we aspire to further refine and expand upon the model, enhancing its applicability and performance across diverse scenarios and datasets within the domain of image steganalysis. One potential avenue of exploration involves the further tuning of the RL framework to more effectively navigate and learn from datasets with varied levels of imbalance and complexity. The current model, while innovative, may benefit from a deeper exploration into alternative strategies for managing sparse and imbalanced data, including more sophisticated sampling techniques or the integration of additional learning paradigms. Moreover, optimizing the computational efficiency of the model will be crucial to enable its deployment in more constrained computational environments and real-time applications. Potential strategies may encompass the development of lightweight versions of the model that maintain a balance between performance and computational demands, or the exploration of distributed computing solutions to manage the computational load more effectively. Furthermore, in addressing the challenges associated with employing GANs for data augmentation, future endeavors may explore the utility of alternative generative models or the development of more robust training protocols to ensure stability and convergence during training. Beyond these facets, future efforts will also explore the adaptability and efficacy of the model across varied steganographic techniques, ensuring its applicability and effectiveness are maintained across the evolving landscape of steganographic methods and countermeasures.

References

You

Zhang

Zhao

. A Siamese CNN for image steganalysis. IEEE Transactions on Information Forensics and Security. 2020; 16: 291-306.

Fridrich

. Steganography in digital media: principles, algorithms, and applications. Cambridge University Press. 2009.

Dehdar

Keshavarz

Parhizgar

. Image steganalysis using modified graph clustering based ant colony optimization and Random Forest. Multimedia Tools and Applications. 2023; 82(5): 7401-7418.

Denemark

Sedighi

Holub

Cogranne

Fridrich

. Selection-channel-aware rich model for steganalysis of digital images. In: 2014 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE; 2014, pp. 48-53.

Juarez-Sandoval

Cedillo-H Fridrich

. Steganography in digital media: principles, algorithms, and applications. Cambridge University Press. 2009.

Zhong

Liu

. Residual convolution network based steganalysis with adaptive content suppression. In: 2017 IEEE international conference on multimedia and expo (ICME). IEEE; 2017, pp. 241-246.

Singhal

Bedi

. Multi-class blind steganalysis using deep residual networks. Multimedia Tools and Applications. 2021; 80: 13931-13956.

Maulana

Widianto

Sasongko

. Steganography based on the B217AN Algorithm for secret messages on flip horizontal and resize image. 2023.

Boroumand

Chen

Fridrich

. Deep residual network for steganalysis of digital images. IEEE Transactions on Information Forensics and Security. 2018; 14(5): 1181-1193.

10.

Xie

Ren

Marshall

Zhao

Chen

. Self-attention enhanced deep residual network for spatial image steganalysis. Digital Signal Processing. 2023; 139: 104063.

11.

Chen

Han

Tong

. Image steganalysis with multi-scale residual network. Multimedia Tools and Applications. 2023; 82(14): 22009-22031.

12.

Tsang

Fridrich

. Steganalyzing images of arbitrary size with CNNs. Electronic Imaging. 2018; 2018(7): 121-1-121-8.

13.

Hou

Zhang

. Combating highly imbalanced steganalysis with small training samples using feature selection. Journal of Visual Communication and Image Representation. 2017; 49: 243-256.

14.

Han

Wang

Mao

. Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: International conference on intelligent computing. Springer; 2005; pp. 878-887.

15.

Chen

Tsai

Moon

Ahn

Young

Chen

. Decision threshold adjustment in class prediction. SAR and QSAR in Environmental Research. 2006; 17(3): 337-352.

16.

Goodfellow

, et al. Generative adversarial nets. Advances in neural information processing systems. 2014; 27.

17.

Radford

Metz

Chintala

. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv: 1511.06434. 2015.

18.

Tanaka

Aranha

. Data augmentation using GANs. arXiv preprint arXiv: 1904.09135. 2019.

19.

Van Schyndel

Tirkel

Osborne

. A digital watermark. In: Proceedings of 1st international conference on image processing. IEEE; 1994. vol. 2. pp. 86-90.

20.

Cogranne

Zitzmann

Fillatre

Retraint

Nikiforov

Cornu

. A cover image model for reliable steganalysis. In: Information Hiding: 13th International Conference, IH 2011, Prague, Czech Republic, May 18–20, 2011, Revised Selected Papers 13. Springer; 2011. pp. 178-192.

21.

Sedighi

Cogranne

Fridrich

. Content-adaptive steganography by minimizing statistical detectability. IEEE Transactions on Information Forensics and Security. 2015; 11(2): 221-234.

22.

Holub

Fridrich

. Digital image steganography using universal distortion. In: Proceedings of the first ACM workshop on Information hiding and multimedia security. 2013; pp. 59-68.

23.

Shi

Dong

Wang

Qian

Zhang

. SSGAN: Secure steganography based on generative adversarial networks. In: Advances in Multimedia Information Processing – PCM 2017: 18th Pacific-Rim Conference on Multimedia, Harbin, China, September 28–29, 2017, Revised Selected Papers, Part I 18. Springer; 2018. pp. 534-544.

24.

Filler

Judas

Fridrich

. Minimizing additive distortion in steganography using syndrome-trellis codes. IEEE Transactions on Information Forensics and Security. 2011; 6(3): 920-935.

25.

Pevný

Filler

Bas

. Using high-dimensional image models to perform highly undetectable steganography. In: Information Hiding: 12th International Conference, IH 2010, Calgary, AB, Canada, June 28–30, 2010, Revised Selected Papers 12. Springer; 2010. pp. 161-177.

26.

Holub

Fridrich

. Designing steganographic distortion using directional filters. In: 2012 IEEE International workshop on information forensics and security (WIFS). IEEE; 2012. pp. 234-239.

27.

Wang

Huang

. A new cost function for spatial image steganography. In: 2014 IEEE International conference on image processing (ICIP). IEEE; 2014. pp. 4206-4210.

28.

Chaumont

. Deep learning in steganography and steganalysis. In: Digital media steganography. Elsevier; 2020; pp. 321-349.

29.

Wang

Jiang

Zheng

. A novel image steganography method via deep convolutional generative adversarial networks. IEEE Access. 2018; 6: 38303-38314.

30.

Tang

Tan

Huang

. Automatic steganographic distortion learning using a generative adversarial network. IEEE Signal Processing Letters. 2017; 24(10): 1547-1551.

31.

Yang

Ruan

Huang

Kang

Shi

Y-Q

. An embedding cost learning framework using GAN. IEEE Transactions on Information Forensics and Security. 2019; 15: 839-851.

32.

Tang

Tan

Barni

Huang

. CNN-based adversarial embedding for image steganography. IEEE Transactions on Information Forensics and Security. 2019; 14(8): 2074-2087.

33.

Zhu

Kaplan

Johnson

Fei-Fei

. Hidden: Hiding data with deep networks. In: Proceedings of the European conference on computer vision (ECCV). 2018; pp. 657-672.

34.

Qian

Dong

Wang

Tan

. Learning and transferring representations for image steganalysis using convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP). IEEE; 2016. pp. 2752-2756.

35.

Vijjapu

Vinod

Murty

Raju

Satyanarayana

Kumar

. Steganalysis using Convolutional Neural Networks-Yedroudj Net. In: 2023 International Conference on Computer Communication and Informatics (ICCCI). IEEE; 2023. pp. 1-7.

36.

Liu

Zhang

Wang

Yang

Hua

Zhang

. Image Steganalysis of Low Embedding Rate Based on the Attention Mechanism and Transfer Learning. Electronics. 2023; 12(4): 969.

37.

Chen

Wang

. CCNet: CNN model with channel attention and convolutional pooling mechanism for spatial image steganalysis. Journal of Visual Communication and Image Representation. 2022; 88: 103633.

38.

H-Z

Shi

Y-Q

. Structural design of convolutional neural networks for steganalysis. IEEE Signal Processing Letters. 2016; 23(5): 708-712.

39.

Yang

Liu

Kang

Wong

Shi

. Steganalysis based on awareness of selection-channel and deep learning. In: Digital Forensics and Watermarking: 16th International Workshop, IWDW 2017, Magdeburg, Germany, August 23–25, 2017, Proceedings 16. Springer; 2017. pp. 263-272.

40.

. Deep learning hierarchical representations for image steganalysis. IEEE Transactions on Information Forensics and Security. 2017; 12(11): 2545-2557.

41.

Yedroudj

Comby

Chaumont

. Yedroudj-net: An efficient CNN for spatial steganalysis. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2018. pp. 2092-2096.

42.

Wei

Ferreira

Tan

. ReST-Net: Diverse activation modules and parallel subnets-based CNN for spatial image steganalysis. IEEE Signal Processing Letters. 2018; 25(5): 650-654.

43.

Tang

Gao

Karlinsky

Sattigeri

Feris

Metaxas

. OnlineAugment: Online data augmentation with less domain knowledge. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16. Springer; 2020. pp. 313-329.

44.

Xiao

Ouyang

. Improved generative adversarial networks with reconstruction loss. Neurocomputing. 2019; 323: 363-372.

45.

François-Lavet

Henderson

Islam

Bellemare

Pineau

. An introduction to deep reinforcement learning. Foundations and Trends® in Machine Learning. 2018; 11(3-4): 219-354.

46.

Barron

Ishii

. The Bellman equation for minimizing the maximum cost. Nonlinear Anal. Theory Methods Applic. 1989; 13(9): 1067-1090.

47.

Zhang

Zhuo

. Coordinated proximal policy optimization. Advances in Neural Information Processing Systems. 2021; 34: 26437-26448.

48.

Schulman

Wolski

Dhariwal

Radford

Klimov

. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347. 2017.

49.

Heess

, et al. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv: 1707.02286. 2017.

50.

Reinel

, et al. GBRAS-Net: a convolutional neural network architecture for spatial image steganalysis. IEEE Access. 2021; 9: 14340-14350.

51.

Özdemir

Sönmez

. Weighted cross-entropy for unbalanced data with application on covid x-ray images. In: 2020 Innovations in Intelligent Systems and Applications Conference (ASYU). IEEE; 2020. pp. 1-6.

52.

Huang

Zhu

. Balanced Symmetric Cross Entropy for Large Scale Imbalanced and Noisy Data. arXiv preprint arXiv: 2007.01618. 2020.

53.

Sun

Meng

Liang

. Dice loss for data-imbalanced NLP tasks. arXiv preprint arXiv: 1911.02855. 2019.

54.

Salehi

SSM

Erdogmus

Gholipour

. Tversky loss function for image segmentation using 3D fully convolutional deep networks. In: Machine Learning in Medical Imaging: 8th International Workshop, MLMI 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 10, 2017, Proceedings 8. Springer; 2017. pp. 379-387.

55.

Taghanaki

, et al. Combo loss: Handling input and output imbalance in multi-organ segmentation. Computerized Medical Imaging and Graphics. 2019; 75: 24-33.

56.

Martín

, et al. Evolving Generative Adversarial Networks to improve image steganography. Expert Systems with Applications. 2023; 222: 119841.

57.

Yang

Liao

. ACGIS: Adversarial Cover Generator for Image Steganography with Noise Residuals Features-Preserving. Signal Processing: Image Communication. 2023; 113: 116927.

58.

, et al. CCNet: CNN model with channel attention and convolutional pooling mechanism for spatial image steganalysis. Journal of Visual Communication and Image Representation. 2022; 88: 103633.

59.

Rana

Singh

Goyal

. SNRCN2: Steganalysis noise residuals based CNN for source social network identification of digital images. Pattern Recognition Letters. 2023; 171: 124-130.

Enhanced image steganalysis through reinforcement learning and generative adversarial networks

Abstract

Keywords

1. Introduction

2. Related works

3.1 Data augmentation

3.1.1 Activation function

Table 1 The distribution of the used activation function and its details

3.2.1 Deep RL

4.1 Cross validation

Table 2 Overview of key hyperparameters, their potential values, and the optimization process employed through cross-validation for the proposed models

Table 3 Comparative analysis of the proposed model against five existing deep learning models and two modified versions

Table 4 Comparative analysis of the proposed model against five existing deep learning models using TL

Table 5 Comparative analysis of diverse loss functions in addressing data imbalance

5. Conclusion

References

Table 1
The distribution of the used activation function and its details

Table 2
Overview of key hyperparameters, their potential values, and the optimization process employed through cross-validation for the proposed models

Table 3
Comparative analysis of the proposed model against five existing deep learning models and two modified versions

Table 4
Comparative analysis of the proposed model against five existing deep learning models using TL

Table 5
Comparative analysis of diverse loss functions in addressing data imbalance