Stereo image quality assessment using deformed pixels and Sobel magnitude

Abstract

The application of 3D technology is rapidly expanding, and stereoscopic imagery is typically used to display 3D data. However, compression, transmission, and other necessary processes may reduce the quality of these images. Stereo image quality assessment (SIQA) has gained more attention to guarantee that customers have a positive watching experience. In order to provide the highest level of experience, it is necessary to develop a quality evaluation mechanism for stereoscopic content that is both dependable and precise. A full-reference method for SIQA is presented in this paper. Compared to previous measures, this method gives users more freedom to use distorted pixel metrics and edge similarity. The binocular summation map is calculated by adding the left and right images for a stereo pair. Improved gradient similarity based distorted pixel measure (SGSDM) is used to calculate the quality of binocular summation. The scored 3D LIVE IQA database is used to evaluate the correlation of the proposed metric with the DMOS subjective score given by the database. The proposed method’s efficacy is demonstrated by experimental comparisons.

Keywords

1 Introduction

Over the past several decades, the stereo and multiview systems have seen rapid development and widespread applications. Stereoscopic images, like 2D images, can be affected by a few common degradations. Due to their numerous applications in printing, compression, communication, analysis, registration, restoration, and enhancement, 2D image quality assessment has received a lot of attention [1 –3]. The goal of quality assessment (QA) measures is to automatically estimate the quality of images in accordance with human quality judgments. There are two types of IQA: subjective and objective. Metrics of objective image quality are primarily used to determine the difference between the test and reference images. The best way to judge the quality of an image is through subjective evaluation. The human observers carry it out.

The full-reference measure, the no-reference measure, and the reduce-reference measure are the three categories that can be used to group the majority of the methods that have been suggested for providing an objective assessment of image quality. Full-reference techniques (FR) [21], in which the algorithm has access to a flawless version of the picture to compare the distorted version, are addressed in the first class of methods. The second class, the no-reference methods (NR) [23], simply have access to the test picture and must judge the quality of the image without knowledge of the ideal version. They do not require the reference image for IQA. The reduced-reference approach (RR), which only uses a portion of the reference picture, is the third category of image quality evaluation. It is important to note that stereoscopic 3D pictures fall under this group as well.

The aim of this paper is to use the gradient similarity and distorted pixel measure to evaluate stereo image quality.

The remainder of the paper is organized as follows. In Section 2, some algorithms of image quality measures are presented. In Section 3, the proposed image quality measure is defined. In section 4, performance of the proposed method is compared with others measures using images with different types of distortion. The conclusion is reported in Section 5.

2 Related works

In the literature many efforts have been completed to develop SIQA measures. The next section breaks out these state-of-the-art SIQA measures per category.

2.1 FR methods of stereoscopic image quality assessment

The combination of two quality measures—the difference between original (left or right) and distorted images—and depth information—the difference between its corresponding disparity maps—was the basis for the approach proposed by Benoit et al. [11]. As measures for evaluating image quality, The SSIM [5] and C4 [20] metrics have been chosen.

The authors of [13] propose to investigate the capacity of 2D image quality metrics to integrate disparity information to estimate stereo image quality. There have been eleven 2D image quality metrics used.

A stereoscopic image quality metric is proposed in [14]. The latter, takes into account the Human Visual System (HVS)’s sensitivity to changes in contrast and luminance at high spatial frequency, and matches regions of high spatial frequency between the left and right views of the stereo pair. It is used to evaluate compressed image quality.

A method for objectively assessing the quality of stereo images was initially proposed in [16]. This strategy surveys sound system pictures from two viewpoints: the objective evaluation of the stereo sense between the viewpoint pairs and the image quality. As a result, there are two components to the indicator of stereo image objective assessment: stereo sense and image quality.

Based on the most recent advancements in human visual system (HVS) physiological and psychological research, the authors of [17] proposed a perceptual metric for assessing stereo video quality. A multi-channel vision model based on 3D wavelet decomposition is proposed after an examination of a number of major HVS properties that are associated with stereo video.

Utilizing the Contrast Sensitivity Function (CSF) and multichannel decomposition with cortex filters to derive perceptual views, Hachicha et al. [19] propose a Stereoscopic Image Quality Assessment measure based on HVS modeling. Additionally, the Binocular Just Noticeable Difference (BJND) model is used in the proposed metric to calculate the smallest distortion in each view of the reference stereo pair. The BJND model was likewise used to demonstrate the binocular concealment hypothesis.

A novel region segmentation algorithm for dividing three-dimensional images into occluded and non-occluded regions is first proposed in [30]. The binocular just noticeable difference model reveals the binocular vision of the human visual system in non-occluded regions, while the just noticeable difference model is utilized on occluded regions to formulate monocular vision. In order to address the unstable segmentation issue encountered by conventional approaches, disparity information and Euclidean distance between stereo pairs are utilized.

A Rich Structural Index (RSI) based on multi-scale perception characteristics is proposed in [35]. Later, this incorporates characteristics of visual perception and the image pyramid. In addition to multi-scale depth edge structures, the approach also takes into account the edge structures and hierarchical structures of multi-scale cyclopean maps. To begin, sensitive images of varying resolution are obtained by placing the stereo pair in the CSF-based image pyramid. On gradient maps, the luminance masking and contrast masking are taken into account, and then a locally adaptive local Luminance and Structural Index (LSI) is obtained. Singular Value Decomposition (SVD) is also used to get the Sharpness and Intrinsic Structural Index (SISI), which help to effectively capture the image’s changes

2.2 RR methods of stereoscopic image quality assessment

Using the extracted edge information, Hewage et al. [12] proposed a Reduced-reference quality metric for the transmission of 3D depth maps.

Wan et al. [24] created the Reduced-reference measure by simulating the brain’s visual perception with sparse representation and natural scene statistics. In particular, the visual information that is closely associated with the hierarchical progressive process of human visual perception. And, it is measured using the distribution statistics of the classified visual primitives extracted by sparse representation.

In [32], the reduced-reference 3D quality assessment evaluator goes through two important technical steps: the perceptual properties of the human visual system (HVS) and the statistical characteristics of three-dimensional images.

Wang et al. [34] utilized disparity and cyclopean images based on depth perception and binocular visual mechanism to complement as monocular perception and used luminance and chroma images that corresponded to the original image to mimic monocular perception. The entropy of the chromaticity map, texture features of the third phase, energy features, energy differences features, and MSCN coefficients from the high frequency sub-band are among the quality-aware features extracted using the quaternion wavelet transform (QWT). A heterogeneous ensemble model using support vector regression (SVR), extreme learning machine (ELM), and random forest (RF) is built to predict quality score, and bootstrap sampling and rotated feature space are used to increase the diversity of the data distribution.

2.3 NR methods of stereoscopic image quality assessment

The reference-less SIQA metrics also have attracted researchers. For instance, For JPEG-coded stereoscopic images, Akhter et al [18] proposes an assessment of image quality with no reference. Local information extraction of disparity and artifacts serves as the foundation for this metric.

A cyclopean image and saliency map-based no-reference stereoscopic IQA has been proposed in [25]. Asymmetrical distortion has been taken into account with the cyclopean image, and saliency aims to select relevant patches from the cyclopean image to focus on the most perceptual relevant regions. In order to estimate the quality, these patches are then fed into a modified version of a CNN model that has already been trained.

A blind stereoscopic IQA metric has been suggested in [26]. The model is based on a sophisticated machine-learning algorithm and human binocular perception. The gradient magnitude (GM), relative gradient orientation (RO), and relative gradient magnitude (RM) maps have been used to extract effective perceptual features. A multiscale gradient map of the cyclopean image was used to account for the various viewing conditions and resolutions of the stereo image. The stereo image features are mapped to the quality score using the AdaBoost neural network.

In [27], another profound element extraction approach has been investigated for NR SIQA. The proposed metric takes into account the phenomenon of binocular rivalry and employs the cyclopean image hypothesis. The cyclopean image is then used to extract a bank of features using four CNN models. Following that, a support vector regression (SVR) is used to map this bank to a quality score.

A cyclopean image and saliency map-based blind stereoscopic IQA has been proposed in [28]. Asymmetrical distortion has been taken into account with the cyclopean image, and saliency aims to concentrate on the most perceptually relevant areas. After that, patches from the cyclopean image’s saliency regions were selected, and a reworked version of the pre-trained vgg-19 [33] model was used to estimate the patches’ quality.

A StereoIF-Net is suggested in [29]. The StereoIF-Net has taken into account all of the human cortex regions’ visual responses to stereoscopic visual signals. It offers a fitting and intricate architecture for replicating the intersection of right and left visual signals in visual cortex regions.

The hypothesis that the perceived quality of the natural stereoscopic view is close to that of the cyclopean image is tested in [31], which generates an intermediate image from the left and right views. The naturalness image quality evaluator score and the entropy score for each subband are calculated using multi-steerable decomposition on cyclopean images.

An algorithm for [36] is proposed that makes use of the characteristics of the spatial domain and the complex contourlet. The across-scale and across-orientation features in the complex contourlet domain as well as the natural scene statistics features in the spatial domain are among the monocular views from which monocular features from the components of CIELAB color space are extracted. The cyclopean image is used for binocular perception to extract energy, energy difference, structural correlation in the complex contourlet domain, and statistics distribution in the spatial domain. The disparity image of stereopairs is also used to extract statistical features related to visual comfort. Finally, the KELM regressor is used to map these features to the objective score.

3 Proposed method

The complete view of the degraded stereo pair is compared with reference stereo pair in order to use a full-reference image quality metric to evaluate the quality of the test image. Our method’s process can be broken down into four steps:

Obtain binocular summation map.

Compute the distorted pixel.

Calculate the gradient similarity.

Obtain the overall assessment as the standard deviation (Fig. 1 shows a flowchart of how the proposed measure is calculated).

Fig. 1

Block diagram of proposed image quality assessment method.

Each section and the structure of the proposed metric are interpreted in details in the following parts. The proposed method uses gradient similarity and Ruderman [4] operator to form map.

The following variables and abbreviations used in the proposed method and the rest of manuscript are defined as:

I: reference image.

J: test image.

I_L: left reference image.

I_R: right reference image.

J_L: left test image.

J_R: right test image.

M × N: image size.

$\hat{I}$ : distorted reference.

$\hat{J}$ : test distorted image

μ (i, j): local mean within the 3 × 3 block surrounding position (i, j).

σ (i, j): standard deviation of (i, j) within the block.

DM _ map:distorted map.

G_I: gradients magnitude of reference image.

G_J: gradients magnitude of test image.

G _ map: Gradient map.

NR: no-reference image quality assessment.

RR: reduced-reference image quality assessment.

FR: full-reference image quality assessment.

IQA: image quality assessment.

HVS: human visual system.

SGSDM: gradient stereo gradient similarity based distorted pixel measure.

CC: Pearson’s linear correlation coefficient.

RMSE: Root mean square prediction error.

ROCC: Spearman’s rank order correlation coefficient.

DMOS_P: predicted Difference Mean Opinion Score.

MOS: mean opinion score.

VQEG: the standard performance of the video quality experts group.

3.1 Binocular summation

Both additive impairments and detail losses are summed in the test stereo pair’s binocular summation map. Additive impairments refer to redundant visual information that does not exist in the original but only appears in the test pair, whereas detail losses refer to the loss of useful visual information in the test stereo pair. We can assess the quality of the binocular summation by separating the test image’s binocular summation from the detail losses and additive impairment and comparing the difference to the reference summation.

Before introducing the notion of the proposed measure, some useful concepts must be explored. The stereo reference and test images are represented by I_L (I_R) and J_L (J_R) respectively. Where L is left image and R is right one. Whereas M × N are image dimensions. The binocular summation map (The reference and test images I and J respectively) of a pair of stereo images can be calculated based on Eq (1) (shown in Fig 2).

Fig. 2

Summation of stereo pair.

$\begin{matrix} I = I_{L} + I_{R} \\ J = J_{L} + J_{R} \end{matrix}$ (1)

3.2 Distorted pixels measure

A distorted pixels measure derived from the reference and test images are used to reflect the local differences between I and J [4]. This cycle can be applied to given intensity image I (i, j) to make: $\hat{I} (i, j) = \frac{I (i, j) - μ (i, j)}{σ (i, j) + 1}$ (2) Where, i ∈ 1, 2 . . . M, j ∈ 1, 2 . . . N are spatial indices, and

$\begin{matrix} μ (i, j) = \\ \frac{1}{(2 K + 1) (2 L + 1)} \sum_{k = - K}^{K} \sum_{l = - L}^{L} I (i + k, j + l) \end{matrix}$ (3) $\begin{matrix} σ (i, j) = \frac{1}{(2 K + 1) (2 L + 1)} \\ \sqrt{\sum_{k = - K}^{K} \sum_{l = - L}^{L} [I (i + k, j + l) - μ (i, j)]} \end{matrix}$ (4) where μ (i, j) is the local mean within the 3 × 3 block surrounding position (i, j), and σ (i, j) is the standard deviation of I (i, j) within the block. The distorted pixel measure of I and J is calculated at the conclusion of this section. In our implementation, K = L = 1; the same procedure is used for J. The distorted test image $\hat{J}$ and the distorted reference image $\hat{I}$ , which represent the test and reference images, are produced as outcomes. The following formula is used to calculate the distortion map (DM _ map):

$DM_map = \frac{2 \hat{I} . \hat{J} + C_{1}}{{\hat{I}}^{2} + {\hat{J}}^{2} + C_{1}}$ (5) where C₁ is positive constant.

3.3 Gradient similarity

Information can be extracted from images using a gradient. To generate gradient mages, this section introduces 5 × 5 Sobel operators; It explains as: $\begin{matrix} G (x) = [\begin{matrix} - 1 & - 2 & 0 & 2 & 1 \\ - 2 & - 3 & 0 & 3 & 2 \\ - 3 & - 5 & 0 & 5 & 3 \\ - 2 & - 3 & 0 & 3 & 2 \\ - 1 & - 2 & 0 & 2 & 1 \end{matrix}] \frac{1}{24} \end{matrix}$

$\begin{matrix} G (y) = [\begin{matrix} 1 & 2 & 3 & 2 & 1 \\ 2 & 3 & 5 & 3 & 2 \\ 0 & 0 & 0 & 0 & 0 \\ - 2 & - 3 & - 5 & - 3 & - 2 \\ - 1 & - 2 & - 3 & - 2 & - 1 \end{matrix}] \frac{1}{24} \end{matrix}$ These are used to identify image vertical and horizontal edges and are made up of a pair of 5×5 convolution kernels. Where the image’s partial derivatives are G (x) and G (y). The following expressions provide the reference (G_I) and test (G_J) images’ gradient magnitudes: $\begin{matrix} G_{I} = \sqrt{G_{I} (x)^{2} + G_{I} (y)^{2}} \\ G_{J} = \sqrt{G_{J} (x)^{2} + G_{J} (y)^{2}} \end{matrix}$ (6)

The Gradient map (G _ map) is formed as a result of our method for calculating gradient similarity: $G_map = \frac{2 G_{I} . G_{J} + C_{2}}{G_{I}^{2} + G_{J}^{2} + C_{2}}$ (7) where C₂ is positive constant.

3.4 Global error

Last but not least, the stereo gradient similarity based distorted pixel measure map (SGSDM _ map) can be expressed as follows:

$SGSDM_map = (DM_map)^{α} . G_map$ (8) The standard deviation of the SGSDM map is what is referred to as the total gradient similarity based distorted pixel measure (SGSDM): $\begin{matrix} SGSDM = \\ \sqrt{\frac{1}{N . M} \sum_{p = 1}^{M} \sum_{q = 1}^{N} (SGSD - SGSDM_map (p, q))^{2}} \end{matrix}$ (9)

Where $SGSD = \frac{1}{N . M} \sum_{p = 1}^{M} \sum_{q = 1}^{N} SGSDM_map (p, q)$ (10) Figure 3 presents a flowchart of High-level overview of how the proposed method assesses the quality of distorted stereo images. Also, it summarize the suggested measure’s calculation all steps.

Fig. 3

A flowchart showing a high level overview of the proposed image quality assessment method.

4 Results

4.1 The used image databases and measurement procedures

To examine SGSDM’s performance; The video quality experts group (VQEG)’s [6] standard performance is followed. Non-linear mapping was carried out between the subjective scores [12] and the objective scores. The five parametric non-linear mappings (θ₁, θ₂, θ₃, θ₄, and θ₅) are used to transform the objective quality measure set of quality ratings into the predicted Difference Mean Opinion Score (DMOS) values (DMOS_P). The mapping function, which is a logistic function, is described in equation (11) [7]. $\begin{matrix} {DMOS}_{p} (VQR) = \\ θ_{1} [\frac{1}{2} - \frac{1}{\exp (θ_{2} (VQR - θ_{3})}] + θ_{4} VQR + θ_{5} \end{matrix}$ (11)

Where VQR is the objective quality rating and θ₁, θ₂, θ₃, θ₄, and θ₅ are the ones chosen for the best fit. Three metrics are used after the regression: the Spearman rank-order correlations coefficient (ROCC), the Pearson linear correlation coefficient (CC), and the Root mean square prediction error (RMSE). The first is the Pearson linear correlation coefficient (CC), which measures the relationship between subjective (DMOS) and objective (DMOSP) scores. It provides an assessment of the accuracy of the prediction and is defined as: $\begin{matrix} CC = \frac{\sum_{i = 1}^{n} Obj . Sub}{\sqrt{\sum_{i = 1}^{n} {Obj}^{2}} \sqrt{\sum_{i = 1}^{n} {Sub}^{2}}} \\ Obj = (DMOS (i) - \bar{DMOS}) \\ Sub = ({DMOS}_{p} (i) - \bar{{DMOS}_{p}}) \end{matrix}$ (12) Where the image sample is denoted by the index i and the number of samples by the number n. The Spearman rank-order correlations coefficient (ROCC) [10] test for conformity between the rank orders of model predictions (DMOS_P) and DMOS is the second index. This correlation measure only assumes that the two quantities have a monotonic relationship. In fact, it does not require the assumption of any particular functional form in the relationship between data and predictions. This type of correlation is advantageous. It is defined as follows: $\begin{matrix} ROCC = 1 - \frac{6 \sum (DMOS (i) - {DMOS}_{p} (i))^{2}}{n (n^{2} - 1)} \end{matrix}$ (13) The third one is the Root mean square prediction error (RMSE) between subjective (DMOS) and objective (DMOS_P) scores. It is defined by: $\begin{matrix} RMSE = \sqrt{\frac{1}{n} \sum_{i = 1}^{n} (DMOS (i) - {DMOS}_{p} (i))^{2}} \end{matrix}$ (14) Prediction consistency is the ability of an objective aptitude model to consistently make accurate predictions for all types of images and not fail significantly for a subset of them. The monotonicity of predictions is evaluated by ROCC. The prediction accuracy is evaluated using CC and RMSE. With values that are closer to 1 or -1, ROCC and CC perform better. As a result, RMSE performs better when its values are low.

To test the suggested approach; the LIVE 3D Database of the University of Texas at Austin consists of two phases is utilized: LIVE 3D Phase I Database [22] and LIVE 3D Phase II Database [38]. 20 reference images (shown in Fig. 4) and 365 distorted images (80 each for JEPG, JEPG2000 (JP2K) compression, additive white noise (WN), fast fading (FF) and 45 with Blur) are included in the LIVE 3D Phase I Database.

Fig. 4

reference images used in the subjective study. Shown here are only the left-views.

Compression using the JPEG and JPEG2000 compression standards, additive white Gaussian noise, Gaussian blur, and a fastfading model based on the Rayleigh fading channel were the distortions that were simulated. The JPEG compression utility in MATLAB was used to simulate JPEG compression, while the Kakadu encoder was used to simulate JPEG2000 compression. The "quality" parameter and the bitrate were the only variables. The imnoise command in MATLAB was used to simulate additive white Gaussian noise, which was applied equally to the R, G, and B planes. By applying a Gaussian low-pass filter to each of the color planes, Gaussian blur was also simulated. The variance of the Gaussian was the control parameter for both WN and Blur. A JP2K compressed image was used to transmit fast-fading distortion over a Rayleigh fading channel, with the channel Signal-to-Noise ratio (SNR) serving as the control parameter.

LIVE 3D Phase II Database contains 8 reference images and 360 distorted images (72 each for JP2K, JPEG, WN, Blur, and FF).

To illustrate the performance of the proposed method SGSDM, we compare it with 21 state-of-the-art measures, including Benoit [11], Hewage [12], You [13], Gorley [14], Shen [15], Yang [16], Zhu [17], Akhter [18], Hachicha [19], PSNR, SSIM [5], MS-SSIM [37], Messai01 [25], Messai02 [27], Jianwei20 [30], KELM [36], Jianwei22 [29], R3DQAE [32], MO-NIQE [31], Wang2021 [34] and Zhang2022 [35].

4.2 Gradient operator selection

Table 1 shows a comparison study with the classic Sobel [8], Prewitt [8], Scharr [9], and the 5 × 5 Sobel. The 5 × 5 Sobel may perform better than the other three.

Table 1
ROCC values using four gradient operators

Gradient operator ROCC

Sobel 0.9261

Prewitt 0.9274

Scharr 0.9234

5 × 5 Sobel 0.9379

Gradient operator	ROCC
Sobel	0.9261
Prewitt	0.9274
Scharr	0.9234
5 × 5 Sobel	0.9379

4.3 Performance comparison

On each 3D IQA database, the ROCC, CC, and RMSE performance of the image quality assessment methods are compared in Tables 2 , and 4. Each assessment measure’s top three measures are highlighted in bold.

Table 2
Spearman’s Rank Ordered Correlation Coefficient (ROCC)

LIVE-PI LIVE-PII

Algorithm JP2K JPEG WN Blur FF All JP2K JPEG WN Blur FF All

SGSDM 0.965 0.964 0.974 0.957 0.953 0.953 0.967 0.972 0.975 0.964 0.964 0.964

Hachicha 0.917 0.660 0.941 0.950 0.775 0.925 - - - - - -

Benoit 0.910 0.603 0.929 0.931 0.699 0.899 0.923 0.751 0.867 0.455 0.773 0.728

Hewage 0.856 0.500 0.896 0.690 0.545 0.814 0.880 0.598 0.736 0.028 0.684 0.501

You 0.860 0.439 0.940 0.882 0.588 0.879 0.909 0.894 0.795 0.813 0.891 0.786

Gorley 0.420 0.015 0.741 0.750 0.366 0.142 0.875 0.110 0.027 0.770 0.601 0.146

Shen 0.213 0.244 0.892 0.659 0.267 0.068 - - - - - -

Yang 0.150 0.133 0.847 0.327 0.143 0.079 - - - - - -

Zhu 0.771 0.293 0.465 0.794 0.475 0.639 - - - - - -

Akhter 0.866 0.675 0.914 0.555 0.639 0.383 0.714 0.724 0.649 0.682 0.559 0.543

PSNR 0.301 0.313 0.650 0.608 0.510 0.510 0.865 0.757 0.768 0.728 0.725 0.725

SSIM 0.458 0.466 0.694 0.501 0.362 0.362 0.800 0.777 0.788 0.751 0.757 0.757

MS-SSIM 0.502 0.489 0.688 0.495 0.365 0.365 0.796 0.801 0.805 0.783 0.772 0.772

Messai01[25] 0.899 0.625 0.941 0.887 0.777 0.930 0.842 0.837 0.943 0.913 0.925 0.913

Messai02[27] 0.964 0.943 0.834 0.953 0.803 0.956 0.959 0.888 0.875 0.935 0.945 0.948

Jianwei20 [30] 0.921 0.674 0.959 0.918 0.850 0.955 0.911 0.875 0.959 0.920 0.951 0.939

KELM [36] 0.921 0.674 0.959 0.918 0.850 0.955 0.911 0.875 0.959 0.920 0.951 0.939

Jianwei22 [29] 0.998 0.894 0.999 0.999 0.900 0.966 0.976 0.931 0.972 0.960 0.956 0.953

R3DQAE [32] 0.807 0.869 0.963 0.933 0.947 0.932 0.900 0.669 0.929 0.903 0.746 0.921

MO-NIQE [31] 0.869 0.480 0.710 0.849 0.587 0.801 0.854 0.561 0.708 0.890 0.800 0.669

Wang2021 [34] 0.936 0.841 0.952 0.900 0.856 0.950 0.916 0.876 0.961 0.965 0.952 0.953

Zhang2022 [35] 0.927 0.676 0.925 0.934 0.816 0.943 0.964 0.909 0.940 0.932 0.918 0.945

	LIVE-PI	LIVE-PII
SGSDM	0.965	0.964	0.974	0.957	0.953	0.953	0.967	0.972	0.975	0.964	0.964	0.964
Hachicha	0.917	0.660	0.941	0.950	0.775	0.925	-	-	-	-	-	-
Benoit	0.910	0.603	0.929	0.931	0.699	0.899	0.923	0.751	0.867	0.455	0.773	0.728
Hewage	0.856	0.500	0.896	0.690	0.545	0.814	0.880	0.598	0.736	0.028	0.684	0.501
You	0.860	0.439	0.940	0.882	0.588	0.879	0.909	0.894	0.795	0.813	0.891	0.786
Gorley	0.420	0.015	0.741	0.750	0.366	0.142	0.875	0.110	0.027	0.770	0.601	0.146
Shen	0.213	0.244	0.892	0.659	0.267	0.068	-	-	-	-	-	-
Yang	0.150	0.133	0.847	0.327	0.143	0.079	-	-	-	-	-	-
Zhu	0.771	0.293	0.465	0.794	0.475	0.639	-	-	-	-	-	-
Akhter	0.866	0.675	0.914	0.555	0.639	0.383	0.714	0.724	0.649	0.682	0.559	0.543
PSNR	0.301	0.313	0.650	0.608	0.510	0.510	0.865	0.757	0.768	0.728	0.725	0.725
SSIM	0.458	0.466	0.694	0.501	0.362	0.362	0.800	0.777	0.788	0.751	0.757	0.757
MS-SSIM	0.502	0.489	0.688	0.495	0.365	0.365	0.796	0.801	0.805	0.783	0.772	0.772
Messai01[25]	0.899	0.625	0.941	0.887	0.777	0.930	0.842	0.837	0.943	0.913	0.925	0.913
Messai02[27]	0.964	0.943	0.834	0.953	0.803	0.956	0.959	0.888	0.875	0.935	0.945	0.948
Jianwei20 [30]	0.921	0.674	0.959	0.918	0.850	0.955	0.911	0.875	0.959	0.920	0.951	0.939
KELM [36]	0.921	0.674	0.959	0.918	0.850	0.955	0.911	0.875	0.959	0.920	0.951	0.939
Jianwei22 [29]	0.998	0.894	0.999	0.999	0.900	0.966	0.976	0.931	0.972	0.960	0.956	0.953
R3DQAE [32]	0.807	0.869	0.963	0.933	0.947	0.932	0.900	0.669	0.929	0.903	0.746	0.921
MO-NIQE [31]	0.869	0.480	0.710	0.849	0.587	0.801	0.854	0.561	0.708	0.890	0.800	0.669
Wang2021 [34]	0.936	0.841	0.952	0.900	0.856	0.950	0.916	0.876	0.961	0.965	0.952	0.953
Zhang2022 [35]	0.927	0.676	0.925	0.934	0.816	0.943	0.964	0.909	0.940	0.932	0.918	0.945

Table 3

Linear Correlation Coefficient (CC)

	LIVE-PI						LIVE-PII
Algorithm	JP2K	JPEG	WN	Blur	FF	All	JP2K	JPEG	WN	Blur	FF	All
SGSDM	0.948	0.955	0.978	0.949	0.952	0.952	0.991	0.977	0.979	0.971	0.970	0.970
Hachicha	0.940	0.657	0.950	0.962	0.847	0.934	-	-	-	-	-	-
Benoit	0.940	0.641	0.925	0.949	0.747	0.903	0.926	0.784	0.853	0.535	0.807	0.784
Hewage	0.904	0.531	0.896	0.798	0.670	0.830	0.891	0.664	0.734	0.450	0.746	0.558
You	0.878	0.487	0.941	0.920	0.730	0.881	0.912	0.905	0.830	0.784	0.915	0.800
Gorley	0.485	0.312	0.796	0.853	0.365	0.451	0.322	0.372	0.874	0.934	0.706	0.515
Shen	0.504	0.390	0.899	0.685	0.483	0.574	-	-	-	-	-	-
Yang	0.201	0.274	0.870	0.626	0.282	0.391	-	-	-	-	-	-
Zhu	0.807	0.379	0.518	0.777	0.504	0.626	-	-	-	-	-	-
Akhter	0.906	0.729	0.905	0.618	0.660	0.427	0.929	0.772	0.776	0.786	0.795	0.674
PSNR	0.506	0.399	0.781	0.756	0.437	0.437	0.880	0.763	0.770	0.732	0.733	0.733
SSIM	0.482	0.437	0.790	0.687	0.535	0.535	0.820	0.786	0.795	0.771	0.773	0.773
MS-SSIM	0.537	0.494	0.755	0.653	0.519	0.519	0.822	0.819	0.817	0.802	0.794	0.794
Messai01[25]	0.926	0.668	0.941	0.935	0.845	0.939	0.835	0.859	0.953	0.978	0.925	0.922
Messai02[27]	0.970	0.960	0.845	0.962	0.865	0.955	0.959	0.887	0.888	0.981	0.931	0.941
Jianwei20[30]	0.945	0.746	0.933	0.961	0.955	0.944	0.910	0.901	0.899	0.961	0.940	0.927
KELM [36]	0.967	0.761	0.985	0.974	0.897	0.968	0.918	0.899	0.981	0.982	0.960	0.941
Jianwei22 [29]	0.998	0.960	0.997	0.996	0.996	0.978	1.000	0.970	0.993	0.996	0.996	0.972
R3DQAE [32]	0.793	0.880	0.954	0.972	0.944	0.925	0.725	0.937	0.936	0.915	0.832	0.934
MO-NIQE [31]	0.862	0.488	0.770	0.905	0.692	0.829	0.854	0.603	0.756	0.942	0.816	0.729
Wang2021 [34]	0.965	0.875	0.975	0.966	0.910	0.959	0.943	0.921	0.978	0.994	0.967	0.965
Zhang2022 [35]	0.968	0.785	0.979	0.956	0.886	0.951	0.933	0.945	0.987	0.971	0.963	0.943

Table 4

Root-mean-squared-error (RMSE)

	LIVE-PI						LIVE-PII
Algorithm	JP2K	JPEG	WN	Blur	FF	All	JP2K	JPEG	WN	Blur	FF	All
SGSDM	4.129	3.252	3.299	4.919	5.014	5.014	2.284	2.331	2.260	2.656	2.766	2.766
Hachicha	4.411	4.933	5.186	3.932	6.601	5.855	-	-	-	-	-	-
Benoit	4.427	5.022	6.308	4.571	8.258	7.062	4.028	6.096	3.787	11.763	6.894	7.490
Hewage	5.530	5.543	7.406	8.748	9.226	9.139	10.713	7.343	4.976	12.436	7.667	9.364
You	6.207	5.710	5.622	5.680	8.492	7.746	4.396	4.186	4.086	8.649	4.649	6.772
Gorley	11.323	6.212	10.197	7.562	11.569	14.635	5.202	9.113	6.940	4.988	8.155	9.675
Shen	12.275	6.022	7.294	10.554	10.882	13.547	-	-	-	-	-	-
Yang	12.697	6.289	8.200	12.129	11.946	15.248	-	-	-	-	-	-
Zhu	7.681	6.068	14.720	9.127	10.736	12.782	-	-	-	-	-	-
Akhter	5.484	4.474	7.093	11.387	9.332	14.827	7.416	6.189	4.535	8.450	8.505	9.294
PSNR	11.281	10.173	9.823	10.272	14.607	14.607	4.937	6.848	6.812	7.419	7.506	7.506
SSIM	11.350	9.836	9.611	11.291	13.851	13.851	5.991	6.705	6.644	7.059	7.161	7.161
MS-SSIM	10.926	9.505	10.276	11.771	14.018	14.018	5.968	6.221	6.323	6.622	6.866	6.866
Messai01 [25]	4.867	4.862	5.593	5.104	6.633	5.605	5.396	3.752	3.226	2.859	4.352	4.352
Messai02 [27]	4.013	3.597	3.488	3.943	6.223	4.865	3.002	4.526	3.366	2.685	4.176	3.817
Jianwei20 [30]	-	-	-	-	-	5.313	-	-	-	-	-	6.194
KELM [36]	-	-	-	-	-	4.211	-	-	-	-	-	3.998
Jianwei22 [29]	-	-	-	-	-	2.608	-	-	-	-	-	2.277
R3DQAE [32]	-	-	-	-	-	4.295	-	-	-	-	-	5.869
MO-NIQE[31]	8.423	5.705	8.259	6.138	8.960	9.149	5.563	7.829	4.796	4.672	6.830	7.813
Wang2021 [34]	-	-	-	-	-	4.089	-	-	-	-	-	4.498
Zhang2022 [35]	-	-	-	-	-	5.003	-	-	-	-	-	4.286

From these tables, the proposed method ranks the top three on LIVE 3D Phase II database. In the LIVE 3D Phase I database, the performance of SGSDM slightly less than the performance of Jianwei22, Messai02 and Wang2021 works. The combination the distorted pixel measure and gradient similarity features in this paper can describe the situation of distortion usefully. Moreover, the introduction of gradient similarity feature can reinforce the discrimination of several types of distortion. The proposed method is well correlated with human subjective perception, as shown by the preceding analysis.

The proposed measure and other SIQA algorithms are also tested in the LIVE 3D IQA database on a variety of distortion types. The results of ROCC and CC in LIVE 3D IQA database are listed in Tables 3, in which the top three results are highlighted in bold.

Our approach provides the highest CC performance on JP2K and FF in LIVE 3D Phase I and on JPEG, JP2K and FF in LIVE 3D Phase II. On JPEG, JP2K in LIVE 3D Phase I, the proposed method is only inferior to the Messai02 model when compared to Wang2021, MO-NIQE, and Messai02 methods. In addition, in LIVE 3D Phase I, the KELM model outperforms the proposed model on WN, Blur, and JP2K, JPEG, and FF. However, our model’s prediction accuracy is competitively robust and stable across all distortion types.

For the ROCC performance, our measure provides the best three performance on JP2K, JPEG, WN, Blur and FF in LIVE 3D Phase I and LIVE 3D Phase II, and performs well under all kinds of distortions, demonstrating the monotony of the proposed model. In LIVE 3D Phase I, the proposed model outperforms the Jianwei22 model on JP2K distortion, JPEG distortion, and Blur, but it outperforms the Jianwei22 model on all distortions in LIVE 3D Phase II, with the exception of JP2K distortion. The fact that the proposed model performs best overall suggests that it can predict the quality of a 3D image under a variety of distortions.

Fig. 5 displays the subjective DMOS scatter distributions in comparison to the SGSDM database’s projected scores. The curve in Fig. 5 was created by a nonlinear fitting using (11).

Fig. 5

Scatter plots of subjective scores versus scores from the proposed scheme on IQA databases.

The curves in Figure 5 show that the SGSDM values are very close to DMOS, indicating that this measure is effective. The classification of all measures’ performance based on their ROCC values in Table 2 demonstrates the SGSDM’s dependability. However, the comparison yields an intriguing result. The CC and ROOC values are closer to 1. Based on our examination of the results obtained with Hachicha, we conclude that SGSDM performs significantly better than Hachicha. Our SGSDM measure clearly performs well and is competitive with other IQA measures, as shown by the results.

4.4 Determination of parameters

The proposed model performs well without test settings and learning strategy. Also, this later outperforms significantly higher consistency with subjective opinions.

SGSDM necessitates the determination of three parameters (see Equations (5), and (8)).We adjusted the parameter in this way using data from the 3D LIVE IQA database. The parameter value that led to a higher ROCC was chosen as the adjusting measure. Consequently, the required parameter for the proposed method was set to: α=0.88, C₁=170, and C₂=180.

Moreover, we try to study the outcomes of SGSDM using equation (8) by computing the standard deviation or the mean, as result its performance is higher with standard deviation.

4.5 Efficiency evaluation

To compare the effectiveness of various measures, the execution time is calculated on an image from the 3D LIVE IQA database. Asus Intel(R) Core (TM) i5-4200U CPU @ 1.60 GHz and 2.30 GHz 4G RAM is used to run IQA measures. Additionally, the platform for the software is MATLAB R2015a. Table 5 displays every result that was obtained. SGSDM is observed to take more time than SSIM. The other tests are slower than SGSDM.

Table 5
Running time of the competing IQA models

Metrics PSNR SSIM MS-SSIM SGSDM

Time (second) 0.123877 0.102500 0.426418 0.115346

Metrics	PSNR	SSIM	MS-SSIM	SGSDM
Time (second)	0.123877	0.102500	0.426418	0.115346

5 Conclusion

Due to two binocular interactions known as binocular fusion and suppression, the human visual system (HVS) is able to distinguish between two retinal images in order to construct a mental image with depth perception. In this paper, we proposed a SGSDM algorithm employing the binocular summation channel. Additionally, the proposed metric employs the distorted pixel measure and gradient similarity for IQA. Exploratory outcomes show that the anticipated consequences of the proposed model are exceptionally predictable with human subjective perception. Our subsequent research will focus on the application of additional HVS characteristics to the quality evaluation of stereoscopic images. Test some of the hypotheses regarding the visual characteristics of stereoscopic images by applying our metric to the evaluation of stereoscopic video quality.

References

winkler

, Digital Video Quality: Vision Models and Metrics, Wiley Editorial Offices, 2005.

Yim

and Bovik

A.C.

, Quality assessment of de-blocked images, IEEE Trans Image Processing 20(1) (2011), pp. 88–98.

Saad

M.A.

, Bovik

A.C.

and Charrier

, A DCT Statistics-Based Blind Image Quality Index, IEEE Signal Processing Letters 17(6) (2010), pp. 583–586.

Ruderman

D.L.

, The statistics of natural images, Netw Comput Neural Syst 5(4) (1994), pp. 517–548.

Wang

, Bovik

A.C.

, Sheikh

H.R.

and Simocelli

E.P.

, Image quality assessment: From error measurement to structural similarity, IEEE Trans Image Processing 13(4) (2004), pp. 600–612.

Brunnstrom

, Hands

, Speranza

and Webster

, VQeg validation and ITU standardization of objective perceptual video quality metrics [standards in a nutshell], IEEE Signal Processing Magazine 26(3) (2009), pp. 96–101.

Sheikh

H.R.

, Sabir

M.F.

and Bovik

A.C.

, A statistical evaluation of recent full reference image quality assessment algorithms, IEEE Trans on Image Processing 15(11) (2006), pp. 3440–3451.

Jain

, Kasturi

and Schunck

B.G.

, Machine Vision, NewYork: McGraw-Hill, 1995.

Jahne

, Haubecker

and Geibler

, Handbook of Computer Vision and Applications, New York: Academic, 1999.

10.

Chok

N.S.

, Pearson’sVersus Spearman’s and Kendall’s Correlation Coefficients for Continuous Data, Master’s Thesis, University of Pittsburgh (2010).

11.

Benoit

, Le Callet

and Campisi

, Quality assessment of stereoscopic images, EURASIP Journal on Image and Video Processing 2008, 2009.

12.

Hewage

C.T.E.R.

and Martini

M.G.

, Reduced-reference quality metric for 3d depth map transmission, in 3DTV-CON, Tampere, Finland, (2010), pp. 1–4.

13.

You

, Xing

, Perkis

and Wang

, Perceptual quality assessment for stereoscopic images based on 2d image quality metrics and disparity analysis, inVPQM,Scottsdale, Arizona, USA, 2010.

14.

Gorley

and Holliman

, Stereoscopic image quality metrics and compression, in IS&T/SPIE Electronic Imaging 6803 (2008), pp. 680305.

15.

Shen

, Yang

and Zhang

, Stereo picture quality estimation based on a multiple channel hvs model, in CISP, Tianjin, China, (2009), pp. 1–4.

16.

Yang

, Hou

, Zhou

, Zhang

and Guo

, Objective quality assessment method of stereo images, in3DTV-CON, Potsdam, Germany, 2009, pp. 1–4.

17.

Zhu

and Wang

, Perceptual distortion metric for stereo video quality evaluation, WSEAS 5(7) (2009), pp. 70.

18.

Akhter

, Sazzad

Z.M.P.

, Horita

and Baltes

, No reference stereoscopic image quality assessment, in IS&T/SPIE Electronic Imaging 7524 (2010).

19.

Hachicha

, Beghdadi

and Alaya

F.C.

, Stereo image quality assessment using a binocular just noticeable difference model, In: IEEE International Conference on Image Processing (2013).

20.

Carnec

, Le Callet

and Barba

, An image quality assessment method based on perception of structural information, IEEE International Conference on Image Processing 2 (2003), 185–188.

21.

Saha

and Wu

Q.M.J.

, Full-reference image quality assessment by combining global and local distortion measures, Signal Processing 128 (2016), 186–197.

22.

Moorthy

A.K.

, Su

C.-C.

, Mittal

and Bovik

A.C.

, Subjective evaluation of stereoscopic image quality, Signal Processing: Image Communication 28(9) (2013), pp. 870–883.

23.

Zhang

, Le

T.M.

, Ong

S.H.

and Nguyen

T.Q.

, No-reference image quality assessment using structural activity, Signal Processing 91 (2011), 2575–2588.

24.

Wan

, Gu

and Zhao

, Reduced reference stereoscopic image quality assessment using sparse representation and natural scene statistics, IEEE Trans Multimedia 22(8) (2020), 2024–2037.

25.

Messai

, Chetouani

, Hachouf

and Seghir

Z.A.

, 3D saliency guided deep quality predictor for no-reference stereoscopic images, Neurocomputing 478 (2022), 22–36.

26.

Messai

, Hachouf

and Ahmed Seghir

, AdaBoost neural network and cyclopean view for no-reference stereoscopic image quality assessment, Elsevier, Signal Processing: Image Communication 82 (2020), 115772.

27.

Messai

, Chetouani

, Hachouf

and Seghir

Z.A.

, No-reference Stereoscopic Image Quality Predictor using Deep Features from Cyclopean Image, in IS&T International Symposium on Electronic Imaging 2021, Image Quality and System Performance XVIII, pp. 297–304.

28.

Messai

, Chetouani

, Hachouf

and Seghir

Z.A.

, Deep Quality evaluator guided by 3D Saliency for Stereoscopic Images, in IS&T International Symposium on Electronic Imaging 2021, Image Quality and System Performance XVIII, pp. 110–115.

29.

, Huang

, Yang

, Lin

and Pan

, A no-Reference Stereoscopic Image Quality Assessment Network Based on Binocular Interaction and Fusion Mechanisms, in IEEE Transactions on Image Processing 31 (2022), pp. 3066–3080. doi: 10.1109/TIP.2022.3164537.

30.

, Yang

, Huang

, Pan

and Su

, A full-reference stereoscopic image quality assessment index based on stable aggregation of monocular and binocular visual features, IET Image Processing 2021.

31.

Appina

, A complete blind no-reference stereoscopic image quality assessment algorithm, in 2020 International Conference on Signal Processing and Communications (SPCOM). IEEE, (2020), pp. 1–5.

32.

, Xu

and Han

, Reduced-Reference 3D Image Quality Measurement via Spatial to Gradient Domain Feature Aggregation, J Electr Eng Technol 17 (2022), 1389–1405.

33.

Simonyan

and Zisserman

, Very deep convolutional networks for large-scale image recognition, arXivpreprint arXiv:1409.1556, 2014.

34.

Wang

, Li

, Guan

and Zhao

, Noreference stereoscopic image quality assessment using quaternion wavelet transform and heterogeneous ensemble learning, Displays 69 (2021), 102058.

35.

Zhang

, Hu

, Gou

, Zhang

, Zheng

and Shen

, Rich Structural Index for Stereoscopic Image Quality Assessment, Sensors (Basel) 22(2) (2022), 499. doi: 10.3390/s22020499. PMID: 35062460; PMCID: PMC8780543.

36.

Guan

, Li

, Zheng

, Zhao

and Wu

, No-reference stereoscopic image quality assessment on both complex contourlet and spatial domain via Kernel ELM, Signal Processing: Image Communication 101 (2022), 116547.

37.

Wang

, Simoncelli

E.P.

and Bovik

A.C.

, Multi-scale structural similarity for image quality assessment, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers 2 (2003), 1398–1402.

38.

Moorthy

A.K.

, Su

C-C.

, Mittal

and Conrad Bovik!!

, Subjective evaluation of stereoscopic image quality, SSignal Processing: Image Communication 28 (2013), pp. 870–883.

Stereo image quality assessment using deformed pixels and Sobel magnitude

Abstract

Keywords

1 Introduction

2 Related works

2.1 FR methods of stereoscopic image quality assessment

2.2 RR methods of stereoscopic image quality assessment

2.3 NR methods of stereoscopic image quality assessment

3 Proposed method

4.1 The used image databases and measurement procedures

Table 1 ROCC values using four gradient operators Gradient operator ROCC Sobel 0.9261 Prewitt 0.9274 Scharr 0.9234 5 × 5 Sobel 0.9379

4.5 Efficiency evaluation

Table 5 Running time of the competing IQA models Metrics PSNR SSIM MS-SSIM SGSDM Time (second) 0.123877 0.102500 0.426418 0.115346

References

Table 1
ROCC values using four gradient operators

Gradient operator ROCC

Sobel 0.9261

Prewitt 0.9274

Scharr 0.9234

5 × 5 Sobel 0.9379

Table 5
Running time of the competing IQA models

Metrics PSNR SSIM MS-SSIM SGSDM

Time (second) 0.123877 0.102500 0.426418 0.115346