Abstract
Blind image deconvolution has attracted growing attention in image processing and computer vision. The total variation (TV) regularization can effectively preserve image edges. However, due to lack of self-adaptability, it does not perform very well on restoring images with complex structures. In this paper, we propose a new blind image deconvolution model using an adaptive weighted TV regularization. This model can better handle local features of image. Numerically, we design an effective alternating direction method of multipliers (ADMM) to solve this non-smooth model. Experimental results illustrate the superiority of the proposed method compared with other related blind deconvolution methods.
Introduction
Image deconvolution is a fundamental problem in image processing, which has attracted increasing interest in recent years. Its aim is to reconstruct a clean image from a noisy blurred image. The blurred image is usually modeled as a linear convolution of an image and a blur kernel also called the point spread function (PSF). Mathematically, the image degradation process can be described as follows:
To solve the blind deconvolution problem, many regularization techniques have been studied. You et al. [32] proposed a blind deconvolution method by employing the H1 norm to minimize the image u and the blur kernel k. The H1 norm minimization problem for u and k can be expressed as
Recently, blind image deconvolution has attracted growing attention in image processing [14, 37]. Money et al. [17] applied the shock filter to pre-process the degraded image, and then used the pre-processed image as an initial condition for TV minimization blind deconvolution. This method can save the computational time, compared with the AM method in [4]. To further reduce the computational cost, the split Bregman algorithm was proposed in [15] to solve the TV blind deconvolution problem (1.3). Specifically, this method has better recovery effect on various types of blur than the proposed method in [17]. Although the TV regularization performs well on preserving edges, it is not good at keeping textures of image. Therefore, Zuo et al. [37] proposed an adaptive non-local TV method for blind image restoration. This method can fully utilize the spatial information distributed in different image regions via the non-local TV operator, and thus can preserve more details of image. Owing to the fact that cartoon and texture components can be represented differently in images, Wang et al. [27] used cartoon-texture decomposition technique for textures preservation in blind image deconvolution. Additionally, the TV norm is easy to convert the smooth signal into piecewise constant, which leads to staircase effects. To eliminate the staircase effects, Li et al. [14] proposed a non-convex high-order TV model for blind image deconvolution. However, the high-order TV regularization is not ideal for preserving image edges. In [35], Zhang et al. proposed a blind deconvolution model based on edge-preserving of L0-regularized gradient prior. To better maintain the edges, an adaptive strategy was adopted to reduce the difficulty of parameter setting. The blur kernel estimated by this method is more accurate, which improves the performance of deblurring. In addition, many improved methods based on TV regularization were also proposed [3, 36].
Although most of the previously studied methods can obtain relatively satisfactory results, it is not ideal to process local structures of the image. Recently, Pang et al. [21, 22] proposed some new regularization techniques combining an adaptive weighted matrix with gradient operator for image denoising. The adaptive weighted matrix can rotate the direction of the gradient operator and make it tend to a larger weight, so it can better describe the local features of image. This fact motivates us to apply this adaptive weighted TV regularization for blind deconvolution. The major contributions of the paper are three-fold: We propose an adaptive weighted TV model for blind image deconvolution. Owing to the use of an adaptive weighted matrix, the proposed model can better deal with local structures of image. We design an effective ADMM to solve the proposed model. The proposed algorithm may yield a unique solution. Extensive experiments are carried out under different blur types and noise levels to show that our proposed method has better performance than other two methods for blind image deconvolution.

Diagram of the basic idea for our proposed method.
The block diagram of the basic idea of our proposed method is shown in Fig 1. The rest of this paper is organized as follows. In Section 2, we first introduce some relevant preliminaries, and then propose an adaptive weighted TV blind image deconvolution model. Section 3 presents an efficient ADMM to solve the proposed model. In Section 4, we give some related experimental results to demonstrate the effectiveness and superiority of the proposed method. Finally, we end the paper with some conclusions in Section 5.
This section is devoted to presenting a new model for blind image deconvolution, where an adaptive weighted matrix is integrated with the gradient operator. Then we make a comparison with other two methods on purpose, strengths and weaknesses.
Comparative analysis of different methods
Comparative analysis of different methods
Generally speaking, different regions of image have different structures. It is crucial to preserve local structures for the proposed model. Since the finite difference scheme of gradient only depends on the horizontal and vertical directions, traditional TV-based models cannot couple with local structures of image efficiently. To overcome this drawback, some adaptive techniques have been proposed in many image applications [20, 29]. Specifically, in [22], the authors proposed an anisotropic TV (ATV) model for Gaussian image denoising, which can better diffuse along the tangent direction of local features. To couple with local structures more effectively, they constructed different weights by adding an adaptive weighted matrix
For blind image deconvolution, we consider combining the adaptive weighted matrix
In this section, we design an ADMM to solve the proposed model (2.2). The solution of the initial problem is transformed into alternating calculation of several simple subproblems. The ADMM has been widely used in convex and non-convex optimization, especially for the problems with non-differentiable functionals [7, 34]. It also has many applications in the field of blind image deconvolution [8, 24].
To apply the ADMM, we introduce three auxiliary variables
The k-subproblem can be expressed as
The u-subproblem can be written as
For the
The
We note that the solution (k, u) of the proposed model (2.2) is not unique. Hence some constraints on k and u are needed to impose so as to obtain a unique reasonable solution. Similar to the previous blind deconvolution methods [4, 15], we normalize the estimated PSF and consider the non-negativity of image intensity during the iterative process. Namely, we utilize
The whole algorithm for solving the proposed model (2.2) is summarized in Algorithm 3.1.
Step 1. Input and initialization:
Step 2. Compute kn+1 by (3.5).
Step 3. Impose constraints on kn+1 by (3.15) and (3.17).
Step 4. Compute un+1 by (3.7).
Step 5. Impose constraint on un+1 by (3.16).
Step 6. Compute
Step 7. Compute
Step 8. Compute
Step 9. Update
Step 10. Stop if the stopping criterion is satisfied. Otherwise, let n : = n + 1 and go to Step 2.

Test images.

Blur kernels for image degradation: (a) linear motion, (b) Gaussian, (c) out-of-focus, (d) customized.
Selected parameter sets for different models under different blur types and noise levels
In this section, comprehensive experimental results for our proposed method are presented to illustrate the effectiveness and superiority over other related blind deconvolution methods: TV-BDSB [15] and EP-L0RG [35]. For simplicity of presentation, we denote the newly developed method as AWTV-BD. All the numerical experiments are conducted in MATLAB environment on a PC with 3.20GHz AMD Ryzen 7 5800H CPU and 16GB RAM. We use the peak signal-to-noise ratio (PSNR), the improved signal-to-noise ratio (ISNR) and the structural similarity (SSIM) as quantitative indexes to evaluate the quality of restoration results. Their definitions are expressed as
Numerical results for blind deconvolution under Gaussian noise with standard variance of 0.001
Numerical results for customized blur under Gaussian noise with standard variance of 0.001

Blind deconvolution results of the Satellite image obtained by different methods under noise level 0.001.

Blind deconvolution results of the Lena image obtained by different methods under noise level 0.001.
In experiments, there are four test images including Satellite (256 × 256), Peppers (256 × 256), Lena (256 × 256) and House (256 × 256) [7, 33], as shown in Fig 1. The intensities of these images are first scaled into the range between 0 and 1. To obtain degraded images, the Matlab command “fspecial" is used to generate three different blur kernels, including linear motion blur (length of 8 pixels and direction of 45 degree), Gaussian blur (size of 7 × 7 pixels, variance of 25) and out-of-focus blur (radius of 3 pixels). The last customized blur is a simulated irregular motion blur, which is manually set to ([0 1 1 1 1 0 0; 1 1 0 0 0 1 0; 0 0 0 0 1 1 0; 0 0 1 1 0 0 0; 0 1 1 0 0 1 1; 0 1 0 0 1 1 0; 0 1 1 1 1 0 0]/22). The blur kernels we used are presented in Fig. 2.
Before the proposed algorithm is executed, we need to set seven parameters manually, i.e., α, β, ι, δ, r1, r2 and r3. α controls the weight of the image regularization term. If α is too large, it will cause the restored image too smooth and lose some details of the image. On the contrary, if α is too small, the noise will not be removed well. In short, the selection of α is usually related to the degree of noise pollution of the image. When the initial clean image is polluted by high noise, the larger α should be selected. In experiments, we set α ∈ [0.000001, 0.01]. β is a regularization parameter for blur kernel k, which affects the spread of the PSF. If β is excess big, the estimated PSF will have more diffuse. Conversely, the support domain of the estimated PSF will not be fully expanded. We set β ∈ [0.0001, 10] in experiments. To obtain satisfactory deconvolution results, it is crucial to choose appropriate values for these two parameters.
ι is a tuning parameter used to control the local adaptivity, and δ is the standard deviation. In the process of parameter adjustment, we observe that the selection of ι will be affected by the test images and blur types. Therefore, we need to adjust according to specific experiment. In addition, we can find that δ has little influence on the PSNR value of the restored image.
r1, r2 and r3 are three penalty parameters. According to equations (3.10), (3.12) and (3.14), they control the update of
Blind deconvolution
Numerical results for blind deconvolution under Gaussian noise with standard variance of 0.01
Numerical results for blind deconvolution under Gaussian noise with standard variance of 0.01
The computational time of each blind deconvolution method under Gaussian blur and noise level 0.01

Blind deconvolution results of the Peppers image under noise level 0.01.

Blind deconvolution results of the House image under noise level 0.01.
Firstly, we test the blind deconvolution performance of AWTV-BD at low noise level. The blurred images generated by adding four different blur kernels in Fig. 2 to the test images are polluted by additive white Gaussian noise with standard variance of 0.001. For blind image deconvolution, we require to set the support of PSF. As the true support of PSF is unknown, the size of the initial support of PSF should be larger than that of the true one, but smaller than the size of the image. In experiments, we set the size of PSF support as 101×101 pixels. The related numerical results of three methods for linear motion blur, Gaussian blur and out-of-focus blur are listed in Table 1, while displaying the results for customized blur in Table 2. From these two tables, we can see that our proposed AWTV-BD obtains the highest PSNR, ISNR and SSIM values in most cases, compared with other methods. Especially for linear motion blur, Gaussian blur and out-of-focus blur, the PSNR and ISNR values of AWTV-BD are always the highest, which can be easily seen from Table 1.

Non-blind deconvolution results of different methods on two images (“Peppers", “Lena") with noise level 0.01: the restored images (row 1, 3) and the locally enlarged images (row 2, 4).
To visually evaluate the results of different methods, the restored Satellite and Lena images with the selected locally enlarged images placing in the lower left corner are shown in Fig. 3 and Fig. 4, respectively. Furthermore, the restored blur kernels are placed in the lower right corner of the images.
From the enlarged images of Fig 3, we observe that the images restored by EP-L0RG lose a lot of details and it can not get a satisfactory visual effect. Its restored blur kernels are great difference from the original one. Fortunately, the blur kernel recovered by AWTV-BD is the closest to the original blur kernel compared with the other two methods. From the enlarged images of Fig. 4, it is easy to see that the images restored by EP-L0RG are too smooth, which result in the loss of some details of the original image. In addition, some images obtained by TV-BDSB have obvious staircase effects in flat regions. Owing to the use of the adaptive weighted matrix, our AWTV-BD can better preserve local structures of image, especially for the face and the mouth. From the restored blur kernels, we find that AWTV-BD also produces the closest to the original blur kernel.
Further, we explore higher noise level case, say standard variance of 0.01 for the Peppers image and the House image. The blur kernels used for image blur are the same as in previous experiments. The corresponding numerical results are arranged in Table 3. Table 3 shows that AWTV-BD still achieves the highest PSNR, ISNR and SSIM values in most cases. To assess the recovery results from the visual effect, we show the restored Peppers and House images with the recovery blur kernels placing in the lower right corner in Fig. 5 and Fig. 6, respectively. From Fig. 5, we can observe that the restored images obtained by EP-L0RG are also too smooth under high noise level, although for the customized blur, its restored blur kernel is closer to the original one. From Fig. 6, the over-smoothing effect generated by EP-L0RG is very obvious. Our proposed AWTV-BD can overcome this problem. Besides, these two figures show that AWTV-BD produces more natural restoration images compared with other two methods. In addition, Table 6 shows the computational time for restoring the Peppers image and the House image with two sizes. We observe that the CPU time grows fast when the image size becomes larger. Moreover, our proposed method is faster than the EP-L0RG.
Comparison of non-blind deconvolution methods under Gaussian noise with standard deviation of 0.01
In this paper, we proposed an adaptive weighted total variation regularization for blind image deconvolution. To better describe the local structures of image, we combined an adaptive weighted matrix with the gradient operator. Furthermore, an efficient ADMM was applied to solve the proposed model. Our proposed method can recover multiple blur types. Experiments were conducted to demonstrate the superior performance of the proposed method, especially for images degraded by linear motion blur, Gaussian blur and out-of-focus blur. For some images generated by using complex non-linear motion blur kernel such as the customized blur, our blind deconvolution method sometimes cannot get satisfactory results. This is also the part that we need to improve in the future work.
In the future, we would like to explore some acceleration techniques to further reduce the computational cost. Besides, we will focus on extending the adaptive strategy to other image processing problems, e.g., image segmentation, hyperspectral image super-resolution and hyperspectral unmixing.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Competing Interests
The authors declare that they have no competing interests.
Conflict of Interest
The authors declare that they have no conflict of interest.
Acknowledgments
This work was partly supported by the Natural Science Foundation of Jiangxi Province (20192BAB211005), the Science Foundation for Post Doctorate of China (2020M672484), and the NNSF of China (61865012).
