Improved color image defogging algorithm based on dark channel prior

Abstract

An improved algorithm of image defogging was proposed based on dark channel prior in order to solve the low efficiency and color distortion in the bright area using original algorithm. If the image contains large areas of bright areas such as sky, white clouds or partial white objects and water surface, we can know that the dark channel prior theory does not apply to these areas. Firstly, it is necessary to clear the bright area of the image. According to principle that he adjacent pixel attributes have similarity, the image transmittance of the local region also has similarity, Block function is Consruted. Applied the dark channel prior, judging whether each block includes a bright area by the absolute value of difference of atmospheric intensity and dark channel, the dark and bright areas of the image are obtained. So the estimation value of the adaptive space transmittance are also obtained. Secondly, the transmittance of bright region is small and it causes deviation, so the enhancement formula is used to modify it dynamically. In order to preserve the edge details after image restoration, for bright areas, using texture function to optimize transmittance independently, for others, using gradient and texture function together. Finally, it restored the fog-free image applying the atmospheric scattering model. The experimental results showed that the restored image had obvious details and rich color and fast processing speed through the proposed algorithm. The algorithm can also be applied to outdoor visual systems, such as video surveillance, intelligent traffic and so on.

Keywords

Dark channel prior (DCP)image defogging gradient information texture information transmittance atmospheric scattering model

1 Introduction

With the more and more foggy weather, the digital images obtained from outdoor often have the phenomenon of low contrast, blurred, and color distortion, which has brought a great obstacle to the Image recognition processing. For example, vehicle identification in fog traffic management, detection and identification of fog traffic signs and so on. Even air transportation by making the outdoor surveillance system abnormal [1, 2], it also causes some monitoring systems based on feature extraction ineffective, such as production monitoring system [3]. Recently, we are studying the identification of the vehicle involved in the fog video, it causes some monitoring systems based on feature extraction ineffective. Therefore, image defogging is particularly important. At present, the technology of image defogging is divided into two categories: image enhancement and image restoration method. Image enhancement method can improve the visual effect of images by mainly increasing the contrast of image and highlighting the details of image. Its essence does not achieve the purpose of fog removal. Therefore, Like the Retinex method [4 –6], such methods usually cause image distortion. By analyzing the imaging principle of fog image with considering the atmosphere, the image restoration method establishes the fog image model to solve the problem of image defogging. Therefore, the current hot technology in the field of image defogging is the image restoration method because of a clear idea and imaging principle, especially Restored image with natural color and rich details.

In recent years, the image priori information has been applied to the restoration of foggy images. By constructing the atmospheric scattering model and user interaction for the necessary image information, Narasimhan et al. restore the foggy image [7, 8]. The algorithm can only restore the image globally. Based on the method of independent component analysis, Fattal estimates the reflectivity of the scene and then deduces the foggy image [9]. The method is based on mathematical statistics with many parameters but not easy to adjust. This method has a good effect on the mist foggy image processing, and there will be serious distortion in the heavy. Tarel and Hautiere think that the contrast of the image without fog is stronger than the foggy image, so the image is restored by maximizing the local contrast [10]. Such method can obviously enhance the contrast, but it is easy to cause the block effect and the image serious distortion. In 2009, He et al. put forward a priori knowledge of outdoor natural images from a new perspective [11]. They concluded that there are some low luminance pixels in some color channel for most outdoor fog free images through the statistical experiments on the large natural images. Based on the experimental results, He et al. proposed a dark channel prior model [11, 12]. So the image restoration describes rough estimating transmittance by dark channel prior theory and refining it through the soft matting or guide filter, then getting the restored image with help of the atmospheric scattering model. At present, it is the restored image with the vivid color and more details that the method is the most effective. But this method is invalid for the bright areas, such as sky and clouds, which are close to the atmospheric light value, there is a serious distortion. Although using the soft matting or guide filtering algorithm to improve transmittance, there exists low efficiency and serious distortion. Other methods are proposed and also have an outstanding effect on defogging. Such Yang et al. and Chen et al proposed a good methods with some shortcoming [3 , 14]. Zhu et al. exploited the characteristics of brightness and saturation of the pixels in hazy image, proposing a new linear model and learning the parameters of the model by using a supervised learning method to estimate the depth information [15]. Their experimental results indicated that dehazing effects are good and efficient, but the insufficient estimation of transmission map is still an unsolved problem. Li et al. proposed a novel improved image clearness method to resolve color shift in light color areas because of inaccurate estimation of transmittance [3]. It uses 33 fixed region to calculate transmission map.Through some experiments, it gets some restoration module parameters to correct the transmission map. The experiment results show that the method can resolve the color shift in light color areas effectively.

In summary, Based on DCP, an improved color image defogging algorithm is proposed, which correct transmittance in bright color area. The remaining parts of this paper are arranged as follows: in Section 2, atmospheric scattering model and the original defogging algorithm based on DCP theory will be described in detail; in Section 3 It constructs an improved algorithm. This algorithm includes image block processing and the method of obtaining transmittance used block by block instead of pixel by pixel, and optimizing the transmittance with the gradient and texture information structure function of image blocks. Then it illustrates the validity of the improved algorithm by the comparison of the experiment results between our defogging algorithm and other algorithms. And then, Section 4 summarizes our work and discusses the future direction [16, 17].

2 Dark channel prior defogging theory

2.1 Atmospheric scattering model

In order to illustrate the imaging principle of foggy images, the atmospheric scattering model is used in the field of computer vision as shown in Equation (1) [10]. $I (x) = J (x) t (x) + A (1 - t (x))$ (1)

In Equation (1), I (x) is foggy image, and J (x) is clear image, and A is atmospheric intensity, and t (x) is medium transmission rate (called transmittance). So if the parameters A and t (x) are known, J (x) is restored from I (x) which is called image restoration. The key lies in the solution of the parameter A and t (x).

2.2 Dark channel prior defogging model

Some low luminance pixels in some color channel for most outdoor fog free images are very low to 0 through the statistical experiments on the large natural images [3, 11, 12]. The empirical regularities are called the dark channel prior. Its mathematical model is as shown in Equation (2). $\overset{dark}{J} (x) = min_{c \in (r, g, b)} (min_{y \in Ω (x)} (J^{c} (y))) \to 0$ (2)

In Equation (2), Ω (x) is the square neighborhood taking x as the center, J^c is one of tricolor channel, and $\overset{dark}{J} (x)$ is dark channel prior in the image neighborhood.

Research shows that, except for bright areas such as sky, white clouds, etc., the value of $\overset{dark}{J}$ is always very low and tends to 0. However, the intensity of the dark channel prior will increase for fogging image and determine the value of the transmittance.

2.3 Fog removal process by dark channel prior

The first step: Supposed that the atmospheric light is known and in a local region Constant transmittance exists. The Equation (1) is transformed in the C channel and taken the minimum operator as shown in Equation (3). $min_{y \in Ω (x)} (\frac{I^{c} (y)}{A^{c}}) = t (x) min_{y \in Ω (x)} (\frac{J^{c} (y)}{A^{c}}) + (1 - t (x))$ (3)

Then select the minimum in three color channels as shown in Equation (4). $\begin{matrix} min_{c} (min_{y \in Ω (x)} (\frac{I^{c} (y)}{A^{c}})) = t (x) \\ min_{c} (min_{y \in Ω (x)} (\frac{J^{c} (y)}{A^{c}})) + (1 - t (x)) \end{matrix}$ (4)

The second step: plugging the Equation (2) into Equation (4) could estimate the transmittance t′ where $\overset{c}{A}$ is positive number. The formula for solving t′ is as shown in Equation (5). $t^{'} (x) = 1 - min_{c} (min_{y \in Ω (x)} (\frac{I^{c} (y)}{A^{c}}))$ (5)

If completely removed fog of image, the image looks unnatural and lacks of scene distance. So add a constant w in Equation (5) for properly retaining the mist that covers a distant scene where the value if w (ω (0 < ω ⩽ 1)) is 0.9–0.96. So the new transmittance is as shown in Equation (6). $t^{'} (x) = 1 - w * min_{c} (min_{y \in Ω (x)} (\frac{I^{c} (y)}{A^{c}}))$ (6)

The third step: the estimation of atmospheric light is defined the maximum value of I (x) corresponding to in the front of greatest strength 0.1% pixels in $\overset{dark}{J} (x)$ [12–15 , 18].

The fourth step: J (x) is obtained where Equation (1), Equation (6), A, and I (x) are known as shown in Equation (7). $J (x) = \frac{I (x) - A}{max (t^{'} (x), 0.1)} + A$ (7)

Where, the value of Lower limit of transmittance is 0.1 for avoiding of bringing a lot of noise when the transmittance is close to 0.

Through the description of above process, the value of Equation (6) is rough. He proposed using the soft matting or guide filtering method to improving the transmittance and achieved a good effect.

3 Improved dark channel prior defogging algorithm

Through the previous analysis, if the image contains large areas of bright areas such as sky, white clouds or partial white objects and water surface, we can know that the dark channel prior theory does not apply to these areas. Because the pixel value of areas are very large and we couldn’t find the dark channel color point with the pixel value near to 0 either under a foggy or clear condition. In order to make the dark channel theory more robust, it is necessary to clear the bright area of the image first. Secondly block processing of images are important, because the image transmittance of the local region is similar, like the similarity of the adjacent pixel attributes. Therefore, the method of obtaining transmittance used block by block instead of pixel by pixel reduces the time complexity of the algorithm. Finally instead of soft matting or filter, using the image gradient and texture information to optimize the transmittance, it can achieve the effect of keeping the edge and smoothing the noise.

3.1 Image block processing

Based on the above analysis, the image is divided into blocks first. If the more selected the local area larger, the more satisfied the dark channel prior theory. But because of the dark areas expanding, the edge spillover is serious. If the selected local area is smaller, it may lead to errors that do not exist in dark pixels. So a compromise between the two, use Equation (8) to block the image. $block = max (w, l) / K + 1$ (8)

Where w and l denote the width and length of the image, General value of K is from 40 to 60.

3.2 The judgment and processing of bright area

Due to the dark channel prior theory does not apply to the bright region, or bright region of the dark color value is very high closed to the atmospheric intensity value. Therefore, the transmittance of the bright area is small and there is a big deviation. Such caused color distortion after image restoration. Modify the transmittance of the bright area is necessary. Through a large number of experiments, if |A - J^dark (x) | < 80 is satisfied, this area is a bright area and need to enhancement the transmittance as shown in Equation (9). $t^{'} (x) = \frac{80}{| A - J^{dark} |} (1 - w * min_{c} (min_{y \in Ω (x)} (\frac{I^{c} (y)}{A^{c}})))$ (9)

3.3 Optimizing transmittance by constructing Gradient-Texture function

The transmittance is obtained for the block image, and optimized by using the constructed function. The gradient information of the image can highlight the image edge in image processing [19, 20]. So construct the gradient smooth conduction function as shown in Equation (10). $β = \frac{1}{1 + \sqrt{\frac{| \nabla f (x, y) |}{S}}}$ (10)

In Equation (10), ∇f (x, y) denotes gradient of image, S denotes Gradient nonlinear empirical coefficient, and I_mean = mean2 (| ∇ f (x, y) |), $S = \frac{100}{e^{\frac{I_{mean}}{10}}}$ . Because of ∫β > 0, β must be a smooth function.

Like the gradient smooth conduction function, constructing texture function better keeps the details of the image as shown in Equation (11). $α = \frac{1}{1 + \sqrt{\frac{τ f}{h}}}$ (11)

In Equation (11), h denotes texture Nonlinear empirical coefficient, h₁ denotes convolution mask, τ denotes convolution image, The value of τf and h is obtained by the Equation (12) and Equation (13) and Equation (14). $h_{1} = [\begin{matrix} - 1 & - 1 & - 1 \\ - 1 & 9 & - 1 \\ - 1 & - 1 & - 1 \end{matrix}], τ = f * h_{1}$ (12) $τ f = \frac{τ - min (min (τ))}{max (max (τ)) - min (min (τ))}$ (13) $τ f_{mean} = mean 2 (τ f), h = \frac{1}{10 \times e^{10 τ f_{mean}}}$ (14)

So, for bright areas, using texture function to optimize transmittance independently, for others, using gradient and texture function together as shown in Equation (15). $t^{″} = t^{'} \times α \times β$ (15)

3.4 Experimental results and analysis

In order to verify the validity and reliability of the algorithm, the experiment is compared with the dark channel color prior and algorithm by the paper processed. The experimental image comes from Baidu website or a camera image by myself with around 600*400 size. And the MATLAB R2013a tool is used for the experimental analysis. Figure 1 shows the comparison of defogging effect between the two methods and all get good results for mist image. Figure 2 shows the comparison of defogging effect between the two methods for dense fog image.

Fig. 1

The comparison of defogging effect between two methods.

Fig. 2

Comparison of defogging effect between the two methods for dense fog image.

Through the comparison of the above processing results, only from the visual effect, we can reach a conclusion that using dark channel prior algorithm achieved very good results. But from the details, as seen in Fig. 1(c) and Fig. 1e, especially the middle of two rows of trees, there are some fogs in the sky in Fig. 1(c), but the sky is clear and bright in Fig. 1e. The leaf part of the upper right of image, there are some difference. One covered a layer of mist in the leaves, the other shows green leaves. From the transmittance image in Fig. 1b and Fig. 1d, one image has a clear texture and the other is blurred. For dense fog image such as Fig. 2(a), Not only is the good defogging effect, but also the color is rich. Especially the sky and the tree of the image, in Fig. 2(c), the sky is blue and the color natural transition between the sky and ground. The whole image color is rich and no distortion. The tree of image in Fig. 2(c) shows clear texture but no lack of sense of distant view [21, 22].

Next, three objective technical indexes used to evaluate results of processing in the experiment as shown Table 1 [20, 23].

Table 1

Quality evaluation index of two algorithms

	Original image	The algorithm in the reference [8]	The algorithm in this paper
PSNR(image1)	15.145	17.032	18.689
K_blur(image1)	–	0.256	0.2885
Q(image1)	–	0.0125	0.0194
PSNR(image2)	10.123	11.332	12.689
K_blur(image2)	–	0.146	0.1485
Q(image2)	–	0.0218	0.0384

PSNR (peak signal to noise ratio) and fuzzy coefficient (Kblur) and quality index (Q) included. Through the three indexes comparison of images, as for as PSNR value of restoration image, we can see that The PSNR value of the latter is higher than that of the former and the PSNR value of two algorithms are higher than that of original image. As for as K_blur value, we found that The K_blur value of the latter is lower than that of the former because introducing a lot of noise in the processing of modifying transmittance could enhance some edge energy resulting get the small value. As for as Q value, the higher value has been obtained in the latter algorithm than the former. From the above objective index, the algorithm of this paper has achieved good results.

4 Conclusion

In this paper, an improved color image defogging algorithm is proposed based on dark channel prior. Because of dark channel prior theory unsuited for the bright area, it needs differential progressing. Using block ideas instead of pixel by pixel has higher efficiency. First it judged the bright area according to the empirical value of the atmospheric intensity and the dark channel prior difference value. By using the image texture, it constructed a smooth function to modify the transmittance of the bright area. By using the image gradient and texture, it constructed a smooth function to modify the transmittance of the non-bright area. Different smoothing functions used in different parts not only reduce the complexity of the algorithm, but also effectively overcome the shortcomings of image distortion in bright area in the existing algorithm. The experiment shows that the algorithm in this paper has a fast processing speed, at the same time, after the restoration, it has highlight detail and rich color. The algorithm can also be applied to outdoor visual systems, such as video surveillance, intelligent traffic and so on.

On the other hand, the image segmentation based on experience without considering the whole image situation. Such as image contains large range of white objects which small image block is not good. Maybe the image blocks are automatically divided according to the statistical attributes of the image pixels. It would be worth studying in the future.

Data availability statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of interest

It is declared by the authors that this article is free of conflict of interest.

Footnotes

Acknowledgments

This work is supported by Key Researched Development Program of Shaanxi (Program: No.2021NY-211).

References

Huang

, Fang

, Zhao

et al., An improved image clearness algorithm based on dark channel prior. In Proceedings of the 33rd Chinese Control Conference, (2014), 7350–7355.

Huang

D.R.

, Fang

and Zhao

, An improved defogging algorithm combined with edge detection, Journal of Shanghai Jiaotong University 49(6) (2015), 861–867.

A.X.

, Fang

, Mi

An improved defogging algorithm based on dark color theory combined with selfadaptive threshold mechanism, Journal of Control Science and Engineering (2018), 1–11.

and Zhu

Q.S.

, The latest research progress of image dehazing, Acta Automatica Sinica 41(2) (2015), 221–239.

Kimmel

, Elad

, Shaked

et al., A variation framework for Retinex, Computer Vision 52(1) (2003), 7–23.

Rahman

, Jobson

D.J.

and Woodel

G.A.

, Retinex processing for automatic image enhancement, Journal of Electronic Imaging 13(1) (2004), 100–110.

Narasimhan

S.G.

and Nayar

S.K.

, Vision and atmosphere, International Journal of Computer Vision 48(3) (2002), 233–254.

Narasimhan

S.G.

and Nayar

S.K.

, Contrast restoration of weather degraded image, IEEE Transactions on Pattern Analysis and Machine Intelligence 25(6) (2003), 713–724.

Fattal

, Single image dehazing, ACM Transactions on Graphics 27(3) (2008), 1–9.

10.

Tarel

J.P.

, Hautiere

Fast visibility restoration from a single color or gray level image, Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Piscataway, NJ: IEEE, (2009), 2201–2208.

11.

K.M.

, Sun

and Tang

X.O.

, Guided image filtering, IEEE Transactions on Pattern Analysis and Machine Intelligence 35(6) (2013), 1397–1409.

12.

Yang

, Bian

, Feng

et al., Optimized design of 1fast single image dehazing algorithm. in Proceedings of the 14th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), (2017), 171–174.

13.

Chen

, Li

, Deng

An adaptive image dehazing algorithm based on dark channel prior, IEEE Control and Decision Conference (2017), 7472–7477.

14.

Zhu

, Mai

and Shao

, A fast single image haze removal algorithm using color attenuation prior, IEEE Transactions on Image Processing 24(11) (2015), 3522–3533.

15.

Zhou

, Wang

and Wan

, Ore image classification based on improved CNN, Computers and Electrical Engineering 99 (2022), 1.

16.

Wang

, Chen

, Yuan

Simple low-light image enhancement based on Weber–Fechner law in logarithmic.

17.

Wang

G.Y.

, Ren

G.H.

, Jiang

L.H.

et al., Single image dehazing algorithm based on sky region segmentation, Information Technology Journal 12(6) (2013), 1168–1175.

18.

Mahajan

and Pandit

A.K.

, Image segmentation and optimization techniques: A short overview, Medicon Engineering Themes 2(2) (2022), 47–49.

19.

Mahajan

, Abualigah

, Pandit

A.K.

et al., Fusion of modernmeta-heuristic optimization methods using arithmetic optimizationalgorithm for global optimization tasks, Soft Computing 26 (2022), 6749–6763.

20.

Yin

, Wang

, Huang

et al., Haze grading using the convolutional neural networks, Atmosphere 13(4) (2022), 522.

21.

, Zheng

, Chen

et al., Joint embedding VQA model based on dynamic word vector,e, PeerJ Computer Science 7 (2021), 353.

22.

Huang

W.H.

, Chen

R.L.

and Zhang

J.M.

, Improvement and implementation of objective digital video quality measurement, Journal of Beijing University of Posts and Telecommunications 28(4) (2005), 87–90.