A new fractional-order variational approach for joint de-hazing and segmentation of images

Abstract

Segmenting outdoor images in the presence of haze, fog or smog (which fades the colors and diminishes the contrast of the observed objects) has been a challenging task in image processing with several important applications. In this paper, we propose a new fractional-order variational model that will be able to de-haze and segment a given image simultaneously. The proposed method incorporates the atmospheric veil estimation based on the dark channel prior (DCP). This transmission map can reduce significantly the edge artifacts and enhance estimation precision in the resulting image. The transmission map is then changed over to the high-quality depth map, with which the new fractional-order variational model can be framed to look for the haze free segmenting image for both grey and color outdoor images. An explicit gradient descent scheme is employed to find efficiently the minimizer of the proposed energy functional. Experimental tests on real world scenes show that the proposed method can jointly de-haze and segment hazy or foggy images effectively and efficiently.

Keywords

Foggy or hazy images fractional-order total variation image de-hazing image segmentation inhomogeneous intensity object detection

1 Introduction

Because of the presence of an atmosphere, the light reflected from a scenario is constantly dissipated before it arrives the camera focal point, and the light received by any camera focal point is normally mixed with the air light. This leads to inescapable degradation of image such as increased noise, deterioration of visual quality (visibility), low contrast, and loss of color constancy. This sort of contamination is especially severe when the climate conditions are poor, that is when the pressurized canned products (such as fog, haze, rain, fumes or dust) are available. For example, as a common climate phenomenon, haze or fog may deliver an albedo impact, which prompts ambiguity, which has somewhat adverse consequences on extraction and comprehension of contents from the images and makes numerous computer vision tasks more difficult such as image segmentation, detection, object tracking and stereo estimation, etc. In this manner, accurate estimation and removal of haze or fog from images becomes an important component and a challenging task because of its ill-posed nature in a post-processing step and needed urgently in real world applications. Recently, haze or fog removal from natural images has pulled in a lot of consideration in imaging science. The upsides of such operations are clear. Firstly, the haze-free images are more appealing visually, meaningful and realistic. Second, the de-haze or defog images are increasingly appropriate for some significant use such as image segmentation, image fusion and feature extraction, and so forth.

During the most recent decade, a few studies have analyzed restoration of image visibility in hazy climate conditions. Generally speaking, these investigations can be partitioned into two significant categories: those assessing (a) multiple-image de-hazing and (b) single image de-hazing. Multiple image based restoration methods utilize either multiple images taken in a similar scene or pre-existing geo-referenced digital terrain and in depth based models an estimate 3-D geometrical model of the scene is required to evaluate the density (thickness) of the atmospheric veil. All these can deliver great outcomes acquiring 3-D geometrical data and multiple images can be an issue in natural world haze removal applications [1] and especially in [2].

Recently, single image restoration based de-hazing methods have been developed to obtain maximum application independence. As only one image is used for input data, usually these methods utilize either prior strategy to estimate the density of the atmospheric veil (transmission map) and reduce the amount of haze or contrast modification to expel haze formation directly from images contaminated with haze. Xu et al. [3] built up their image visibility restoration approach by applying contrast limited adaptive histogram equalization to redistribute histograms. They improved the contrast of lower visibility segment of foggy or hazy images. The methodology utilized by Ancuti and Ancuti [4], upgrades the visibility of hazy images directly by employing a multi-scale fusion method that joins two hazy images determined by using the shades of gray color constancy approach with a contrast improving method containing luminance, chromatic and saliency weight maps into a haze-free image. Tan’s [5] approach sees the approximation of the atmospheric veil as an ill-posed problem and incorporates a cost function for approximating the atmospheric veil in the Markov random field framework, in which the solution can be obtained by maximizing the local contrast of the haze free image.

Through the un-correlation principle, Fattal’s [6] method estimates the optical transmission map which asserts that scene transmission and surface shading are statistically uncorrelated locally; it utilizes an optical transmission map also to obtain de-hazed image. Although, it fails to work with those images which contain heavy haze or fog. Tarel et al. [7], utilized a prior strategy, adopts hybrid filters based on two constraints namely: (a) it is lower than the minimum of RGB color parts and (b) the atmospheric veil is pure white, to approximate the atmospheric veil. However, images recovered with this approach appear to be un-natural with loss of brightness, particularly in sky regions. He et al. [8], developed a dark channel prior (DCP) based on the assumption that there is at least one color channel with very low intensity mostly in non-sky and free of haze scenes. Based on the physical properties of air-light, Cho et al. [9] introduced a model for the estimation and refinement of the air-light map in order to restore a fog-free image. By taking fractional derivative on both sides of the atmospheric physics scattering model, Wendan et al. [10] established a new space fractional-order PDE based model for haze removal (having the spatial global correlation and good amplitude frequency characteristics of fractional-order differential). The advantage of the fractional order differential is that the proposed algorithm exhibits more detailed information and keeps more texture details of the smoothing area comparing to integer-order partial differential equations based image enhancement algorithms. For more details on the subject, the interested reader is referred to [11]. Next, we provide a review of the basic hazy image formation model and dark channel prior.

1.1 Hazy image formation model

The occurrence ray of light is absorbed and dissipated from surfaces of turbid media (for instance, particulate matter) present in the atmosphere during poor climate conditions before it arrives the camera. This marvel involves the deflection of light from its original course of spread. Under this phenomenon, Koschmieder [2] introduced the haze effect model to demonstrate further, the formation of a hazy or foggy image pointed out by [12] through Fig. 1, can be formulated mathematically as follows

$I (x) = T (x) J (x) + I_{\infty} (1 - T (x)) .$ (1) Our mission is to restore J (x) from I (x), where I (x) is the observed hazy or foggy luminance, J (x) is the scene or surface radiance, I_∞ is the global atmospheric light which is supposed to be globally constant over the whole image and x represents the image coordinates. T (x) = e^-βd(x) is the un-scattered projected light (also called transmission represent the portion of the light arrives to the camera without distortion) with degradation coefficient β and scene depth d (x) which describes the distance of the given object with intrinsic luminance and is always non-negative. In mostly clear weather, I ≈ J being β ≈ 0. Note, β (non-negligible for hazy or foggy images) is known as the medium extinction coefficient and set it usually as 1. Since T varies with regard to depth of scene or space. Therefore, without any prior knowledge, direct estimation of T from I is useless. In [6, 13], the highest intensity of the input image is set to be I_∞. In this work, we tackle this problem with the application of a dark channel prior [14], which is explained in next section.

Fig. 1

Hazy image formation model.

1.2 Dark Channel Prior (DCP)

The dark channel prior introduced by He et al. [14] is based on the assumption that there exist some dark pixels whose intensities are closed to zero for at least one color channel within an image patch of the non-sky region of the haze-free outdoor images. For an image J, the dark channel J^dark can be defined as follows $\begin{matrix} J^{dark} (x) = min_{c \in {r, g, b}} (min_{y \in Ω^{*} (x)} J^{c} (y)), \end{matrix}$ (2) where J^c is a single band (color) image of J (x), c ∈ (r, g, b) is a color channel, and Ω^∗ (x) is a local patch centered at pixel x. The approximate value J^dark (x) ≈0 for each pixel of the dark channel is called the dark channel prior (DCP). According to the DCP approximation, $\tilde{T} (x)$ can be represented as $\tilde{T} (x) = 1 - min_{c \in {r, g, b}} (min_{y \in Ω^{*} (x)} \frac{I^{c} (y)}{I_{\infty}^{c}}) .$ (3) Additionally, the value of I_∞ can be picked as the pixels with the highest intensity level in I among the pixels which have 0.1 percent brightness in J^dark (x). Note, the color of the sky is nearly close to I_∞ in hazy images and if the fog or haze removed completely from the scene, the image may look un-natural [14]. we have $min_{c \in {r, g, b}} (min_{y \in Ω^{*} (x)} \frac{I^{c} (y)}{I_{\infty}^{c}}) \approx 1 and \tilde{T} (x) \approx 0 .$ (4) A constant 0 < ω ≤ 1 is adopted to preserve some depth information of the real scene and is usually adjusted as 0.95 or 0.98. Considering that the transmission is a constant in Ω^∗ (x), and performing the important operation in the local patch and three color channels on the haze scene I. Therefore, an estimated value of T (x) defined by $\tilde{T} (x)$ can be obtained as follows $\tilde{T} (x) = 1 - ω min_{c} (min_{y \in Ω^{*} (x)} \frac{I^{c} (y)}{I_{\infty}^{c}}) .$ (5) After obtaining $\tilde{T} (x)$ , a de-hazed image can be recovered according to (1). Next, we provide a background of image segmentation and related methods.

1.3 Overview of image segmentation and related methods

Image segmentation, refers to the process of finding (locating) boundaries and objects (lines,curves, etc.) in images and is one of significant problems in image processing and computer vision. Recently, active contour models (ACM) [15] have been developed widely and have a few noteworthy advantages over classical image segmentation models. By and large, the current ACM-methods can be characterized into two groups namely edge based models [15 –17] and region based models [18 –22]. The choice of these models relies on results adequacy for every image and issue on each specific segmentation application. The region based active contour models tend to depend on intensity homogeneity. For example, the well-known Chan-Vase (CV) model [21], which has been effectively implemented in binary phase segmentation with the assumption that each image region is homogenous statistically. Vase and Chan broadened their work and introduced the piecewise constant models (PCM). For representing multiple regions, this strategy uses level set functions (multi-phase). Both the above models fail in case of in-homogeneous region. Li et al [23] introduced the region scalable fitting model (RSFM), which employs the image local information as imperatives, can successfully segment objects with intensity homogeneity. Zhang et al. [24] improved and extend the similar strategy to segment images with intensity inhomogeneity, however this method is tedious and time consuming. Zhang et al. [25], developed a local image fitting model (LIFM), which reduces computational complexity over RSFM-model. Song et al. [26], combined Laplace zero crossing function and set forward a regularized gradient flow method, which improves edge detection. Based on Bayes rule, Wang et al. [27] introduced a non-linear weighted item that is able of adaptive evolution, therefore can handle boundary leak very well. Nonetheless, the above strategies do not tackle well the problem of sensitivity to the initial evolution curve.

Indeed, fractional differentiation has been proved to be an effective tool for strengthening the components of medium and high frequency, while in a non-linear manner, reserves the part of very low frequency. In image processing, these features are used widely in preserving weak edges and texture details. For instance, Ren et al. [28] employed fractional differential operator to CV-model and combined a fractional fitting term with it. Tian et al. [29] introduced fractional divergence prior to CV-model. Both strategies enhance the weak edges location capacity and the performance of weak edges segmentation. Moreover, both the above mentioned methods are more efficient to noise. Although, these methods fail to perform well while dealing with those images having intensity inhomogeneity, because they use local information only.

To improve the CV-model, another attempt was made by Wu et al. [30] and introduced a variational based convex image segmentation model, within the interval [-1, 1] with a unique global solution for the corresponding energy functional. Akram et al. [31] replaced the signed pressure force function (SPF) in Zhang et al. [25] model by a local-SPF function improving the performance by stopping the contours at blurred or weak edges, which is the modification of Zhang model for scenes with inhomogeneous intensity but the model was not able to tackle the images with multiple intensity objects. Ali et al. [32] combined the SPF function and the generalized average concept and proposed the image segmentation model. In the segmentation of multi-region images with inhomogeneous intensities, the model shows good performance over Akram et al. [31] model. For more details on the subject, the interested reader is referred to [33, 34].

On the other hand, segmentation of real world scenes in the presence of haze, fog or smog have attracted much attention in image processing recently. To the best of our information, in the literature, there is only one recent published work of Ali et al. [35] for joint de-hazing and segmentation of hazy or foggy images. This variational model (based on estimating the atmospheric veil and level set function) is capable of de-hazing and segmenting a given image simultaneously. Although, it has some disadvantages as well. For instance, the employed integer-order total variational prior favors piecewise constant solution and hence, it fails to preserve textures properly and often leads to blocky effect, halo artifacts and color distortion.

To handle the above issues, fractional calculus has gotten wide consideration in recent years because of its remarkable advantages. The studies show that fractional differential has the properties of upgrading high frequency, reducing low frequency, and preserving intermediate frequency for signal and can protect the texture feature and other fine details of low frequency part in non-linear manner when enhancing the high frequency part. Accordingly, when compared with the integer order differential, fractional differential image enhancement can effectively preserve the texture details of the smoothing area as well as make the image edge more prominent. At present, fractional calculus has been generally applied in image de-noising, image enhancement, image segmentation, edge detection, face recognition and other aspects [36]. In addition, the fractional-order differential operator has a “non-local” behavior because the fractional-order differential depends upon the properties of the entire function at a point and not simply the values in the vicinity of the point [37], which is advantageous to improve the exhibition of texture preservation. It has been demonstrated in [38] that the fractional-order derivative fulfills the lateral inhibition principle of the biological visual framework better than the integer order derivative. In [50], Pu et al. have talked about the kinetic physical meaning of the fractional-order derivative and introduced fractional inspection strategies for texture details of images. For more details, see [42, 43].

The aim of this paper, is to introduce a new fractional-order variational model for jointly de-hazing and segmenting a given hazy or foggy image. It incorporates the atmospheric veil estimation dependent on the dark channel prior. This transmission map can minimize the edge artifacts and enhance estimation precision in the resulting image. The transmission map is then reduced to the depth map, with which the new fractional-order variational model is developed to obtain the haze free segmenting image for both grey and color outdoor images. The joint strategy takes advantage of both fractional-order total variational filter and dark channel prior since it can preserve edges while reducing the blocky and halo artifacts in smooth areas. Moreover, we employed the explicit gradient descent scheme. We report experimental results on both grey and color images which show that the proposed model accomplishes state of the art de-haze and segmented results simultaneously.

The outline of this work is as follows. In Section II, our novel model for addressing the problem is proposed. For solving the highly non-linear Euler Lagrange fractional-order PDEs, explicit gradient descent scheme is applied in Section III. In order to demonstrate the performance of the proposed model, some image segmentation and restoration results of outdoor scenes are given in Section IV. Finally, we give some concluding remarks and future directions in Section V.

2 The proposed model

Fractional-order derivative performs well for avoiding staircase effect, preserving sharp edges, structures with low gradients and textures. This is accepted to be a better approximation than a piecewise constant estimation in smooth regions for a natural scene. Here, a new fractional-order variational model is proposed for simultaneously de-hazing and segmenting a given hazy or foggy image in the presence of homogeneous or inhomogeneous intensity. This model uses clearly advantages of fractional order total variation regularizer (FOTVR) and dark channel prior, which leads to good de-hazing and segmented results and has the upside of better preserving image regions containing textures, colors and other fine features, leads to a more natural scene while reducing the blocky effect in smooth areas. Hence applying this model, better improvement in entropy and Jaccard similarity index (JSI) values is gotten. Let Ω be a bounded open subset of R² with boundary ∂Ω and C : [0, 1] → R² be a parameterized curve. The following fractional-order based variational energy functional is proposed to jointly de-haze and segment a given hazy or foggy image $\begin{matrix} F_{r}^{f} (ϕ, A_{L}, c_{1}, c_{2}) = μ \int_{Ω} δ (ϕ) | \nabla^{α} ϕ | d x d y \\ + ν \int_{Ω} (| \nabla^{α} A_{L} |) d x d y + λ^{2} \int_{Ω} H (W - A_{L}) d x d y \\ + λ_{1} \int_{Ω} {(\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{1})}^{2} H (ϕ) d x d y \\ + λ_{2} \int_{Ω} {(\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{2})}^{2} (1 - H (ϕ)) d x d y, \end{matrix}$ (6) where μ, ν, λ, λ₁ and λ₂ are parameters, α ∈ (0, 1) is the fractional derivative order. H, δ and φ are the Heaviside, one-dimensional Dirac and level set functions, respectively. c₁ and c₂ are the averages of $\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}}$ inside and outside C. In the level set formulation, C ⊂ Ω is represented by the zero level set of a Lipschitz function φ : Ω ⟶ R, such that $\begin{matrix} C = {x \in Ω : φ (x) = 0} \\ inside (C) = {x \in Ω : φ (x) > 0} \\ outside (C) = {x \in Ω : φ (x) < 0} . \end{matrix}$ The Heaviside function and one-dimensional Dirac function are defined by $H (z) = {\begin{matrix} 1 & if z ≥ 0 \\ 0 & otherwise \end{matrix}, δ (z) = \frac{d}{dz} H (z) .$ The following slight version of H and δ denoted by H_ɛ and δ_ɛ is used in (6). $H_{ɛ} (z) = \frac{π + 2 arctan (\frac{z}{ɛ})}{2 π}, δ_{ɛ} (z) = \frac{ɛ}{π (ɛ^{2} + z^{2})} .$ The second and third terms in (6) represent the physical properties of air-light A_L whereas the other terms in (6) are variational segmentation fitting and smoothness terms. The physical properties of air-light demonstrate that A_L > 0 and A_L ≤ W. For each image pixel, W = min (I) is the whiteness or minimal components of an image I. The average values c₁ and c₂ can be formulated as follows $c_{1} (φ) = \frac{\int_{Ω} (\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}}) H (φ) dxdy}{\int_{Ω} H (φ) dxdy},$ (7) $c_{2} (φ) = \frac{\int_{Ω} (\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}}) (1 - H (φ)) dxdy}{\int_{Ω} (1 - H (φ)) dxdy} .$ (8) Due to the fractional order gradient terms in the proposed energy functional (6), the associated fractional-order variation terms can be uniformly re-written as follows $F_{r}^{f} (ϕ) = \int_{Ω} | \nabla^{α} ϕ | d x d y,$ (9) $F_{r}^{f} (A_{L}) = \int_{Ω} | \nabla^{α} A_{L} | d x d y .$ (10) The gradient flow equations corresponding to the fractional-order optimizing problem in (6) can be formulated as follows $\frac{\partial φ}{\partial τ} = δ (φ) [\begin{matrix} μ {div}^{α} (\frac{\nabla^{α} φ}{| \nabla^{α} φ |}) - λ_{1} {(\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{1})}^{2} \\ + λ_{2} {(\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{2})}^{2} \end{matrix}],$ (11) $\begin{matrix} \frac{\partial A_{L}}{\partial τ} = \\ [\begin{matrix} ν {div}^{α} (\frac{\nabla^{α} A_{L}}{| \nabla^{α} A_{L} |}) - λ^{2} δ^{*} (A_{L}) \\ - 2 λ_{1} (\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{1}) (\frac{I - I_{\infty}}{I_{\infty} {(1 - \frac{A_{L}}{I_{\infty}})}^{2}}) H (φ) \\ - 2 λ_{2} (\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{2}) (\frac{I - I_{\infty}}{I_{\infty} {(1 - \frac{A_{L}}{I_{\infty}})}^{2}}) (1 - H (φ)) \end{matrix}], \end{matrix}$ (12) where δ^∗ (A_L) = - δ (W - A_L). The boundary conditions are $D^{α} φ . \vec{n} |_{\partial Ω} = 0$ and $D^{α} A_{L} . \vec{n} |_{\partial Ω} = 0$ , where $\vec{n}$ is the outward unit normal vector of boundary ∂Ω. The fractional divergence operator div^α (·) can be written uniformly as ${div}^{α} (φ) = (- 1)^{α} (D_{x}^{α *} φ + D_{y}^{α *} φ),$ (13) where $D_{x}^{α *}$ and $D_{y}^{α *}$ are the adjoints of $D_{x}^{α}$ and $D_{y}^{α}$ can be defined as $D_{x}^{α *} [Φ (x, y)] = D_{x}^{α} [Φ (- x, y)]$ (14) $D_{y}^{α *} [Φ (x, y)] = D_{y}^{α} [Φ (x, - y)] .$ (15) Equation (12) can be equivalently rewritten as $\frac{\partial A_{L}}{\partial τ} = [\begin{matrix} ν {div}^{α} (\frac{\nabla^{α} A_{L}}{| \nabla^{α} A_{L} |}) - λ^{2} δ^{*} (A_{L}) \\ - 2 (\frac{I - I_{\infty}}{I_{\infty} (1 - \frac{A_{L}}{I_{\infty}})^{2}}) [λ_{1} (\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{1}) H (φ) \\ + λ_{2} (\frac{I - A_{L}}{1 - \frac{A_{L}}{I_{\infty}}} - c_{2}) (1 - H (φ)) \end{matrix}] .$ (16) Next, we present explicit gradient descent method for solving the non-linear fractional-order Euler-Lagrange equations (12).

3 Numerical implementation

In this section, explicit gradient descent method is employed for solution of the fractional-order based non-linear Euler-Lagrange PDEs (11) and (16) arisen from the minimization of functional (6). In the implementation of (11) and (16), the important problem is the discretization of fractional-order gradient operator D^α. Here, the α-order Grünwald-Letnikov definition [35 –37] of fractional differential is used to discretize the fractional-order gradient operator D^α, defined as $\overset{GL}{t_{0}} D_{t}^{α} g (t) = \frac{1}{(Δ x)^{α}} \overset{[(t - t_{0}) / Δ x]}{\sum_{j = 0}} ϖ_{j}^{(α)} g (t - j Δ x) .$ (17) Where t and t₀ represents the upper and lower variants of the derivative respectively, $ϖ_{j}^{(α)} = (- 1)^{j} α j = (- 1)^{j} \frac{α!}{(α - j)! j!}$ are the coefficients of the Taylor series expansion of the corresponding “generating” functions [39], which can be formulated as follows $\begin{matrix} ϖ_{1}^{(α)} (z) & = (1 - z)^{α} \\ ϖ_{2}^{(α)} (z) & = (\frac{3}{2} - 2 z + \frac{1}{2} z^{2})^{α} \end{matrix}$ (18) $\begin{matrix} ϖ_{3}^{(α)} (z) & = (\frac{11}{6} - 3 z + \frac{3}{2} z^{2} - \frac{1}{3} z^{3})^{α} \\ ϖ_{4}^{(α)} (z) & = (\frac{25}{12} - 4 z + 3 z^{2} - \frac{4}{3} z^{3} + \frac{1}{4} z^{4})^{α} . \end{matrix}$ For the sake of simplicity, the four directions fractional-order masks are constructed that is $D_{x}^{α}$ , $D_{y}^{α}$ , $D_{x}^{α *}$ and $D_{y}^{α *}$ in x - axis⁺ and y - axis⁺ directions and x - axis^- and y - axis^- directions respectively. It should be noted that the mask size is in direct proportion to the degree of accuracy. Here, the size is being selected to 5 given in Table 1.

Table 1

Masks in four directions: (a) $D_{x}^{α}$ ; (b) $D_{y}^{α}$ ; (c) $D_{x}^{α *}$ ; (d) $D_{y}^{α *}$

0	0	ϖ ₄	0	0	0	0	0	0	0
0	0	ϖ ₃	0	0	0	0	0	0	0
0	0	ϖ ₂	0	0	ϖ ₀	ϖ ₁	ϖ ₂	ϖ ₃	ϖ ₄
0	0	ϖ ₁	0	0	0	0	0	0	0
(a)					(b)
0	0	ϖ ₀	0	0	0	0	0	0	0
0	0	ϖ ₀	0	0	0	0	0	0	0
0	0	ϖ ₁	0	0	0	0	0	0	0
0	0	ϖ ₂	0	0	ϖ ₄	ϖ ₃	ϖ ₂	ϖ ₁	ϖ ₀
0	0	ϖ ₃	0	0	0	0	0	0	0
0	0	ϖ ₄	0	0	0	0	0	0	0
(c)					(d)

where

$\begin{matrix} ϖ_{0} & = 1 \\ ϖ_{1} & = 1 - α \\ ϖ_{2} & = \frac{3}{2} - 2 α + \frac{1}{2} α^{2} \\ ϖ_{3} & = \frac{11}{6} - 3 α + \frac{3}{2} α^{2} - \frac{1}{3} α^{3} \\ ϖ_{4} & = \frac{25}{12} - 4 α + 3 α^{2} - \frac{4}{3} α^{3} + \frac{1}{4} α^{4} . \end{matrix}$

The remaining terms in (11) and (16) can be easily discretized. The update form of the fractional-order partial differential equations (16) are formulated by $A_{L_{i, j}}^{(n + 1)} = A_{L_{i, j}}^{(n)} + Δ τ \cdot Q_{A_{L_{i, j}}^{(n)}},$ (19) $φ_{i, j}^{(n + 1)} = φ_{i, j}^{(n)} + Δ τ \cdot Q_{φ_{i, j}^{(n)}},$ (20) where $Q_{A_{L_{i, j}}^{(n)}} = [\begin{matrix} ν {div}^{α} (\frac{\nabla^{α} A_{L_{i, j}}^{(n)}}{| \nabla^{α} A_{L_{i, j}}^{(n)} |}) - λ^{2} δ^{*} (A_{L_{i, j}}^{(n)}) \\ - 2 (\frac{I_{i, j} - I_{\infty_{i, j}}}{I_{\infty_{i, j}} (1 - \frac{A_{L_{i, j}}^{(n)}}{I_{\infty_{i, j}}})^{2}}) \times \\ {λ_{1} (\frac{I_{i, j} - A_{L_{i, j}}^{(n)}}{1 - \frac{A_{L_{i, j}}^{(n)}}{I_{\infty_{i, j}}}} - c_{1} (φ_{i, j}^{(n)})) H (φ_{i, j}^{(n)}) \\ + λ_{2} (\frac{I_{i, j} - A_{L_{i, j}}^{(n)}}{1 - \frac{A_{L_{i, j}}^{(n)}}{I_{\infty_{i, j}}}} - c_{2} (φ_{i, j}^{(n)})) (1 - H (φ_{i, j}^{(n)}))} \end{matrix}],$ $Q_{φ_{i, j}^{(n)}} = [\begin{matrix} δ (φ_{i, j}^{(n)}) {μ {div}^{α} (\frac{\nabla^{α} φ_{i, j}^{(n)}}{| \nabla^{α} φ_{i, j}^{(n)} |}) \\ - λ_{1} (\frac{I_{i, j} - A_{L_{i, j}}^{(n + 1)}}{1 - \frac{A_{L_{i, j}}^{(n + 1)}}{I_{\infty_{i, j}}}} - c_{1} (φ_{i, j}^{(n)}))^{2} \\ + λ_{2} (\frac{I_{i, j} - A_{L_{i, j}}^{(n + 1)}}{1 - \frac{A_{L_{i, j}}^{(n + 1)}}{I_{\infty_{i, j}}}} - c_{2} (φ_{i, j}^{(n)}))^{2}} \end{matrix}] .$ There is problem of selecting the ideal step size Δτ which would guarantee quick convergence. Due to the non-linearity of the equations, anyway such an optimal step size is hard to get theoretically and costly computationally. We adopted a trial strategy and noted that a decent step size is in the range of [0.0001, 1] and tolerance (Tol) is in the range of [10^-6, 10^-4] respectively.

To summarize, the following steps can be performed to obtain the most important steps of the level set evolution and air-light map estimation

Initialize φ to its zero-level curve φ₀ and $A_{L}^{0}$ by A_L
₀. $φ^{0} (x, y) = {\begin{matrix} - β^{*} & if (x, y) \in Ω - \partial Ω \\ 0 & if (x, y) \in \partial Ω \\ β^{*} & if (x, y) \in Ω \end{matrix}$ $\begin{matrix} A_{L}^{0} (x, y) = \\ {\begin{matrix} β^{★} \min_{c \in (r, g, b)} I^{c} (x, y) for color image \\ β^{★} I (x, y) for gray - scale image \end{matrix} \end{matrix}$ where β^∗ > 0 and β^★ > 0 are constants, Ω and ∂Ω are image domain and boundary of image domain respectively.

Compute $c_{1} (φ_{i, j}^{(n}))$ and $c_{2} (φ_{i, j}^{(n)})$ using (7).

Compute $φ_{i, j}^{(n + 1)}$ and $A_{L_{i, j}}^{(n + 1)}$ by using (19) and (20).

If $\frac{∥ \hat{φ} - φ ∥^{2}}{∥ φ ∥^{2}} \leq Tol$ ; then stop.

Output $φ_{i, j}^{(n + 1)}$ and $A_{L_{i, j}}^{(n + 1)}$ .

Determine whether the solution is stationary. If not, proceed towards another step (n=n+1).

End procedure

4 Experimental results and analysis

In this section, some numerical tests are presented to demonstrate the effective performance of the proposed model. We analyze and describe the results on some clean and hazy or foggy images. The image de-hazing and segmentation results are compared with some state of the art models namely, SPF [25], GSPF [32], Cho et al. [9], Wu et al. [30] and Ali et al. [35] through various outdoor scenes. All the experiments are performed in MATLAB R2013a in Haier Win8.1 PC, Intel core i3 CPU @ 1.70GHz with 4.00 GB RAM. To obtain optimal results, the parameters are adjusted and tuned according to the thickness of haze or fog in the given image. The proposed model has been tested from the aspects of segmentation performance, de-hazing performance, robustness to initial curve location and its comparison with the recent state of the art methods.

4.1 Image segmentation performance

This section focuses on testing image segmentation performance of the proposed model on grey and color clean images. The results obtained by our model are contrasted with those of the GSPF [31] model as well as the Ali et al. [35] model as shown in Figs. 3. Figure 2 shows grey-scale haze free images and Fig. 3 shows vector-valued haze free images. The first column of the figures show the given images. The second, third and fourth columns of the figures represent GSPF [32], Ali et al. [35] and proposed models segmentation results, respectively. From the experimental results, we notice that our method performs well for images having low contrast, low visibility, weak edges and inhomogeneous intensity and successfully segment the object edges whereas GSPF [32] and Ali et al. [35] models can not segment the object edges completely. GSPF [32] and Ali et al. [35] models segment the extra and undesired region of the images with object edges as shown in Figs. 2(c), 3(g) and 3(o).

Fig. 2

Comparison of Proposed model with GSPF [32] and Ali et al. [35] models in segmenting grey-scale images. The first column shows the given grey-scale images. The second, third and forth columns show the segmentation results of GSPF [32], Ali et al. [35] and proposed models, respectively.

Fig. 3

Comparison of Proposed model with GSPF [32] and Ali et al. [35] models in segmenting vector valued images. The first column shows the given vector valued images. The second, third and fourth columns show the segmentation results of GSPF [32], Ali et al. [35] and proposed models, respectively.

Fig. 4

The de-fogging or de-hazing performance of the proposed model in comparison with Cho et al. [9] and Ali et al. [35] models. The first column shows the given hazy or foggy images. The second, third and fourth columns show the restoration results of Cho et al. [9], Ali et al. [35] and proposed models, respectively.

Fig. 5

The de-fogging or de-hazing performance of the proposed model in comparison with Cho et al. [9] model, Ali et al. [35] model and clear images. The first column shows the given hazy or foggy images taken from MRFID [40]. The second, third and fourth columns show the restoration results of Cho et al. [9], Ali et al. [35] and proposed models, respectively. The fifth column shows the clear images taken from MRFID [40].

Fig. 6

Segmentation performance of the proposed model in comparison with Wu et al. [30] and Ali et al. [35] models on hazy images. The first column shows the given hazy or foggy images. The second, third and fourth columns show the Wu et al. [30] results, Ali et al [35] results and proposed method results, respectively.

Fig. 7

Segmentation performance of the proposed model in comparison with SPF [25] and GSPF [32] models on hazy images. The first column shows the given hazy or foggy images. The second, third and fourth columns show the SPF [25] results, GSPF [31] results and proposed method results, respectively.

4.2 Image de-hazing performance

In this section, to evaluate the performance of the proposed de-hazing method for haze or fog removal, the de-hazing results are compared with state of the art recent works of Cho et al. [9] and Ali et al. [35]. We have performed experiments on 150 synthetic images of different hazy data-sets brought together from the internet and 150 real images of multiple real-world foggy image data-set (MRFID, http://www.vistalab.ac.cn/MRFID-for-defogging/) [40]. Some of the experimental results are shown in Figs. 5. The first column of the figures show the given hazy or foggy images. The second, third and fourth columns of the figures represent Cho et al. [9], Ali et al. [35] and proposed restoration results, respectively. The fifth column of Fig. 5 shows the clear images of the corresponding hazy or foggy images given in the first column. In the proposed method, the result in the sky region is clearer than other algorithms and the color contrast of the image is increased and preserves edges and textures well which are the advantages of the fractional order derivative based prior over integer order derivative based prior. Since, there is some difference in the contrast and brightness of the clear images and in the de-hazed images of the proposed method i.e., the results of the proposed method are brighter and have some high contrast than the clear images but it can be concluded that our method outperforms other competing methods in most of the cases especially in removing haze or fog from images of multiple real-world foggy image data-set (MRFID) [39] where the competent methods may not work properly. The better performance of our method over other methods can be further confirmed by the quantitative and visual results shown in Table 2, Figs. 5. Table 2 shows the entropy values in terms of mean ± s . d.

Table 2
Quantitative results under entropy measure for the de-hazed images

Models Entropy

Different Hazy MRFID Images

Data-Sets Images

mean±s.d mean±s.d

Cho et al. [9] 5.9765 ± 1.2602 6.0072 ± 0.3143

Ali et al. [35] 6.4034 ± 0.8358 6.3730 ± 0.7782

Proposed 7.3467 ± 0.4743 7.8192 ± 0.0837

Models	Entropy
Cho et al. [9]	5.9765 ± 1.2602	6.0072 ± 0.3143
Ali et al. [35]	6.4034 ± 0.8358	6.3730 ± 0.7782
Proposed	7.3467 ± 0.4743	7.8192 ± 0.0837

Here s.d is the standard deviation showing measurement of how spread out the entropy values are.

4.3 Segmentation performance for hazy or foggy images

This section aims to test the validity and robustness of the proposed model to segment the desired object of the given hazy or foggy image. For this purpose, we have performed tests on various images having light and dense haze or fog as shown in Figs. 7 respectively. The first column of the figures show the given hazy or foggy images. The outcomes of the proposed model (fourth column) are compared with Wu et al. [29] and Ali et al. [35] models (second and third columns) shown in Fig. 6 and SPF [24] and GSPF [31] models (second and third columns) given in Fig. 7. Computational cost of our model and Ali et al. [35] model is nearly same. Experimental results clearly show that the proposed model successfully segment the object boundaries (desired object) better in hazy images as compared with the existing models. In case of dense fog, we noticed that Wu et al. [29], Ali et al. [35], SPF [24] and GSPF [31] give poor results and even undesired results in some images. The optimal results of the proposed model are achieved due to the physical properties of air-light map estimation through dark channel prior and fractional-order derivative based regularizer incorporated in the model.

Fig. 8

Quantitative analysis of the proposed model in comparison with Wu et al. [30] and Ali et al. [35] models using graphical representation of JSI values. The first column shows the given hazy or foggy images with initial contour. The second, third and fourth column show the graphical representation of JSI values of Wu et al. [30], Ali et al. [35] and proposed models, respectively. The JSI values of the Wu et al. [30], Ali et al. [35] and proposed models of the images (a, e, i, m, q) at final iterations are (0.9824, 0.9452, 0.9287, 0.8433, 0.9805), (0.9830, 0.9502, 0.9145, 0.8498, 0.9723) and (0.9950, 0.9947, 0.9766, 0.9836, 0.9928), respectively, shown from the top images to the bottom images in the second, third and fourth columns.

Fig. 9

Quantitative analysis of the proposed model in comparison with SPF [25] and GSPF [32] models using graphical representation of JSI values. The first column shows the given hazy or foggy images with initial contour. The second, third and fourth column show the graphical representation of JSI values of SPF [35], GSPF [32] and proposed models, respectively. The JSI values of the SPF [25], GSPF [32] and proposed models of the images (a, e, i, m, q) at final iterations are (0.9164, 0.9908, 0.9853, 0.8386, 0.9402), (0.8620, 0.9894, 0.9894, 0.8131, 0.9210) and (0.9232, 0.9963, 0.9950, 0.9379, 0.9807), respectively, shown from the top images to the bottom images in the second, third and fourth columns.

Table 3

JSI values for segmentation accuracy using grey-scale images, vector valued images and hazy images

Models	mean ± s.d
Wu et al. [30]	0.8848 ± 0.1068
SPF [25]	0.8900 ± 0.1001
GSPF [32]	0.9040 ± 0.0752
Ali et al. [35]	0.9095 ± 0.0637
Proposed	0.9629 ± 0.0359

Table 4

Average values of e, $\bar{r}$ , $\bar{σ}$ , MSE and PSNR for de-hazing performance using synthetic images

Models	e	$\bar{r}$	$\bar{σ}$	MSE	PSNR
Cho et al. [9]	0.0485	1.0014	0.1963	3.2237	18.8888
Ali et al. [35]	0.0509	1.0013	0.1100	3.1999	20.9386
Proposed	0.1549	1.0013	0.0013	1.9266	24.0000

Table 5

Average values of e, $\bar{r}$ , $\bar{σ}$ , MSE and PSNR for de-hazing performance using multi real world foggy image data-set (MRFID)

Models	e	$\bar{r}$	$\bar{σ}$	MSE	PSNR
Cho et al. [9]	0.0162	1.0013	0.2640	2.3623	14.4255
Ali et al. [35]	0.0272	1.0011	0.1600	2.0044	16.6161
Proposed	0.0988	1.0013	0.0010	1.0005	20.8329

4.4 Quantitative comparison of proposed model with existing methods

In this work, a variety of quality measures/metrics (entropy, Jaccard similarity index (JSI), the ratio of newly visible edges e, the average gradient ratio $\bar{r}$ , percentage of completely black or white pixels after restoration $\bar{σ}$ , mean square error (MSE) and peak signal to noise ratio (PSNR)) are taken to evaluate the image restoration and image segmentation results of the proposed model in comparing with other existing methods. The first one is the entropy to measure the de-haze image quality and the second one is the Jaccard similarity index (JSI) to show the segmentation accuracy of the proposed model. The JSI, e, $\bar{r}$ , $\bar{σ}$ , MSE and PSNR are given by $\begin{matrix} JS (S_{r}, G_{t}) = \frac{S_{r} ⋂ G_{t}}{S_{r} ⋃ G_{t}} \in [0, 1], e = \frac{n_{r} - n_{0}}{n_{0}}, \\ \bar{r} = exp [\frac{1}{n_{r}} \overset{\sum}{i} log (r_{i})], \bar{σ} = \frac{n_{s}}{\dim_{x} \times \dim_{y}}, \\ MSE = \frac{1}{\dim_{x} \times \dim_{y}} \overset{\dim_{x}}{\sum_{i}} \overset{\dim_{y}}{\sum_{j}} [I (i, j) - J (i, j)], \\ PSNR = 10 {log}_{10} \frac{I_{\max}^{2}}{MSE} . \end{matrix}$ where S_r, G_t, n_r, n₀, $r_{i} = \frac{Δ J_{i}}{Δ I_{i}}$ , n_s, dim_x, dim_y, and I_max represents segmentation result, ground truth, number of visible edges in haze-free image J, number of visible edges in hazy image I, ratio of the gradients of the restored image J and hazy image I, number of entirely black or white pixels, width of image, height of image and maximum intensity of the image, respectively. Note, JSI is the ratio of overlapped areas and union of areas showing better segmentation result if closer to 1 and imperfect result if closer to 0.

Entropy results and the values of e, $\bar{r}$ , $\bar{σ}$ , MSE and PSNR of the proposed model using synthetic images collected from the internet and multi real world foggy images (MRFID) are compared with Cho et al. [9] and Ali et al. [35] models given in Tables 2 , 4 and 5. High entropy results, High values of e and PSNR and low values of $\bar{σ}$ and MSE show the outclass de-hazing performance of the proposed model over Cho et al. [9] and Ali et al. [35] models. Although Cho et al. [9] model has generated high values of $\bar{r}$ than Ali et al. [35] and proposed models, the de-hazing or visual results are not very interesting due to the incomplete removal of haze as shown in Figs. 5. The JSI values of the proposed model (fourth column) are compared graphically with Wu et al. [30] and Ali et al. [35] models (second and third columns) shown in Fig. 8 and with SPF [25] and GSPF [32] models (second and third columns) displayed in Fig. 9. High values of JSI shown graphically in Figs. 9 and quantitative results in Table 3 clearly show the better performance, efficiency and effectiveness of the proposed model. Therefore, it is reasonable to conclude that the proposed fractional-order based variational model is best in the sense that it has sharp edges, piecewise smooth intensities, reduce staircase effects and preserves other fine image details better than others.

Fig. 10

Best values of parameters λ₁, λ₂, μ and fractional order α with optimal jaccard values for Fig. 6(a) showing sensitivity of the proposed model. The best parameter values are λ₁ = 750, λ₂ = 890, μ = 2.3 and α in the range of 0.4 and 0.5.

4.5 Computational complexity and running time analysis

As discussed earlier, the proposed algorithm involves joint learning of two fields: image de-hazing and image segmentation. Therefore, a variety of hazy and haze-free images with different structures (natural/real, synthetic, sharp edges) and sizes were tested in order to show the effectiveness, potency and speed advantage of the proposed algorithm. Different modules of de-hazing, segmentation and jointly de-hazing and segmentation algorithm were used. The proposed and state of the art algorithms are run in MATLAB R2013a in Haier Win8.1 PC, Intel core i3 CPU @ 1.70GHz with 4.00 GB RAM. The same value of atmospheric light I_∞ (the top 0.1 percent brightest pixel value of the image) is fixed as initial guess. The running speed of an algorithm or algorithm complexity is always a linear function of size, nature (real, synthetic), haze density ρ, number of sharp edges ν_s and number of iterations N_i required to de-haze or segment the test images.

For a dim_x × dim_y size image, the complexity of the proposed joint visibility restoration and segmentation algorithm with haze density ρ and number of sharp edges ν_s is O (N_idim_xdim_y), whatever the value of ρ and ν_s are. The running speed of the proposed joint de-hazing and segmentation model is compared with SPF [25], GSPF [32], Cho et al. [9], Wu et al. [30] and Ali et al. [35] through various outdoor scenes (synthetic, real) having sharp edged objects. The running speed of the proposed fractional order divergence operator based algorithm outperforms other state of the art models especially in de-hazing and segmenting the hazy real world images of MRFID because the state of the art algorithms has the lowest operational efficiency on MRFID.

Various experiments are conducted on more than 300 images, all the images are resized to 150 × 150. The average running time of more than 300 images of the proposed algorithm and other state of the art algorithms is given in Table 6. The running time of the proposed model and Ali et al. [36] model exceeds other state of the art models due to the joint de-hazing and segmentation algorithm.

Table 6
Average running time for de-hazing and segmenting grey-scale images, vector valued images and hazy images (synthetic and real images(MRFID)). D: De-hazing algorithm, S: Segmentation algorithm, JT: De-hazing + Segmentation algorithm

Models Time (s)

Wu et al. Model [30] (S) 10.7485

SPF Model [25](S) 9.0045

GSPF Model [32](S) 7.6666

Cho et al. [9](D) 6.0071

Ali et al. Model [35] (JT) 18.9015

Proposed Model (JT) 11.0054

Models	Time (s)
Wu et al. Model [30] (S)	10.7485
SPF Model [25](S)	9.0045
GSPF Model [32](S)	7.6666
Cho et al. [9](D)	6.0071
Ali et al. Model [35] (JT)	18.9015
Proposed Model (JT)	11.0054

4.6 Choice of the regularization parameters/parameters sensitivity

The proposed model needs a suitable balance between the regularization term and data fitting term, to obtain better restoration and segmented results. In fact selection of parameters is a challenging task. Sometimes a small deviation from the optimal value of the parameter or choosing a wrong value may lead to poor results. Although, finer tuning of the parameters may lead to better performance. For the sake of convenience, we adjust or tune the number of iterations and empirical values of the parameters (λ₁, λ₂, μ, α) according to the size and nature of the test images. We have tested on various images and have selected the parameter values at which optimal jaccard and entropy values are obtained. Figure 10 reports the best parameter values (λ₁ = 750, λ₂ = 890, μ = 2.3 and α in the range of 0.4 and 0.5) with optimal jaccard values for Fig. 6(a) affirming dependency of the proposed model on parameters. The parameter values selected for proposed model are detailed in Table 7.

Table 7
Choice of parameter values for the Proposed Model

Figures μ ν Δτ λ λ ₁ λ ₂ α ɛ

Fig.2(a) .0001 .01×10^- 6 1 10 350 280 0.9 0.005

Fig.2(e) .01 .0001 1 15 140 350 0.9 0.005

Fig.2(i) .02 .01×10^- 11 .01 10 10 10 0.1 1

Fig.2(m) .01 .01 1 1 119 120 0.9 0.005

Fig.3(a) .0001 .0001 1 10 110 100 0.9 1

Fig.3(e) 10 10 1 5 40 390 0.9 0.005

Fig.3(i) .02 .01×10^- 11 0.1 25 400 390 0.9 1

Fig.3(m) .02 .01×10^- 11 0.1 10 10 10 0.7 1

Fig.4(q) .02 .01×10^- 11 0.1 10 10 10 0.7 1

Fig.5(a) .01 .01×10^- 4 0.5 10 50 20 0.9 0.05

Fig.5(f) .01 .01 1 13 40 30 0.9 0.005

Fig.5(k) .02 .01×10^- 5 .01 1 10 10 0.1 1

Fig.5(p) .01 .01 1 10 20 10 0.9 0.05

Fig.5(u) .0001 .0001 1 10 110 100 0.9 1

Fig.6(a) .0001 .0002 1 15 400 200 0.9 0.005

Fig.6(e) 1 0.01 0.01 1 30 40 0.5 1

Fig.6(i) 1 1 0.01 1 10 10 0.7 1

Fig.6(m) .02 .01×10^- 11 1 10 100 110 0.9 1

Fig.6(q) 1 1 1 5 350 550 0.9 0.005

L Fig.7(a) .0001 1 0.001 15 90 300 0.9 1

Fig.7(e) .0001 1 0.001 40 400 500 0.9 1

Fig.7(m) 0.0001 0.0001 1 10 550 200 0.9 1

Fig.7(q) 0.0001 1 0.0001 5 170 350 0.9 1

Figures	μ	ν	Δτ	λ	λ ₁	λ ₂	α	ɛ
Fig.2(a)	.0001	.01×10^- 6	1	10	350	280	0.9	0.005
Fig.2(e)	.01	.0001	1	15	140	350	0.9	0.005
Fig.2(i)	.02	.01×10^- 11	.01	10	10	10	0.1	1
Fig.2(m)	.01	.01	1	1	119	120	0.9	0.005
Fig.3(a)	.0001	.0001	1	10	110	100	0.9	1
Fig.3(e)	10	10	1	5	40	390	0.9	0.005
Fig.3(i)	.02	.01×10^- 11	0.1	25	400	390	0.9	1
Fig.3(m)	.02	.01×10^- 11	0.1	10	10	10	0.7	1
Fig.4(q)	.02	.01×10^- 11	0.1	10	10	10	0.7	1
Fig.5(a)	.01	.01×10^- 4	0.5	10	50	20	0.9	0.05
Fig.5(f)	.01	.01	1	13	40	30	0.9	0.005
Fig.5(k)	.02	.01×10^- 5	.01	1	10	10	0.1	1
Fig.5(p)	.01	.01	1	10	20	10	0.9	0.05
Fig.5(u)	.0001	.0001	1	10	110	100	0.9	1
Fig.6(a)	.0001	.0002	1	15	400	200	0.9	0.005
Fig.6(e)	1	0.01	0.01	1	30	40	0.5	1
Fig.6(i)	1	1	0.01	1	10	10	0.7	1
Fig.6(m)	.02	.01×10^- 11	1	10	100	110	0.9	1
Fig.6(q)	1	1	1	5	350	550	0.9	0.005
L Fig.7(a)	.0001	1	0.001	15	90	300	0.9	1
Fig.7(e)	.0001	1	0.001	40	400	500	0.9	1
Fig.7(m)	0.0001	0.0001	1	10	550	200	0.9	1
Fig.7(q)	0.0001	1	0.0001	5	170	350	0.9	1

5 Conclusion

In this paper, a new fractional-order variational model is introduced for joint restoration and segmentation of (both grey and color) hazy or foggy images. For utilizing the accurate air-light map and recovering desired edges of the target object, we introduce fractional-order edge operator as well as local binary fitting energy terms. The proposed method estimates a transmission map using the dark channel prior which is then converted to the depth map. Using this depth map, we construct a new energy functional to seek the final haze or fog free segmented image simultaneously. Explicit gradient descent method was exploited for solving the non-linear fractional-order PDEs emerged from the proposed functional minimization. Experimental results on numerous real scenes uncovered that the proposed model improves entropy, Jaccard similarity index (JSI), and can preserve edges, textures, other fine features and reduces the staircase effect better than other competing methods. Especially in images with heavy haze or fog, our method outperform other methods. Therefore, we can conclude that the proposed method can jointly remove haze or fog and segment the desired image efficiently and effectively.

6 Applications:

The applications of the proposed method may be extended to cover image inpainting, image registration, inland river image processing, road scenes image processing under homogeneous and heterogeneous haze or fog, defense and surveillance images, underwater image processing, and hazy or foggy videos processing. Additionally, it can also be extended in numerous possible ways, for example the application of fuzzy membership framework and for inhomogeneous haze or fog.

7 Future Work:

Developing methods to deal with the sky regions more accurately as well as implementing fast numerical schemes for solving partial differential equations arisen from the proposed model minimization might be considered in future research.

References

Fang

, Li

and Zeng

, Single image de-hazing and denoising: afast variational approach, SIAM J Imaging Sciences 7(2) (2014), pp. 969–996. DOI: 10.1137/130919696.

Koschmieder

, Theorie der horizontalen Sichtweite, Beitrĺage zur Physik der freien Atmosphĺ are, Keim and Nemnich, Frankfurt, Germany (1924).

, Liu

and Ji

, Fog removal from color images using contrast limited adaptive histogram equalization, in Proc IEEE Int Congr, Image Signal Process., Oct. 2009, pp. 1–5, DOI: 10.1109/CISP.2009.5301485.

Ancuti

C.O.

and Ancuti

, Single image dehazing by multiscalefusion, IEEE Trans Image Process 22(8) (2013), pp. 3271–3282. DOI: 10.1109/TIP.2013.2262284.

Tan

R.T.

, Visibility in bad weather from a single image, in Proc IEEE Conf Comput Vis Pattern Recognition, (2008), pp. 1–8, DOI: 10.1109/CVPR.2008.4587643.

Fattal

, Single Image Dehazing, ACM transactions on graphics(TOG) 27(3) (2008), pp. 1–9. DOI: https://doi.org/10.1145/1399504.1360671

Tarel

J.-P.

and Hautière

, Fast visibility restoration from a single color or gray level image, in Proc IEEE Conf Comput Vis (2009), pp. 2201–2208, DOI: 10.1109/ICCV.2009.5459251.

, Sun

and Tang

, Single image haze removal using darkchannel prior, IEEE Trans Pattern Anal Mach Intell 33(12) (2011), pp. 2341–2353. DOI: 10.1109/TPAMI.2010.168.

Cho

W.H.

, Seop

I.N.

, Seo

S.C.

, Kim

S.K.

and Park

S.Y.

, Single imagede-fogging method using variational approach for edge-preservingregularization, World cademy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation,Control and Information Engineering 7(6) (2013), pp. 829–833. DOI: doi.org/10.5281/zenodo.1076846.

10.

Xue

and Zhao

, Haze image enhancement based on space fractionalorder partial differential equation, Ninth InternationalConference on Digital Image Processing (ICDIP) 10420N (2017), pp. 1–8. DOI: https://doi.org/10.1117/12.2281752.

11.

Narasimhan

S.G.

and Nayar

S.K.

, Vision and the Atmosphere, Int’l J. Computer Vision 48 (2002), pp. 233–254. DOI: https://doi.org/10.1023/A:1016328200723.

12.

W. Wang and X. Yuan. “Recent Advances in Image Dehazing,” it IEEE/CAA JOURNAL OF AUTOMATICA SINICA 4(3) (2017), 410–436. DOI: https://doi.10.1109/JAS.2017.7510532.

13.

Tan

, Visibility in bad weather from a single image, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway, NJ., Jun. (2008), pp. 1–8, DOI: 10.1109/CVPR.2008.4587643.

14.

, Sun

and Tang

, Single image haze removal using dark channel prior, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR, Miami., Jun. (2009), pp. 1956–1963, DOI: 10.1109/CVPR.2009.5206515.

15.

Kass

, Witkin

and Terzopoulos

, Snakes: active contourmodels, International Journal of Computer Vision 1(4) (1988), 321–331.

16.

Yuan

and He

, Adaptive active contours without edges, Mathematical and Computer Modelling 55(5-6) (2012), 1705–1721. DOI: https://doi.org/10.1016/j.mcm.2011.11.014.

17.

Caselles

, Kimmel

and Sapiro

, Geodesic active contours, International Journal of Computer Vision 22(1) (1997), 61–79. DOI: https://doi.org/10.1023/A:1007979827043.

18.

Mumford

and Shah

, Optimal approximations by piecewise smoothfunctions and associated variational problems, Communicationson Pure and Applied Mathematics 42(5) (1989), 577–685. DOI: 10.1002/cpa.3160420503.

19.

Mumford

and Shah

, Boundary detection by minimizing functionals, In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Jun. (1985), 22–26.

20.

Mumford

and Shah

, Boundary detection by minimizing functionals, In: Ullman S, Richards W (eds) Image Understanding. Springer, Berlin, 1989, pp. 19–13.

21.

Chan

and Vase

, Anactive contour model without edges, SCALE-SPACE ’99: Proceedings of the Second International Conference on Scale-Space Theories in Computer Vision, Sep (1999), 141–151.

22.

Zhang

, Zhu

, Liu

and Chen

, Image Segmentation based on fractional differentiation and RSF model, Proceedings of the ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference IDETC/CIE 2017, Cleveland, Ohio, USA, Aug (2017), 1–8, DOI: 10.1115/DETC2017-67110.

23.

, Kao

C.Y.

, Gore

J.C.

and Ding

, Implicit active contours driven by local binary fitting energy, IEEE Transactions on Image Processing (2007), 1–7, DOI: 10.1109/CVPR.2007.383014.

24.

Zhang

, Zhang

, Lam

K.M.

and Zhang

, A level set approach toimage segmentation with intensity inhomogeneity, IEEETransactions on Cybernet 46(2) (2016), 546–557. DOI: 10.1109/TCYB.2015.2409119.

25.

Zhang

, Song

and Zhang

, Active contours driven by localimage fitting energy, Pattern Recognition 43(4) (2010), 1199–1206. DOI: https://doi.org/10.1016/j.patcog.2009.10.010.

26.

Song

, Active contours driven by regularised gradient flux flowsfor image segmentation, Electronics Letters 50(14) (2014), 992–994. DOI: https://doi.org/10.1049/el.2014.1710.

27.

Wang

, Gao

, Tao

and Li

, A nonlinear adaptive level setfor image segmentation, IEEE Transactions on Cybernetics 44(3) (2014), 418–428. DOI: 10.1109/TCYB.2013.2256891.

28.

Ren

, Adaptive active contour model driven by fractionalorderfitting energy, Signal Processing 117 (2015), 138–150. DOI: https://doi.org/10.1016/j.sigpro.2015.05.009.

29.

Tian

, Zhang

and Fan

, A fractional-order level set model forimage segmentation, International Journal of Digital ContentTechnology and its Applications(JDCTA) 7(2) (2013), 622–630. DOI: 10.4156/jdcta.vol7.issue2.76.

30.

and He

, A convex variational level set model for imagesegmentation, Signal Processing 106 (2015), 123–133. DOI: https://doi.org/10.1016/j.sigpro.2014.07.013.

31.

Akram

, Kim

J.H.

, Lee

C.G.

and Choi

K.N.

, Segmentation of regionsof interest using active contours with SPF function, Computational and Mathematical Methods in Medicine 2015(2015), 1–14. DOI: 10.1155/2015/710326.

32.

Ali

, Badshah

, Chen

, Khan

G.A.

and Zikria

, Multiphase segmentation based on new signed pressure force functions and onelevel set function, Turkish Journal of Electrical Engineeringand Computer Science 25 (2017), 2943–2955. DOI: 10.3906/elk-1606-260.

33.

Bertero

and Boccacci

, Introduction to inverse problems in imaging, London: Taylor and Francis, CRC press, Jan. 1998.

34.

Chan

and Vase

, Active contours without edges, IEEETransactions on Image Processing 10(2) (2001), 266–277. DOI: 10.1109/83.902291.

35.

Ali

, Sher

, Zikria

and Rada

, A joint image dehazing andsegmentation model, Turk J Elec Eng and Comp Sci 27(2019) (2019), 1652–1666. DOI: 10.3906/elk-1810-1813.

36.

Ullah

, Chen

and Khan

M.A.

, A new variational approach forrestoring images with multiplicative noise, Computers andMathematics with Applications 71(10) (2016), 2034–2050. DOI: https://doi.org/10.1016/j.camwa.2016.03.024.

37.

Zhang

, hui

and Xiao

, A fast adaptive reweighted residualfeedback iterative algorithm for fractional order total variation regularized multiplicative noise removal of partly-textured images, Signal Process 98(4) (2014), 381–395. DOI: https://doi.org/10.1016/j.sigpro.2013.12.009.

38.

Y.F.

, Fractional differential analysis for texture of digitalimage, J Algorithms Comput Technol 1(3) (2007), 357–380. DOI: https://doi.org/10.1260/174830107782424075.

39.

Oldham

K.B.

and Spanie

, The fractional calculus, Academic Press, New York, NY, USA, Sep. 1974.

40.

Liu

, Zhou

, Lu

, Duan

and Qiu

, Image defogging qualityassessment: Real-world database and method, Ieee TransactionsOn Image Processing 30 (2020), 176–190. DOI: 10.1109/TIP.2020.3033402.

41.

Y.F.

, Wang

, Zhou

, Wang

and Jia

, Fractionaldifferential approach to detecting textural features of digitalimage and its fractional differential filter implementation, Sci China Ser F Inf Sci 51(9) (2008), 1319–1339. DOI: 10.1007/s11432-008-0098-x.

42.

Tamanini

, Optimal approximation by piecewise constant functions, Progress in non-linear differential equations and theirApplications, Birkhauser Verlag 25 (1996), 73–85. DOI: https: //doi:org//10:1007//978 –3 –0348 –9244 —56.

43.

Ullah

, Chen

, Sun

and Khan

M.A.

, A modified multigridalgorithm for a novel variational model to remove multiplicativenoise, J Vis Commun Image R 40 (2016), 485–501. DOI: 10.1016/j.jvcir.2016.07.016.

Models	Entropy
	Different Hazy	MRFID Images
	Data-Sets Images
	mean±s.d	mean±s.d
Cho et al. [9]	5.9765 ± 1.2602	6.0072 ± 0.3143
Ali et al. [35]	6.4034 ± 0.8358	6.3730 ± 0.7782
Proposed	7.3467 ± 0.4743	7.8192 ± 0.0837

A new fractional-order variational approach for joint de-hazing and segmentation of images

Abstract

Keywords

1 Introduction

1.1 Hazy image formation model

2 The proposed model

4.1 Image segmentation performance

Table 2 Quantitative results under entropy measure for the de-hazed images Models Entropy Different Hazy MRFID Images Data-Sets Images mean±s.d mean±s.d Cho et al. [9] 5.9765 ± 1.2602 6.0072 ± 0.3143 Ali et al. [35] 6.4034 ± 0.8358 6.3730 ± 0.7782 Proposed 7.3467 ± 0.4743 7.8192 ± 0.0837

6 Applications:

7 Future Work:

References