An Improved Multiexposure Image Fusion Technique

Abstract

Multiexposure image fusion (MEF) is an effective approach to generate high dynamic range images from multilevel exposures taken from ordinary cameras. In this article, a novel MEF algorithm is proposed to gain maximum visual details as well as vivid colors from the captured scene. This algorithm first decomposes the input images with multiple exposures into the base and detail layer. The weights for the base and detail layers are computed by using exposedness function and then both the layers are combined to generate the final fused image. The proposed multiexposure technique requires fewer computational operations, preserves edges, and also reduces spatial artifacts. The proposed technique has been evaluated quantitatively using image quality assessment model based on structure similarity index measure for MEF. By the extensive experimental results, it has been illustrated that in addition to significantly outperforming other state-of-the-art techniques, the proposed technique is much faster and can achieve better image quality.

Introduction

Recent advancement in technology has undergone enormous evolution in almost all the fields of life and to stay relevant with this evolution, every field needs to keep pace. This aspect has resulted in an increased interest in computer vision and image processing, both of which have gained popularity throughout the past few years.^1–3 In today's technological world, a large variety of devices have been invented that are capable of capturing digital images, which can be effectively used in various technological fields and industries.^4,5 The main objective of all these devices is to capture the image of a scene as much similar as possible when viewed via human natural visual system.

In general, the dynamic range of a natural scene is often large as compared to the dynamic range of commercially available imaging products. Therefore, in digital image processing, it is a challenging task to render a natural scene, which has a high dynamic range (HDR) to an imaging display which typically has a low dynamic range (LDR). In general, for a natural scene, the dynamic range can be characterized as the ratio of radiance among the brightest to the darkest point in the scene.

In the real-world scenes, the illuminance values are of several orders of magnitude that cannot be spanned by LDR. For instance, the radiance of the natural scenes spans the range 1:500,000, which cannot be spanned by the cameras available now-a-days, even though they have sensors with pixel depth within the range of 8 to 14 bits that represents digital values from 256 to 16,384.⁶

To overcome this limitation, HDR imaging techniques are used. In these techniques, a series of images of the same scene are captured at different exposure levels and the HDR image is then reconstructed by inverting the camera response function (CRF). In these techniques, estimating the CRF is the main problem.

Different constraints, for example, exposure time and some particular parameters are required for breaking the exponential and self-similar ambiguities.^7,8 In HDR imaging, multiple images are captured of the same scene, with all of them having distinct exposure levels. These images are then fused to give the resultant LDR image. The HDR imaging technique can be considered as an image enhancement technique that can be divided into two subcategories, tone mapping-based image enhancement techniques⁹ and fusion-based image enhancement techniques.¹⁰ Basically, in a tone mapping-based method, numerous LDR images are taken of the same scene but at distinct exposure levels. These images are then converted to intermediate HDR images by using CRF and finally converted into LDR images using tone mapping operator.¹¹

An effective alternative to resolve the problems of tone mapping-based technique is provided by the multiexposure image fusion (MEF) technique, which circumvents the gap between HDR imaging and LDR displays. In MEF, various images are taken of the same scene under distinct exposure levels as its input sequence. These images are then integrated together to produce a resultant fused image, which is sharper, perceptually appealing, and has more details, compared to any of the individual input image.^12,13 Multiexposure fusion aims at merging various images having distinct exposure levels into a single composite image having high quality so that the full dynamic range contains details and texture of LDR images.¹⁴

The main contribution of the proposed framework can be summarized as follows:

An innovative technique for MEF to extract comprehensive information from the input source images and also remove the blurring and halo artifacts.

A novel technique with a fast and accurate bilateral filter used for preserving edge information and estimation of weight maps uses the exposedness function required for fusion.

By the extensive experimental results, it has been illustrated that the proposed technique generates fused images that are better both qualitatively and quantitatively. The complexity analysis and comparison of execution time proved that it provides substantial perceptual gains, while maintaining an achievable computational complexity.

The proposed technique reduces the computational cost and ensures minimum manual interaction when compared with the state-of-the-art methods.

The organization of the rest of the article is as follows: In Related Work section, existing MEF techniques are explored. Proposed MEF technique is discussed in Proposed Methodology section. Performance Evaluation section compares performance evaluation of proposed MEF technique with existing MEF techniques, both qualitatively and quantitatively. Conclusion section concludes the article.

Related Work

Over the past few years, various research studies have been carried out on MEF due to which it has attained great importance as a developing research area.^15–22 Some of these recent researches have been taken as a reference for the development of the proposed framework.

MEF was introduced in 1980s.¹⁵ Most of MEF techniques are pixelwise that are generally formulated as: $Ŷ (i) = \sum_{n = 1}^{N} W_{n} (i) Y_{n} (i)$ (1)

where, n denotes the number of input images with different exposure levels of the same source sequence. In the nth exposure image, $W_{n} (i)$ and $Y_{n} (i)$ represents the weight and intensity values at the ith pixel, respectively; $Ŷ$ indicates the resultant fused image. In transform domain, a direct extension of this method is the replacement of $Y_{n} (i)$ with coefficient of transformation. W_n, weighting map, contains information on the preservation of the structure as well as the pixel-level visual significance of the nth input image. In image fusion, estimation of accurate weight map is very important. Various MEF methods^16–31 adopted different ways to find weight maps. The resultant fused image provided by these techniques represents more detailed information about the scene being more effective for human as well as machine perception.

Mertens et al.¹⁶ computed weight maps in terms of saturation, contrast, and well exposedness measure. This being appropriate for the preservation of global contrast, still fails to preserve fine details in case of overexposed or underexposed areas. Therefore, various techniques were proposed for edge preservation and weight refinement. The MEF technique proposed by Raman and Chaudhuri¹⁷ used the bilateral filter for preservation of edge information, and then weights were calculated to guide the fusion process. For accurate estimation of the weight map, recursive filtering-based techniques^18–21 were proposed. Li et al.²⁰ used Laplacian filtering for construction of weighting maps followed by guided filtering²¹ for weight refinement.

In some cases, gradient information of input images is used to estimate the impact on the resultant reconstructed image. Similarly, different techniques^22–24 were proposed in which gradient field from the structure of the input images was extracted which was subsequently modified iteratively to calculate the final fusion weights. Production of halo artifacts was the common drawback of these techniques.

Most algorithms for MEF could be categorized as: single-scale algorithms²⁵ and multiscale algorithms.^26,27,32 In single-scale exposure fusion, the minute details are preserved well for the HDR scenes, however, it causes the fused images to appear flat. Contrary to this, in multiscale exposure fusion, the images obtained are of better quality compared to the ones obtained from single-scale exposure fusion. Multiscale exposure fusion does not preserve the details in brightest and darkest areas besides producing halo artifacts. To overcome these problems, other techniques were introduced, which are known as the edge preserving smoothing techniques that are appropriate for the design of exposure fusion algorithm.^28–30 On one hand, such techniques can preserve the edges, and on the other hand, they are not able to reduce halo artifacts which arise in the fused image.

Other techniques^14,31,33 proposed in literature for image fusion are MEF pyramid decomposition. In these techniques, down sampling or blurring operation is used to decompose the source images into successive subimages. Fusion rules are then applied on these successive subimages to synthesize the final fused image. In such techniques, proper selection of the decomposition levels in the darkest or the brightest regions is required to retain the fine details as well as the global contrast. One major drawback of such techniques is that the details of the fused image are preserved regardless of the increased computational complexity. The algorithm proposed by Yang et al.³⁴ takes multiexposed images as its input to obtain a virtual image by using an intensity mapping function and subsequently use the existing pyramid-based technique. This algorithm provides a better way to preserve brightness, however, it can possibly produce halo artifacts.

Later, it was found that compared to the pyramids, wavelets can produce better results, therefore, discrete wavelet transform (DWT)³⁵ was used to achieve image fusion. Being shift invariant, DWT can produce artifacts in the resultant fused image. To reduce such artifacts, shift invariant DWT-based exposure fusion technique was developed.³⁶ Another drawback in the DWT-based exposure fusion techniques was that, compared to the images in the source sequence, the details of the fused image were not fine enough. This problem was solved by use of non-subsampled multiscale analysis technique.³⁷ Selection of proper decomposition level and production of halo artifacts are the main drawbacks of this technique. These methods were time consuming. A few existing techniques for image fusion^19,38,39 add an optimization framework to the MEF technique.

In these techniques, an estimate of the weight maps can be obtained by solving an energy function. A proper balance is found between the color consistency and the local contrast when combining the details of the scene³⁶ where a framework for the generalized random walk is proposed. A fusion scheme based on a probabilistic model approach is used³⁹ that can preserve the maximum visible contrast, as well as the gradient consistency in the resulting fused image. Li et al.¹⁹ proposed a method in which a quadratic optimization problem is solved so as to get the minute details present in the input images, which is then augmented in an intermediate reconstructed image to make it sharp. Patch-based exposure fusion techniques^40–43 were also proposed in the literature. These techniques give better quality of fused image, but they were consuming more time.

From the previous research carried out on image fusion, it can be concluded that most of the techniques used for MEF basically aim at finding the weight map. In these techniques, an innovative weighting factor is designed by considering various quality measures; contrast, color saturation, and luminance of a pixel to build a blending function for the input images to obtain a better fused image. Therefore, existing MEF techniques preserve edges at the expense of computational complexity and produces halo artifacts. Furthermore, most of the existing MEF techniques are validated with a confined dataset without extensive database verifications that include sufficient image content discrepancies.

In this work, an improved MEF technique is proposed, which results in output having most of visual details, vivid colors, and also preserves edges. Beside fusion performance, computing efficiency is very important in real applications. The goal of proposed technique is to synthesize a high-quality image with less computation time. By applying the proposed technique on various image datasets, it is concluded that, compared to existing technique, the proposed approach is memory efficient, less complex, and very fast that makes it suitable to be used in embedded systems and other real-world applications.

Proposed Methodology

This section covers the detailed discussion of the proposed MEF technique along with the mathematical modeling involved at each step. The new MEF framework is comparatively efficient to fuse multiple images. The gist of the proposed technique is the decomposition of sequence of input images into two scales defined as the base layer and the detail layer, followed by the calculation of weight map. The proposed framework is depicted in Figure 1.

FIG. 1.

Block diagram of Proposed Technique.

Two-scale decomposition

In the proposed technique, two-scale image decomposition has been exploited to separate each source sequence into two layers, on which fusion steps are then applied. This decomposition step focuses on separating each image of the input sequence into a smooth layer known as the base layer that accounts for the large-scale intensity variations and a detail layer for the small-scale intensity variations.

Let the input images of a source sequence with multiple exposure be denoted as $I_{n}, n = 1, 2, 3 \dots N$ . For each of the input sequence $I_{n}$ , the first step is computation of its luminance component denoted as L_n, by means of the weighted summation of color channels, that is, red, green, and blue of I_n. These luminance components thus obtained are utilized to compute the fusion weights.

Base layer

To obtain the base layer, an averaging filter can be applied to the luminance component. However, in case of a large noise floor, more pixels are needed to be averaged to suppress the noise. In such scenario, these averaging filters tend to over-smooth sharp image features such as edges and corners. Gaussian and box filters typically work well in applications where the amount of smoothing required is small. However, this over-smoothing issue can be alleviated by using the edge preserving smoothing filter where the image features are used to control the quantum of smoothing. Bilateral filter is one such edge-preserving smoothing technique that can effectively blur the image but retains the sharpness of edges. Unlike conventional filters, it defines the closeness of two pixels based on geometric as well as the radiometric distance. In the recent years, the bilateral filter has gained attention in the perspective of image denoising.

In the proposed technique, we use an extension of the bilateral filter for edge preserving and smoothening of images.⁴⁴ Conventionally, O(S) operations per pixel are required by a bilateral filter, where S denotes spatial filter support size. In proposed work, a fast and accurate bilateral filter has been used. In the proposed method, the bilateral filter proposed by Chaudhury and Dabhade⁴⁴ with O(1) operation per pixel is used. Therefore, in the proposed work, the fast bilateral filter is used to obtain the base layer represented by b_n, as follows: $b_{n} = f_{b F} (L_{n})$ (2)

Detail layer

In the proposed technique, two-scale decomposition takes place. The large-scale intensity variations are included in the base layer, whereas the small-scale information is included in the detail layer. After obtaining the base layer from Equation (2), the detail layer is easily formed from original input image sequences and base layer. $d_{n} = I_{n} - b_{n}, n = 0, 1, 2 \dots N$ (3)

where, $d_{n}$ represents the detail layer of all three red, green, and blue channels of the input images I_n.

Estimation of the weight map

In MEF, exposedness feature gain significant importance to get more detailed and comprehensive final resultant image. The weights used in the proposed technique are estimated efficiently by utilizing a generalized exposedness function f, defined as follows: $f (φ, σ) = - e x p (\frac{φ - c_{e}}{2 σ^{2}})$ (4)

where, $φ$ indicates the exposure feature and σ controls the spread of Gaussian. Similarly, c_e denotes constant of well-exposedness taken to be the middle value of the intensity range. From the Equation (4), weights are computed based on exposedness function indicates how close is the exposure feature to its desired value c_e.

The base layer obtained from Equation (2) mainly expresses the large-scale structural details of image luminance. To construct the fusion weights for this layer, the exposure quality of the local as well as luminance is considered. For the assessment of global exposure quality, the mean value of luminance of the entire image is used for estimating the exposedness feature. Thus, the global exposedness weight $E_{n}^{b, g}$ of the nth input image is computed as: $E_{n}^{b, g} (i, j) = e x p (- \frac{{({\hat{L}}_{n} - 0.5)}^{2}}{2 σ_{g}^{2}})$ (5)

where $σ_{g}$ is used to control the spread of Gaussian along mean luminance. In Equation (5) ${\hat{L}}_{n}$ represents the value of mean luminance for the entire image.

However, base layer itself can be used as an exposure feature, to achieve structural consistency among the base layer and its weight map. For each pixel location, the value of local mean of luminance can be utilized as the exposure feature for the assessment of the local exposure quality of the base layer. Correspondingly, $E_{n}^{b, l}$ represents the local exposure weight of the base layer of the nth input image, which is computed as: $E_{n}^{b, l} (i, j) = e x p exp (- \frac{{(b_{n} (i, j) - 0.5)}^{2}}{2 σ_{l}^{2}})$ (6)

where $σ_{l}$ is used to control the spread of profile along base layer. In Equation (6), exposedness feature is the base layer. Therefore, the weights $E_{n}^{b, g} (i, j)$ and $E_{n}^{b, l} (i, j)$ at each pixel position $(i, j)$ are combined together as to compute the weight map of base layer: $E_{n}^{b} (i, j) = E_{n}^{b, g} (i, j) . E_{n}^{b, l} (i, j)$ (7)

Thus, for the nth source sequence of input images, $E_{n}^{b} (i, j)$ indicates the weight map for the base layer.

For the detail layer, at each pixel position the exposedness feature $φ_{n}^{d} (i, j)$ is estimated, taken to be the average luminance in a small local neighborhood. In Equation (8), $σ_{d}$ is used to control the spread of profile along $φ_{n}^{d} (i, j)$ . Certainly, the exposure quality of image details is evaluated using the mean value of local intensity variations. In this setting, for nth input image, the image weights for every pixel position located in the detail layer are defined accordingly and for any image I, the exposure weight is computed: $E_{n}^{d} (i, j) = exp (- \frac{{(φ_{n}^{d} (i, j) - 0.5)}^{2}}{2 σ_{d}^{2}})$ (8)

Once the weight maps are constructed for all the input images, their normalization is done so that for each pixel position a unit sum is obtained. Finally, a summation of the weighted averages of both the base and detail layers of the input images gives the resultant fused image, which is given as: $R = \sum_{n = 1}^{N} E_{n}^{b} b_{n} + α \sum_{n = 1}^{N} E_{n}^{d} d_{n}$ (9)

where, $α$ is a controlling factor that controls the degree of local contrast and detail strength of the resultant fused image which was set as $α = 1.1$ . Therefore, $α$ should be greater or equal to 1. All the parameter values in this work were decided on the bases of fusion performance.

Performance Evaluation

For performance evaluation of the proposed technique, we have tested it on various images of static natural scenes, each having a different exposure level and compared its results with the different existing techniques for MEF. In this work, we selected 17 natural source image sequences of high quality to span a diverse image content, consisting of man-made architectures, natural sceneries, and outdoor and indoor views. All the source sequences are enlisted in Table 1. These different techniques are implemented on MATLAB R2015 on a computer with Intel Core I3, 1.61 GHz CPU, and 8 GB RAM, running Microsoft window 10 operating system.

Table 1.

Information of input image Sequences

Source sequences	Size	Image courtesy
Balloons	339 × 512 × 9	Erik Reinhard
Belgium House	384 × 512 × 9	Dani Lischinski
Cadik Lamp	384 × 512 × 15	Martin Cadik
Candle	364 × 512 × 10	HDR Projects
Cave	384 × 512 × 4	Bartlomiej Okonek
Chinese Garden	340 × 512 × 3	Bartlomiej Okonek
Farm House	341 × 512 × 4	HDR Projects
House	340 × 512 × 4	Tom Mertens
Kulki	341 × 512 × 4	Bartlomiej Okonek
Lamp	342 × 512 × 6	HDR Projects
Landscape	341 × 512 × 3	HDRsoft
Lighthouse	340 × 512 × 3	HDRsoft
Madison Capitol	384 × 512 × 30	Chaman Singh Verma
Memorial	512 × 381 × 16	Paul Debevec
Office	340 × 512 × 6	MATLAB
Tower	512 × 341 × 3	Jacques Joffre
Venice	341 × 512 × 3	HDRsoft

HDR, high-dynamic range.

As described previously, for a fair comparison, the parameter values used in the fusion process are taken to be same for all the images. It is demonstrated that for all test sequences, perceptually appealing results are produced by the proposed technique, both qualitatively as well as quantitively.

Figure 2 compares existing MEF algorithm with proposed method by applying them on the “Madison Capitol” sequence. In the fused image produced by Gu et al.,²² it can be seen that the color in the uniform areas of the image appears dreary. The resultant image produced by Li et al.¹⁹ gives sharper contrast and looks artificial. The image represented by Li et al.²⁰ is unable to recover precise details about painting as well as pattern on the walls and the image produced by Raman and Chaudhuri¹⁷ has suffered from lack of brightness and detail loss. Therefore, it can be seen that, compared to the previous techniques, the proposed technique has not only produced a more sharp image, but has preserved the patterns on the wall as well.

FIG. 2.

Images are shown for subjective analysis of the scene. (a1, a2, and a3) Represent multiexposed images of the scene, (b) represents the resultant image of Gu et al.²² (c) Represents the resultant image of Li et al.¹⁹ (d) Represents the resultant image Li et al.²⁰ (e) Represents the resultant image of Merten et al.¹⁶ result, (f) represents the resultant image of Raman and Chaudhuri¹⁷ result, (g) represents the resultant image of Li and Kang.¹⁸ (h) Represents the resultant image of Ma and Wang.⁴⁰ (i) Represent the resultant image of Shen et al.³⁸ and (j) Represents the resultant image of the proposed technique.

In Figure 3, we compared the results obtained by applying Mertens et al.,¹⁶ Gu et al.,²² Li and Kang,¹⁸ and Raman and Chaudhuri¹⁷ and the proposed technique on the “Tower” sequence. Compared with Mertens et al.,¹⁶ the proposed technique generates a fused image with various perceptual gains such as better preservation of top structure of the tower as well as the brightest cloud area, as shown in the red box. Furthermore, the sky and meadow regions give a more natural color appearance that is consistent with the source sequence. In the uniform areas of the fused image obtained by Gu et al.,²² as represented in Figure 3b, the color appears dreary, such as at the top of the tower, greenery, and the sky. Figure 3c represents the result obtained by Li and Kang.¹⁸

FIG. 3.

Results of the proposed technique compared with other MEF techniques on “Tower” sequence. (a) Represents the resultant image of Mertens et al.¹⁶ (b) Represents the fused image obtained by Gu et al.²² (c) Represents the resultant image obtained by Li and Kang.¹⁸ (d) Represents the resultant image of Raman and Chaudhuri.¹⁷ (e) Represents the resultant image obtained by Proposed technique. MEF, multiexposure image fusion.

Li and Kang's¹⁸ is basically a detail-enhancement form of Mertens et al.,¹⁶ but detail enhancement might not inevitably result in perceptual gains, particularly when the limit of camera acquisition is neglected. Therefore, the resultant image obtained after fusion from Li and Kang¹⁸ technique does not look natural around the edges such as marked in the box. Figure 3d compares the results obtained by Raman and Chaudhuri.¹⁷ As it can be seen that image has suffered from lack of brightness and detail loss as it gives less information about color and texture. On the contrary, Figure 3e is obtained by the proposed technique. The image looks sharper and brighter, as it can be seen in the red boxes. Also, greenery and the structure of the tower are more clear.

Hence, from the subjective analysis, it can be concluded that the proposed technique produced a fused image that looks brighter, perceptually appealing, and more informative.

For the objective performance evaluation, quality assessment model,¹³ which is proposed recently, is utilized. This model is using a metric based on structural similarity index measure for MEF. This quality indicator is used to measure the local structure preservation at the fine scales and also captures luminance consistency at coarser scales.

The results obtained in the proposed work of Ma et al.¹³ show that, compared to the existing measures for the assessment of MEF quality, this quality measure outperforms in terms of correlation with subjective judgments. This quality measure is used to compare the performance of the proposed technique with already prevailing 12 different MEF techniques, which include Raman and Chaudhuri,¹⁷ Li and Kang,¹⁸ Li et al.,¹⁹ Li et al.,²⁰ Gu et al.,²² Zhang and Cham,²³ Liu and Wang,²⁴ Song et al.,³⁹ Ma and Wang,⁴⁰ Ancuti et al.,²⁵ Xu et al.,⁴⁵ and Zhang et al.⁴⁶ Table 2 enlists the results of this comparison for a dataset of 17 multiexposure image sequences. The range of the quality score is from 0 to 1, with the higher value indicating better quality.

Table 2.

Performance comparison of proposed multiexposure image fusion technique with existing multiexposure image fusion technique using image quality assessment model¹³

Source sequences	Raman and Chaudhuri¹⁷	Li and Kang¹⁸	Gu et al.²²	Song et al.³⁹	Zhang and Cham²³	Li et al.¹⁹	Li et al.²⁰	Ma and Wang⁴⁰	Liu and Wang²⁴	Ancuti et al.²⁵	Xu et al.⁴⁵	Zhang et al.⁴⁶	Proposed
Balloons	0.767	0.944	0.912	0.917	0.944	0.940	0.947	0.962	0.926	0.929	0.717	0.931	0.962
Belgium House	0.810	0.946	0.895	0.872	0.947	0.953	0.964	0.965	0.930	0.938	0.897	0.942	0.953
Cadik Lamp	0.728	0.931	0.875	0.943	0.948	0.945	0.928	0.944	0.902	0.941	0.675	0.939	0.948
Candle	0.841	0.937	0.898	0.960	0.964	0.966	0.868	0.952	0.936	0.951	0.789	0.933	0.924
Cave	0.693	0.960	0.935	0.917	0.951	0.922	0.977	0.980	0.949	0.972	0.805	0.955	0.971
Chinese Garden	0.911	0.981	0.926	0.967	0.977	0.951	0.983	0.987	0.971	0.974	0.636	0.949	0.984
Farmhouse	0.876	0.976	0.931	0.946	0.971	0.958	0.985	0.982	0.966	0.976	0.746	0.948	0.980
House	0.770	0.921	0.875	0.947	0.958	0.934	0.956	0.944	0.926	0.893	0.751	0.935	0.938
Kulki	0.901	0.964	0.921	0.938	0.968	0.948	0.967	0.961	0.958	0.949	0.693	0.928	0.954
Lamp	0.863	0.936	0.870	0.828	0.931	0.933	0.933	0.945	0.925	0.877	0.793	0.916	0.943
Landscape	0.953	0.971	0.940	0.943	0.983	0.947	0.943	0.990	0.977	0.939	0.663	0.889	0.989
Lighthouse	0.937	0.952	0.933	0.970	0.971	0.967	0.950	0.968	0.964	0.957	0.645	0.925	0.963
Madison Capitol	0.764	0.917	0.863	0.944	0.951	0.949	0.967	0.973	0.927	0.907	0.769	0.958	0.964
Memorial	0.616	0.944	0.870	0.963	0.934	0.946	0.966	0.940	0.901	0.936	0.802	0.948	0.952
Office	0.906	0.971	0.901	0.960	0.967	0.953	0.966	0.985	0.966	0.957	0.748	0.913	0.973
Tower	0.895	0.983	0.931	0.938	0.973	0.950	0.985	0.980	0.975	0.962	0.698	0.927	0.976
Venice	0.891	0.951	0.889	0.941	0.961	0.936	0.953	0.977	0.953	0.932	0.652	0.924	0.965
Average	0.830	0.952	0.903	0.936	0.960	0.947	0.955	0.966	0.944	0.940	0.734	0.932	0.961

From the comparison shown in Table 2, it is clear that for all the 17 image sequences, proposed technique mostly gives better performance in terms of the quality measure proposed by Ma et al.¹³ Therefore, it can be concluded that the proposed method and Zang and Cham²³ and Ma and Wang⁴⁰ have the best performance.

Since the MEF techniques are mainly implemented in digital cameras, along with fusion quality, the fusion speed is an equivalently significant factor. For this purpose, we made a comparison among the proposed method and the existing best MEF algorithms, for which the source codes are available, on the basis of their fusion speed. In the proposed work, computational time is reduced by using a fast bilateral filter⁴⁴ having constant computation complexity to obtain base layer.

Moreover, we reduced computations by using two-scale decomposition technique instead of multiscale decomposition technique. Table 3 and Figure 4 include the comparison of the average computational time of the proposed technique over 17 MEF image sequences with the 7 existing methods, which include Yang et al.,³⁴ Ma and Wang,⁴⁰ Li and Kang,¹⁸ Liu and Wang,²⁴ Li et al.,³² Zhang et al.,⁴⁶ and Li et al.²⁰ From comparison, it can be concluded that the proposed technique is computationally efficient.

FIG. 4.

Bar graph of average execution time comparison of eight fusion methods.

Table 3.

Comparison of execution time of different multiexposure image fusion algorithms

Techniques	Average time (seconds)
Yang et al.³⁴	3.1105
Ma and Wang⁴⁰	2.6846
Li and Kang¹⁸	2.3146
Liu and Wang²⁴	2.2324
Li et al.³²	1.7124
Zhang et al.⁴⁶	1.2851
Li et al.²⁰	1.1655
Proposed	0.9011

Conclusion

In this article, an efficient technique for MEF is proposed. In particular, it decomposes the input images into two scales, and also for preservation of edges, we used fast algorithm and then computed the blending weights efficiently by using exposedness function. The proposed technique is also less time consuming for digital electronics. By experimental results, we compared the proposed technique against other existing MEF techniques. It shows that the proposed technique significantly outperforms in terms of objective as well as subjective evaluation. Moreover, proposed technique is computationally efficient and applicable for real-world applications.

Footnotes

Authors' Contributions

N.Z.: writing—original draft (lead) and editing (equal). A.M.S.: conceptualization (Support) and review and editing (equal). A.A.: methodology (Support) and review. W.I.: methodology (lead) and writing—review and editing (equal). M.I.: conceptualization (supporting), writing—original draft (supporting), and writing—review and editing (equal). I.T.: formal analysis (lead) and review. W.M.: software (lead) and writing—review and editing (equal).

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

Abbreviations Used

References

Zhu

, Wei

, Hu

, et al. A novel fast single image dehazing algorithm based on artificial multi-exposure image fusion. IEEE Trans Instrum Meas. 2021; 7:1–23.

Wang

, Sun

, Zhang

, et al. Multi-Exposure decomposition fusion model for high dynamic range image saliency detection. IEEE Trans Circuits Syst Video Technol. 2020; 30:4409–4420.

Hou

, Zhang

, Zhou

. Reconstructing a high dynamic range image with a deeply unsupervised fusion model. IEEE Photon J. 2021; 13:1–10.

Rani

, Pellegrini

, Nerina

, et al. New devices for imaging in nuclear medicines. Cancer Biother Radiopharm. 2004; 19:121–128.

Stacy

, Dearth

. Multimodality imaging approaches for evaluating traumatic extremity injuries: Implications for military medicines. Adv Wound Care. 2017; 6:1–11.

Youm

, Cho

, Hong

. High dynamic range video through fusion of exposure-controlled frames. In: Proceedings of 9th IAPR Conference on Machine Vision Application (MVA), 2005. The Eurographics Association: Netherlands; 2005. pp. 546–549.

Grossberg

, Nayar

. Determining the camera response from images: What is knowable?. IEEE Trans Pattern Anal Mach Intell. 2003; 25:1455–1467.

Lee

, Matsushita

, Shi

, et al. Radiometric calibration by rank minimization. IEEE Trans Pattern Anal Mach Intell. 2013; 35:144–156.

Rana

, Singh

, Valenzise

, et al. Deep tone mapping operator for high dynamic range images. IEEE Trans Image Process. 2020; 29:1285–1298.

10.

, Huang

, Liu

. An improved algorithm of multi-exposure image fusion by detail enhancement. Multimedia Syst. 2021; 27:1–12.

11.

, Yeganeh

, Zeng

, et al. High dynamic range image compression by optimizing tone mapped image quality index. IEEE Trans Image Process. 2015; 24:3086–3097.

12.

Burt

PJ.

The pyramid as a structure for efficient computation. Springer Series Inf Sci. 1984; 12:6–35.

13.

, Zeng

, Wang

. Perceptual quality assessment for multi-exposure image fusion. IEEE Trans Image Process. 2015; 24:3345–3356.

14.

Shen

, Zhao

, Yan

, et al. Exposure fusion using boosting Laplacian pyramid. IEEE Trans Cybern. 2014; 44:1579–1590.

15.

Burt

, Adelson

. The Laplacian pyramid as a compact image code. IEEE Trans Commun. 1983; 31:532–540.

16.

Mertens

, Kautz

, Van Reeth

. Exposure fusion: A simple and practical alternative to high dynamic range photography. Comput Graph Forum. 2009; 28:161–171.

17.

Raman

, Chaudhuri

. Bilateral filter based compositing for variable exposure photography. Eurographics Short Pap. 2009; 2009:1–4.

18.

, Kang

. Fast multi-exposure image fusion with median filter and recursive filter. IEEE Trans Consum Electron. 2012; 58:626–632.

19.

, Zheng

, Rahardja

. Detail-enhanced exposure fusion. IEEE Trans Image Process. 2012; 21:4672–4676.

20.

, Kang

, Hu

. Image fusion with guided filtering. IEEE Trans Image Process. 2013; 22:2864–2875.

21.

, Sun

, Tang

. Guided image filtering. IEEE Trans Pattern Anal Mach Intell. 2013; 35:1397–1409.

22.

, Li

, Wong

, et al. Gradient field multi-exposure images fusion for high dynamic range image visualization. J Vis Commun Image Represent. 2012; 23:604–610.

23.

Zhang

, Cham

. Gradient-directed multiexposure composition. IEEE Trans Image Process. 2012; 21:2318–2323.

24.

Liu

, Wang

. Dense SIFT for ghost-free multi-exposure fusion. J Vis Commun Image Represent. 2015; 31:208–224.

25.

Ancuti

, Ancuti

, De Vleeschouwer

, et al. Single-scale fusion: An effective approach to merging images. IEEE Trans Image Process. 2017; 26:65–78.

26.

Farbman

, Fattal

, Lischinski

, et al. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Graph. 2008; 27:1–10.

27.

Jiang

, Wang

. Image fusion using multiscale edge-preserving decomposition based on weighted least squares filter. IET Image Process. 2014; 8:183–190.

28.

Shreyamsha Kumar

BK.

Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process. 2015; 9:1193–1204.

29.

Kou

, Li

, Wen

, et al. Multi-scale exposure fusion via gradient domain guided image filtering. In: Proceedings of IEEE International Conference on Multimed Expo. IEEE: Hong Kong, China; 2017. pp. 1105–1110.

30.

Nejati

, Karimi

, Soroushmehr

SMR

, et al. Fast exposure fusion using exposedness function. In: Proceedings of International Conference on Image Process (ICIP) 2017. IEEE: Beijing, China; 2018. pp. 2234–2238.

31.

Kou

, Li

, Wen

, et al. Edge-preserving smoothing pyramid based multi-scale exposure fusion. J Vis Commun Image Represent. 2018; 53:235–244.

32.

, Ma

, Yong

, Zhang

. Fast multi-scale structural patch decomposition for multi-exposure image fusion. IEEE Trans on Image Process. 2020; 29:5805–5816.

33.

, Xie

, Li

, et al. Multi-exposure image fusion based on improved pyramid algorithm. In: 2020 IEEE 4th Information Technology, Networking, Electron and Automat Control Conference (ITNEC). IEEE: Chongqing, China; 2020. pp. 2028–2031.

34.

Yang

, Cao

, Wu

, et al. Multi-scale fusion of two large-exposure-ratio images. IEEE Signal Process Lett. 2018; 25:1885–1889.

35.

Malik

, Gilani

SAM

, Anwaar ul

. Wavelet based exposure fusion. World Congr Eng, 2008; 2008:688–693.

36.

Wang

, Xu

, Lang

, et al. Exposure fusion based on shift-invariant discrete wavelet transform. J Inf Sci Eng. 2011; 27:197–211.

37.

Zhang

, Liu

, Wang

, et al. Multi-exposure image fusion based on wavelet transform. Int J Adv Robot Syst. 2018; 15:1–19.

38.

Shen

, Cheng

, Shi

, et al. Generalized random walks for fusion of multi-exposure images. IEEE Trans Image Process. 2011; 20:3634–3646.

39.

Song

, Tao

, Chen

, et al. Probabilistic exposure fusion. IEEE Trans Image Process. 2012; 21:341–357.

40.

, Wang

Multi-exposure image fusion: A patch wise approach. In: Proceedings of IEEE International Conference on Image Process. IEEE: Quebec City, QC, Canada; 2015, pp. 1717–1721.

41.

, Sun

, Zheng

, et al. A novel Multi-exposure image fusion based on adaptive patch Structure. Entropy. 2018; 20:935.

42.

Zheng

, Qi

, Zhu

, et al. Image dehazing by an artificial image fusion method based on adaptive structure decomposition. IEEE Sensors J. 2020; 20:8062–8072.

43.

, Chang

, Luo

, et al. A precise Multi-exposure image fusion method based on low-level features. Sensors. 2020; 20:1597.

44.

Chaudhury

, Dabhade

. Fast and provably accurate bilateral filtering. IEEE Trans Image Process. 2016; 25:2519–2528.

45.

, Ma

, Zhang

. MEF-GAN: Multi-exposure image fusion via generative adversarial networks. IEEE Trans on Image Process. 2020; 29:7203–7216.

46.

Zhang

, Liu

, Sun

, et al. IFCNN: A general image fusion framework based on convolutional neural network. Inf Fusion. 2020; 54:99–118.