An improved clarity evaluation function for precise focusing and its application on vision measuring machine

Abstract

Four clarity evaluation functions are introduced. Their merits and deficits in sensitivity and speed were tested by using four photos with different clarity. Based on the test, a new evaluation function was proposed, which is adopted by Fourier spectrum evaluation function. Test results show that the new function has the better sensitivity on clarity evaluation compared to the Fourier spectrum evaluation function. This new function was applied to the vision measuring machine for clarity evaluation. Gauss curve fitting method is used to get the exact focusing position after acquiring the evaluation value of each position. It’s shown that the new evaluation function has good performance on machine’s auto-focusing action. It’s suitable to be used for the precise focusing situation.

Keywords

Precise focusing clarity evaluation function fourier spectrum function gauss curve fitting

1. Introduction

Vision measuring system based on microscope has been widely applied to PCB and precise components checking. Vision machine and AOI testing equipment are the typical products in the Chinese domestic market. The calibration template is used to get the correspondence between world coordinates and image coordinates. The calibration coefficient is saved and called out for measurement. Before the workpiece is measured, it should be focused to obtain a clear image. A blurred image has bad influence on image features detection and boundary elements extraction. It would also reduce the measuring accuracy because of the inaccurate calibration.

In order to obtain a clear image, it’s necessary to accurately adjust the distance between the lens and the measuring object for an Image acquisition system. The key point is to determine which image is clear. That the image is clear or blurred represents the energy concentration and diffusion by the energy point of view. While by the frequency domain point of view, it is the increase and decrease of high frequency components. All of these have become the important basis for the design of focus evaluation functions. How to design or select an appropriate evaluation function has become one of the key technologies for image focusing.

2. Introduction of the clarity evaluation function

With the rapid development of electronics and information technology, the computer’s computing power is increased and the related focus evaluation algorithms are invented. The common focus evaluation algorithms are Brenner function, Robert operator, Tenengrad function, variance evaluation function, vollathF4, vollathF5 and so on, which are based on spatial domain. Chung-Feng and Jeffrey found that the Brenner gradient and threshold pixel counting methods were the optimal methods for acquiring high quality metaphase chromosome images from the bone marrow and blood specimen [1]. It’s also found that VollathF4 is suitable for medical image focusing situation. All of these tradition methods have its’ special applicable environment. The evaluation algorithms based on frequency domain include fast Fourier spectrum analysis, wavelet multi-scale decomposition analysis. Spatial domain algorithms are generally simple and have small calculation [2, 3, 4, 5]. But they are often not sensitive enough especially near the focus position. It is easy to take the near focus position as the exact one. Frequency domain algorithms are relatively complex and have large amount of calculation, but they are more sensitive at the near focus position than spatial domain algorithms [6, 7]. So the frequency domain algorithms are more suitable for the precise focusing situation if not considering time consumption. That the acceleration algorithms and enhanced computer’s computing power shorten the computing time makes it’s possible to be used on real-time industry situations.

A good focus evaluation function should have the following characteristics: (1) unbiased; (2) single peak; (3) high sensitivity; (4) high signal to noise ratio; (5) small calculation. The following parts will describe several evaluation functions applied in the image measuring instrument.

2.1 Brenner function

The Brenner gradient function is a simple gradient evaluation function, which simply sums the squares of the difference between two adjacent pixels. The evaluation value could reflect gray-level value changes of the adjacent pixels. The higher value it gets, the better contrast the image has. So a clear image has high value of the Brenner gradient function. So we can use the Brenner gradient function to evaluate the image clarity. The Brenner gradient function of an $M\times N$ size image can be expressed as follows:

$\displaystyle F_{\text{brenner}}=\sum\nolimits_{j=1}^{N}\sum\nolimits_{i=1}^{M% }|f(i+2,j)-f(i,j)|^{2}$ (1)

2.2 Variance evaluation function

The variance evaluation function represents the degree to which the signal deviates from the mean. A clear image has more gray-level difference and signal deviation than a blurred image. So this function can be used as a clarity evaluation function. The variance evaluation function is shown in

Figure 1.

Coin surface image with 640 $\times$ 480 pixels, image (a) to image (d) is from blurred to clear.

Eqs (2) and (3), where $\mu$ represents the mean pixel gray-level value of the image and $F_{\textit{var}}$ represents the pixel gray-level variance of the image.

$\displaystyle\mu=\frac{1}{MN}\sum\nolimits_{j=1}^{N}\sum\nolimits_{i=1}^{M}{f(% i,j)}$ (2) $\displaystyle F_{\textit{var}}=\frac{1}{MN}\sum\nolimits_{j=1}^{N}\sum% \nolimits_{i=1}^{M}|f(i,j)-\mu|^{2}$ (3)

2.3 VollathF4 function

Vollath proposed two kinds of functions, VollathF6 and VollathF4, which have a good suppression effect on the image noise. VollathF4 is based on the auto-correlative function. A clear image has weak correlation between each pixel. This method is considered to be a preferred method in the medical image focusing, and its evaluation function is shown in Eq. (4).

$\displaystyle F_{\textit{var}}=\sum\nolimits_{i=1}^{N-1}\sum\nolimits_{j=1}^{M% -1}{f(i,j)}\times f(i+1,j)-\sum\nolimits_{i=1}^{N}\sum\nolimits_{j=1}^{M-2}{f(% i,j)}\times f(i+2,j)$ (4)

2.4 Fourier spectrum evaluation function

Fourier spectrum evaluation function is based on the theory of Fourier transform and its theory. It’s theoretical basis is that a clear image contains more high-frequency components than a blurred one. Frequency domain analysis can be done if an image is transformed by the two-dimensional Fourier transform. The discrete Fourier transform of an image with $M\times N$ pixels can be expressed by Eq. (5). Function $f(i,j)$ is the gray-level value of pixel which locates at the coordinates $(x,y)$ in an image. Parameter $u$ and $v$ are independent variables.

$\displaystyle F(u,v)=\frac{1}{MN}\sum\nolimits_{j=1}^{N}\sum\nolimits_{i=1}^{M% }{f(i,j)}e^{-j\left(\frac{2\pi ui}{M}+\frac{2\pi vj}{N}\right)}∼{}∼{}u=1,2% \ldots M,v=1,2\ldots N$ (5)

$R(u,v)$ is the real part of the function $F(u,v)$ and $I(u,v)$ is the imaginary part. We define the Fourier spectrum as follows:

$\displaystyle|F(u,v)|=[R^{2}(u,v)+I^{2}(u,v)]^{\frac{1}{2}}$ (6)

That a clear image has more sharp edge means it have more high frequency components than a blurred one. It also means more energy. So energy spectrum can be used to construct a focus evaluation function. It can be defined as follows:

$\displaystyle P(u,v)=|F(u,v)|^{2}=R^{2}(u,v)+I^{2}(u,v)$ (7)

The sum of the energy of an image is used to evaluate an image’s clarity. It is defined as Eq. (8).

$\displaystyle F=\sum\nolimits_{u=1}^{M}\sum\nolimits_{v=1}^{N}{P(u,v)}$ (8)

3. Results of focus evaluation function test and its’ analysis

Figure 1 includes four 24-bit colour BMP images of coin metal surface with 640 $\times$ 480 pixels. From image (a) to image (d), the clarity is gradually enhanced. The colour image has three components of red, green and blue colours. The red, green and blue components of each pixel should be converted to gray level value according to Eq. (9) before the evaluation. This could reduce the computing time of the evaluation job.

$\displaystyle I_{1}=0.299*\text{RedValue}+0.587*\text{GreenValue}+0.114*\text{% BlueValue}+0.5$ (9)

Since the DFT (direct Fourier transform) take up long calculation time, It’s not suitable for engineering applications that require real-time processing. FFT (fast Fourier transform) replaces it in actual application for less time cost. There should be point number $2^{n}$ for FFT calculation. So an image with 640 $\times$ 480 pixels, not all pixels in which are involved in the FFT operation. The maximum pixel number should be 256 $\times$ 256.

Four images in Fig. 1 were evaluated by the above four methods. Fourier spectrum analysis uses the window with 256 $\times$ 256 pixels. The window centre is the image centre. Other functions use full image pixels. The results are obtained as shown in Table 1.

Table 1

Evaluation results

Method	Fig. 1a		Fig. 1b		Fig. 1c		Fig. 1d
	Time/ms	Value	Time/ms	Value	Time/ms	Value	Time/ms	Value
Brenner	7	129.65	6	203.99	6	1317.6	6	2869.91
Variance	7	3740.91	7	3996.46	7	4889.33	7	5212.63
VollathF4	20	5.51E $+$ 06	19	1.13E $+$ 07	20	9.14E $+$ 07	19	1.83E $+$ 08
Fourier	58	1.40E $+$ 06	58	1.82E $+$ 06	58	3.745E $+$ 06	59	4.956E $+$ 06

Figure 2.

Clarity evaluations of the focusing an defocusing images with four different methods.

As can be seen from Table 1, evaluation functions based on spatial domain have less time-consuming. The time cost of Brenner function, Power function, VollathF4 function and Fourier spectrum function increases in turn. Obviously, the evaluation value of the clear image is larger than the blurred one.

In order to test the sensitivity of these four functions, more images are acquired in our experiments. We fix the distance between the CCD lens and the coin surface first, and then adjust the focusing distance with a fixed step to capture two sets of images from low clarity to high clarity and from high to low clarity again. Totally the number of the images is 13. All these images are saved in order and evaluated by these four functions. The evaluation values are normalized after the evaluation. Figure 2 shows the curves of these image evaluation values from low clarity to high clarity and from high clarity to low clarity.

From this figure, we can see that the curve of Fourier spectrum function has maximum slope near the focus position without local extreme point. It means that it has high sensitivity and single peak on the near focus evaluation. The curve of variance evaluation function has minimum slope and the small value changes near the focus position. Although the curve of Brenner and variance are sharp, but it’s slope is smaller than the Fourier spectrum function near the focus position.

Figure 3.

Spectrum maps (a)–(d) which are gotten by FFT to image (a)–(d).

In a word, the Fourier spectrum function has large time consuming, but it has good sensitivity and single peak especially near the focus position. The time consumption of Fourier spectrum analysis to an image with 640 $\times$ 480 pixels is within 60 milliseconds. When the occasion demands precise focus it is also an acceptable choice. Considering all these test results, an improved evaluation function based on the Fourier spectrum analysis is created in order to obtain more favourable performance for precise focusing.

4. Improved Fourier spectrum analysis evaluation function and its test

The spectral information could be obtained by the fast Fourier transform to the image gray-level information. However, the transformed $u$ and $v$ do not represent the true frequency. The frequency centre must be shifted to the image centre. Four maps in Fig. 2 is gotten by the FFT transform and the spectrum centre shift to the four maps centre in Fig. 1. It can be seen that the high frequency contents increase with the image clarity. This means CCD obtain more energy in the clear image. From image (a) to image (d), more low frequency parts get high Amplitude. This could be seen from Fig. 3 that more points near the centre area become bright from image (a) to image (b). However, it is found that sometimes the sum of the low-frequency components energy have a large proportion on the whole image energy especially near the real focus position, which often leads to low sensitivity on focus evaluation because of using the sum of the energy of an image to evaluate the clarity just as Eq. (8) describes. Sometimes the image with maximum function value is not the clearest image. This happens more often at an image with background noise. So it’s very important to find a method to reduce the negative influence cause by the energy of low frequency components.

Figure 4.

Filter’s 3-D model.

Figure 5.

Spectrum maps (a)–(d), gotten by using improved evaluation function to transform image (a)–(d).

High-pass filter could suppress the low frequency content in an image. It’s suitable to apply a high pass filter to deduce the low frequency influence at near focus position. There is a problem to find the cut-off frequency for the high pass filter function because of the different image content. It’s hard to say which frequency is high. The improved spectral evaluation function is shown as follows:

$\displaystyle F_{\textit{var}}=\frac{1}{MN}\sum\nolimits_{v=1}^{N}\sum% \nolimits_{u=1}^{M}{\sqrt{\left(u-\frac{M}{2}\right)^{2}+\left(v-\frac{N}{2}% \right)^{2}}|F^{\prime}(u,v)|}\times K$ (10)

where $F^{\prime}(u,v)$ is the Fourier transform function with centre shift, $\sqrt{\left(u-\frac{M}{2}\right)^{2}+{\left(v-\frac{N}{2}\right)}^{2}}$ is the spectrum radius, $K$ is a weighted item to leave $F_{\textit{var}}$ as an accepted value.

The spectrum radius is used as a high pass filter. The three-dimensional surface shape of it is shown in Fig. 4. It can be seen from the figure, the farther it is away from the centre region, the greater the weight gain. High frequency components get higher gain and low frequency components are opposite. It greatly suppresses the interference caused by the low frequency component and overcomes the shortcomings of other high filters that need to determine the filter radius in advance. When the actual application is processed, the coefficient $K$ is often used to keep $F_{\textit{var}}$ below 255 and the $K$ value is a coefficient less than 1. The valve exceed 255 couldn’t be shown on the spectrum picture. Most occasions, the maximum value of the spectrum is used as a comparison standard, which multiple $K$ should be 255.

Figure 6.

Clarity evaluations of the focusing an defocusing images with five different methods.

All images in Fig. 1 are processed by the improved evaluation function and the spectrum images are then obtained. These spectrum images are shown in Fig. 5. It can be seen from the figure that the number of points with large brightness from left to right is increased obviously. More points in Fig. 3 near the image centre are with low brightness compared with Fig. 2. This means the energy of low frequency components is suppressed. Compared with Fig. 5, it can be seen that more high frequency components are strengthened and the contrast are significantly enhanced. So it’s easier to evaluate the image clarity.

We apply the improved clarity evaluation method to compute the clarity values of these images captured before. For comparison, we draw the curve in the Fig. 2 together after the evaluation value is normalized. Figure 6 shows the curves of these image evaluation values. From this figure, we can see that the curve of improved function is sharper than the curve of Fourier function at the near focus position, which would result in the accurate detection of the best focus in the auto focusing steps.

Compared to the traditional methods, the improved clarity evaluation method has the best sensitivity and single peak, which could reduce the negative effect of background noise and increase the focusing accuracy. Compared to the wavelet-based method, which is computation expensive in decomposing the image into different frequency bands and computation of transform [10], the improved method has less time consuming and more simple and applicable in the engineering practice. Different from the LSF method, based on line detection, which is effective only for image with lines and sensitive to noise in edge line detection [11], the improved function can be used in various types of image with enough information of background and target.

5. Focused application research on improved evaluation function

In order to further test the effectiveness of the improved evaluation function, it is applied to the automatic vision measuring machine to do the focus evaluation test. The test scenario is shown in Fig. 7. The test equipment is VMC250 of 3D-Family, which is an auto vision measuring machine. It has the optical lens with 0.7–4.5 magnification and color CCD vision system. The lighting system chooses led ring lamp. One Yuan coin is chosen to be the focus target. Image focus window is the 256 $\times$ 256 pixels area.

Figure 7.

Auto focusing test environment.

The machine drives Z axis motor to move at a fixed step. At each position, it stops and does the image acquirement and evaluation. It records position value $x_{i}$ and the evaluation value $y_{i}$ . The moving sequence is defocus-focus-defocus. The evaluation value is low–high–low. That motion control algorithm is the typical Mountain-Climbing algorithm [12, 13, 14, 15]. Not all positions with the maximum evaluation value are the right focus position [8, 9]. In many cases it may be the local extreme point and is just near the focus position. In order to quickly find the exact location of the focus, the data fitting method is used to find the maximum value of the location, that is, the real focus position. Quadratic curve fitting method is a widely used fitting method. But the evaluation function value increases quickly because of the high frequency filter. So the quadratic curve fitting method is not ideal in this occasion. After observing the measured data points, it is found that the shape of the data curve is very similar to the shape of the Gaussian curve. Therefore, the focus test is carried out by Gaussian data fitting method.

Equation (11) is the expression of the Gaussian curve, where the parameters $y_{\max}$ , $x_{\max}$ and $S$ are the peak, peak position and half width information of the Gaussian curve. The peak position $x_{\max}$ is exactly the position with highest value $y_{\max}$ .

$\displaystyle y_{i}=y_{\max}\times e^{\left[-\frac{{(x_{i}-x_{\max})}^{2}}{s}% \right]}$ (11)

If both sides of Eq. (11) take natural logarithm, it’ll get Eq. (12) as follows:

$\displaystyle\ln y_{i}=\left(\ln y_{\max}-\frac{x_{\max}^{2}}{S}\right)+\frac{% 2x_{i}x_{\max}}{S}-\frac{x_{i}^{2}}{S}$ (12) $\displaystyle\text{If we let }\ln y_{i}=z_{i},a_{0}=\ln y_{\max}-\frac{x_{\max% }^{2}}{S},a_{1}=\frac{2x_{\max}}{S},a_{2}=-\frac{1}{S}$ (13)

If all record points ( $x_{i}$ , $y_{i}$ ) are substituted in equation, we can get matrix as follows:

$\displaystyle\begin{bmatrix}z_{1}\\ z_{2}\\ \vdots\\ z_{i}\end{bmatrix}=\begin{bmatrix}1&x_{1}&x_{1}^{2}\\ 1&x_{2}&x_{2}^{2}\\ \vdots&\vdots&\vdots\\ 1&x_{i}&x_{i}^{2}\end{bmatrix}\begin{bmatrix}a_{0}\\ a_{1}\\ a_{2}\end{bmatrix}$ (14)

The matrix can be expressed as: $Z=XA$ , its solution is:

$\displaystyle A=(X^{T}X)^{-1}X^{T}Z$

If the coefficient matrix $A$ is obtained, the extreme position can be taken out according to Eq. (11). For more accurate judgment, after obtaining the extreme position, the machine drive Z axis motor to move to the extreme position. The image is read and the evaluation function is executed. Compared with the maximum value of the record points, if the evaluation value is greater than the maximum value, it means that the probability of obtaining the correct focus position is very large. If the value is smaller than the maximum value, continue to collect a small number of points near the extreme position at a small interval step, the position with the highest evaluation value can be took as the real focus position. Figure 8 shows Gaussian fitting curve according to the collected point in an image focus experiment. After the fitting, the extreme point position is obtained and machine move to this position to acquire image and do evaluation. After comparing the extreme position evaluation data with the recorded maximum data, it is confirmed that the extreme position gotten by the data fitting is the correct focus position.

Figure 8.

Gauss curve fitting to the acquired point group.

Using a various test samples and different fixed steps to repeat the test, it is found that the focus success rate could arrive 99.96% if having appropriate fixed step. There is a greater focusing failure probability in the choice of a large fixed step, Which is mainly due to the sampling points in the entire collection procedure are too few. So the improved evaluation function has good sensitivity, unbiased and single peak. But if choosing small fixed steps, such as 0.2 mm, the focus time is relatively long. If start focusing at a position with 5 mm distance from the focus point, it took an average of 8.5 seconds to get the right focus position. Considering its focusing performance, in an vision measuring instrument, its practical application is divided into the following two cases: one is to align the lens to the location where contains rich feature information of the workpiece and narrow the focus evaluation window to reduce the computing time; another method is to combine the improved clarity evaluation function with other small computational evaluation function and use rough and precise two-step focusing method [16, 17, 18].

6. Conclusion

There is a huge demand to high focus precision and high focus speed in modern AOI instruments. It depends on both the design of a good evaluation function and the design of good focus motion control algorithms. How to design a faster, more efficient and accurate evaluation function is one of the key. It is a practical way to improve traditional focus functions or introduce other excellent algorithms according to the focusing characteristics. In this paper, the improved Fourier spectrum evaluation function is very excellent in focusing sensitivity, unbiased and single peak performance. It has been successfully applied in the vision measuring machine. After the practical application test, it’s shown that the evaluation function is suitable for high precision focusing. It could also be applied to other occasions where precise focusing is required.

References

Qiu

et al., Evaluations of auto-focusing methods under a microscopic imaging modality for metaphase chromosome image analysis, Analytical Cellular Pathology 36 (2013), 37–44.

Jin

and Ma

, Study on definition evaluation function in auto-focusing process, Optical Instruments 34(1) (2012), 59–64.

Chen

et al., Efficiency contrast of digital image definition functions for general evaluation, Computer Engineering and Applications 49(14) (2013), 152–155.

Pan

et al., Image entropy autofocus sharpness evaluation function simulation and analysis under Matlab, Infrared Technology 38(10) (2016), 838–844.

Chen

et al., Detection method of the LGP warpage degree based on edge detection evaluation function, Manufacturing Automation 37(11) (2015), 1–4.

Wang

Tan

and Tian

, A new kind of clarity-evaluation-function of image, Journal of Wuhan University of Technology 29(3) (2007), 124–126.

Lee

et al., Development of an auto-focus system based on the moiré method, Measurement 44(10) (2011), 1793–1800.

et al., Study on auto-focusing algorithm of micro-vision system, Computer Engineering and Applications 52(14) (2016), 197–201.

Gao

Zhu

and Zhang

, Auto focusing method for camera in machine vision fatigue crack propagation test, Computer Measurement & Control 24(11) (2016), 258–264.

10.

Makkapati

, Improved wavelet-based microscope auto focusing for blood smears by using segmentation, 5th Annual IEEE Conference on Automation Science and Engineering (2009), 208–211.

11.

et al., Auto-focus using LSF in aerial push-broom remote-sensing camera, Infrared and Laser Engineering 41(7) (2012), 1808–1814.

12.

Gao

Jiang

and Zhu

, Auto focusing algorithm based on most gradient and threshold, Journal of Electronic Measurement and Instrument 21(5) (2007), 49–54.

13.

Zhou

Sun

and Zhang

, Auto focusing and control of micro vision system, Optical Precision Engineering 21(3) (2013), 807–812.

14.

Jiang

et al., Study on auto focusing algorithm for automatic microscope, Chinese Journal of Image and Graphics 9(4) (2004), 396–401.

15.

Wang

et al., Study on auto focusing algorithm for microscope based on digital image processing, Chinese Journal of Scientific Instrument 30(6) (2009), 1290–1294.

16.

Wang

Chen

and Wu

, Implementation of a multiple automatic focusing algorithm for video measuring system, China Mechanical Engineering 18(21) (2007), 2556–2560.

17.

Liu

and Yang

, An image auto-focusing algorithm for industrial image measurement, EURASIP Journal on Advances in Signal Processing (2016), 2–16.

18.

Kuo

and Chiu

, Auto-focus control of a CMOS image sensing module, Journal of Intelligent & Fuzzy Systems 18 (2007), 405–415.