The study of binarization algorithm about digital rubbings image based on threshold segmentation and morphology

Abstract

This paper discusses the binarization restoration algorithms of ancient Chinese Rubbings image. According to the location of the image features, such as color, edge, stroke width and pixel location feature, the binary restoration algorithm of digital rubbings based on threshold segmentation and morphology is discussed. If the difference between the text and the background is larger and more obvious distinguish in images, Otsu threshold segmentation method can be directly used. For images with more background noise, shadows and uneven illumination, threshold segmentation cannot achieve effective segmentation and reasonable denoising, a set of algorithms associated with the pixel position of adaptive segmentation, morphological method, connected domain and mean filtering combined of binarization restoration are needed. The experimental results show that this algorithm is very effective for image with complex and large area noise.

Keywords

Otsu threshold segmentation rubbings image connected domain morphology

1. Introduction

Ancient Chinese calligraphy inscription images have unique expression form with high degree artistic achievement and rich cultural connotation. Due to various reasons, a lot of inscriptions have the status such as dirty surface of paper, broken inscriptions, which make the writing diffuse, incomplete. It is difficult to achieve purpose by using the usual image segmentation method. With the help of image processing, pattern recognition and other technologies for binary processing, can make it more convenient to analysis and process the Chinese characters and calligraphy strokes etc. artistic elements, so that accurately identify these words from a variety of background. At the same time, it provides accurate and complete Chinese character model with high artistic and historical value, reproduce as much as possible the ancient rubbings cultural relics.

Image binarization is to propose what we are interested in from the background image [1], that is, image segmentation. Image segmentation is widely used, but up to now, no one segmentation method suitable for all images. Depending on the characteristics of the image, different division methods are required. Different images need using different segmentation methods according to their own characteristics [2]. At present, image segmentation methods mainly include threshold method, [3, 4], edge detection [5, 6], mathematical morphology [7], and so on. These classical image segmentation methods, such as simple threshold segmentation, cannot solve the problem of noise. Mathematical morphology is mainly used for expansion and corrosion; edge detection cannot get coherent strokes. These cannot adapt to the complex segmentation of rubbing images. With the development of technology, some new algorithms have begun to be applied to image segmentation. Such as fuzzy clustering [8, 9, 10], support vector machine [11], genetic algorithm [2], neural network [12, 13], etc. In order to obtain the ideal segmentation effect, it is usually necessary to comprehensively use multiple segmentation methods. Clustering and support vector machines are mostly suitable for multi-class classification, and neural network algorithm takes too long time. For Rubbings image segmentation algorithm, the threshold method is more suitable, but the simple threshold method cannot achieve good results, many researchers have done a lot of research on the synthesis algorithm based on the threshold method. On the basis of Otsu, [14, 15] respectively adopt iterative algorithm and enhanced target variance method to segment foreground and background, which have better effect on weak target recognition, but cannot solve the text part. In [16], a new adaptive region histogram correction technique is proposed, which can automatically enhance the contrast of the image [17, 18] have studied the binarization of document images with shadows and degradation, but [17] only aims at the regular document processing, and [18] only applies to the threshold method which can distinguish the foreground and background images, not to the multi-noise images in rubbing images [19, 20] proposed the binarization of the degraded document images. In [19] is based on the combination of Otsu and NiBlack document image binarization hybrid algorithm to binaries the degraded broken documents. In [20] uses the concept of boundary and edge detection method to locate the region of interested object, which is also not suitable for rubbing images. The image of ancient rubbing images in China has a unique font structure and high artistic value. In addition to the appropriate threshold selection, it also needs to remove its unique noise and reproduce its artistic value.

The remaining parts of this paper are arranged as follows: In Section 2, the first step of feature extraction about rubbing Image – gray scale, then threshold segmentation algorithm is researched. In Section 3, rubbing image binarization algorithm, our method is proposed, some of experimental results are shown. In Section 4, the results of experimental are analyzed and discussed. Conclusions are described in Section 5.

2. Threshold segmentation of rubbing image

2.1 Image pre-processing

Image binarization is a key step from image processing to reducing computation. Image pre-processing is the premise of digital image processing. Usually, many images need to be converted into grey images before binary processing. One pixel has three colours in colour image, and only one grey value needs to store. Grayscale processing of images is an important technology in computer vision. Accurate binarization not only reproduces the artistic value of the image, but also provides the basis for further processing (such as recognition), which is an important step in machine vision processing.

It extracts image information through computer technology; determines whether each image has a certain image characteristics. The result is that the points on the image are divided into different subsets, which often belong to isolated points, continuous curves or continuous regions. In order to binary colour image, usually, colour image is first converted to grey image. Histogram graphical display different pixel values on the frequency of occurrence of different brightness values, for the colour image can display three colours independently histogram, as shown in Fig. 1. By greying the colour image, the subsequent processing process can be accelerated; the conversion formula is shown in Eq. (1).

$\displaystyle\textit{Gray}=R\times 0.299+G\times 0.587+B\times 0.114$ (1)

The results are rounded, for the grey image range is [0 $\sim$ 255], the grey image and grey histogram are shown in Fig. 2.

Figure 1.

Rubbing image and color histogram.

Figure 2.

Grey-scale image and its histogram.

2.2 Threshold segmentation algorithm research

The advantage of threshold segmentation is that the calculation is simple and the operation efficiency is high. It has been widely used in the occasion of needing pay attention to the operational efficiency. Image threshold segmentation is one of the most common methods and is the most basic and widely used technique in image segmentation. It is particularly useful for target and background of image have different levels of grey scale.

Not only does it greatly reduce the amount of data, it also greatly simplifies subsequent analysis and processing. In many cases, it is also the necessary reprocessing process for image analysis, feature extraction and pattern recognition [21] etc. According to the characteristics of rubbing image, the threshold binarization algorithm is adopted in the initial binarization.

2.2.1 Iterative threshold segmentation

Iterative algorithm adopts the idea of approximation algorithm. Concrete steps are as follows:

Step1:
Find the minimum grey value in the image $Z_{\min}$ and the maximum grey value $Z_{\max}$ , set the threshold initial value

$\displaystyle T_{0}=(Z_{\min}+Z_{\max})/2;$ (2)
Step2:
The image is divided into the target and background according to the threshold initial value $T_{k}$ , obtained the average grey value of two parts $Z_{A}$ and $Z_{B}$ .

$\displaystyle Z_{A}=\frac{\sum\limits_{z(i,j)<T_{k}}{z(i,j)\times N(i,j)}}{% \sum\limits_{z(i,j)<T}{N(i,j)}}$ (3) $\displaystyle Z_{B}=\frac{\sum\limits_{z(i,j)>T_{k}}{z(i,j)\times N(i,j)}}{% \sum\limits_{z(i,j)<T}{N(i,j)}}$
Step3:
Get the new threshold $T_{k+1}$ . $T_{k+1}=\frac{Z_{A}+Z_{B}}{2}$ .
Step4:
If $T_{k}=T_{k+1}$ , then the operation is over, otherwise, $k+1\to k$ , go to step2.
Step5:
After step4, $T_{k}$ is the optimal threshold.

Iterative threshold segmentation is an improvement on bimodal algorithm, which is easy to be optimized.
2.2.2 Otsu threshold segmentation algorithm

Otsu is proposed by the Japanese scholar Otsu. Its essence is the maximum inter-class variance algorithm based on grey image to find the global threshold. According to the particularity of the rubbings image [22], this paper adopts Otsu as initial binarization algorithm. And on this basis, the subsequent denoising and restore the image are carried out.

The grey level is set [0, …, $L$ ], the probability of the pixel’s grey level is $p_{i}=n_{i}/N$ , $N$ is the total number of pixels in the image. $C$ [0, …, $L$ ] is the total pixel of the image. Its total grey-scale mean is denoted as $\mu=\sum_{i=0}^{L}i\cdot p_{i}$ . If $T$ is the threshold, the pixels $C$ [0, …, $L$ ] are classified into two categories: $C_{0}$ and $C_{1}$ , which represent the background and foreground, $C_{0}=\{0,1,2,\cdots,k\}$ , $C_{1}=\{k+1,k+2,\cdots,L\}$ . The mean of $C_{0}$ and $C_{1}$ are denoted as $\mu_{0}$ , $\mu_{1}$ , the number of pixels respectively is $N_{1}$ , $N_{2}$ , the occurrence probability of $C_{0}$ is $\omega_{0}$ , the occurrence probability of $C_{1}$ is $\omega_{1}$ , then:

$\displaystyle\omega_{0}=p_{0}+p_{1}+\cdots+p_{k}=\sum\limits_{i=1}^{k}{p_{i}}=% \omega(k)$ (4) $\displaystyle\omega_{1}=\sum\limits_{i=T+1}^{J}i\cdot p_{i}=1-\omega(k)$ (5)

The average grey value of $C_{0}$ is

$\displaystyle\mu_{0}=\sum\limits_{i=0}^{k}i\times\frac{p_{i}}{\omega(k)}=\frac% {\mu(k)}{\omega(k)}$ (6)

The average grey value of $C_{1}$ is

$\displaystyle\mu_{1}=\sum\limits_{i=k+1}^{L}i\times\frac{p_{i}}{1-\omega(k)}=% \frac{\mu-\mu(k)}{1-\omega(k)}$ (7)

The mathematical expectations of two categories: $\mu_{0}\omega_{0}+\mu_{1}\omega_{1}=\mu$ .

Figure 3.

Threshold segmentation.

Figure 4.

Otsu threshold segmentation.

So the variance $\sigma^{2}(k)$ between-cluster can be get

$\displaystyle\sigma^{2}(k)=\omega(k)[\mu(k)-\mu]^{2}+\omega_{1}(k)[\mu_{1}(k)-% \mu]^{2}=\frac{[\mu\omega(k)-\mu(k)]^{2}}{\omega(k)[1-\omega(k)]}$ (8)

According to the standard of the maximum between-cluster variance, change $k$ from 1 to $L$ , and calculate the variance between the classes, then the $k$ corresponding to the largest variance, is the threshold $T$ of the region division.

$\displaystyle T=\arg\mathop{\max}\limits_{0\leqslant k\leqslant L}\sigma^{2}(k)$ (9)

As an effective way, histogram is often used to find the binarization threshold of grey image. If a histogram in grey image is displayed as two peaks, the threshold of binarization is between the two peaks. Figure 3 shows the image of the threshold segmentation result, (a) the global binarization segmentation based on the midpoint of the two peaks in the grey scale as a threshold, (b) an iterative algorithm, and (c) using the Otsu algorithm. Figure 4 shows the image of the threshold segmentation of Otsu.

3. Rubbing image binarization algorithm

The difference between the text and the background is larger and more obvious distinguish tablet images, we can directly use the Otsu [23] for the difference between the text and the background is larger and more obvious distinguish tablet images. For the text and the background is not much difference, the image is more noisy, simple Otsu segmentation can’t meet the requirements, as shown in Fig. 5.

Figure 5.

Otsu threshold segmentation method.

Figure 6.

Algorithm flow chart.

From Fig. 5, a lot of noise with illegible handwriting can be seen after binarization in the image, which caused great distress for us on protection, learning, researching the rubbing image. Because of the particularity of the rubbing image, usually there is much noise in the image after Otsu segmentation, and the text itself still have a lot of loopholes. The research on binarization restoration of digital inscriptions image is mainly focused on noise elimination, such as eliminating noise in image, smoothing the inscription image, and eliminating the “hole” in the text. Mean filter is the best linear filter using the mean square error criterion. It eliminates the image noise and blurs the edge of the target, weakening the details of the text, which is not conducive to the accurately extract the text. Choosing different nonlinear filters (morphological [24], median filtering), can remove the salt and pepper noise of document image. However, when large structural elements are used, morphological operations may damage the integrity of the edge of the text and graphics components, can also lead to edge degradation. Adaptive median filter can preserve the edge features but requires a long time [25]. After removing the noise, we need to extract the feature information of the image, and then take the appropriate method to binary the original image, the next problem is how to remove the noise while maintain the edge of the image.

In order to not only maintain the edge of the image characteristics, but also to eliminate background noise, remove the text inside the hole. After repeated experiments, mathematical morphology open operation (first corrosion denoising, re-expansion of the hole) is used for the initial filtering. Selecting the appropriate structural element can preserve the edge of the image, but there is a lot of noise after filtering. That is shown as in Figs 8 and 9. If the background noise is needed to remove more completely, we need to increase the structural element; this will destroy the image edge. In Figs 8 and 9, whether 3 $\times$ 3 median filter, or 5 $\times$ 5 mean filter, it can be seen from the figure, after filtering still left a lot of noise. The flow chart of algorithm is shown in Fig. 6.

3.1 Morphology algorithm

Morphology is composed by algebraic operators, its basic operations have expansion (or expansion), corrosion (or erosion) expansion after corrosion is the open operation; corrosion after expansion is closure operation. Opening operations generally smooth out the contours of the image, weakening the narrowed portion, and removing fine protrusions. Closed operation is also smooth the outline of the image, in contrast to the open operation, it generally fuses the narrow gap to remove the hole and fills the gap on the contour.

3.1.1 Corrosion

Corrosion is the most basic operation of morphology. Its operation is realized based on the meaning of filling structure unit. The process of using structure unit padding depends on a basic Euclidean space operation-shift [26]. The translational distance $x$ of a set $A$ can be expressed as $A+x$ , defined as:

$\displaystyle A+x=\{a+x,a\in A\}$ (10)

From a geometric point of view, $A+x$ represents the distance that $A$ translates along the vector $x$ , the purpose of the detection is to mark the position of the image, which can be filled into the structure unit.that $A$ along the vector $x$ translation a distance, the purpose of detection is to mark the position of the image that can be filled into structure elements unit. Set $B$ corrodes set $A$ can be expressed as $A\Theta B$ , defined as

$\displaystyle A\Theta B=\{X|B+x\subset A\}$ (11)

Equation (11) is a geometric sense of the definition of corrosion operations, there is a more important form of expression is defined from the arithmetic point of view.

$\displaystyle A\Theta B=\cap\{A-b|b\in|B\}$ (12)

As can be seen from Eq. (12), corrosion can be obtained by translating the input image by $-b$ ( $b$ is a structural element) and computing the intersection of all translations.

3.1.2 Expansion

Expansion is the dual operation of corrosion, which can be defined by the erosion of the complement. $A$ is expanded by $B$ and can be expressed as $A\oplus B$ , which is defined as:

$\displaystyle A\oplus B=[A^{c}\Theta(-B)]^{c}$ (13)

Where $A^{c}$ denotes the complement of $A$ , in order to use $B$ expanding $A$ , $B$ can be rotated 180 ${}^{\circ}$ relative to the origin and get $-B$ to corroded $A^{c}$ , corrosion results is the desired result of complement.

The equivalent of expansion is defined as:

$\displaystyle A\oplus B=\cup\{A+b,b\in B\}$ (14)

Taking into account the need to retain the text of the stroke information, the algorithm uses morphological open operation, which can not only remove a lot of noise, while preserving image edges. It is prepared for the next connected domain filtering.

By labelling the connected area of white pixels (objects) in a binary image, each individual connected area is identified as a block, and we can further obtain the block outline, rectangles, centroid, invariants And other geometric parameters. The filtering step of the connected region is as follows:

Step1:

Calculate the area of the image connected domain.

Step2:

Calculate the number of image connected domains.

Step3:

Find the filter threshold $T_{l}$ for more noise image, $T_{l}$ can be set to 1/2 the average area of the connected domain, and the connected domain pixels are set 0 if area below the threshold.

Step4:

The filtered image is filtered with a mean of 5 $\times$ 5 to further remove the burr.

Step5:

Output image.

Figure 7.

Segmentation effect of various algorithms.

Figure 8.

Experimental results.

Figure 9.

Experimental results.

4. Experimental results

Test is used Lenovo T460 notebook, CPU for Intel Core i5 6200U, 4G running memory, the algorithm runs on matlab2016a. Many new algorithms have appeared in recent years, but they are not well suitable for our rubbing images segmentation. Figure 7 shows several algorithms, including Otsu, median filtering, mean filtering and genetic algorithm. As can be seen, the segmentation effect is not ideal, there are many noises. Figure 8 from left to right, Fig. 9 from top to bottom, are the original colour image, threshold segmentation method of binary initial map, median filter, mean filter, morphological filtering and my method. From the figure we can see that this method is significantly better than other methods. Since there is no standard graphics library to verify the accuracy of segmentation, here we can see that the method used in this paper for complex images, segmentation effect is very good; almost all the noise is removed, and the edge of the text is retained.

5. Conclusions

Based on the threshold segmentation and the combination of the connected domain and the morphology, this paper segments rubbing image with a lot of serious noise, not only it removed much of background noise, but also fill in the small hole in the stroke, While preserving the edge of the image, and the original image is restored as much as possible. A set of algorithms associated with the pixel position of adaptive segmentation, morphological method, connected domain and mean filtering combined of binarization restoration are needed. The experimental results show that this algorithm is very effective for image with complex and large area noise.

Footnotes

Acknowledgments

The authors are thankful to the science and technology project of Jiangxi Provincial Education Department in 2016 (2016 GJJ16110) for providing the necessary funds for this work.

References

X.Z.

Shi

Z.Z.

and Jia

W.K.

, New theory and new method of image segmentation, Journal of Electronics 38(S1) (2010), 76–82.

Y.L.

Huang

Z.K.

et al., Research on Image Segmentation of Digital Rubbings Based on OTSU Threshold and Genetic Algorithm, in: 2017 5th International Conference on Mechanical, Automotive and Materials Engineering (CMAME), 2017.

Zhou

and Xia

Z.L.

, An improved Otsu threshold segmentation algorithm, Journal of China Jiliang University 27(3) (2016), 319–323.

Yang

and Lu

, Poor grey image threshold operator and improved Otsu algorithm combined with the segmentation and implementation, Instrument Technique and Sensor Research (3) (2015), 104–106.

Zhang

Gao

and Xu

, Image segmentation algorithm based on fusion threshold and edge detection, Journal of the Academy of Armored Forces Engineering (5) (2014), 73–76.

Zhang

Z.Q.

and Song

H.S.

, Application of Otsu improved Canny operator image edge detection method, Computer and Digital Engineering 42(1) (2014), 122–128.

and Cai

H.F.

, Watershed image segmentation based on morphological mark connected, Journal of Electronic Measurement and Instrument 25(25) (2011), 864–869.

Fan

J.L.

and Zhao

, An improved image segmentation algorithm based on C-mean clustering, Journal of Xi’an University of Posts and Telecommunications (5) (2014), 56–60.

Zhao

Cheng

and Cheng

, Neighbourhood weighted fuzzy C-means clustering algorithm for image segmentation, Iet Image Processing 8(3) (2014), 150–161.

10.

Zhong

C.F.

and Liu

, An improved image segmentation algorithm based on Genetic Fuzzy C mean clustering, Journal of Changchun University of Science and Technology (Natural Science Edition) (2) (2014), 62–67.

11.

Yin

Dai

and Yuan

, Identification method of freshwater fish species using multi-kernel support vector machine with bee colony optimization, Transactions of the Chinese Society of Agricultural Engineering 30(16) (2014), 312–319.

12.

Zhou

P.F.

and Zhang

Y.Q.

, Research on image segmentation method based on fuzzy BP neural network, Computer Engineering and Design (7) (2014), 2423–2426.

13.

Zhou

P.F.

, Research on image segmentation based on Improved Fuzzy BP neural network, Taiyuan University of Technology, 2014.

14.

Cai

Yang

Cao

Xia

and Xu

, A new iterative triclass thresholding technique in image segmentation, IEEE Trans Image Process 23(3) (2014), 1038–1046.

15.

Yuan

X.C.

L.S.

and Peng

, An improved Otsu method using the weighted object variance for defect detection, Applied Surface Science 349 (2015), 472–484.

16.

Adhikari

and Dasgupta

, A Local Adaptive Region-wise Histogram Correction and Thresholding Technique for Very Poorly Illuminated Images, IEEE, 2018.

17.

Liang

Lin

et al., eds. Document image binarization via optimized hybrid thresholding, in: IEEE International Symposium on Circuits & Systems, 2017.

18.

Ingle

P.D.

and Kaur

, eds. Adaptive thresholding to robust image binarization for degraded document images, in: International Conference on Intelligent Systems & Information Management, 2017.

19.

Vijayarani

M.A.S.D.S.

, A hybrid approach for document image binarization, in: Proceedings of the International Conference on Inventive Computing and Informatics (ICICI 2017), IEEE Xplore Compliant – Part Number, CFP17L34-ART.

20.

Wang

H.X.

R.F.

Gao

Y.F.

Cao

C.Q.

L.Z.

and Xie

, Target Recognition and Localization of Mobile Robot with Monocular PTZ Camera, Journal of Robotics, 2019.

21.

Y.L.

and Huang

Z.K.

, The study of segmentation of rubbings image based on OTSU & histogram differential algorithm, in: International Conference on Computer, Information and Telecommunication Systems (CITS), 2017.

22.

Chutani

Patnaik

and Dwivedi

, An improved approach for automatic denoising and binarization of degraded document images based on region localization, in: International Conference on Advances in Computing, 2015.

23.

Jin

and Song

, Characteristic analysis of Otsu threshold and its applications, Pattern Recognition Letters 32(7) (2011), 956–961.

24.

Hendriks

C.L.L.

Borgefors

and Strand

, Mathematical morphology and its applications, Signal and Image Processing 7883(4) (2015), 384.

25.

Abdeldayem

A.R.

Hamou

A.K.

and Elsakka

M.R.

, Novel adaptive filtering for salt-and-pepper noise removal from binary document images, Lecture Notes in Computer Science 3212 (2004), 191–199.

26.

Y.L.

Zeng

and Li

, License Plate Location Research Based on Texture Analysis & Mathematics Morphology, Advanced Materials Research, 2011.