In order to solve the problems of large error and low performance of traditional progressive image model matching information checking methods, an automatic progressive image model matching information checking method based on machine learning is proposed. The generation method of progressive image is analyzed, and the target image sample is obtained. On this basis, machine learning algorithm is used to segment progressive image samples. In each image segmentation part, crawler technology is used to automatically collect progressive image model matching information, and under the constraint of image model matching information checking standard, automatic checking of progressive image model matching information is realized from geometric structure, image content and other aspects. Experimental results show that the verification error of the design method is reduced by 0.687 Mb, and the quality of progressive image is improved.
Progressive image is a JPEG format image. Compared with ordinary image, progressive image first shows the fuzzy outline of the whole image. With the increase of scanning times, the image becomes clearer and clearer. Its main advantage is that when the network speed is slow, the outline of the picture is first seen, so as to know the picture information being loaded in advance. From the compression principle, the common image compression method is from left to right and from top to bottom line by line, while the progressive image compression method is based on wavelet transform, which stores the outline content of low-frequency image first, and then stores the details of high-frequency image. In this way, the image drawing process is a process from blur to clarity. When that network traffic is limit, the progressive image can still be downloaded, thus achieve the purpose of reducing the image resolution and traffic.
Because progressive image adopts the form of pattern matching in the loading process, that is, under the fixed image model, the number of pixels is increased according to the position of image pixels, and the definition of image pixels is improved, which produces progressive pattern matching information [1]. Image pattern matching is an important and basic problem in the field of computer vision and pattern recognition. Its main task is to match and transform the points in two images of the same scene which satisfy certain geometric transformation relationship, so as to recognize and locate the target object in the scene. In the existing literature, there are many research methods of point pattern matching, and there are many classification methods. According to different solving methods, the research methods of point pattern matching can be roughly divided into: iterative relaxation based method, Hausdauff distance based method, clustering based method, geometric invariant based method, spectral graph theory based method, computational geometry based method, neural network based method, bionic computational intelligence based method, etc. According to the different geometric transformation relations and application approaches between point sets, point pattern matching algorithms are divided into rigid body/affine transformation method, non-rigid body/elastic body transformation method and dynamic sequence method [2]. However, due to the influence of environmental factors, the above-mentioned progressive image matching methods may produce errors in the implementation process, resulting in information errors in the final display results of progressive images. Therefore, it is necessary to automatically check the image matching information in the progressive image generation and loading process.
The research status of image model matching information detection methods at home and abroad is analyzed. Reference [3] puts forward a hybrid invariant local feature description method for image registration, which is composed of binary robust invariant extensible key points, accelerated robust features and accelerated segment test features, thus extracting corresponding relations from multiple sets of features and realizing image processing. Reference [4] proposes to use adaptive redundant key point elimination method, automatically adjust the threshold and remove redundant key points, and determine the maximum value of the threshold combined with histogram, so as to realize image matching with scale-invariant feature transformation. However, the traditional detection methods have serious detection errors, which leads to serious quality problems of the final progressive image. In order to solve a series of problems existing in traditional detection methods, a machine learning algorithm is introduced, and a progressive pattern matching information checking method is proposed. An encoder is compiled by using a cyclic neural network to generate a progressive image. Using neural network algorithm in machine learning to segment images; Calculate the maximum similarity for coarse matching and fine matching of image model; Set the calibration standard of registration information, with the similarity above 0.6 and the difference above 0.5; UKF filter is used to filter matching information, which reduces noise interference and improves the calibration effect and accuracy of automatic calibration method.
Design of automatic verification method for progressive image model matching information
Generating progressive images
In the process of progressive image generation, firstly, the encoder uses convolutional neural network to extract the features of three different positions of the image, and uses attention mechanism to determine the coding position and content of the current step at different time steps. The decoder uses convolutional neural network, so that continuous coding samples can transfer information between the encoder and the encoder. The encoder at the current time is affected by the decoder output at the previous time; Secondly, the output of the encoder is continuously added to the distribution used to generate data, unlike other variational automatic encoders, which generate the whole picture at one time; Finally, attention mechanism is introduced into encoder and decoder. In the encoding phase, attention mechanism is used to limit the encoding position of each time step. In the decoding phase, attention mechanism is used to determine the reconstruction position and content. At the initial time, saliency detection algorithm is used to determine the position of the main information of the picture, which provides the initial state position information for the encoding and decoding network. The specific architecture of progressive image generation is shown in Fig. 1.
Block diagram of progressive image generation.
In the figure, the dotted line above is the decoding part, which is the model actually used to generate the picture. It extracts the noise z by probability , and finally generates picture by conditional probability . The part below the dotted line is the coding part, which is used to help train the model. There are two purposes: One is to ensure that the output of the decoder is consistent with the input of the encoder as much as possible, and the other is to make the probability of the sampling layer as close as possible to the probability P(z) of the extracted noise. Read operation is added to the coding network, and write operation is added to the decoding network. They are different representations of attention mechanism in the coding and decoding network. Among them, the read operation is to enable the encoder to encode different image positions with emphasis and selection in each time step. The same write operation is to limit the position and content of each time step decoding output. Firstly, convolutional neural network is used to extract the features of the input image X, and three feature vectors a of different positions are obtained, in which the length of each feature vector is D. Then one or more of the three position features are selected for encoding by read operation in each time step. The convolutional neural network adopts the long-term and short-term memory model, which is the hidden layer output of the decoding network, that is, the hidden vector. Each cycle will accept the hidden vector of the previous time as the input parameter to help generate new output [5]. At each time step t, the encoder receives the picture X and the decoder hidden vector of the previous time as the input. Meanwhile, the accurate form of the encoder input mainly depends on the read operation. The output vector of the encoder is used to determine the parameters of the distribution of the hidden space variable . In the proposed network model, the hidden space distribution is defined as skew Gaussian :
In the automatic encoder, Bernoulli distribution is used more than Gaussian distribution, but a great advantage of Gaussian distribution is that the gradient of sample function obtained from distribution parameter sampling is easier to obtain by means of re parameterization [6]. At each time , the model will do the following operations in a cycle:
In the formula, the generation of the output of the encoding part is affected by the at the current time and the picture feature a, which is actually the weighted sum of the picture feature vectors a at three different positions. is the output of the read layer, which is jointly determined by the of the previous time and the decoding hidden state . represents the weight of the image feature at the ith position at the time t when the image is generated [7]. In each time step, a sample will be sampled from the hidden distribution and transmitted to the decoder, that is to say, the input image will be encoded into a Gaussian distribution hidden space. Then a sample is sampled from this distribution and sent to the decoder for decoding. Finally, the output of the decoder is added to the canvas matrix through the write operation, where represents the result of a painting, and a complete picture is finally generated by iterating this process [8].
Segmentation of progressive image using machine learning algorithm
Generally, for a machine learning classifier, the input is the mapping of the original sample data in the feature space, and the output is the class label. Therefore, for progressive image segmentation task, to use machine learning to predict tumor region, the first task is to extract features [9]. In order to make the extracted feature distribution more discriminative, it is necessary to normalize the original image before extracting features. The progressive image segmentation framework based on method of this paper is shown in Fig. 2.
Image segmentation block diagram based on machine learning algorithm.
The incremental image is used as the input of the machine learning algorithm, and the input image is set as the feature vector in the d-dimensional space, and the output y is the prediction label. Suppose that the parameter space of machine learning algorithm is , then the prediction model is as follows:
The goal of machine learning algorithm training is to find the optimal parameters [10]. The neural network algorithm of machine learning algorithm is adopted, and the calculation formula of neuron is defined as:
Neuron is the basic unit of neural network. The process of neuron processing information is carried out according to certain rules and principles. Firstly, the neuron inputs the connected upper layer neurons to carry out the weighted sum operation; Then, a certain deviation processing is set in the previous calculation results to ensure that the results are more accurate and reasonable; Finally, through the specific function processing, and then output the results. Neural network training needs specific methods, so as to ensure the effectiveness of training [11]. In the first step, normalization is needed. Normalization can reduce the range of data, the activation function suitable for network training can improve the speed of network training. The second step is to set the weight, deviation and starting value. The third step, start training, through the input of the upper layer, calculate the output of the next layer, one layer by one, until the final result is output. The fourth step is to compare the gap between the expected value and the output value, adjust the weight and deviation, and train repeatedly to achieve the desired effect.
Automatic collection of progressive image model matching information
In the process of image pattern matching, the size of the template is , and the mask sizes of the template, interleaved template, record matrix and interleaved template are all . The specific matching process of image mode is shown in Fig. 3.
Flow chart of image mode matching.
Progressive graph pattern matching can be roughly divided into two steps, namely coarse matching and fine matching. In the process of coarse matching, the root node of the current document pattern and the root node of the i-th pattern in the pattern library are read respectively [12]. According to Eq. (5), the similarity between image pattern and root node is calculated.
In Eq. (5), the parameters and represent the structure sequence and aspect ratio of progressive image respectively, and pn is the number of image regions. If the calculated is greater than the set threshold, is selected as the candidate matching pattern. Judge whether is less than , so as to judge whether to end the rough matching of patterns [13]. The input of pattern fine matching process is the pattern t to be matched and the candidate pattern set M1. The similarity of the corresponding pattern tree of pattern and pattern is calculated, and the best matching pattern is the one with the largest similarity [14]. In the process of progressive pattern matching, crawler technology is used to collect the real-time registration data, which is used as the initial data of registration verification.
Setting the calibration standard of image model matching information
Progressive image model matching information verification combines with the information content in the image for specific verification, so it is necessary to formulate certain verification standards to ensure the correctness of the verification results [15]. On the basis of the current progressive image loading results, the similarity between the registration information and the known information data should not be less than 0.6, and the difference between the amount of registration information and the known information of the pixel should not be less than 0.5. The content of registration information must be based on geometric structure and objective theory.
Realize automatic information checking
According to different types of progressive image model matching information, different checking procedures are executed. Firstly, a UKF filter is embedded in the progressive image generator. UKF filtering is a kind of nonlinear filtering method based on unscented transformation, which directly uses the nonlinear model to avoid the introduction of linearization error, so as to improve the filtering accuracy. The UKF method has a good effect on nonlinear state estimation with large misalignment angle. However, when the matching information is lost, the standard UKF method will increase the estimation error due to substituting the wrong measurement update information into the update process of state estimation and estimation mean square error matrix [16]. For this reason, when there is packet loss in the process of data transmission, by setting a binary control variable , when there is observation data in the k-th step in the filtering process, the value of is 1; If the observation data in the k-th step is lost, the value of is 0, and then the filtering update process is processed accordingly. The nonlinear checking equation is expressed as:
where and are system noise and measurement noise respectively. After initialization, allocation prediction and update, the corresponding filtering steps are processed by judging whether the matching information is lost [17]. That is, when the matching information of the primary INS is transmitted to the sub ins at the k-th moment, that is, the matching information is not lost, the value of is 1, and the filtering steps are the same as those of the basic UKF. When the matching information of the master inertial navigation system is lost in the transmission at the kth moment, is 0. At this time, one-step prediction of the state and the mean square error matrix is used to replace their estimated values respectively, and the wrong matching information is not used to update the filter state [18]. Through the above operation, the adjustment error caused by environmental factors can be basically solved. For the distortion of geometric information and the content information of the image adjustment, further check is needed. Taking the tilt correction in several distortion of the adjustment information as an example, the checking process can be expressed as follows:
The checking principle of Eq. (7) is to reflect the content of progressive image as the trend of straight line through the method of line detection. The transformation in Eq. (7) maps in Cartesian coordinate system to point in Hough space, and maps to the same point after transformation. All pixel points on the layout are mapped to Cartesian coordinate system to obtain a series of linear clusters. The most angle of lines is considered as the inclined angle of the layout. Then, the local adaptive method is used to check the content information of the coordination. Firstly, the neighborhood range m is determined, and the binary threshold of pixels is calculated in M range. The average gray level of the region is used to subtract the fixed gray offset. The regional average gray information can often reflect the illumination of the region, so the use of small area gray average can effectively eliminate the impact of uneven illumination, and the global use of the same gray offset C can ensure the global consistency to a certain extent [19]. The specific calculation formula is shown in Eq. (8).
where is the neighborhood with as the center and r as the radius, IV is the total number of pixels in the neighborhood, and C is the offset constant of the threshold [20]. Finally, a comprehensive verification result of the progressive image model matching information is obtained by synthesizing various verification operations.
Verification performance test experiment
According to the application performance of this method in actual image processing, the detection performance test experiment is designed.
Set up performance test environment
The purpose of the verification performance test is to find as many running defects as possible, find out the errors in each stage of the verification program development cycle, analyze the nature of the errors and correct them. The experimental environment of performance test is divided into two parts: hardware and software. The hardware part includes the system working host, multiple method running application computers, and is set as administrator host, database host and entry host. In terms of software, MySQL 5.0 database is connected to the main test computer, and the operating system is Windows XP or higher.
Prepare progressive image samples
The progressive image samples used in the experiment are from MINIST database, PASCALVOC database and CIFAR-10 database. MNIST is the standard data set for machine learning, with 10 categories and 0–9 handwritten digits. It contains 60000 binary training images of 28 28 and 10000 binary test images of 28 28. PASCAL VOC data set includes 20 categories: human; animals (birds, cats, cattle, dogs, horses, sheep); vehicles (planes, bicycles, boats, buses, cars, motorcycles, trains); indoor (bottles, chairs, dining tables, potted plants, sofas, television). PASCAL VOC dataset has good image quality and complete annotation, which is suitable for testing algorithm performance. In addition, CIFAR-10 database contains 10 categories, 60000 32 32 color images, 6000 images per category. Among them, 50000 are training pictures and 10000 are test pictures. In the three databases, 5000 images were randomly selected to establish a data set, and 80% of the images were randomly selected for training. The number of iterations was set to 800, the initial learning rate was set to 0.1, and the learning rate change factor was set to 0.2. After training, the remaining 20% images were tested as a test set. The progressive images in the test set are divided into 8 groups, and the number of samples in each group is not fixed. The progressive images are divided into 8 groups, and the number of samples in each group is not fixed. In addition, in order to ensure the efficient operation of the verification method, the format and size of the prepared progressive image samples need to be processed in batch.
Set check performance test index
In order to ensure the credibility of the test results, the calibration error, image quality and processing speed are set as the three quantitative indexes. The image quality and processing speed are the test indexes to check the application performance. Check error is the amount of error data that still exists in the image model matching information after the check is completed. The quantitative data result of check error can be obtained by comparing the error amount with the check data amount. To check the evaluation of image quality, we need to get the conclusion from the image resolution, the number of distorted pixels and other aspects. The higher the image resolution is, the better the image quality is. The more the number of distorted pixels in the image is, the worse the image quality is. In addition, the verification processing speed can be directly obtained by reading the background data in the main test computer, and the final quantitative test results can be obtained by calculating the time difference between the progressive image samples and the output verification results.
Verification performance test process description
The method in this paper is transformed into program code and imported into experimental environment, and the direct operation interface of the verification method is.
In this interface, the image samples are input in turn, and the check results are obtained by running. In order to reflect the advantages of the design verification method, the methods in Reference [3] and Reference [4] are taken as two comparison methods in the experiment, which are imported into the experimental environment in parallel, and the image samples processed by the three verification methods are ensured to be the same, so as to control the uniqueness of experimental variables.
Verification performance test results
Through the statistics of relevant data, the test results of verification performance are obtained respectively, and the test comparison results of verification error are shown in Table 1.
According to the analysis of the data in Table 1, the average verification errors of Reference [3], Reference [4] and the method in this paper are 0.825 Mb, 0.450 Mb and 0.138 Mb, respectively. It can be found that the method in this paper has high accuracy and can accurately verify the matching information of image models.
Combined with the verification results of progressive image model matching information, the progressive image is adjusted, and the application performance test results including image quality are obtained, as shown in Table 2.
Application performance test results of verification method
From the test results in Table 2, it can be seen directly that the resolution of all images can be guaranteed to be 1280*1280 after image processing by using the method in this paper, while the errors of the methods in Reference [3] and Reference [4] are large, resulting in lower image resolution. According to the number of distorted pixels, the average values of distorted pixels corresponding to the methods in Reference [3], Reference [4] and this paper are 155.2 pixels, 113.9 pixels and 53.1 pixels, respectively. Summarizing the results of image resolution and distortion rate, it can be shown that the proposed method improves the image quality. The results in Table 2 also show that the average verification time of Reference [3], Reference [4] and this method is 5.75 s, 4.04 s and 2.11 s respectively, which means that the processing speed of this method is faster.
Conclusion
In this paper, the neural network algorithm of machine learning algorithm is used to verify the registration information of progressive images, and an image pattern matching method with high efficiency and accuracy is designed, which has certain application value and can provide some help for image processing. However, it can be seen from the experimental results that the verification error of the design method is lower than that of the traditional method, but it is not close to zero, so it needs to be further optimized in the future research work.
Footnotes
Conflict of interest
Guizhou Power Grid Co., Ltd. technology projects for No. GZKJXM20190715.
References
1.
VergariCGajnyLCourtoisIEbermeyerESkalliW. Quasi-automatic early detection of progressive idiopathic scoliosis from biplanar radiography: A preliminary validation. European Spine Journal.2019; 28(9): 1970-1976.
2.
KhelifaMBoughaciDAmeurE. A new approach based on graph matching and evolutionary approach for sport scheduling problem. Intelligent Decision Technologies.2020; 14(1): 1-16.
3.
ChandrappaDNAnilNS. Satellite image matching and registration using affine transformation and hybrid feature descriptors. International Journal of Advanced Intelligence Paradigms.2021; 1(1): 1.
4.
Hossein-NejadZAgahiHMahmoodzadehA. Image matching based on the adaptive redundant keypoint elimination method in the SIFT algorithm. Pattern Analysis and Applications.2021; 24(2): 669-683.
5.
PassiasPGPierceKENaessigSAhmadWLafageV. 291. Adjustment of the global alignment and proportion scores accounting for frailty in adult spinal deformity surgical patients. The Spine Journal.2020; 20(9): S144-S145.
6.
ChirkovONTsipinaNVSlinchukSAVorobyevEI. Automated expert support complex based on a machine learning semantic processor. Journal of Physics: Conference Series.2020; 1691(1): 012062.
7.
ZhaoJ. Efficiency of corporate debt financing based on machine learning and convolutional neural network. Microprocessors and Microsystems.2021; 83(1): 103998.
8.
IslamARMTTalukdarSMahatoSZiaulSLinhNTT. Machine learning algorithm-based risk assessment of riparian wetlands in Padma River Basin of Northwest Bangladesh. Environmental Science and Pollution Research.2021; 28: 34450-34471.
9.
Revel-VilkSChodickGShalevVGadirN. Study design: Development of an advanced machine learning algorithm for the early diagnosis of Gaucher disease using real-world data. Blood.2020; 136(1): 13-14.
10.
BonakdariHJamshidiAPelletierJPAbramFMartel-PelletierJ. A warning machine learning algorithm for early knee osteoarthritis structural progressor patient screening. Therapeutic Advances in Musculoskeletal Disease.2021; 13: 1-16.
11.
KalpanaJJKVenkatalakshmiK. Medical image pattern matching process using expectation maximization with fuzzy rules. Journal of Medical Imaging and Health Informatics.2019; 9(3): 495-501.
12.
KhaninezhadRGolmohammadiAJafarpourB. A pattern-matching method for flow model calibration under training image constraint. Computational Geosciences.2019; 23(2): 813-828.
13.
FanDZDongYZhangYS. Satellite image matching method based on deep convolutional neural network. Acta Geodaetica et Cartographica Sinica.2018; 47(6): 844-853.
14.
JeonHGParkJChoeGParkJSBokYTaiYWKweonIS. Depth from a light field image with learning-based matching costs. IEEE Transactions on Pattern Analysis and Machine Intelligence.2019; 41(2): 297-310.
15.
DhimanSChawlaRGuptaS. A novel video shot boundary detection framework employing DCT and pattern matching. Multimedia Tools and Applications.2019; 78(24): 34707-34723.
16.
ChangHLSuKYGoodmanSDChouNALinKCChengWCLinLIChangSYFangWH. Proofreading of single nucleotide insertion/deletion replication errors analyzed by MALDI-TOF mass spectrometry assay. DNA Repair.2020; 88: 102810.
17.
BartlettDWGilbertAM. A kinetic proofreading model for bispecific protein degraders. Journal of Pharmacokinetics and Pharmacodynamics.2020; (10): 1-15.
18.
AlhudhaifACmertZPolatK. Otitis media detection using tympanic membrane images with a novel multi-class machine learning algorithm. PeerJ Computer Science.2021; 7(7): e405.
19.
MauryaRSinghNJindalTPathakVKDuttaMK. Machine learning-based identification of radiofrequency electromagnetic radiation (RF-EMR) effect on brain morphology: A preliminary study. Medical & Biological Engineering & Computing.2020; 58: 1751-1765.
20.
XuSWangJZhuQ. Progressive image painting outside the box with edge domain. Journal of Intelligent and Fuzzy Systems.2020; 39(1): 371-381.