An effective latent fingerprint enhancement and recognition system using dictionary learning and LCPnet mechanisms

Abstract

Latent fingerprint recognition plays an essential role for law enforcement agencies to detect criminals and security purposes. One of the key stages utilized in the latent fingerprint recognition model is to automatically learn consistent minutiae from fingerprint images. However, the existing state-of-the-art recognition approaches are not adequate since live-scan fingerprint images and enhancements are necessary for each step of the recognition process. Hence, an automated recognition system along with appropriate minutiae learning algorithm is required for matching the latent fingerprint exactly. In this paper, an efficient recognition system using dictionary learning and Local Context-Perception deep neural network (LCPnet) has been proposed to enhance the accuracy of latent fingerprint recognition. Primarily, the Total Variation decomposition model is utilized to remove the smooth background noise and dictionary learning contributes to the extraction of multiple patches. Afterward, the LCPnet is trained for 12 patch types to develop a salient minutiae descriptor where every descriptor is trained using LCPnet with a particular patch size at a location surrounding the minutiae. The proposed detection system has been tested through two latent public datasets. Here, three different types of templates (LCPnet minutiae, LCPnet texture, and LCPnet minutiae+texture) are analyzed for evaluating the proposed fingerprint detection system. The performance results manifest that the proposed system acquires a superior recognition accuracy of 99.44% and 99.58% under two different datasets.

Keywords

Latent fingerprint dictionary learning ridge enhancement minutiae extraction deep neural network

1 Introduction

In the recent era, the latent fingerprint matching technology system has been highly essential for security and investigation purposes. Latent fingerprints are collected from the location where the crime took place [1]. Among those impressions, some impressions of the skin of the finger are left at the spot of the crime incident by the person involved in the crime, such incidental impressions of fingers may occur at the surface of an object. The impressions are taken from the crime spots are generally not clear with a smudgy and noisy background. The quality of the latent image is very poor and the particulars collected are not trustworthy [2].

At present, many biometric techniques are successfully developed such as face recognition, iris recognition, hand geometry, palmprints, and signature. Still, Latent fingerprint identification is a superior technique because of its uniqueness, distinctness, universality, and invariableness [3]. The fingerprint patterns will not change during the life period of any person except if any wounds happen at the fingertip. At the fingerprint, the features are extracted in various categories such as frequency map, orientation map, pores, singular points, dots, etc.

The three major pattern classifications in the fingerprint are Arch, Loop, and Wheel. These patterns are further categorized into the arch, tent arch, double loop, left loop, right loop, left pocket loop, right pocket loop, and mixed figure. During the enrolment, the quality of fingerprints should be taken through the inking or scanning method. The fingerprint recognition process is associated with a variety of tasks such as identification, feature extraction, indexing, verification, and classification [4].

In the case of recognizing such latent fingerprints, the images have to be segmented and enhanced before doing the similarity matching. For segmentation purposes, the latent fingerprints are separated into background and foreground images. In the background image, the noises and unwanted contents are present. But, in the foreground image, the valid information is available at the Region of Interest (ROI). Similarly, enhancement is significant to remove the over looping and recover the corrupted ridge structures [5]. Table 1 enumerates the major terms and Abbreviations utilized in the proposed work.

Table 1
Key terms and abbreviations

Notation Abbreviations

LCPnet Local Context-Perception deep neural network

ROI Region of Interest

TV Total Variation

SSIM Structural Similarity Index Matrix

CSSC Convolution Structure Sparse Coding

NIST SD National Institute of Standards and Technology - Special Database

WVU West Virginia University

CNN Convolutional Neural Network

SSD Single Shot Detector

SVD Singular Value Decomposition

LCP Local Coherence Pattern

RELU Rectifier Linear Unit

Notation	Abbreviations
LCPnet	Local Context-Perception deep neural network
ROI	Region of Interest
TV	Total Variation
SSIM	Structural Similarity Index Matrix
CSSC	Convolution Structure Sparse Coding
NIST SD	National Institute of Standards and Technology - Special Database
WVU	West Virginia University
CNN	Convolutional Neural Network
SSD	Single Shot Detector
SVD	Singular Value Decomposition
LCP	Local Coherence Pattern
RELU	Rectifier Linear Unit

In general, Fingerprints are defined through two phases where the first phases are described as topological or global features [6]. The stated features have a rough ridge line and singular points which are helpful for the fingerprint categorization. However, they cannot be utilized for accurate and operative matching. A second phase will be established by the occurrence of local ridge parts known as minutiae. Minutiae are described using the ridge endings and the divergences. Minutiae are normally the significant information in a fingerprint that is exploited to attain reliable results while a minutiae mining is considered. Nevertheless, the detection process is a challenging task owing to the association of low-quality fingerprints in the case of latent fingerprint or rolled fingerprint [7].

Typically, the input images comprise texture and noise. The texture is considered as a repetitive and influential structure of small patterns whereas noise is considered as unrelated random patterns. The remaining portions of the input image are called cartoon that encompasses object hues and shrill boundaries. The low-quality images yield false minutiae points during the output phase and it also avoids some valid minutiae points which affect the performance of the recognition process. Thus, a reliable fingerprint enhancement model is necessitated for enhancing the low-quality images and improving the accurate identification of minutiae points.

In the last few decades, many researchers have developed various enhancement algorithms for increasing the quality of fingerprint images [8]. Some of these algorithms are designed based on spatial and frequency domain models. But, an individual level processing either in spatial or in the frequency domain is not sufficient because of the variance in ridge arrangement of individuals. To address this constraint, a few modern models have employed a hybrid technique that combines both the spatial and frequency domain models to acquire superior enhancement outcomes.

In [9], the segmentation model has been suggested that are based on the accurate mining of background information. This model formulates three minutiae templates and one texture template applied in both NIST 27 and WUV datasets. It is relatively problematic to exactly learn ridge orientation and frequency detail from lower quality fingerprint images. Owing to the inaccurate evaluation of this information, enriched fingerprint images can yield some false minutiae points and avoid some valid minutiae points. Most of the stated methods offer sufficient enhancement for higher-class fingerprint images but they may not be capable to provide appropriate enhancement for lower-class fingerprint images [10]. Consequently, the key objective of the proposed work is to develop a superiority enhancement model for fingerprint images of different classes (high and low).

In the proposed scheme, a novel identification based on deep learning and LCPnet is introduced to identify the fingerprint enhancement and recognition. Since fingerprint matching plays a vital role in theft crime cases, achieving better precision is important. The features in the latent fingerprint can only be physically recognized by skilled experts. Therefore, the quality of the initial images should be upgraded to get a superior image. For which, the contrast enhancement part does the major role. Otherwise, further process like feature extraction and similarity matching finds difficult. Hence, in this work, feature extraction is performed before the matching technique where the similarity matching algorithm is executed to find the exact matches during the classification phase.

The novel contributions of the current stated work are as follows:

The proposed Total Variation (TV) decomposition methodology has been employed in the enhancement phase to remove the smooth background noise present in the input images. It also assists to achieve a perfect extraction of ROI, ridge flow, and ridge spacing.

A new Structural Similarity Index Matrix (SSIM) score is computed between the corrupted input and references latent image in order to obtain the enhanced image.

In order to develop a salient minutiae descriptor, the robust LCPnet is trained for 12 patch types and each descriptor is trained using LCPnet with a particular patch size at a location surrounding the minutiae.

A proper feature extraction strategy can be selected in the proposed detection system so that only 3 out of the 12 LCPnets are sufficient to attain a superior identification accuracy with the reduced computational burden.

A new similarity matching has been introduced in the proposed system where one texture template and one minutiae template are learned for every latent. By selecting each minutia and texture template from the total templates, the best recognition accuracy is achieved.

The remaining section of this proposed work is formatted as follows: The existing works related to fingerprint enhancement and recognition are discussed in Section 2. Section 3 explains the proposed methodology and its steps in detail. Section 4 enumerates the simulation results and discusses the validated performances. Finally, Section 5 provides the conclusion of the proposed work.

2 Related work

The following section describes the various existing works related to fingerprint enhancement and recognition models. Zhang et al. [11] have proposed a novel method in image fusion based on Convolution Structure Sparse Coding (CSSC). Initially, the sparse coding is combined with the degradation relationship of multispectral and panchromatic images. The restoration model is then established to show the correlation in multispectral hands. Finally, image fusion took place by computing the feature maps through alternative optimization. However, the CSSC model failed to attain better recognition accuracy due to its complex convolution structure in the evaluation phase.

In [12], an end-to-end fingerprint separation using deep learning (FinSNet) has been introduced to address the detection issues. Here, a network is trained using a neural network and the real fingerprints are separated from the dataset fingerprints. The FinSNet employs artificial fingerprints and managed to form a training dataset. Finally, this method is evaluated using the software of fingerprint identification. Nevertheless, the FinSNet model is not able to implement efficient enhancement algorithms for enhancing the lower quality figures.

In [13], a new Feedback Framework (FF) model has been established to separate and identify latent fingerprints from complex backgrounds. A dual mechanism is applied to enhance the ridge quality of the input image. In the first mechanism, the large quality area is enhanced according to the priority and these high-quality areas are feedback to improve the nearby areas. A spectral auto-encoder will be implemented in the second mechanism for learning the good fingerprint spectrum and this area is feedback to mechanism one to further improve the enhanced output. But, the FF model lagged to restore and keep vital parameters for enhancement because the latent fingerprints normally consist of complex background and noise.

Deerada et al. [14] have introduced a Systematic Feedback (SF) model that estimates a reliable pose to find the best solution to the corrupted information in the latent fingerprint. The automated algorithm will locate the potential pose and enhance the poor edge ridges. Experiments were conducted on two different datasets. The SF model produces an improved accuracy with the aid of an automated algorithm. Nonetheless, the SF model does not detect the weak friction ridges which degrade the performance of the detection system.

Gu et al. [15] have discussed a Non-Minutiae Registration (NMR) model for identifying the latent fingerprint. Initially, the special transformation is estimated using a dense fingerprint patch algorithm and matching procedure. Here, the minutiae extraction step is eliminated. The highlighted patch alignment and algorithm for matching will compare all pairs of sampling points. Finally, the set of consistent correspondence is formed through spectral clustering. This spectral clustering offers better detection accuracy and lesser computational complexity. At the same time, the NMR model is failed to obtain superior recognition efficiency.

In [16], a Fingerprint Quality Assessment (FQA) model has been established to assign different qualities of class for recognizing the fingerprints. Primarily, each quality class has been assessed using a two-phase fingerprint enhancement mechanism. In the first phase, a quality adaptive pre-processing mechanism is exploited to alleviate the unwanted noise information from the fingerprint dataset. Afterward, the pre-processed fingerprint datasets are enriched by implementing adaptive enhancement techniques. A new statistical-based feature forecasting technique is invoked to acquire the distinct discriminative attributes that can exactly distinguish fingerprint images according to their features. The FQA considers seven different features to group the fingerprint datasets into appropriate quality classes. Nevertheless, the consideration of these seven features makes the suggested detection system more complex in nature.

In [17], a Convolutional Neural Network (CNN) model has been presented to retrieve fingerprint edge structures from corrupted input images. The CNN model comprises two fields that recreate the input image and orientation field concurrently. The superior fingerprint is further developed with the assist of orientation field details. Further, a corrupted fingerprint dataset is created to enable the training and testing phase of the CNN model. The anticipated model produces a better performance by retrieving the fingerprint edge structures in the input images. However, the CNN model is not able to retrieve some fragments of the fingerprint region when the input image contains poor contrast and noisy content.

In [18], a robust latent fingerprint enhancement model based on the FingerNet mechanism has been proposed to address the fingerprint detection constraints. The FingerNet model can be influenced by multi-layer CNN. The anticipated model comprises three different levels: one shared convolution level and two dissimilar deconvolution levels that include the enhancement field and the orientation field. The convolution level is to learn fingerprint attributes predominantly for enrichment determination. The enhancement deconvolution field is utilized to mitigate structured noise whereas the orientation deconvolution field accomplishes the task of controlling enhancement using a multi-task mining technique. Nevertheless, the FingerNet model does not extract the poor-quality images which cause the overall performance of the suggested system.

According to [19], the detection accuracy of latent fingerprint recognition is obtained by using extended features which are marked manually. It includes minutiae, ridge shaping, ROI, ridge flow, and skeleton. Still, there are some disadvantages like more time consumption and finding difficulty in poor quality latent images.

Deshpande et al. [20] have introduced a latent fingerprint recognition technique using the clustered minutiae patterns. Initially, the latent minutiae attribute set is formed from the minutiae features. Later, the hash table is constructed by the proposed hashing method. Finally, a fingerprint retrieval procedure is adopted to match the query latent fingerprints. This method shows an improved performance in terms of matching speed and accuracy.

In [21], a latent fingerprint matching algorithm has been discussed to detect the fingerprint in which a local and global matching technique is implemented and results are evaluated. While testing with the common dataset NIST SD-27(A) the identification accuracy is 74%. Still, there are some disadvantages like the manual marking of ROI and skeleton-based minutiae extraction. Additionally, this method introduces a large amount of spurious minutiae and texture templates. In order to fulfill the challenges in the minutiae-based approaches, orientation-based regulation is focused on [22].

Paulino et al. [23] have discussed a new fingerprint matching technique that is developed for matching the latent. A descriptor-based Hough Transform is designed to arrange the fingerprints and to measure the similarity between the fingerprints. Two latent databases: NIST SD-27 and WVU are employed to test and this method shows a better performance than the other three fingerprint methods. As a result, in the proposed scheme both frequency and orientation estimation are collectively implemented in the feature extraction stage.

In the above-discussed literature, the estimated minutiae are very few due to minor operative regions and imprecise location or direction in the latent fingerprints. In case the fingerprint images are significantly mixed with the background noise, it finds challenging to meet the latent fingerprints with poor quality. Moreover, due to the wrong evaluation of direction in virtual minutiae, it will much disturb the output of virtual descriptors. If the descriptor is learned from the local image patches, it gets aligned based on the direction.

In many literatures, the extended latent features are manually marked and if latent is of low quality, then it consumes more time and sometimes it might not be feasible. They have not applied any correlation between ridge frequency and ridge orientation. Most of the existing models do not able to provide optimum enhancement and recognition mechanisms for poor as well as high-quality fingerprint images. To overcome these issues, an efficient LCPnet-based detection system has been proposed in this work. The proposed system employs the proper similarity matching algorithm to find the exact matches during the classification phase.

3 Proposed system

The proposed system consists of six phases: Noise removal, patch extraction, feature extraction, enhancement, LCPnet, and Similarity Matching Phase. Figure 1 demonstrates the sequential flow of the proposed system in which various functions are performed under matching of the latent fingerprint. Based on the proposed approach, the input image is initially subjected to noise removal using the TV decomposition technique. Simultaneously, dictionary learning is applied to the reference image for the extraction of patches. Both outputs from noise removal and patch extraction are subjected to feature extraction. In this stage, the orientation and frequency field components are extracted. Hence, the enhanced latent fingerprint is achieved as the output of feature extraction.

Fig. 1

Sequential flow of the proposed system.

Then, the proposed LCPnet has been executed to design and train the network to obtain the trained images. Finally, the extracted images are trained images that are fused through the image fusion technique i.e., Single Shot Detector (SSD). Thereby, the matched latent fingerprints are developed from the corrupted latent fingerprints.

3.1 Noise removal phase

For the removal of noises in the latent fingerprint images, the TV decomposition model is more suitable to remove the smooth background noise which is shown in Fig. 2. Initially, the input image (y) is categorized into blocks of 16 × 16 patches P_I. We decomposed input image y into two components: y = u+v. Here, u denotes the cartoon or geometric component of y and the textured component of y is represented by v [24]. For example, v is chosen in optical flow evaluation as it is frequently free of shading shadows and reflections. Meanwhile, u is frequently learned for improving the constancy of depth evaluation. Each patch P_I is subjected to TV decomposition in order to obtain texture and cartoon components. Equation (1) is applied to decompose the given input image ‘y’.

Fig. 2

(a) Latent fingerprint for noise removal, (b) Noise filtered texture image.

$\min {TV (u) {∥ u - y ∥}_{B} ⩽ ɛ}$ (1) where TV (u) implies the total variation of u and ∥ . ∥ _B intends the norm. ɛ symbolizes the trivial positive constants used for scaling. Finally, ∥u - y ∥ _B ⩽ ɛ represents the fidelity term which forces u to be nearer to y.

The texture component v is a weak extracted signal which is further enhanced for various processing. Simultaneously, the cartoon component ‘u’ is removed as a structured noise. The texture component of each patch P_I is allowed for the consecutive process to estimate the orientation field ‘φ’ and frequency field ‘f’. Once after the TV decomposition, the effect of varying illumination gets reduced significantly in each patch along with the structured noises. Furthermore, in the texture component of each patch, some missing patterns and broken ridges are found. An enhancement is needed in order to restore the missing patterns and broken ridges. Otherwise, it finds difficult to extract the minutiae in the latent image.

3.2 Patch extraction phase

Before the enhancement phase, the reference images are subjected to the patch extraction stage. The image patches for the training set are generated using the dictionary learning algorithm. In order to extract the patches, around 500 high-quality latent images are acquired from the fingerprint dataset [28]. A ridge quality map is formed in five quality levels, if the quality level is 4, it is denoted as high and 0 represents the low quality.

Traditionally, two types of dictionary learning are available such as coarse level dictionary and fine level dictionary. In the coarse level dictionary, image patches are formed using an 64 × 64 pixel sliding window in the input reference image. For each input patch, the mean of the intensity is estimated and it is compared with the predefined threshold of T = 3.75. We find this value is the best cutoff value for patch selection because the proposed method yields better accuracy for the particular T value. Therefore, the predefined threshold value has been fixed as T = 3.75. If the threshold is T > 3.75, then the corresponding image patch is considered for a coarse level dictionary in the training set. Similarly, in the fine level dictionary, the 32 × 32 sliding window is applied to generate image patches in the input reference image.

The mean is estimated for each image patch and compared with the predefined threshold of T = 3.75. If the threshold T <3.75, then the corresponding image patch is neglected from the training set. In this work, 16 orientation-specific patch size dictionaries are utilized for coarse level and fine level estimation of ridge quality maps. In the further step, vector normalization $\tilde{(s})$ can be carried out to change the image matrix to vector form. Mean intensity ‘μ’ and standard deviation ‘σ’ are estimated for each patch ‘s’. $\tilde{s} = \frac{(s - μ_{s})}{σ_{s}}$ (2) where, ‘s’ indicates a set of patches and thus huge dimensional patches are obtained. But, a set of training patches are selected randomly as 40,000 patches from coarse level and 5000 patches from fine level dictionaries.

The dimension of dictionary L is given by n_s × n_l where n is the number of dictionary elements. Here, a two-step learning algorithm named as Singular Value Decomposition (SVD) is applied for dictionary updating. In the first stage, sparse coding is performed and in the second stage, dictionary update is processed. By solving the following optimization in Equation (3), ‘γ_i’ is obtained from training set ‘s’ containing all patches. $min_{L, C} {∥ s - L ∥}_{F}^{2} s . t \forall_{i}, {∥ γ_{i} ∥}_{0} ⩽ P_{I}$ (3) where, C denotes the column of the matrix N_L × N. Using Equation (4), SVD one column of dictionary ‘s’ is minimized by SVD fixing. $min_{γ_{i}} {∥ s_{i} - L_{γ_{i}} ∥}_{2}^{2} s . t \forall_{i}, {∥ γ_{i} ∥}_{0} ⩽ P_{I}$ (4) where D is the dictionary of a single column updated at a time using SVD by fixing all other columns of D. A total number of elements on the coarse level dictionary is 1024, whereas, in the fine level dictionary it is 64.

A non-overlapping 16 × 16 blocks are constructed from each patch P_I. The orientation and frequency of each block are obtained which results in the patch orientation field φ^p and frequency field f^p. These are used to estimate the orientation and frequency field of each patch P_I in the latent image. The coarse estimates of each block with orientation $φ_{cb}^{p}$ and frequency $f_{cb}^{p}$ are given by Equation (5) and (6). $φ_{cb}^{p} = \frac{1}{2} {tan}^{- 1} (\sum_{j = 1}^{nb} q_{j} \sin 2 φ j + \sum_{j = 1}^{nb} q_{j} \cos 2 φ j)$ (5) $f_{cb}^{p} = \frac{1}{\sum_{j = 1}^{nb} q_{j}} \sum_{j = 1}^{nb} q_{j} f_{j}$ (6) where nb indicates the number of patches covering the block ‘cb’. Since the coarse level dictionary is reliable to local noise, the information of detailed ridge is unable to extract from the patches p of the latent image. Therefore, the orientation and frequency fields are computed for fine-level dictionaries.

3.3 Enhancement Phase

Once after the decomposition of the input latent image, it is of low quality with missing ridges which is difficult to extract the minutiae from the latent image. In order to obtain the enhanced image, the SSIM score is computed between the corrupted input and the reference latent image. Normally, the SSIM score lies between 0 and 1 for digital images. If the value is greater than 0.7, then the similarity is more between the input corrupted image and the reference image. Henceforth, the input image patch ‘P_I’ is replaced by the reference image patch of coarse level or fine level.

A global threshold T_g is estimated by the normalization of Otsu’s method. This threshold is used to normalize the enhanced map where the enhanced map is computed as the average of coarse-level quality and fine-level quality. If the quality of the threshold block is lesser than T_g, then it is considered as background 0 and other blocks are considered as the foreground. Besides, morphological operations like dilation and opening are applied to fill the holes in the foreground and to remove the small foreground blocks are computed to get a final segmented output.

The latent text image in the foreground region is developed to enhance the text image. This is executed by means of Gabor filtering [25]. This method is also used to tune and enhance the ridge structures by estimating the ridge spacing and ridge flow. In the Gabor filter, the standard deviation is set as 5 as the Gaussian envelope. Figure 3 depicts the segmented output of the enhancement fingerprint image.

Fig. 3

Segmented output of the enhancement image.

3.4 Minutiae extraction phase

The minutiae extraction is performed in good quality regions where ROI>0. If ROI< =0, it denotes other than fingerprint region. The example for minutiae extraction is demonstrated in Fig. 4. The process of minutiae extraction can be carried out in five steps:

Step 1: The quality regions are binarized based on the EM algorithm.

Fig. 4

Minutiae extracted image.

Step 2: In the binary image, fill all the small holes and remove small islands.

Step 3: Make the binary image much thinner to form the skeleton.

Step 4: After removing the spurs and broken ridges are connected.

Step 5: Using the method proposed in [26], detect the minutiae present in the skeleton.

3.5 Local coherence pattern

In order to estimate the coherence in the latent fingerprint image, the intensity-gradient distribution is computed initially and it is measured as, $C = [c_{1,} c_{2,} \dots . c_{Z}]$ (7) $C_{k} = [I (k), I_{x} (k), I_{y} (k)]^{T}$ (8) where I(k) represents the intensity of the image. I_x (k) and I_y (k) symbolize the image gradient along with horizontal and vertical directions at every pixel location (x_k, y_k). Here, the intensity computes on image whereas image gradient computes on horizontal and vertical direction on the image. Next, Z implies the number of pixels corresponding to the local region; for instance: Z = 3 ×3 pixels. The obtained intensity value using Equation (7) plays a key role in the involvement of textural information. The gradient distribution is projected by the intensity value into a higher feature space.

In a further step, the dominant orientation and the corresponding coherence have to be estimated using the intensity-gradient distribution expressed in Equation (7). Later, the given distribution is decomposed into independent axes using the SVD method where the SVD is applied and the decomposition is performed with the corresponding energy. Therefore, the energy of intensity gradient distribution is computed using Equation (9). $C = U \times W \times V^{T} = U [\begin{matrix} w_{1} & 0 & 0 \\ 0 & w_{2} & 0 \\ 0 & 0 & w_{3} \end{matrix}] {[v_{1} v_{2} v_{3}]}^{T}$ (9) where U implies the auxiliary matrix (Z × Z), W denotes the matrix containing energies (3 × 3), V represents a 1 × 3 matrix on directions of v₁, v₂, and v₃. Here, the column vector v₁ indicates the dominant orientation for the particular distribution. Since the variance between the energies on every direction can signify the degree of concentration in the intensity-gradient distribution, we suggest to describe our coherence using the Equation 10. $c = w_{1} - (w_{2} + w_{3}), where w_{1} ⩾ w_{2} ⩾ w_{3}$ (10) where w₁, w₂, w₃ intends the energies on directions of v₁, v₂, and v₃ respectively. The values of w depend on the structure of the image region. The higher values are obtained for the well-directed regions and lower values are acquired for the noisy regions. Further, the c value is computed for these different regions. Among the c values, the highest coherence gives the directional coherence of valleys and ridges. For example, if w₁ = 12, w₂ = 3, and w₃ = 2 for the well-directed region and w₁ = 6, w₂ = 1.5, and w₃ = 1 for the noisy region, then the coherence according to Equation 10 will be 7 for the texture region and 3.5 for the noisy region. Hence, the texture features can be extracted efficiently for latent fingerprints based on the proposed coherence pattern method.

After computing coherence, the Local Coherence Pattern (LCP) is constructed in such a way that the relation between coherences of neighbour pixels is encoded as follows: $f_{LCP} (x, y) = \sum_{1 ⩽ i ⩽ n} 2^{i - 1} {LCP}^{i} (x, y)$ (11) ${LCP}^{i} (x, y) = {\begin{matrix} 1, & if c (x, y) > c (x_{i}, y_{i}) \\ 0, & otherwise \end{matrix}}$ (12) where, n intends the number of sampling pixels (3 × 3) in the neighbouring region i.e., n = 8. c (x, y) is calculated using Equation 8. Further, i denotes the index of the neighbouring pixel pointed at the current position (x, y) here, i∈{1, 2, …, 8}.

3.6 Extraction of descriptors using LCPnet

The three different types of templates (LCPnet minutiae, LCPnet texture, and LCPnet minutiae + texture) are analyzed for evaluating the proposed fingerprint detection system. The first minutiae set is extracted from the approach in [9]. The proposed LCPnet model will extract deep texture features in the first layer of CNN. The convolution in the first layer is performed based on the LCP values of the pixels, but not on the original greyscale values. Since the descriptor from the LCPnet is more robust it improves the result of the deep neural network. In addition, the LCP descriptors work with the high-level texture features of the training data. The configuration of the LCPnet is as follows:

The first convolution layer of Lenet-5 is modified by incorporating the LCP descriptor.

A Rectifier Linear Unit (RELU) layer eliminates the negative values by performing the inner product of an original image.

The two-node fully connected layers perform an inner product and classify the input image using the softmax classifier.

Figure 5 illustrates the working flow of the proposed LCPnet model. The test image of 80 × 80 size is given input to the first convolution layer. The size of the input image is not fixed because the proposed model also supports the larger size input images. The first layer convolution operation estimates the LCP-based values for the pixels and generates 96 outputs with the size of 38 × 38. The pixels are again convolute with different kernels with the size of 11 × 11, produces an output image with a stride of 1. The convolution operation is given by:

Fig. 5

Working flow of the proposed LCPnet model.

$C_{j} (t) = \sum_{\forall s \in N (t)} LCP (I (S)) \times K_{j} (I)$ (13) where, S = (x_s, y_s) symbolizes the LCP value of the pixels S, whereas, t = (x_t, y_t) represents the LCP values of pixel t, C_j (t) implies the value of LCP in the respective position of ‘S’ in the output feature map C_j, where, j = (1,2,3 . . . . 96), K_j (I) denotes the kernel values, where j = (1,2, 3 . . . . 96) and I = (1, 1) (2, 2) … . . (11, 11). These kernels are initialized with weights of inversely proportional input size. The weights of a neuron are adjusted during learning in CNN. Initially, the biases of neurons are set to zero and the weights adjusted during learning and optimized during training of the network. The pooling operation is given by $P_{j} (r) = max_{\forall p \in N (r)} {C_{j} (P)}$ (14) where, C_j (p) with p = (x_p, y_p) indicates the value in position p of the neighbourhood position r = (x_r, y_r). The above two convolutions and pooling operations are repeated in the second layer of LCPnet. In this layer, LCP is not calculated but, kernels with size 5 × 5 are only applied. To generate 128 two-dimensional feature maps of size 14 × 14.

LCPnet Algorithm
Input: Fingerprint minutiae patches
Output: Minutiae descriptor extraction
Step 1: Construct C at every pixel position using Equation (7).
Step 2: Conduct SVD of C and compute the coherence c
using Equation (10).
Step 3: Compute f_LCP using Equation (11)
Step 4: Local fingerprint patches around these selected
minutiae are extracted to train the LCPnet
Step 5: Smaller patches are resized to 80×80 pixels to
ensure that we can use the same LCPnet with
80×80 images as input
Step 6: For each LCPnet, its 128-dimensional output of the
last fully connected layer is considered as a feature
vector

The RELU layer is performed by taking the inner product with each 3 × 3 structures to generate the rectified image as output without negative values. $RELU (q) = \max {0 : q}$ (15) where q represents the weighted sum of neuron signals from the previous layer of LCPnet.

A fully connected layer with two neurons is connected to the neurons of the RELU layer and performs an inner product operation. The activations in the fully connected layer are defined by applying the softmax function and it is evaluated as $S_{k} = \frac{e^{uk}}{(e^{u 0} + e^{u 1})}$ (16) where k is an element represents (0,1) to the weighted sum of values from the previous RELU layer. This function normalizes the input of neurons and makes their output as 1. It is also used to extract the minutiae of the original image. Figure 6 shows the LCPnet-based descriptor.

Fig. 6

LCPnet-based descriptor.

The neighbors set around the pixel in LCPnet are also considered in the proposed model. The estimation of neighbors set [27] is as follows:

$LCP = \sum_{i = 0}^{G - 1} F (a_{i} - a_{c}) 2^{i}$ (17) $F (x) = {\begin{matrix} 1, & if a_{i} - a_{c} > 0 \\ 0, & otherwise \end{matrix}}$ (18) where G denotes the number of neighbors for central pixel, a_c implies the intensity of the central pixel illumination and a_i symbolizes the illumination intensity of the ith neighbor pixel. F(x) indicates the sign function which is utilized to produce the binary values and is computed using Equation 18.

3.7 Similarity matching phase

For similarity matching, one texture template and one minutiae template are learned for each latent. There are two parts of similarity between the reference minutiae template and latent minutiae template such as (i) minutiae similarity and (ii) ridge flow similarity. Here, minutiae similarity is the similarity of descriptors for matched minutiae correspondences. In order to obtain the ridge initially, the two ridge flow maps are aligned through the minutiae correspondences and then by estimating the orientation similarity in overlapping blocks.

One minutiae template similarity score (S_m) are evaluated by comparing the latent minutiae templates with respect to the reference minutiae template. The texture template similarity (S_T) contains all the similarities of matched minutiae correspondences. Besides, the texture similarity is computed by comparing the latent and reference print texture templates. The weighted sum of S_m and S_T provides the final similarity score S between the latent and the reference print using the expression. $S = ρ_{1} S_{m} + ρ_{2} S_{T}$ (19) where ρ₁ and ρ₂ denote the weights and the values of ρ₁ and ρ₂ are empirically determined as 1 and 2 respectively.

4 Simulation results and discussions

The developed system has been processed in MATLAB platform, version 16, executed by i7 core processor at 2.5 GHz and 16 GB RAM. Table 2 exposes the simulation setting and its parameter values. The following section discussed the detailed description of the dataset, performance evaluation of the proposed fingerprint recognition model, and comparison of the proposed system over existing models.

Table 2
Simulation setting and parameter values

Parameters Values

Learning rate 10^–2

Epochs 10

Batch size 32,64

Threshold 3.75

Weight decay 0.035

Momentum 0.95

Gabor filter 2-5

Parameters	Values
Learning rate	10^–2
Epochs	10
Batch size	32,64
Threshold	3.75
Weight decay	0.035
Momentum	0.95
Gabor filter	2-5

4.1 Dataset

The proposed detection system has been tested through two latent public datasets: NIST SD 27 [28] and WVU [29].

In these two datasets, the images are chosen for their different characteristics with respect to ridge clarity, number of minutiae, and background noise. The variation in the images provides a challenging task for the recognition problem.

NIST SD 27: In this dataset, there are 258 grayscale latent fingerprint images along with the reference matching rolled fingerprints. The database images are taken from the department of forensic agencies.

WVU: This dataset is collected by the students of West Virgina University under the laboratory setting. It consists of 449 latent images along with their reference mated images.

4.2 Evaluation of fingerprint registration using mated minutiae pair

Three different types of templates have been conducted for evaluating the performance of the proposed fingerprint registration such as LCPnet minutiae, LCPnet texture, and LCPnet minutiae+texture. The performances of the three templates are compared with the existing model stated in [9]. They have developed three minutiae templates and one texture template applied in both NIST 27 and WUV datasets. But, one minutiae template, one texture template, and one output template are constructed in the proposed system for acquiring the registration result.

The empirical cumulative distribution function (E(t)) has been evaluated to analyze the performance of different latent subsets and it is calculated using Equation 20. This function is based on deviation in location and deviation in direction. $E_{k} (t) = \frac{1}{k} \sum_{i = 1}^{k} x_{i} ⩽ t$ (20) where x_i signifies the deviation in location of the ith latent and k intends the total number of latent subsets.

The E(t) results are enumerated in Figs. 7 and 8. In concrete analysis, the performance under three subsets namely good, bad, and ugly subsets are considered. Table 3 shows the accuracy of registration in different methods on three subsets. For the evaluation purpose, the direction and location difference of ground truth is used for matching minutiae once alignment is made. Here, the threshold for deviation from ground truth is fixed as 20 pixels and 15 degrees for direction and location respectively.

Fig. 7

Empirical cumulative distribution function of location (a) Subset of good latent (b) Bad latent and (c) Ugly latent.

Fig. 8

Empirical cumulative distribution functions of direction (a) Subset of good latent (b) Bad latent and (c) Ugly latent.

Table 3

Registration accuracy with respect to location and Direction

Method	Location				Direction
	Good	Bad	Ugly	All	Good	Bad	Ugly	All
Cao-minutiae	84.55	74.51	62.30	76.66	89.36	82.29	80.01	85.24
Cao-texture	83.13	77.84	66.06	77.75	90.97	85.47	85.41	88.16
Proposed-minutiae	84.65	79.61	72.76	82.76	88.46	84.39	82.11	86.34
Proposed-texture	85.24	81.95	76.17	83.86	89.08	85.58	86.52	87.27
Proposed	87.33	84.62	78.64	84.59	90.74	86.03	88.64	88.93
minutiae + texture

Based on Table 3, it is evident that the proposed model has achieved a superior result on the good subset and improved performance in bad and ugly subsets. Furthermore, the texture template outperforms the minutiae templates. This is due to the execution of the appropriate pre-processing and patch learning phases in the proposed system. It also utilized the TV decomposition methodology in the enhancement phase to remove the smooth background noise present in the input images. This proper noise removal facilitates to achieving a perfect extraction of ROI, ridge flow, and ridge spacing.

Besides, the proposed model considers both texture and minutiae templates for fingerprint registration. It does not avoid any relevant ridge information during the evaluation phase. In contrast, the existing models considered the texture and minutiae templates separately. This paves a way to obtain lesser performance than the proposed model. The performance of the proposed system is evaluated in two major steps (a) At the enhancement stage and (b) At the matching stage. At the enhancement stage, the text image is enhanced in order to improve the overall efficiency. Also in the matching stage, two valuable methods are adopted so as to provide a better result.

The matching performance of various registration methods for both datasets is illustrated in Figs. 9 and 10. From the obtained results, it is noticed that the proposed model attains a better matching performance than the existing models. The key reason behind this achievement is that the proposed model implemented the unique similarity matching algorithm in which one texture template and one minutiae template are extracted for each latent. The proper extraction yields accurate outcomes in the matching stage.

Fig. 9

Matching performances of different registration methods on NIST27 database.

Fig. 10

Matching performances of different registration methods on WVU database.

4.3 Comparison of proposed fingerprint recognition system over existing models

In this work, the proposed system has been compared with the existing techniques using two different datasets (NIST 27 and WVU). Every latent image in NIST 27 is matched with the impressions on the background image. In addition, the rank 1 accuracy, specificity, recognition rate, and error rate are estimated to validate the efficiency of the proposed system. The accuracy states that the degree to which the detection result of a fingerprint evaluation adapts to the precise value [30 –34].

The performance comparison of the proposed system over the recently stated models is depicted in Table 4. It is observed from Table 4 that the proposed system reveals better performance than other existing models by learning the most vital fingerprint ridges with 99.58% accuracy and 99.24% specificity. The proposed system eradicates superior results for both datasets. The major reason behind this improvement is the execution of the dictionary learning algorithm for extracting the image patches in the proposed system. Further, the SSIM score is computed between the corrupted input and reference latent image in order to obtain the enhanced image. This appropriate computation strategy paves a way to prolong the accuracy during the multi-scale dataset.

Table 4
Performance comparison of proposed system over existing systems

Dataset Model Accuracy (%) Specificity (%) Recognition rate (%) Error rate

NIST 27 CNN [17] 86.5 84.2 79.6 1.98

FQA [16] 88.1 87.8 85.7 1.36

NMR [15] 92.0 91.5 89.3 1.22

SF [14] 85.0 88.7 83.2 1.82

FF [13] 88.2 90.9 86.5 1.88

FinSNet [12] 96.4 95.1 95.2 0.78

CSSC [11] 96.9 97.5 96.2 0.77

Proposed 99.44 99.06 98.62 0.82

WVU CNN [17] 88.6 85.2 80.4 1.92

FQA [16] 90.2 88.7 86.5 1.24

NMR [15] 93.7 92.4 90.3 1.11

SF [14] 87.0 89.5 84.2 1.62

FF [13] 89.2 93.6 90.3 1.78

FinSNet [12] 97.4 96.0 96.3 0.89

CSSC [11] 98.1 98.6 97.2 0.88

Proposed 99.58 99.24 98.94 0.81

Dataset	Model	Accuracy (%)	Specificity (%)	Recognition rate (%)	Error rate
NIST 27	CNN [17]	86.5	84.2	79.6	1.98
	FQA [16]	88.1	87.8	85.7	1.36
	NMR [15]	92.0	91.5	89.3	1.22
	SF [14]	85.0	88.7	83.2	1.82
	FF [13]	88.2	90.9	86.5	1.88
	FinSNet [12]	96.4	95.1	95.2	0.78
	CSSC [11]	96.9	97.5	96.2	0.77
	Proposed	99.44	99.06	98.62	0.82
WVU	CNN [17]	88.6	85.2	80.4	1.92
	FQA [16]	90.2	88.7	86.5	1.24
	NMR [15]	93.7	92.4	90.3	1.11
	SF [14]	87.0	89.5	84.2	1.62
	FF [13]	89.2	93.6	90.3	1.78
	FinSNet [12]	97.4	96.0	96.3	0.89
	CSSC [11]	98.1	98.6	97.2	0.88
	Proposed	99.58	99.24	98.94	0.81

The proposed system constructs the optimal LCP descriptors which work with the high-level texture features of the training data. It produces minimal error rates in the proposed system. The proposed system exactly learns the different characteristics of latent with respect to direction and location. Better extraction of various characteristics of latent is obtained with enhanced recognition rate and efficiency.

The accuracy of the existing models is insignificant owing to the perpetuation of the mismatched error rate in the detection phase. Particularly, the SF and FF models lagged in learning the influential latent features of the fingerprint. As a result, it causes a poor accuracy of 87% and 87.2% respectively when compared with the proposed system. Likewise, the traditional CNN model attains minimal accuracy of 86.5% because of linking a large number of hidden layers to compute convolution and pre-processing mechanisms to learn the ridge information.

According to Figs. 11 and 12, it is manifest that the proposed system yields superior precision of 98.98% and the recall of 99.54% when compared with the existing stated models. This is because of the establishment of morphological operations like dilation and opening in the proposed system to fill the holes in the foreground. It also removes the small foreground blocks that are computed to get a final segmented output. Further, Gabor filtering is also utilized to tune and enhance the ridge structures by estimating the ridge spacing and ridge flow. The texture features can be extracted efficiently for latent fingerprints based on the proposed coherence pattern method. In order to develop a salient minutiae descriptor, the robust LCPnet is trained for 12 patch types and each descriptor is trained using LCPnet with a particular patch size at a location surrounding the minutiae. This proper patch and feature extraction offer lesser false prediction during the detection phase.

Fig. 11

Comparison of Precision value for different models with respect to two datasets.

Fig. 12

Analysis of recall value for various models under different datasets.

On the contrary, the existing models failed to obtain the finest salient minutiae and texture features in the fingerprint recognition. The CNN and FQA models are reliable to local noise in which the information of detailed ridge is unable to extract from the patches of the latent image. As a result, they produce lesser precision of 82% and 92% respectively as compared with the proposed system.

Similarly, the SF and FF models follow a feedback strategy to extract the texture features that create unwanted complexity and increase the false error rate in the enhancement stage. This higher error rate causes a minimum precision value of 91% and 85% respectively. Alternatively, the NMR, FinSnet, and CSSC models employ optimum filtering techniques to remove the smooth background noise. This paves a way to achieve the higher precision value of 92%, 93%, and 94% respectively. At the same time, they lagged to implement the proper similarity matching algorithm which causes a minimal precision value than the proposed system.

5 Conclusion and future work

In this work, a latent fingerprint recognition system was developed using Dictionary learning and LCPnet techniques. The model is completely automated and able to overcome the drawbacks of manual searching. The TV decomposition model is preferred in the pre-processing stage to eradicate the background noise in the fingerprint image. Thereafter, the segmentation phase using global threshold and enhancement through Gabor filtering aids to enhance the minutiae extraction. Moreover, the implementation of an appropriate local coherence pattern net mechanism will extract the deep texture features. Finally, a new similarity matching was introduced in the proposed system where one texture template and one minutiae template are learned for every latent.

The efficacy of the proposed system has been evaluated using the MATLAB platform under two different datasets. The performance results of the proposed system outperform well as compared with the existing state-of-the-art approaches. Specifically, the proposed system has acquired a higher Specificity of 99.24% and 99.06% under NIST SD27 and WVU datasets respectively. This specificity result proved that the proposed system exactly recognized and matched the latent fingerprint. Furthermore, the proposed system yields a superior precision value of 98.98% and a recall of 99.54%. These results validated that the proposed system is more suitable to detect the latent fingerprint for larger as well as smaller datasets.

In the future, the proposed system can be extended to manipulate some other frequency and spatial domain attributes to enhance the detection performance.

References

Liu

and Qian

, Automatic Segmentation and Enhancement of Latent Fingerprints Using Deep Nested UNets, IEEE Transactions on Information Forensics and Security 16 (2021), 1709–1719.

Yang

, Hu

and Li

, Recognition and classification of damaged fingerprint based on deep learning fuzzy theory, Journal of Intelligent& Fuzzy Systems 38 (2020), 3529–3537.

Oduah

, Kevin

I.F.

and Oluwole

D.O.

, Towards a high-precision contactless fingerprint scanner forbiometric authentication, Array 11 (2021), 100083–17.

Zhao

, Ge

and Zhao

, Fingerprint pre-processing and feature engineering to enhance agricultural products categorization, Future Generation Computer Systems 125 (2021), 944–948.

Labati

R.D.

, Genovese

and Muñoz

, A novel pore extraction method for heterogeneous fingerprint images using Convolutional Neural Networks, Pattern Recognition Letters 113 (2018), 58–66.

Yun

and Cho

, Adaptive fingerprint image enhancement with fingerprint image quality analysis, Image and Vision Computing 24 (2006), 101–110.

Chen

, Fan

and Yang

, Small-size fingerprint recognition method based on densely connected convolutional network, Computer Engineering and Applications 56 (2020), 134–140.

Wang

, Jiang

, Zhai

and Shi

, Traffic Identification Method for SSL VPN Encryption, Computer Engineering and Applications 58 (2022), 143–151.

Cao

, Liu Anil

and Jain

, Segmentation and Enhancement of Latent ingerprints: A Coarse to Fine Ridge Structure Dictionary, IEEE Transactions on Pattern Analysis and Machine Intelligence 36 (2014), 1847–1859.

10.

Zhao

, Ge

, Zhao

and Xiang

, Fingerprint pre-processing and feature engineering to enhance agricultural products categorization, Future Generation Computer Systems-The International Journal of Science 125 (2021), 944–948.

11.

Zhang

, Wang

and Yang

, Convolution Structure Sparse Coding for Fusion of Panchromatic and Multispectral Images, IEEE Transactions on Geoscience and Remote Sensing 57 (2019), 1117–1130.

12.

Yoo

, Cho

and Lee

, FinSNet: End-to-End Separation of Overlapped Fingerprints Using Deep Learning, IEEE Access 8 (2020), 209020–209029.

13.

Horapong

, Srisutheenon

and Areekul

, Progressive and Corrective Feedback for Latent Fingerprint Enhancement Using Boosted Spectral Filtering and Spectral Autoencoder, IEEE Access 9 (2021), 96288–96308.

14.

Deerada

, Phromsuthirak

and Rungchokanun

, Progressive Focusing Algorithm for Reliable Pose Estimation of Latent Fingerprints, IEEE Transactions on Information Forensics and Security 15 (2019), 1232–1247.

15.

, Feng

and Lu

, Latent Fingerprint Registration via Matching Densely Sampled Points, IEEE Transactions on Information Forensics and Security 16 (2021), 1231–1244.

16.

Sharma

, Dey

Two-stage quality adaptive fingerprint image enhancement using FuzzyC-means clustering based fingerprint quality analysis, Image and Vision Computing (2019), 1–16

17.

Wong

and Lai

, Multi-task CNN for restoring corrupted fingerprint images, 03:, Pattern Recognition 101 (1072), 1–18.

18.

, Feng

and Kuo

C.C.

, Deep convolutional neural network for latent fingerprint enhancement, Signal Processing: Image Communication 60 (2018), 52–63.

19.

Choi

, Boaventura

I.A.G.

and Jain

A.K.

, Automatic segmentation of latent fingerprints, IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems 14 (2015), 1–6.

20.

Deshpande

and Malemath

V.S.

, Experimental Study on Latent Fingerprint Matching Using Clustered Minutiae Patterns, Part of the Communications in Computer and Information Science Book Series 1036 (2019), 381–394.

21.

Jain

A.K.

and Feng

, Latent fingerprint matching, IEEE Trans Pattern Anal Mach Intell 33 (2011), 88–100.

22.

Liu

, Chen

and Wang

, Latent Fingerprint Enhancement via Multi-ScalePatch Based Sparse Representation, IEEE Transactions on Information Forensics and Security 10 (2015), 6–15.

23.

Paulino

A.A.

, Feng

and Jain

A.K.

, Latent fingerprint matching using descriptor-based hough transform, IEEE Transactions on Information Forensics and Security 8 (2013), 1–6.

24.

Armi

and Fekri-Ershad

, Texture Image Analysis and Texture Classification Methods –A Review, International Online Journal of Image Processing and Pattern Recognition 2 (2019), 1–29.

25.

Manickam

, Devarasan

and Manogaran

, Score level based latent fingerprint enhancement and matching using SIFT feature, Multimedia Tools Application 78 (2019), 3065–3085.

26.

Jan

, Ali

, Mahmood

Statistical Descriptors based Automatic Fingerprint Identification: Machine Learning Approaches, CoRR-ArXiv, abs/1907.12741 (2019), 1–10.

27.

Fekri-Ershad

, Cell phenotype classification using multi threshold uniform local ternary patterns in fluorescence microscope images, Multimedia Tools and Applications 80 (2021), 12103–12116.

28.

https://www.nist.gov/itl/iad/image-group/nist-special-database-2727a.

29.

https://databases.lib.wvu.edu/.

30.

Sekar

and Aruchamy

, An Efficient Clinical Support System For Heart Disease Prediction Using TANFIS Classifier, Computational Intelligence 38 (2022), 610–640.

31.

Prasanth

, Certain Investigations on Energy-Efficient Fault Detection and Recovery Management in Underwater Wireless Sensor Networks, 137:, Journal of Circuits, Systems and Computers 30 (2150), 1–20.

32.

Jayachitra

and Prasanth

, Multi-Feature Analysis for AutomatedBrain Stroke Classification Using Weighted Gaussian Naïve Baye’sClassifier, Journal of Circuits, Systems, and Computers 30 (2021), 2150178:1–26.

33.

Kalli

and Suresh

, An effective motion object detection usingadaptive background modeling mechanism invideo surveillance system, Journal of Intelligent& Fuzzy Systems 41 (2021), 1777–1789.

34.

Bhaskar

K.B.

, Prasanth

, Saranya

An Energy Effi-cient Blockchain Approach for Secure Communication in IoT-Enabled Electric Vehicles, International Journal of Communication System (2022), 1–30.

An effective latent fingerprint enhancement and recognition system using dictionary learning and LCPnet mechanisms

Abstract

Keywords

1 Introduction

3 Proposed system

Table 2 Simulation setting and parameter values Parameters Values Learning rate 10–2 Epochs 10 Batch size 32,64 Threshold 3.75 Weight decay 0.035 Momentum 0.95 Gabor filter 2-5

4.2 Evaluation of fingerprint registration using mated minutiae pair

References

Table 2
Simulation setting and parameter values

Parameters Values

Learning rate 10^–2

Epochs 10

Batch size 32,64

Threshold 3.75

Weight decay 0.035

Momentum 0.95

Gabor filter 2-5