Multimodal biometric authentication: A review

Abstract

Critical applications ranging from sensitive military data to restricted area access demand selective user authentication. The prevalent methods of tokens, passwords, and other commonly used techniques proved deficient as they can be easily stolen, lost, or broken to gain illegitimate access, leading to data spillage. Since data safety against tricksters is a significant issue nowadays, biometrics is one of the unique human characteristic-based techniques that may give better solutions in this regard. The technique entails biometric authentication of users based on an individual’s inimitable physiological or behavioral characteristics to provide access to a specific application or data. This paper provides a detailed description of authentication and its approaches, focusing on biometric-based authentication methods, the primary challenges they encounter, and how they have been addressed. The tabular view shows the benefits and downsides of various multimodal biometric systems, and open research challenges. To put it another way, this article lays out a roadmap for the emergence of multimodal biometric-based authentication, covering both the challenges and the solutions that have been proposed. Further, the urge to develop various multi-trait-based methods for secure authentication and data privacy is focused. Lastly, some multimodal biometric systems comprising fingerprint and iris modalities have been compared based on False Accept Rate (FAR), False Reject Rate (FRR), and accuracy to find the best secure model with easy accessibility.

Keywords

Authentication multimodal biometrics identification biometrics

1. Introduction

Due to the overwhelming number of scammers in nearly every sector, extremely secure authentication techniques are crucial for access to confidential data. Strict authentication services with data privacy need to be implemented at this moment in order to accomplish an outstanding level of security. Literature involves various user authentication techniques: traditional password-based authentication, frictionless authentication, unimodal, and multimodal authentication. Out of these, traditional authentication methods are less secure and vulnerable. Various researchers have pointed out that multimodal techniques provide better authentication services than unimodal systems [1–5]. However, most of the multimodal systems designed so far have lower user acceptance rates due to biometric data privacy issues. Therefore, they are only accepted by some organizations as a suitable user authentication method. Numerous researchers have addressed this data privacy issue, but the complete framework for strict authentication and data protection is still an open research problem. This paper provides the strengths and weaknesses of these systems and emphasizes the fact that there is a great need for a complete solution for data security in highly confidential applications. Additionally, a thorough examination of some biometric systems based on fingerprint and iris modalities has been conducted.

Table 1
A tabular comparison of our review work with most recent review works

Authors Main focus Discussion on authentication Methods of authentication Dataset description Advantages & disadvantages of state-of-the-art multimodal system

Shaheed et. al. [6] Deep learning based discussion No No Yes No

Shaveta et. al. [7] Traits-based discussion No No No No

Shervin Minaee et. al. [8] Application of deep learning in biometrics No No Yes No

Abdulrahman et. al. [9] Discussed five traits only No No No No

Chiroma [10] Fingerprint authentication Yes Yes No No

Bala et. al. [11] Fusion techniques No No No Yes

Uwaechia et. al. [12] ECG No No Yes No

Our work A complete knowledge in one paper Yes Yes Yes Yes

Authors	Main focus	Discussion on authentication	Methods of authentication	Dataset description	Advantages & disadvantages of state-of-the-art multimodal system
Shaheed et. al. [6]	Deep learning based discussion	No	No	Yes	No
Shaveta et. al. [7]	Traits-based discussion	No	No	No	No
Shervin Minaee et. al. [8]	Application of deep learning in biometrics	No	No	Yes	No
Abdulrahman et. al. [9]	Discussed five traits only	No	No	No	No
Chiroma [10]	Fingerprint authentication	Yes	Yes	No	No
Bala et. al. [11]	Fusion techniques	No	No	No	Yes
Uwaechia et. al. [12]	ECG	No	No	Yes	No
Our work	A complete knowledge in one paper	Yes	Yes	Yes	Yes

The primary goal of this work is to provide an insight into biometric-based authentication for new researchers in this field, although this description is available in other articles [13–16] too but it is not all-encompassing. The majority of survey publications on biometric authentication focused solely on the various modalities utilized [17–22] or on specific application areas, but a comprehensive survey is still missing. Shaheed et al. [6] offered a systematic review of several physiological biometric features organized into three categories: hand, face, and ocular regions. They also assessed several strategies for these trait categories, but it is still needed to pinpoint the crucial advantages and disadvantages explored in this article. The researchers additionally offered a list of datasets that were readily available, however, it lacked certain vital information that will be uncovered in this article. Furthermore, their research was limited to the above-mentioned feature categories exclusively, leaving a slew of other crucial traits unexplored. Therefore, the main objective of this paper is to provide newcomers with an in-depth overview of biometric authentication in a single place. Table 1 compares this work to prior works and shows how this paper differs from those earlier efforts. To summarize, the significant contributions of this article include:

Reviewing state-of-the-art authentication methods and outlining concerns connected with different authentication techniques identified by the researchers,

Discussing research-based solutions to such problems and outlining the requirement for multimodal biometric systems,

Summarizing publicly accessible datasets for both unimodal and multimodal biometric systems, together with the vital information that has not been mentioned in most survey publications,

Examining existing multimodal biometric systems in terms of their advantages and disadvantages and emphasizing the importance of the biometric database’s stored template security,

Discussing challenges and solutions in the field of authentication followed by future research directions along with the preceding topics,

Comparing five multimodal biometric systems that include fingerprint and iris features in terms of availability and security.

The rest of the article is organized as follows: Section 2 provides a brief description of authentication and its various methods, thereby describing the need for biometric-based authentication systems. A detailed overview of biometric usage for authentication and authorization purposes is given in Section 3. Section 4 addresses the major challenges in biometric systems along with the major focus on data security in such systems, and the quick summary of the complete flow of research in the field of authentication is given in Section 5. Section 6 compares some multimodal systems utilizing the fingerprint and iris of individuals. Finally, Section 7 concludes the paper.

2. Authentication methods and their related issues

Authentication can be defined as the process in which a user claims a particular identity, and the system checks for its genuineness. The various factors used for authentication can be broadly classified into specific information-based authentication and heterogeneous information-based authentication factors.

2.1. Specific information-based authentication

They are also called one-factor authentication, which comprises knowledge-based, possession-based, and biometric-based authentication methods. Knowledge-based authentication, one of the traditional user authentication methods, requires the user to reveal his password-based knowledge. These passwords are unique to a user but are still vulnerable to security breaches [23]. Since knowledge-based methods are less secure and problematic, possession-based authentication methods were introduced. This method is based on “what the user has” but suffers from difficulties like missing cards, stolen cards, etc. It is further broken down into two categories: single-device possession and multi-device possession. Security levels would be high in the case of multi-device possession schemes as compared to single-device possession schemes. The biometric-based authentication method works on “who the user is” i.e., his essential physiological characteristics work as the password to unlock access to the system. It is a widely accepted authentication method nowadays and is gaining increasingly more importance daily.

2.2. Heterogeneous information-based authentication

Due to the high risk of security-critical applications, a combination of two distinct authentication methods based on specific information-based techniques was devised, resulting in heterogeneous information-based authentication methods or two-factor authentication methods [24,25], which include knowledge-possession-based and biometric-possession-based techniques. The primary disadvantage of the knowledge-possession-based strategy is that it lacks user-friendliness and acceptability because it requires an additional step for the user to enter. Since data security is in high demand today, authentication systems that enable users to authenticate securely without leaking their personal information are needed. The biometric-possession-based system uses a person’s biometric characteristics and the card issued to him to provide a high-security grade. For example, Das et al. [26] utilized smart cards and user’s biometric information for remote authentication, providing security against various attacks and making it usable in real-world applications.

Despite the increased level of data security offered by heterogeneous information-based systems, they are still not considered widely accepted means of authentication because, during authentication, the user is not only concerned with the application access, but his requirements also include easily accessible applications without many obstacles. So, the system for authentication needs to be improved to meet this user expectation. Also, the user desires that the system should not violate user privacy terms at any cost and must possess easily understandable properties so that any naive user can use it without difficulties. Furthermore, during the COVID-19 pandemic situations with raised health concerns, users hesitate to touch any sensors for data acquisition. Therefore, the authentication process involved with an application must have some features to enable automatic data acquisition with little human intervention. Deployment of a frictionless authentication system can be one of the promising solutions for this requirement.

2.3. Frictionless multitudinal information-based authentication

For ease of application access in heterogeneous information-based authentication systems, frictionless multi-information-based authentication systems have gained importance in the past few decades. Mustafa et al. suggested a Frictionless Authentication System (FAS), but unfortunately, the system suffers from high computational and communication costs due to the employment of fuzzy extractors [27]. Zhang et al. utilized the location information in [28] for authentication and authorization purposes. Singh et al. also used location information to control access in healthcare management systems [29]. Hernandez et al. [30] proffered an effective privacy-preserving continuous authentication technique that exploited behavioral information from two sensors on the user’s mobile device. The major downside of this approach is that it relies on other authentication processes in the case of a cloud outage.

To assure security, individuals are increasingly turning to multi-information-based approaches, such as biometrics, rather than single-information-based techniques because multi-information-based techniques allow for using two or more authentication criteria for successful authentication. To some extent, these measures would increase security. Luo et al. [31] introduced a new access control system for anonymous person identification approaching critical areas. Bera et al. [32] provided resilient user authentication leveraging passwords, mobile devices, and biometrics in a smart city environment with user anonymity and untraceability properties. Authors in [33,34] explored self-service-based techniques for automatic check-in and other formalities at the airport. Solano et al. [35] employed device identification and behavioral identification for authentication purposes.

From the preceding discussion, it is evident that frictionless procedures are required, but with biometrics added to ensure outstanding security. Thus, combining frictionless approaches with biometrics would be a far more effective bundle.

3. Biometric-based authentication

Human beings themselves are one of the everlasting as well as invulnerable means of authentication for any data, or system access. Their distinguishable characteristics such as ear, nose, face, hand, iris, fingerprint, etc., make them unique from other individuals since no two individuals bear the same features. Mathematically, suppose the biometric image is represented by the function $F (x, y)$ defined as $\begin{matrix} (1) & F (x, y); 1 < = x < = m and 1 < = y < = n \end{matrix}$ During the authentication process, for the input biometric query image Q, a matching image M is searched in the image database in such a way that the distance between their corresponding feature vectors is less than or equal to the specified threshold value ‘t’. $\begin{matrix} (2) & Distance (Q (x, y), M (x, y)) < = t \end{matrix}$ Or $\begin{matrix} (3) & Feature (Q (x, y)) - Feature (M (x, y)) < = t \end{matrix}$ where, $Feature (Q (x, y))$ represents the feature vector of the query image, Q and $Feature (M (x, y))$ represent the feature vector of matching image M. The real-time implementation of a biometric-based authentication system is divided into two phases, namely the Enrollment phase and the Verification or Authentication phase, as shown in Fig. 1. Users input their biometric information in the first phase, which is then recorded in the database and utilized to authenticate them in the second phase whenever they attempt to reaccess the system in the future. In other words, biometric image analysis includes various techniques to analyze the image and extract the required information attributes from it, which can further be used for result declaration based on predefined methods. Wang et al. [36] summarized the advantages and drawbacks of some of the features used for user authentication in these biometric systems. This authentication method suffers from only one disadvantage: the consequences of false acceptance and rejection, resulting in negative identification of a person and convicting an innocent person. This problem can be solved by using multi-factor biometrics, which uses a combination of characteristics and passwords for authentication purposes but involves missing data problems.

Fig. 1.

Working: enrollment and authentication phase.

Fig. 2.

Generic multimodal authentication process based on fusion at various levels.

3.1. Types of biometric system

There are basically two types of biometric systems, which are as follows:

Unimodal Biometric System: These systems are based on a single biometric trait and are inefficient for twin identification [37]. This single modality-based authentication system also suffers from various drawbacks,namely inter-class similarity due to overlapping input data, intra-class variations arising due to incorrect interaction with the sensor, noisy data taken by sensing devices, the danger of spoofing attacks, and a high rate of Failure To Enroll (FTE) error due to the non-universal nature of some modalities [38]. A new brain-wave-based unimodal authentication scheme called SaS-BCI has been proposed by Yousefi et al. [39], which achieved an accuracy of 88% only. The error rates associated with unimodal systems are generally high, making the system unsuitable for safety-critical applications.

Multimodal Biometric System: These systems are driven by a combination of features and function better for twin identification than unimodal biometric systems [13,40]. The system operates in three modes, namely serial mode, parallel mode, and hierarchical mode. Out of these three modes, the hierarchical operating mode provides the best performance, even if the fusion is performed at the decision level. The generic hierarchy-based multimodal authentication process with three biometric traits is depicted in Fig. 2, which shows three sensing devices for taking the input of biometric samples. Information is fused in these systems to achieve high recognition accuracy with less time consumption and computational power. This information fusion in the fusion module can be applied at the sensor, feature extraction, score, or decision levels. The figure shown varies according to the position of the fusion module. For instance, if the feature-level fusion component of the multimodal system is implemented, the feature extraction module will extract the feature vector from the biometric inputs obtained by multiple sensor modules, which will then be fused to generate a single feature set. The decision module then makes a determination based on the input provided by the matcher module after passing this feature set through it. The illustration’s red dotted line (see Fig. 2) represents this entire process. Other levels of fusion can be used on the input biometric features in a similar manner. Among these levels of fusion, feature-level fusion yields the best outcomes. For instance, researchers in [41] exploited face and online signature feature vector fusion for multimodal biometric authentication in the banking industry.

Abo-Zahhad et al. [42] proposed a human identification scheme based on Canonical Correlation Analysis (CCA) by utilizing heart sound as an identifier. They should have considered several factors used in practical applications, such as using an extensive signal database containing individual data with varying ages, emotions, and heart diseases, because their main focus was maximizing the correlation between feature sets alone. Similarly, Xin and Xiaojun employed correlation to fuse biometric data at the feature level [43]. However, the canonical correlation analysis method must highlight the complicated and nonlinear correlation relationship, which is its fundamental shortcoming. In [38], the features of fingerprint, ear, and palm are extracted using the High-Frequency Maximum Significant Bit (HMSB) operator as well as the Gabor filter, and these features are further selected using Oppositional Gray Wolf Optimization (OGWO) algorithm. However, using raw data during enrollment makes the system-stored data less secure and vulnerable to attack.

Table 2
Difference between unimodal and multimodal systems

S. no.	Feature	Unimodal system	Multimodal system
1.	Meaning	Biometric system which works on a single biometric trait for identification and authentication purposes.	Biometric system which works on more than one biometric trait for identification and authentication purposes.
2.	No. of traits required	One only	Multiple
3.	Complexity level	Simple	Complex
4.	Accuracy	Least accurate	More accurate
5.	Flexibility	Less flexible	More flexible
6.	Reliability	Less	More
7.	Security	Low level security	Provides high grade of security
8.	Cost	Cheap	Costly
9.	Storage requirements	Less	More
10.	Recognition accuracy	Less	Increased by the use of multiple traits
11.	User acceptance	Less	Wider
12.	Vulnerability	More vulnerable to spoofing attacks	Less vulnerable to spoofing attacks
13.	FTE rate	More	Less
14.	Processing time	Less	More
15.	Robustness against sensor failure	Less robust	More robust against sensor failures
16.	Information	Some information may be missing or incomplete. For example – occluded faces	Complete

Some points of differences between unimodal and multimodal biometric systems are summarized in Table 2. The table undoubtedly shows that using multimodal systems would lead to high grades of security, high enrollment rates, low false acceptances, and high recognition rates but at the cost of increasing storage and timing requirements. The state-of-the-art methods developed so far using multi-traits are not focusing on these two parameters due to their high values [44,45].

3.2. Biometric identification modes

There are basically two modes of biometric identification which are as under:

Positive. In this, user identity is checked against the stored identity for application access. This is used for authentication purposes or access control applications. Here, the user gives a biometric input, which is then processed to give the desired results. It involves one-to-one comparisons in the database and is also termed the mode of authentication or verification.

Negative. In this, user identity is compared with the list of stored identities to find a suitable match. This is used by Crime Investigation Departments (CID) for criminal identification purposes. This mode does not require user biometric input; instead, this input is acquired from the crime scene for one-to-many comparisons. This mode is also termed as a mode of identification. Many researchers have created several biometric identification systems that use various modalities.

3.3. Biometric identifiers

They can be defined as the human physiological and behavioral traits used for unique identification and verification. They can be classified based on their capability to avoid intrusion: Intrusive and Non-intrusive. Every identifier has pros and cons and applies in various fields, as mentioned in Table 3. The selection of identifiers to be used in various systems is purely based on the system’s requirements to be designed.

Table 3
A summary of biometric identifiers with their advantages, disadvantages and applications

Biometric identifier Advantages Limitations Applications

Finger knuckle print ∙ Difficult to scrap ∙ Contactless acquisition ∙ Resilient to spoofing ∙ Distinct ∙ Reliable ∙ Small area for processing ∙ Access control

DNA ∙ Natural liveness detection ∙ Highly accurate ∙ Provides the possibility of linking relatives to an unknown person ∙ Requires more storage space ∙ Large processing time ∙ Expensive ∙ Not so convenient ∙ Physical and network security ∙ Defense and counter-terrorism ∙ Immigration and counter-human trafficking ∙ Disaster victim identification

ECG ∙ Measured continuously ∙ Natural liveness detection ∙ Robust to hardware variations, noise etc. ∙ Universal ∙ Non-intrusive ∙ Cannot be forged ∙ High operational speed ∙ Affected by the physical and psychological activity of individual ∙ Time-dependent ∙ Medical diagnostics ∙ Consumer electronics

Fingerprint ∙ Inexpensive ∙ Universal ∙ Non-intrusive ∙ Unique ∙ Immutable ∙ Reliable ∙ Performance impacted by cuts, scars, dust, dirt etc. ∙ Accuracy impacted by aging factor ∙ Contact-based ∙ Spoofing possible ∙ High FTE rate ∙ Hygienic issues ∙ Forensics for criminal identification ∙ License and Visa authentication ∙ Aadhaar identification ∙ Attendance management ∙ Border control

Hand vein/ palm vein ∙ Natural liveness detection ∙ Contactless ∙ Non-intrusive ∙ Difficult to forge ∙ Unique ∙ Time-invariant ∙ Hygienic ∙ Relatively expensive ∙ Image quality affected by change in body temperature ∙ Login authentication ∙ Physical access control ∙ Door security system ∙ Banking services

Face ∙ Non-intrusive ∙ Contactless ∙ widely accepted ∙ Accuracy affected by variations in the facial expressions ∙ Affected by aging ∙ Not used for twins identification ∙ Illumination and lighting conditions are to be considered ∙ Can be changed by surgery ∙ Large-sized template ∙ Video surveillance ∙ Airport security ∙ Mobile platform ∙ Identity verification ∙ Criminal identification

Finger vein ∙ Natural liveness detection ∙ Contactless ∙ Easy to sample ∙ Unique ∙ Non-intrusive ∙ Difficult to forge ∙ Low maintenance ∙ Unaffected by external factors such as aging, weather conditions etc. ∙ Time-invariant ∙ Minimized storage and transmission ∙ Hygienic ∙ medium-sized template ∙ Cannot be used for person having arthritis ∙ Relatively expensive ∙ Difficult to extract ∙ Access control ∙ Banking ∙ Login authentication

Biometric identifier	Advantages	Limitations	Applications
Finger knuckle print	∙ Difficult to scrap ∙ Contactless acquisition ∙ Resilient to spoofing ∙ Distinct ∙ Reliable	∙ Small area for processing	∙ Access control
DNA	∙ Natural liveness detection ∙ Highly accurate ∙ Provides the possibility of linking relatives to an unknown person	∙ Requires more storage space ∙ Large processing time ∙ Expensive ∙ Not so convenient	∙ Physical and network security ∙ Defense and counter-terrorism ∙ Immigration and counter-human trafficking ∙ Disaster victim identification
ECG	∙ Measured continuously ∙ Natural liveness detection ∙ Robust to hardware variations, noise etc. ∙ Universal ∙ Non-intrusive ∙ Cannot be forged ∙ High operational speed	∙ Affected by the physical and psychological activity of individual ∙ Time-dependent	∙ Medical diagnostics ∙ Consumer electronics
Fingerprint	∙ Inexpensive ∙ Universal ∙ Non-intrusive ∙ Unique ∙ Immutable ∙ Reliable	∙ Performance impacted by cuts, scars, dust, dirt etc. ∙ Accuracy impacted by aging factor ∙ Contact-based ∙ Spoofing possible ∙ High FTE rate ∙ Hygienic issues	∙ Forensics for criminal identification ∙ License and Visa authentication ∙ Aadhaar identification ∙ Attendance management ∙ Border control
Hand vein/ palm vein	∙ Natural liveness detection ∙ Contactless ∙ Non-intrusive ∙ Difficult to forge ∙ Unique ∙ Time-invariant ∙ Hygienic	∙ Relatively expensive ∙ Image quality affected by change in body temperature	∙ Login authentication ∙ Physical access control ∙ Door security system ∙ Banking services
Face	∙ Non-intrusive ∙ Contactless ∙ widely accepted	∙ Accuracy affected by variations in the facial expressions ∙ Affected by aging ∙ Not used for twins identification ∙ Illumination and lighting conditions are to be considered ∙ Can be changed by surgery ∙ Large-sized template	∙ Video surveillance ∙ Airport security ∙ Mobile platform ∙ Identity verification ∙ Criminal identification
Finger vein	∙ Natural liveness detection ∙ Contactless ∙ Easy to sample ∙ Unique ∙ Non-intrusive ∙ Difficult to forge ∙ Low maintenance ∙ Unaffected by external factors such as aging, weather conditions etc. ∙ Time-invariant ∙ Minimized storage and transmission ∙ Hygienic ∙ medium-sized template	∙ Cannot be used for person having arthritis ∙ Relatively expensive ∙ Difficult to extract	∙ Access control ∙ Banking ∙ Login authentication

Table 3

(Continued)

Biometric identifier	Advantages	Limitations	Applications
Ear	∙ Fast identification ∙ Stable ∙ Appearance is not affected by aging ∙ Less computational complexity ∙ Reduced processing time ∙ Low cost ∙ Suited for long term identification	∙ Accuracy impacted by presence of jewelry, hats etc.	∙ Attendance monitoring ∙ Crime investigation ∙ Surveillance
Signature	∙ Widely accepted ∙ Easy retrieval ∙ Low FAR ∙ Non-invasive ∙ Medium-sized template	∙ Can be forged ∙ Changes over time ∙ Influenced by physical and emotional condition of signing individual	∙ Document verification and authorization ∙ Banking services
Retina	∙ Reliable ∙ Cannot be forged ∙ Quick verification ∙ Distinct for twins ∙ Small-sized template	∙ Disease can affect the accuracy ∙ High equipment cost ∙ Uncomfortable ∙ Slow enrollment and scanning	∙ Security (FBI, CIA, and NASA)
Palmprint	∙ Reliable ∙ Uniqueness ∙ Permanent	∙ High equipment cost ∙ Affected by illumination variation and distortions	∙ Personal identification ∙ Medical diagnosis
Sclera	∙ Non-intrusive ∙ Unique ∙ Low FAR	∙ Require specialized hardware ∙ Invasive ∙ Can be spoofed	∙ Personal identification ∙ Medical diagnosis ∙ Border security
Iris	∙ Contactless ∙ Non-intrusive ∙ No effect of aging ∙ Reliable ∙ Hard to spoof ∙ Fast processing speed ∙ Non-invasive ∙ Small-sized template ∙ Uniqueness	∙ Expensive to use ∙ Acquisition in constrained environment ∙ Infrared radiations of scanner can be harmful ∙ Can be deformed ∙ Obscured by eyelashes, lenses, reflections ∙ Disease can affect the accuracy	∙ Healthcare ∙ Banking applications ∙ Restricted areas ∙ Aadhaar identification
Voice	∙ Low cost ∙ Contactless ∙ No special hardware required ∙ Convenient to use ∙ Small-sized template	∙ Accuracy can be affected by throat problems or illness. ∙ Susceptible to noise ∙ Low accuracy ∙ Can be spoofed ∙ Varies with age ∙ Impacted by emotional state of person ∙ Not good for large-scale identification	∙ Identification ∙ Web based transactions ∙ Entertainment and emergency services. ∙ Door management
Keystroke	∙ Fast identification ∙ No special hardware required	∙ Diseases, gap in days, change of keyboard etc., can change the typing rhythm ∙ Not so comfortable	∙ Multifactor authentication system
Gait	∙ Non-invasive ∙ Convenient ∙ Contactless ∙ Perceived from distance	∙ Computationally expensive ∙ Non-reliable ∙ Can be easily mimicked	∙ Forensics∙ Surveillance

3.4. Biometric datasets

A large variety of datasets for unimodal and multimodal biometric research are available in the biometric market. Table 4 and 5 present an overview of some of these datasets along with some notable points of interest. For multimodal-based research, the researchers preferred to use virtual datasets instead of making realistic ones based on the assumption that each characteristic of the individual bears distinct properties [46,47]. As depicted in Fig. 3, chimeric datasets are formed by combining multiple datasets with different modalities. The number of inter-class variations and intra-class variations in the dataset as well as the application’s requirements is used to choose between these different datasets to create one virtual dataset.

Table 4
List of unimodal datasets with detailed information

Database name Trait Density Conditions Acquisition device

Carnegie Mellon University (CMU) Multi-PIE Database [48] Face 750,000 images of 337 subjects Range of facial expressions was taken Canon EOS 10D (6.3-megapixel CMOS camera) with a Macro Ring Lite MR-14X ring flash.

Olivetti Research Laboratory (ORL) Face Database [49] Face 10 images of 40 subjects Light change, facial details and facial expressions Not Mentioned

North China University of Technology (NCUT) Database [50] Hand vein 150 images Natural lighting (indoor environment) Far-Infrared (FIR) or a Near-Infrared (NIR) technique of imaging

University of Science and Technology Beijing (USTB) Ear Database [51] Ear 3 images of 60 subjects Rotation, different lighting conditions Digital camera

USTB-Helloear Database [52] Ear 610,000 images of 1570 subjects Under uncontrolled conditions with illumination and pose variation Color CCD camera

Fingerprint Verification Competition (FVC) 2002 Database [53] Fingerprint 1680 images of 30 subjects Not mentioned Capacitive sensor

Fingerprint Verification Competition (FVC) 2004 Database [54] Fingerprint 1440 images of 30 subjects Not mentioned Optical sensor and thermal sweeping sensor

LivDet2015 Database [55] Fingerprint 100 subjects of 30 subjects Not mentioned CrossMatch, DigitalPersona, GreenBit and Biometrika

University of Palackeho and Olomouc (UPOL) Database [56] Iris 384 images of 64 subjects, each subject contributing 6 images (3 from left eye and 3 from right eye) Controlled environment with human subject cooperation TOPCON TRC50IA optical device connected with SONY DXC-950P 3CCD camera

Chinese Academy of Science’s Institute of Automation (CASIA) Database [57] Iris 756 images of 108 subjects Unconstrained environments, near infrared illumination CASIA close-up iris camera

Physikalisch-Technische Bundesanstalt (PTB) Database [58] ECG 549 records of 290 subjects Not mentioned Not mentioned

Check Your Biosignals Here initiative (CYBHi) Database [59] ECG 63 subjects Not mentioned Two ECG scanners

Database name	Trait	Density	Conditions	Acquisition device
Carnegie Mellon University (CMU) Multi-PIE Database [48]	Face	750,000 images of 337 subjects	Range of facial expressions was taken	Canon EOS 10D (6.3-megapixel CMOS camera) with a Macro Ring Lite MR-14X ring flash.
Olivetti Research Laboratory (ORL) Face Database [49]	Face	10 images of 40 subjects	Light change, facial details and facial expressions	Not Mentioned
North China University of Technology (NCUT) Database [50]	Hand vein	150 images	Natural lighting (indoor environment)	Far-Infrared (FIR) or a Near-Infrared (NIR) technique of imaging
University of Science and Technology Beijing (USTB) Ear Database [51]	Ear	3 images of 60 subjects	Rotation, different lighting conditions	Digital camera
USTB-Helloear Database [52]	Ear	610,000 images of 1570 subjects	Under uncontrolled conditions with illumination and pose variation	Color CCD camera
Fingerprint Verification Competition (FVC) 2002 Database [53]	Fingerprint	1680 images of 30 subjects	Not mentioned	Capacitive sensor
Fingerprint Verification Competition (FVC) 2004 Database [54]	Fingerprint	1440 images of 30 subjects	Not mentioned	Optical sensor and thermal sweeping sensor
LivDet2015 Database [55]	Fingerprint	100 subjects of 30 subjects	Not mentioned	CrossMatch, DigitalPersona, GreenBit and Biometrika
University of Palackeho and Olomouc (UPOL) Database [56]	Iris	384 images of 64 subjects, each subject contributing 6 images (3 from left eye and 3 from right eye)	Controlled environment with human subject cooperation	TOPCON TRC50IA optical device connected with SONY DXC-950P 3CCD camera
Chinese Academy of Science’s Institute of Automation (CASIA) Database [57]	Iris	756 images of 108 subjects	Unconstrained environments, near infrared illumination	CASIA close-up iris camera
Physikalisch-Technische Bundesanstalt (PTB) Database [58]	ECG	549 records of 290 subjects	Not mentioned	Not mentioned
Check Your Biosignals Here initiative (CYBHi) Database [59]	ECG	63 subjects	Not mentioned	Two ECG scanners

Table 5

Multimodal datasets with detailed information

Dataset name	Biometric traits	Acquisition device used	Data density
BIOMET [60]	Audio, face, signature, fingerprint and hand	SONY DCR-TR20-E Camera	91 individuals
MyIDEA [61]	Audio, face, signature, fingerprint, handwriting and hand geometry	Sony Digital Camera DCR-HC40 recording in DV Quality and regular webcam for audio and face, optical sensor Morphosmart MSO-100 and TocaBit scan-thermal sensor for fingerprint, perfection 1660 scanner for palmprint, Olympus C-370 CCD camera for hand geometry, A4 Intuos2 graphic tablet for signature and handwriting	104 individuals
Biosec [62]	Iris, voice, fingerprint and face	Webcam, Iris Sensor, Crosstalkn Headset, Webcam Microphone	200 individuals
BiosecurID [63]	Speech, iris, face, handwritten signature, handwritten text, fingerprint, hand and keystroke	Plantronics DSP 400, Biometrika FX 2000, Yubee Sensor, LG Iris Access EOU 3000, Scanner EPSON Perfevtion 4990, Philips ToUcam Pro II, Wacom Intuous3 A4/Inking Pen and Labtec Standard Keyboard SE	400 individuals
SDUMLA-HMT [64]	Face, vein, gait, iris and fingerprint	7 Cameras	106 individuals
MMU GASPFA [65]	Face, gait and speech	Digital voice recorder for speech recordings, Digital camera for face images, Digital video cameras, Kinect camera and Accelerometer equipped smart phones for capturing gait data.	82 individuals

If ‘C’ denotes the total number of classes in the given dataset and the total number of images captured per class in the dataset are represented by ‘ $N_{c}$ ’. Then, the total number of intra-class variations is given by the formula $\begin{matrix} (4) & Intra-class variations = ((N_{c} - 1) \times (N_{c} / 2) \times C) \end{matrix}$ and total number of inter-class variations are given by the formula $\begin{matrix} (5) & Inter-class variations = (C \times (C - 1) \times N_{c} \times N_{c}) \end{matrix}$

4. Challenges in biometric-based authentication

Whether it be for banking services, home security services, mobile services, or any other service, biometrics is becoming increasingly significant in our daily lives. However, even biometric solutions encounter numerous challenges since the system has several vulnerable points that can be exploited, as illustrated in Fig. 4. The main challenge in biometric authentication is to enhance the security of the biometric image over insecure public channels and the security of biometric templates stored in the database [66]. Since biometrics are increasingly becoming part of numerous identification and verification applications, various intruder attacks are also prevalent. Thus, specific security mechanisms must be devised to preserve biometric data’s security and integrity. Researchers have found that traditional cryptographic techniques [67] cannot provide better security to digital images than digital messages since cryptographic algorithms are not suitable for direct application to images. This is primarily due to the massive computational overhead associated with image encryption and the fact that images take quite a long time to encrypt due to their enormous size. So, they moved towards fragile watermarking [68,69], hoping that it would provide strict authentication services. But still, the problem persisted, so finally, biometric encryption was used for encrypting the template stored in the database to preserve the biometric image credentials from illegal users. For this, many biometric cryptosystems and feature transform-based algorithms [70] have been presented in the literature. Adler in [71] describes the biometric encryption system’s vulnerability to hill-climbing attacks leading to leakage of secret code and estimation of an enrolled image of the person. Jincey John et al. [72] have proposed a new biometric encryption scheme for biometric data security involving chaotic maps, but the algorithm uses ample-sized key space to provide enhanced security.

Usage of visual cryptographic technique [73] also became prevalent for securing biometric templates, but it involves pixel expansion, leading to increased image size and hence loss in original values of pixels, making the image degradable. The fuzzy vault scheme is also used in many research works [74–79] for providing security to biometric templates. Radha et al. demonstrated the use of a cancellable biometric technique with the added advantage of revocability, i.e., if the templates are compromised, they can be canceled and further reissued [80]. However, this technique also has a limitation, as reported by [81]; recognition accuracy of the system is reduced due to the high variance obtained by the distorted data when a transformation is applied to the user’s biometric data. In [82,83], a face de-identification technique has been introduced, and Bitouk et al. [84] proposed a face-swapping technique for the security of biometric templates of facial images, but both these techniques resulted in the loss of original image. Two schemes – visual cryptography technique without pixel expansion and visual secret sharing scheme for multiple secrets without image size expansion – were used by [85] to encrypt the biometric template and maintain system security to get around the drawbacks of pixel expansion and image size expansion. Nevertheless, these schemes suffer from the drawback of increased storage requirements due to the high-quality target image obtained.

Fig. 3.

Virtual or chimeric dataset.

Fig. 4.

Attack points in biometric system.

Das et al. [86] proposed a new two-factor authentication scheme based on two threat models and further proved that the proposed algorithm is secure against various kinds of attacks such as Denial of Service (DoS) attacks, privileged user attacks, impersonation attacks, the man in the middle attack and replay attack.

According to Tapase [87], an inverse relationship exists between biometric template security and matching performance in current biometric schemes. To address this issue, the authors propose an encryption method that prioritizes high levels of confusion and diffusion. Deploying multimodal biometric systems that utilize multiple biometric features to ensure data security is highly recommended. A comprehensive approach is necessary to develop a cost-effective security mechanism for template protection that can withstand various attacks [66,88]. Despite extensive research conducted by many scholars, they have yet to meet all the aforementioned requirements.

4.1. Alleviating authentication security and privacy concerns through multimodal biometric system

As discussed above, a multimodal biometric system proved advantageous regarding biometric data security. Additionally, they can withstand sensor or subsystem failures. Considering the fact that information fusion makes the system resilient to harmful activities, employing them to assure data security and privacy is a brilliant idea [89]. Numerous examples of it can be found in the literature.

Two biometric characteristics, namely the palmprint and palm vein, were obtained employing the newly developed acquisition system by Michael et al. [90]. Furthermore, they demonstrated that combining these produced a Genuine Acceptance Rate (GAR) of 99.73%, which was superior to the unimodal-based system when FAR was set to 0.001. In [91], a new human identification system for mobile-based devices using two biometric traits, namely palm print and knuckle, has been proposed. The system is designed to work in real-life conditions without any physical contact with the acquisition device. Jaswal et al. [92] used four biometric traits of a human hand, namely major finger knuckle, minor finger knuckle, palm print, and handprint. All these traits were acquired using a single virtual imaging device to reduce the user’s overhead. Results proved that combining different traits gives better performance with an Equal Error Rate (EER) of 0.01%, and the system is resilient against spoof attacks. Regouid et al. [93] created a novel multimodal system using internal (ECG) and external (ear, iris) features of human beings. The system involves normalizing ECG signals for noise reduction purposes and segmenting ear and iris features in the preprocessing step. Results showed 100% Correct Recognition Rate (CRR) with EER of 0.5%, 0% FAR and 1.1% FRR. An android-based user authentication method that makes use of a person’s visual and vocal characteristics was presented by Zhang et al. [94]. The approach must be improved because the required training data prevents it from being used in real applications.

Based on the cascading principle, which splits the authentication procedure into three layers, a multimodal biometric identification system is proposed in [95]. The algorithm would terminate if the first level of recognition is successful; otherwise, it would proceed to the second level, which would trigger the system to seek the third level if the second level failed. They used decision-level fusion with the AND rule applied to two or three different modalities at the second and third levels. However, if the impostor possesses prior knowledge of the modality used for the first level of authentication, the system is quite susceptible to attack. In [96], another multimodal system that used face, fingerprint, and voice signals as three biometric features is discussed. The dual cross pattern approach is used to preprocess face images, binarization and thinning are used to process fingerprint images, and the Mel-Frequency Cepstral Coefficients (MFCC) approach is used to extract the speech data. These obtained attributes are then combined in order to leverage various distance classifiers and machine learning-based techniques for ideal matching. A multimodal biometric identification system utilizing the user’s face and iris attributes was proposed by Xiong et al. [97], but the model suffers from the loss of crucial feature information caused by the conversion from feature tensors to vectors.

Multimodal biometric system usage surpasses unimodal biometric systems in the current scenario because it makes it more difficult for offenders to defeat system security and offers superior recognition rates. Table 6 summarizes various multimodal biometric systems explored in the literature and their shortcomings. The authors of [98–101] demonstrate that there is a critical need for the security of biometric templates being stored at the time of enrollment because this information is essential to authenticate a person afterward.

Table 6
Some state-of-the-art multimodal biometric systems with significant information, results and their limitations

Ref. Traits Technique used Database used Performance metrics Drawbacks Result

[108] Face and ear Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC) for classification, Principal Component Analysis (PCA) for Feature Extraction, Feature level fusion Extended Yale B Database, AR Database and USTB ear Database Balance factor and threshold Vulnerable to spoofing attack, no template protection Performance is 97.83% in the presence of sunglasses on face and 96.71% in the presence of scarf.

[109] Finger print, iris and face Minmax normalization sum, product and weighted sum rules are used to get fusion FVC 2004, CASIA Iris Database and Face94 database Time consumption, FAR, FRR and EER Less secure, external factors not considered Sum rule give an accuracy equal to 98.81%, product rule presents accuracy of 99.31%, and weighted sum rule gives the best accuracy i.e. 99.7%. Weighted sum rule which have 0.54% EER is more reliable than product rule with 0.83% EER and sum rule with 1.20% EER.

[98] Finger print, retina and finger vein Feature level fusion and Rivest–Shamir–Adleman (RSA) key generation Not mentioned FAR and Genuine Acceptance Rate (GAR) Vulnerable to stolen key attack GAR of 95.3% and FAR of 0.01%

[46] Iris, finger vein and fingerprint Backtracking Search Optimization Algorithm (BSA), optimal score level fusion model, Proportional Conflict Redistribution rules (PCR-6) IITD PolyU Iris Database, VERA Finger Vein Database, FVC 2006 Fingerprint Database and CASIA Fingerprint Database. Receiver Operating Characteristics (ROC) Curves, EER, decidability index and standard deviation System needs to be adaptable towards different security levels and traits importance needs to be considered Accuracy of 98.43% and an EER of 1.57%

[111] ECG and finger print Convolution Neural Network (CNN) and Q-Gaussian Multi Support Vector Machine (QG-MSVM), a feature level fusion and a decision level fusion Physikalisch-Technische Bundesanstalt (PTB) Database and Check Your Biosignals Here initiative (CYBHi) Database for ECG, LivDet2015 Database and FVC 2004 Database for Fingerprint Computational costs, Accuracy and EER Large waiting time for authentication in case of sequential system and there is no guarantee of liveness detection in case of parallel system EER is 0.10%, low computational cost

[112] Face, hand vein and ear Incremental Principal Component Analysis (IPCA) Feature Extraction, Multi-Objective Modified Genetic Algorithm (MOM-GA) for feature selection and K-Nearest Neighbour (KNN) classifier ORL Database, USTB Database and NCUT Database FAR and Genuine Acceptance Rate (GAR) Virtual dataset used, Face is sensitive to surroundings, costly Combined traits has higher FAR percentage with GA feature selection by 3.62% for face as well as ear and by 8.75% for face and hand vein. Also, they give higher FAR percentage with MOM-GA feature selection by 0.52% for face and ear and by 3.62% for face and hand vein

Ref.	Traits	Technique used	Database used	Performance metrics	Drawbacks	Result
[108]	Face and ear	Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC) for classification, Principal Component Analysis (PCA) for Feature Extraction, Feature level fusion	Extended Yale B Database, AR Database and USTB ear Database	Balance factor and threshold	Vulnerable to spoofing attack, no template protection	Performance is 97.83% in the presence of sunglasses on face and 96.71% in the presence of scarf.
[109]	Finger print, iris and face	Minmax normalization sum, product and weighted sum rules are used to get fusion	FVC 2004, CASIA Iris Database and Face94 database	Time consumption, FAR, FRR and EER	Less secure, external factors not considered	Sum rule give an accuracy equal to 98.81%, product rule presents accuracy of 99.31%, and weighted sum rule gives the best accuracy i.e. 99.7%. Weighted sum rule which have 0.54% EER is more reliable than product rule with 0.83% EER and sum rule with 1.20% EER.
[98]	Finger print, retina and finger vein	Feature level fusion and Rivest–Shamir–Adleman (RSA) key generation	Not mentioned	FAR and Genuine Acceptance Rate (GAR)	Vulnerable to stolen key attack	GAR of 95.3% and FAR of 0.01%
[46]	Iris, finger vein and fingerprint	Backtracking Search Optimization Algorithm (BSA), optimal score level fusion model, Proportional Conflict Redistribution rules (PCR-6)	IITD PolyU Iris Database, VERA Finger Vein Database, FVC 2006 Fingerprint Database and CASIA Fingerprint Database.	Receiver Operating Characteristics (ROC) Curves, EER, decidability index and standard deviation	System needs to be adaptable towards different security levels and traits importance needs to be considered	Accuracy of 98.43% and an EER of 1.57%
[111]	ECG and finger print	Convolution Neural Network (CNN) and Q-Gaussian Multi Support Vector Machine (QG-MSVM), a feature level fusion and a decision level fusion	Physikalisch-Technische Bundesanstalt (PTB) Database and Check Your Biosignals Here initiative (CYBHi) Database for ECG, LivDet2015 Database and FVC 2004 Database for Fingerprint	Computational costs, Accuracy and EER	Large waiting time for authentication in case of sequential system and there is no guarantee of liveness detection in case of parallel system	EER is 0.10%, low computational cost
[112]	Face, hand vein and ear	Incremental Principal Component Analysis (IPCA) Feature Extraction, Multi-Objective Modified Genetic Algorithm (MOM-GA) for feature selection and K-Nearest Neighbour (KNN) classifier	ORL Database, USTB Database and NCUT Database	FAR and Genuine Acceptance Rate (GAR)	Virtual dataset used, Face is sensitive to surroundings, costly	Combined traits has higher FAR percentage with GA feature selection by 3.62% for face as well as ear and by 8.75% for face and hand vein. Also, they give higher FAR percentage with MOM-GA feature selection by 0.52% for face and ear and by 3.62% for face and hand vein

Table 6

(Continued)

Ref.	Traits	Technique used	Database used	Performance metrics	Drawbacks	Result
[113]	Face and iris	Local Phase Quantization (LPQ) for feature extraction, Linear Discriminant Analysis (LDA) for dimensionality reduction, feature level fusion, KNN (linear and weighted), and Support Vector Machine (SVM) classifiers	MEPCO Biometric Database	Precision, recall, and F-measure	Impact of external factors such as expressions, jewellery etc. are not considered for face, data privacy issues	Recognition accuracy is 99.13 %
[114]	Face, iris and finger vein	Convolution Neural Network (CNN) and Softmax classifier, score level fusion, feature level fusion	SDUMLA-HMT Dataset and FV-USM Dataset	Accuracy	Pretrained model used, No template protection	Recognition performance is 99.39%
[110]	Finger-print and iris	Feature level fusion and Minutiae matcher technique	FVC 2002, FVC 2004 and KVK Fingerprint Database, CASIA Iris Database, Multimedia-University (MMU) Iris Database and KVKR-Iris Database	FAR, FRR, ROC curve, EER and Total Error Rate (TER)	Poor lighting conditions are not considered, virtual dataset used	Overall accuracy of 99.23% with FAR and FRR of 1.3% and 4.01%
[115]	Palm-print and Dorsal Hand Veins (DHV)	Deep Hashing Network (DHN), Hamming distance, Biometric Graph Matching (BGM) and score-level fusion of DHN and BGM	GPDS Database, NCUT Database, and XJTU Dorsal Hand Vein Databases and PolyU Multispectral Palmprint Database	FAR, FRR and EER	Virtual dataset used, high cost, contact-based acquisition	Both FAR and FRR are close to 0, EER of both palmprint recognition and multi-biometrics is equal to 0
[4]	ECG and fingerprint	Neural Network, Fuzzy Logic and Linear Discriminant Analysis (LDA) for Classification, Score level and decision level fusion	MIT-BIH database of ECG, and FVC 2004 Database of Fingerprint	Standard error of the mean (SEM), Area under ROC curve (AUC)	Virtual dataset used, Data privacy issues, used touch-based input acquisition	SEM is 0.00824 for sequential multimodal system and 0.01045 for parallel multimodal system, AUC is 0.951 using Neural Network (NN) classifier, 0.93688 using Fuzzy Logic (FL) classifier and 0.90214 using LDA classifier
[47]	Face and iris	Singular Spectrum Analysis (SSA) with the wavelet transform, 2D Log-Gabor filter, Spectral Regression Kernel Discriminant Analysis (SRKDA), Fuzzy K-Nearest Neighbor (FK-NN) for classification, score level and decision level fusion	CASIA V3 Iris Image Database, ORL and Face Recognition Technology (FERET) Database	Cumulative Match Characteristic (CMC) curve	Virtual dataset used, data privacy issues, aging factor not considered.	Recognition rate of 99.16% on CASIA-ORL Database and 99.33% on CASIA-FERET Database

Numerous models developed in the literature need to be more secure since they incorporate fewer modalities and high complexity levels, making them less useful for various organizations. Additionally, the models have substantial computational costs and do not use template security. Although Singh et al. [102] employed score-level fusion of ECG and fingerprint for authentication purposes, the system exhibits overfitting issues and is computationally complex, making it less useful for practical applications. The overfitting issue also affects the model put forward by [103,104]. Hammad and Wang in [105] suggested a new trustworthy CNN-based authentication system in conjunction with template protection to alleviate such issues in this combination.

To protect templates, Nafea et al. employed a blend of cryptography and watermarking schemes [106]. The proposed method is resistant to biometric attacks, key theft, brute force predictions of a valid template, and estimates of the Hadamard row of that template. The created templates are also diversified, non-invertible, and revocable without impairing system performance. However, the main disadvantage of this strategy is the deterioration of facial characteristics with age and the incapability of providing fingerprints in erratic weather. Authors in [107] developed a novel code called competition code derived from the combination of magnitude and orientation characteristics of finger vein and iris images. These magnitude and orientation features are obtained using Gabor operators, which are further resized and then processed. However, this model’s primary flaw is how susceptible it is to lighting and finger posture changes. As illustrated in Table 7, the multimodal-based authentication system still has several unresolved problems that must be addressed to offer more secure authentication services to different applications. The review of the listed papers [4,47,96,108–110] show that most of the multimodal systems developed so far stored raw biometric data into the database leading to the problem of non-revocability if compromised. The research gaps in the table (see Table 7) also emphasize the need to develop multimodal biometric systems with template protection to address data privacy issues.

Table 7

Research trends in multimodal biometric systems: advantages and research gaps

Ref.	Biometric traits	Addressed challenges	Research gaps
[110]	Fingerprint and iris	Better user acceptance Better performance	No template protection
[96]	Face, speech and fingerprint	Increased Computational speed Minimized FAR	No template protection
[47]	Face and iris	Iris has permanence characteristics Less complex	No template protection Facial recognition is still sensitive to the surroundings
[116]	Fingerprint and finger vein	Improved identification rate More accurate	No template protection
[117]	Hand geometry, palmprint and finger surface	From single image, several features are computed Different sensors are not required for image acquisition	No template protection High FTE rate due to poor image acquisition
[4]	ECG and fingerprint	Performance of different classifiers, different fusion levels and rules are considered. Natural Liveness Detection	No template protection ECG is prone to changes in physical activity, mental stress or exercise
[118]	Finger vein, fingerprint, fingershape and finger knuckle print	Less computational complexity Natural Liveness Detection High recognition performance due to larger distribution distance	No template protection
[119]	Finger vein and ECG	Less computational complexity Natural Liveness Detection High recognition performance Highly secure	No template protection Big data required for high performance First time authentication takes longer time
[120]	Face and ear	Non-invertible	Non-revocable Non-diverse
[121]	Face and palmprint	Non-invertible	Non-revocable Non-diverse
[122]	Iris and fingerprint	Non-invertible	Vulnerable to Channel attack High FAR
[101]	Finger knuckle and retina	Good performance	Non-revocable Vulnerable to attack during each authentication attempt
[123]	Face and iris	Integrity is preserved No performance degradation Non-invertible	Non-revocable
[124]	Fingerprint and iris	Non-invertible	Non-revocable Non-diverse
[106]	Face and fingerprint	Non-invertible Revocable Diverse Good performance	Face is sensitive to surroundings Needs to be tested for different modalities
[125]	Face and fingerprint	UnlinkableRevocable Tokenless	Face inaccuracies while recovering the key
[126]	ECG, sclera and fingerprint	Authentication performance Computational powerAdaptable to trait changes	No template protection

The review of papers [101,120,122–124,127] shows that most of the multimodal systems developed to provide template security are non-revocable. This non-revocability has recently been addressed by [128], but the level of performance is compromised.

5. Comprehensive overview of authentication research and future research directions

As stated in the preceding sections, there are certain issues with various authentication techniques. Traditional authentication methods are less reliable and vulnerable; location-based approaches have device-related problems; unimodal biometric systems are brittle and not widely used; and multimodal biometric systems have data security and privacy concerns. Multimodal biometric systems can replace conventional, frictionless, unimodal biometric systems and template protection strategies can address multimodal biometric system’s data security and privacy concerns. As depicted in Fig. 5, combining multimodal biometric systems and template protection will result in improved security services, with the added benefit of revocability in template protection. [129,130].

Human Action Recognition (HAR) is another growing field that uses various algorithms to recognize and comprehend human actions. It is increasingly widely used in security-critical applications as well. When integrated with HAR, biometrics can provide far more promising solutions by enabling human-computer interaction [131–134]. Islam et al. [135] proposed a Concatenated Action Descriptor (CAD) for efficiently recognizing one or two people. They made use of silhouettes and skeletal frames. A blend of signature-based optical flow, signature-based corner point, and BRISK descriptors are employed in CAD. In [136], the authors presented another technique, ASD-R, incorporating 3D skeletal joint spatial and temporal information. On the SBU Kinect Interaction Dataset, both techniques [135,136] obtained 95.6% accuracy. In [137], they recently suggested a human action recognition network based on SNSP properties. To reduce spatial and temporal redundancy in video-based content, the authors proposed action recognition based on Steered, Diversion, Quadrangular, Intricacy, and Obscurity (SDQIO) features, which allow for quick action detection in a variety of applications [138]. Therefore, it can be concluded from the above discussion that using HAR along with multimodal biometric systems will provide better security in real-time scenarios in the future.

6. Performance comparison of existing multimodal biometric systems based on fingerprint and iris modalities

Fig. 5.

Research flow in authentication.

In this section, the performance parameters of five existing multimodal biometric systems have been compared in the MATLAB environment. Because of the fingerprint’s consistency and acceptance across the board, despite its susceptibility to spoofing attacks, and because the iris has permanence characteristics that can get around a fingerprint’s limitations, multimodal systems that only use the fingerprint and iris modalities were chosen. In addition, fingerprints and irises have a high degree of uniqueness compared to other biometric features. The system is, therefore, ideal for safety-critical applications because its combination offers promising outcomes in terms of data security. These multimodal biometric systems, which are based on features taken from iris and fingerprint characteristics, are explained as follows:

Decision Level Fusion based Multimodal Biometric System (DLF-MBS)

Garg et al. [139] retrieved textural features from fingerprint and iris samples stored in the CASIA dataset. They used KNN and neural classifiers for classification and accomplished decision-level fusion utilizing logical conjunction and disjunction approaches. The neural classifiers with logical conjunction outperformed the other classification methods in terms of recognition accuracy.

Fuzzy Logic based Multimodal Biometric System (FL-MBS)

Abdolahi et al. [140] used weighted scores for both fingerprint and iris. A final judgment is then reached by using fuzzy logic for decision-level fusion. Due to its higher stability factor and more detailed features, the iris biometric modality is given more weight in this study than the fingerprint.

Fuzzy Matching based Multimodal Biometric System (FM-MBS)

Benaliouche and Touahria used the existing mono-modal systems with three distinct matching algorithms, out of which two are classical and one is their own proposal, to build and implement a multimodal biometric system that includes iris and fingerprint [141]. For matching purposes, the authors use three techniques: sum rule matching, weighted sum rule matching, and the proposed matching with fuzzy logic. The latter strategy uses fusion at the decision level, whereas the first two use fusion at the score level. Experimental validation using the CASIA iris database and the FVC 2004 fingerprint database revealed that using fuzzy logic produces superior outcomes to the sum rule and weighted sum rule.

Cascade-based Multimodal Biometric System (C-MBS)

Features of a fingerprint are extracted using minutiae-based techniques, and features of an iris are extracted using a log Gabor filter. A Hough transform is used for boundary detection in the iris, and for segmentation, a canny edge detector is used. Hamming distance is utilized for matching purposes. Experiments are performed on the CASIA-Iris V1 dataset, FVC 2000, and FVC 2002 datasets at various threshold levels, and results are promising when the threshold level is set to 0.55 [142].

High-Speed Frequency based Multimodal Biometric System (HSF-MBS)

Gawande et al. [143] used a log Gabor filter for feature extraction of both traits, which are then fused in the frequency domain and matched using hamming distance. The authors can achieve 0% FAR and increased execution speed due to the separation of templates based on the number of cores.

Table 8 shows results of all these multimodal biometric systems in terms of FAR, FRR and Accuracy.

Table 8

Results of above multimodal systems

Multimodal system	FAR	FRR	Accuracy	Accuracy (scaled by 100)
DLF-MBS	2.040	14.94	91.5	0.915
FL-MBS	2	2	98.3	0.983
FM-MBS	0	0.05	99.97	0.9997
C-MBS	0	0.0571	99.86	0.9986
HSF-MBS	0	4.3	97.85	0.9785

Fig. 6.

Comparison of fingerprint-iris based multimodal systems.

6.1. Analysis and discussions

As depicted in Fig. 6, the multimodal systems FM-MBS, C-MBS, and HSF-MBS are more secure as compared to systems DLF-MBS and FL-MBS in terms of imposter access due to their 0% FAR value, but the system HSF-MBS has a high degree of false rejections making it less usable in the applications that require frequent access by legitimate users. The FAR and FRR values of the systems FM-MBS and C-MBS are nearly comparable, while system FM-MBS offers a comparatively greater accuracy value. System DLF-MBS, which is less accurate and less usable than system FL-MBS, has a high FAR value that makes it susceptible to imposters. It also has a high FRR value that makes it less usable.

7. Conclusion

In the modern world, the incorporation of biometrics into the user authentication process has progressed substantially, and it is now widely used in every sector demanding critical access. Robust multimodal biometric systems are used frequently today for authentication because they enable maximum population coverage, an increased degree of freedom, improved recognition rates (low FAR and FRR values), are less vulnerable to spoof attacks, and have a lower failure to enroll rate than unimodal biometric systems, which have numerous problems in daily life. Despite these tremendous benefits, multimodal systems have storage issues due to the massive datasets, processing time-related issues, and computational demands. The primary advantage of such systems is data privacy and security, which can be achieved using various techniques, thus making them more acceptable in daily life. In this article, various biometric methods, along with measures taken for template security, have been explored. The whole study made us arrive at a common conclusion that using a multi-feature-based biometric authentication method would always be beneficial in terms of better acceptance rates and trickster identification. Additionally, five multimodal systems designed by various researchers are also analyzed to determine the best system in terms of security, recognition performance, and usability. Finally, the analysis’s findings indicate that, in terms of security, accuracy, and accessibility, the FM-MBS system outperforms all other biometric systems under consideration.

Footnotes

Acknowledgements

No funding has been received to carry this survey work.

References

Cherifi ,

Amroun and

Omar , Robust multimodal biometric authentication on IoT device through ear shape and arm gesture, Multimedia Tools and Applications 80 (2021), 14807–14827. doi:10.1007/s11042-021-10524-9.

Ahamed ,

Farid ,

Suleiman ,

Jan ,

L.A.

Wahsheh and

Shahrestani , An intelligent multimodal biometric authentication model for personalised healthcare services, Future Internet 14(8) (2022), 222. doi:10.3390/fi14080222.

Arora ,

Bhatia and

Kukreja , A multimodal biometric system for secure user identification based on deep learning, in: Proceedings of Fifth International Congress on Information and Communication Technology: ICICT 2020, Vol. 1, Springer, London, 2021, pp. 95–103. doi:10.1007/978-981-15-5856-6_8.

S.A.

El_Rahman , Multimodal biometric systems based on different fusion levels of ECG and fingerprint using different classifiers, Soft Computing 24 (2020), 12599–12632. doi:10.1007/s00500-020-04700-6.

Daas ,

Yahi ,

Bakir ,

Sedhane ,

Boughazi and

E.-B.

Bourennane , Multimodal biometric recognition systems using deep learning based on the finger vein and finger knuckle print fusion, IET Image Processing 14(15) (2020), 3859–3868. doi:10.1049/iet-ipr.2020.0491.

Shaheed ,

Mao ,

Qureshi ,

Kumar ,

Abbas ,

Ullah and

Zhang , A systematic review on physiological-based biometric recognition systems: Current and future trends, Archives of Computational Methods in Engineering (2021), 1–44.

Dargan and

Kumar , A comprehensive survey on the biometric recognition systems based on physiological and behavioral modalities, Expert Systems with Applications 143 (2020), 113114. doi:10.1016/j.eswa.2019.113114.

Minaee ,

Abdolrashidi ,

Su ,

Bennamoun and

Zhang , Biometrics recognition using deep learning: A survey, Artificial Intelligence Review (2023), 1–49. doi:10.1007/s10462-023-10534-z.

S.A.

Abdulrahman and

Alhayani , A comprehensive survey on the biometric systems based on physiological and behavioural characteristics, Materials Today: Proceedings 80 (2023), 2642–2646.

10.

Chiroma , Deep learning algorithms based fingerprint authentication: Systematic literature review, Journal of Artificial Intelligence and Systems 3(1) (2021), 157–197. doi:10.33969/AIS.2021.31010.

11.

Bala ,

Gupta and

Kumar , Multimodal biometric system based on fusion techniques: A review, Information Security Journal: A Global Perspective 31(3) (2022), 289–337.

12.

A.N.

Uwaechia and

D.A.

Ramli , A comprehensive survey on ECG signals as new biometric modality for human authentication: Recent advances and future challenges, IEEE Access 9 (2021), 97760–97802. doi:10.1109/ACCESS.2021.3095248.

13.

V.M.

Mane and

D.V.

Jadhav , Review of multimodal biometrics: Applications, challenges and research areas, International Journal of Biometrics and Bioinformatics (IJBB) 3(5) (2009), 90–95.

14.

Bhattacharyya ,

Ranjan ,

Alisherov ,

Choi et al., Biometric authentication: A review, International Journal of u-and e-Service, Science and Technology 2(3) (2009), 13–28.

15.

Unar ,

W.C.

Seng and

Abbasi , A review of biometric technology along with trends and prospects, Pattern recognition 47(8) (2014), 2673–2688. doi:10.1016/j.patcog.2014.01.016.

16.

I.M.

Alsaadi , Physiological biometric authentication systems, advantages, disadvantages and future development: A review, International Journal of Scientific & Technology Research 4(12) (2015), 285–289.

17.

Syazana-Itqan ,

Syafeeza ,

Saad ,

N.A.

Hamid and

Saad , A review of finger-vein biometrics identification approaches, Indian J. Sci. Technol 9(32) (2016), 1–9.

18.

P.L.

Galdámez ,

Raveane and

A.G.

Arrieta , A brief review of the ear recognition process using deep neural networks, Journal of Applied Logic 24 (2017), 62–70. doi:10.1016/j.jal.2016.11.014.

19.

J.P.

Singh ,

Jain ,

Arora and

U.P.

Singh , A survey of behavioral biometric gait recognition: Current success and future perspectives, Archives of Computational Methods in Engineering 28 (2021), 107–148. doi:10.1007/s11831-019-09375-3.

20.

Wang ,

Yang and

Zhu , Review of ear biometrics, Archives of Computational Methods in Engineering 28 (2021), 149–180. doi:10.1007/s11831-019-09376-2.

21.

Kortli ,

Jridi ,

Al Falou and

Atri , Face recognition systems: A survey, Sensors 20(2) (2020), 342. doi:10.3390/s20020342.

22.

M.O.

Diab ,

Seif ,

Sabbah ,

El-Abed and

Aloulou , A review on ecg-based biometric authentication systems, Hidden Biometrics: When Biometric Security Meets Biomedical Engineering (2020), 17–44. doi:10.1007/978-981-13-0956-4_2.

23.

Jakobsson ,

Jakobsson and

Dhiman , The benefits of understanding passwords, Mobile Authentication: Problems and Solutions (2013), 5–24. doi:10.1007/978-1-4614-4878-5_2.

24.

Gupta ,

Prajapati ,

Nedjah ,

Vijayakumar ,

A.A.A.

El-Latif and

Chang , Machine learning and smart card based two-factor authentication scheme for preserving anonymity in telecare medical information system (TMIS), Neural Computing and Applications 35(7) (2023), 5055–5080. doi:10.1007/s00521-021-06152-x.

25.

M.A.

Khan ,

Alhakami ,

A.V.

Shvetsov and

Ullah , A smart card-based two-factor mutual authentication scheme for efficient deployment of an IoT-based telecare medical information system, Sensors 23(12) (2023), 5419. doi:10.3390/s23125419.

26.

A.K.

Das and

Goswami , A robust anonymous biometric-based remote user authentication scheme using smart cards, Journal of King Saud University – Computer and Information Sciences 27(2) (2015), 193–210. doi:10.1016/j.jksuci.2014.03.020.

27.

M.A.

Mustafa ,

Abidin and

E.A.

Rúa , Frictionless authentication system: Security & privacy analysis and potential solutions, 2018, arXiv preprint arXiv:1802.07231.

28.

Zhang ,

Kondoro and

Muftic , Location-based authentication and authorization using smart phones, in: 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications, IEEE, 2012, pp. 1285–1292. doi:10.1109/TrustCom.2012.198.

29.

Singh and

Chatterjee , Lobac: A secure location-based access control model for e-healthcare system, in: Advances in Machine Learning and Computational Intelligence: Proceedings of ICMLCI 2019, Springer, 2021, pp. 621–628. doi:10.1007/978-981-15-5243-4_58.

30.

Hernández-Álvarez ,

J.M.

De Fuentes ,

González-Manzano and

L.H.

Encinas , SmartCAMPP-smartphone-based continuous authentication leveraging motion sensors with privacy preservation, Pattern Recognition Letters 147 (2021), 189–196. doi:10.1016/j.patrec.2021.04.013.

31.

Luo ,

S.-C.S.

Cheung ,

Lazzeretti ,

Pignata and

Barni , Anonymous subject identification and privacy information management in video surveillance, International Journal of Information Security 17 (2018), 261–278. doi:10.1007/s10207-017-0380-2.

32.

Bera ,

A.K.

Das ,

Balzano and

C.M.

Medaglia , On the design of biometric-based user authentication protocol in smart city environment, Pattern Recognition Letters 138 (2020), 439–446. doi:10.1016/j.patrec.2020.08.017.

33.

N.A.R.

Negri ,

G.M.R.

Borille and

V.A.

Falcão , Acceptance of biometric technology in airport check-in, Journal of Air Transport Management 81 (2019), 101720. doi:10.1016/j.jairtraman.2019.101720.

34.

Bawa , Framework to understand and deploy AI-based smart CRM in the airline industry, in: Adoption and Implementation of AI in Customer Relationship Management, IGI Global, 2022, pp. 154–165. doi:10.4018/978-1-7998-7959-6.ch010.

35.

Solano ,

Camacho ,

Correa ,

Deiro ,

Vargas and

Ochoa , Combining behavioral biometrics and session context analytics to enhance risk-based static authentication in web applications, International Journal of Information Security 20(2) (2021), 181–197. doi:10.1007/s10207-020-00510-x.

36.

Wang ,

Yan ,

Zhang and

Zhang , Attacks and defenses in user authentication systems: A survey, Journal of Network and Computer Applications 188 (2021), 103080. doi:10.1016/j.jnca.2021.103080.

37.

S.S.

Barpanda ,

P.K.

Sa ,

Marques ,

Majhi and

Bakshi , Iris recognition with tunable filter bank based feature, Multimedia Tools and Applications 77(6) (2018), 7637–7674. doi:10.1007/s11042-017-4668-z.

38.

Purohit and

P.K.

Ajmera , Optimal feature level fusion for secured human authentication in multimodal biometric system, Machine Vision and Applications 32 (2021), 1–12. doi:10.1007/s00138-020-01119-9.

39.

Yousefi ,

Kolivand and

Baker , SaS-BCI: A new strategy to predict image memorability and use mental imagery as a brain-based biometric authentication, Neural Computing and Applications 33 (2021), 4283–4297. doi:10.1007/s00521-020-05247-1.

40.

Alharbi and

H.S.

Alshanbari , Face–voice based multimodal biometric authentication system via FaceNet and GMM, PeerJ Computer Science 9 (2023), e1468. doi:10.7717/peerj-cs.1468.

41.

Singhal and

Shinghal , Secure deep multimodal biometric authentication using online signature and face features fusion, Multimedia Tools and Applications (2023), 1–20.

42.

Abo-Zahhad ,

S.M.

Ahmed and

S.N.

Abbas , PCG biometric identification system based on feature level fusion using canonical correlation analysis, in: 2014 IEEE 27th Canadian Conference on Electrical and Computer Engineering (CCECE), IEEE, 2014, pp. 1–6.

43.

Xin and

Xiaojun , Correlation-based identification approach for multimodal biometric fusion, The Journal of China Universities of Posts and Telecommunications 24(4) (2017), 34–50. doi:10.1016/S1005-8885(17)60221-8.

44.

S.K.S.

Modak and

V.K.

Jha , A novel multimodal biometric authentication framework using rule-based ANFIS based on hybrid level fusion, Wireless Personal Communications 128(1) (2023), 187–207. doi:10.1007/s11277-022-09949-8.

45.

Pranathi ,

A.M.

Rao ,

Sandhya ,

Inchara ,

Kanakareddy and

G.K.

Devadiga , Multimodal biometric authentication, in: 2023 International Conference on Smart Systems for Applications in Electrical Sciences (ICSSES), IEEE, 2023, pp. 1–6.

46.

G.S.

Walia ,

Singh ,

Singh and

Verma , Robust multimodal biometric system based on optimal score level fusion model, Expert Systems with Applications 116 (2019), 364–376. doi:10.1016/j.eswa.2018.08.036.

47.

Ammour ,

Boubchir ,

Bouden and

Ramdani , Face–iris multimodal biometric identification system, Electronics 9(1) (2020), 85. doi:10.3390/electronics9010085.

48.

Gross ,

Matthews ,

Cohn ,

Kanade and

Baker , Multi-pie, Image and vision computing 28(5) (2010), 807–813. doi:10.1016/j.imavis.2009.08.002.

49.

Senthilkumar and

Gnanamurthy , Face databases for 2D and 3D facial recognition: A survey, IOSR Journal of Engineering (IOSRJEN) 3(4) (2013), 43–48. doi:10.9790/3021-03414348.

50.

Huang ,

Tang ,

Wang ,

Chen and

Wang , Hand vein recognition based on oriented gradient maps and local feature matching, in: Computer Vision–ACCV 2012: 11th Asian Conference on Computer Vision, Daejeon, Korea, November 5–9, 2012, Revised Selected Papers, Part IV, Vol. 11, Springer, 2013, pp. 430–444. doi:10.1007/978-3-642-37447-0_33.

51.

Abaza and

M.A.F.

Harrison , Ear recognition: A complete system, in: Biometric and Surveillance Technology for Human and Activity Identification X, Vol. 8712, SPIE, 2013, pp. 145–155. doi:10.1117/12.2015946.

52.

Zhang ,

Mu ,

Yuan ,

Yu and

Liu , USTB-Helloear: A large database of ear images photographed under uncontrolled conditions, in: Image and Graphics: 9th International Conference, ICIG 2017, Shanghai, China, September 13–15, 2017, Revised Selected Papers, Part II, Vol. 9, Springer, 2017, pp. 405–416. doi:10.1007/978-3-319-71589-6_35.

53.

Maio ,

Maltoni ,

Cappelli ,

J.L.

Wayman and

A.K.

Jain , FVC2002: Second fingerprint verification competition, in: 2002 International Conference on Pattern Recognition, Vol. 3, IEEE, 2002, pp. 811–814.

54.

Maio ,

Maltoni ,

Cappelli ,

J.L.

Wayman and

A.K.

Jain , FVC2004: Third fingerprint verification competition, in: International Conference on Biometric Authentication, Springer, 2004, pp. 1–7.

55.

R.C.A.S.G.L.P.T.L.G.V.

Mura ,

Orrù and

G.L.

Marcialis , Livdet 2015 fingerprint liveness detection competition, in: IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS), IEEE, 2015, pp. 1–6.

56.

Dobes and

Machala , Upol iris image database. Accessed: Jun 15 2020 (2004).

57.

Tan and

Sun , CASIA-FingerprintV5. Accessed: Aug, 2010.

58.

A.L.

Goldberger ,

L.A.

Amaral ,

Glass ,

J.M.

Hausdorff ,

P.C.

Ivanov ,

R.G.

Mark ,

J.E.

Mietus ,

G.B.

Moody ,

C.-K.

Peng and

H.E.

Stanley , PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals, circulation 101(23) (2000), e215–e220. doi:10.1161/01.CIR.101.23.e215.

59.

H.P.

Da Silva ,

Lourenço ,

Fred ,

Raposo and

Aires-de-Sousa , Check your biosignals here: A new dataset for off-the-person ECG biometrics, Computer methods and programs in biomedicine 113(2) (2014), 503–514. doi:10.1016/j.cmpb.2013.11.017.

60.

Garcia-Salicetti ,

Beumier ,

Chollet ,

Dorizzi ,

J.L.L.

Jardins ,

Lunter ,

Ni and

Petrovska-Delacrétaz , BIOMET: A multimodal person authentication database including face, voice, fingerprint, hand and signature modalities, in: Audio-and Video-Based Biometric Person Authentication: 4th International Conference, AVBPA 2003, Guildford, UK, June 9–11, 2003, Proceedings, Vol. 4, Springer, 2003, pp. 845–853.

61.

Dumas ,

Pugin ,

Hennebert ,

Petrovska-Delacrétaz ,

Humm ,

Evéquoz ,

Ingold and

Von Rotz , Myidea-multimodal biometrics database, description of acquisition protocols (2005), 59–62.

62.

Fierrez ,

Ortega-Garcia ,

D.T.

Toledano and

Gonzalez-Rodriguez , BioSec baseline corpus: A multimodal biometric database, Pattern Recognition 40(4) (2007), 1389–1392. doi:10.1016/j.patcog.2006.10.014.

63.

Fierrez ,

Galbally ,

Ortega-Garcia ,

M.R.

Freire ,

Alonso-Fernandez ,

Ramos ,

D.T.

Toledano ,

Gonzalez-Rodriguez ,

J.A.

Siguenza ,

Garrido-Salas et al., BiosecurID: A multimodal biometric database, Pattern Analysis and Applications 13 (2010), 235–246. doi:10.1007/s10044-009-0151-4.

64.

Yin ,

Liu and

Sun , SDUMLA-HMT: A multimodal biometric database, in: Biometric Recognition: 6th Chinese Conference, CCBR 2011, Beijing, China, December 3–4, 2011, Proceedings, Vol. 6, Springer, 2011, pp. 260–268. doi:10.1007/978-3-642-25449-9_33.

65.

C.C.

Ho ,

Ng ,

W.-H.

Tan ,

K.-W.

Ng ,

H.-L.

Tong ,

T.T.-V.

Yap ,

P.-F.

Chong ,

Eswaran and

Abdullah , MMU GASPFA: A COTS multimodal biometric database, Pattern Recognition Letters 34(15) (2013), 2043–2050. doi:10.1016/j.patrec.2013.01.027.

66.

Pahuja and

Goel , State-of-the-art multi-trait based biometric systems: Advantages and drawbacks, in: International Conference on Emerging Technologies in Computer Engineering, Springer, 2022, pp. 704–714.

67.

Stallings , Cryptography and Network Security, 4/E, Pearson Education India, 2006.

68.

Lee ,

Lim ,

Yu ,

Kim and

Lee , Biometric image authentication using watermarking, in: 2006 SICE-ICASE International Joint Conference, IEEE, 2006, pp. 3950–3953. doi:10.1109/SICE.2006.314934.

69.

M.M.A.

Allah , Embedded biometric data for a secure authentication watermarking, in: Proceedings of the Fourth Conference on IASTED International Conference: Signal Processing, Pattern Recognition, and Applications, 2007, pp. 191–196.

70.

J.-H.

Im ,

S.-Y.

Jeon and

M.-K.

Lee , Practical privacy-preserving face authentication for smartphones secure against malicious clients, IEEE Transactions on Information Forensics and Security 15 (2020), 2386–2401. doi:10.1109/TIFS.2020.2969513.

71.

Adler , Vulnerabilities in biometric encryption systems, in: International Conference on Audio-and Video-Based Biometric Person Authentication, Springer, 2005, pp. 1100–1109. doi:10.1007/11527923_114.

72.

John and

A.S.

Raj , Reliable biometric data encryption using chaotic map, International Journal of Advanced Research in Computer and Communication Engineering 4(11) (2015), 52–56.

73.

Ross and

Othman , Visual cryptography for biometric privacy, IEEE transactions on information forensics and security 6(1) (2010), 70–81. doi:10.1109/TIFS.2010.2097252.

74.

Ponce-Hernandez ,

Blanco-Gonzalo ,

Sanchez-Reillo and

Liu-Jimenez , Template protection approaches: Fuzzy vault scheme, in: 2019 International Carnahan Conference on Security Technology (ICCST), IEEE, 2019, pp. 1–5.

75.

Chitra and

Sujitha , Security analysis of prealigned fingerprint template using fuzzy vault scheme, Cluster Computing 22(Suppl 5) (2019), 12817–12825. doi:10.1007/s10586-018-1762-6.

76.

Mehmood and

Selwal , Polynomial based fuzzy vault technique for template security in fingerprint biometrics, Int. Arab J. Inf. Technol. 17(6) (2020), 926–934.

77.

V.S.

Baghel ,

Prakash and

Agrawal , An enhanced fuzzy vault to secure the fingerprint templates, Multimedia Tools and Applications 80 (2021), 33055–33073. doi:10.1007/s11042-021-11325-w.

78.

Rathgeb ,

Merkle ,

Scholz ,

Tams and

Nesterowicz , Deep face fuzzy vault: Implementation and performance, Computers & Security 113 (2022), 102539. doi:10.1016/j.cose.2021.102539.

79.

Albermany and

F.M.

Baqer , EEG authentication system using fuzzy vault scheme, Journal of Discrete Mathematical Sciences and Cryptography 25(8) (2022), 2405–2410. doi:10.1080/09720529.2020.1859798.

80.

Radha and

Karthikeyan , An evaluation of fingerprint security using noninvertible biohash, International Journal of Network Security & Its Applications (IJNSA) 3(4) (2011), 1–11. doi:10.5121/ijnsa.2011.3401.

81.

E.Y.

Du ,

Yang ,

Zhou et al., Key incorporation scheme for cancelable biometrics, Journal of Information Security 2(04) (2011), 185. doi:10.4236/jis.2011.24018.

82.

E.M.

Newton ,

Sweeney and

Malin , Preserving privacy by de-identifying face images, IEEE transactions on Knowledge and Data Engineering 17(2) (2005), 232–243. doi:10.1109/TKDE.2005.32.

83.

Gross ,

Sweeney ,

De la Torre and

Baker , Model-based face de-identification, in: 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06), IEEE, 2006, pp. 161–161. doi:10.1109/CVPRW.2006.125.

84.

Bitouk ,

Kumar ,

Dhillon ,

Belhumeur and

S.K.

Nayar , Face swapping: Automatically replacing faces in photographs, in: ACM SIGGRAPH 2008 Papers, 2008, pp. 1–8.

85.

Nandakumar ,

Prabakar and

Karthik , Securing the images using biometric cryptography technique, International Journal of Computer Trends and Technology 4(3) (2013), 253–257.

86.

A.K.

Das ,

Wazid ,

Kumar ,

A.V.

Vasilakos and

J.J.

Rodrigues , Biometrics-based privacy-preserving user authentication scheme for cloud-based industrial Internet of Things deployment, IEEE Internet of Things Journal 5(6) (2018), 4900–4913. doi:10.1109/JIOT.2018.2877690.

87.

G.T

Pranoti ,

D.R.

Dandekar and

G.N.

Wazurkar , Lossless encryption technique for finger biometric images, International Research Journal of Engineering and Technology (IRJET) 5(4) (2018), 580–582.

88.

Jain and

Kant , Attacks on biometric systems: An overview, International Journal of Advances in Scientific Research 1(07) (2015), 283–288. doi:10.7439/ijasr.v1i7.1975.

89.

Marasco ,

Ding and

Ross , Combining match scores with liveness values in a fingerprint verification system, in: 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS), IEEE, 2012, pp. 418–425. doi:10.1109/BTAS.2012.6374609.

90.

G.K.O.

Michael ,

Connie and

A.B.J.

Teoh , A contactless biometric system using palm print and palm vein features, Advanced Biometric Technologies (2011), 155–177.

91.

Choraś and

Kozik , Contactless palmprint and knuckle biometrics for mobile devices, Pattern Analysis and Applications 15(1) (2012), 73–85. doi:10.1007/s10044-011-0248-4.

92.

Jaswal ,

Nigam ,

Kaul ,

Nath and

A.K.

Singh , Bring your own hand: How a single sensor is bringing multiple biometrics together, Soft Computing 23 (2019), 9121–9139. doi:10.1007/s00500-018-03709-2.

93.

Regouid ,

Touahria ,

Benouis and

Costen , Multimodal biometric system for ECG, ear and iris recognition based on local descriptors, Multimedia Tools and Applications 78 (2019), 22509–22535. doi:10.1007/s11042-019-7467-x.

94.

Zhang ,

Cheng ,

Jia ,

Dai and

Xu , An efficient Android-based multimodal biometric authentication system with face and voice, IEEE Access 8 (2020), 102757–102772. doi:10.1109/ACCESS.2020.2999115.

95.

R.A.

El Mehdi Cherrat and

Bouzahir , A multimodal biometric identification system based on cascade advanced of fingerprint, fingervein and face images, Indonesian Journal of Electrical Engineering and Computer Science 18(1) (2020), 1562–1570.

96.

Kumari ,

Rieit and

Sharma , A novel approach for secure multimodal biometric system using multiple biometric traits, IJRAR-International Journal of Research and Analytical Reviews (IJRAR) 7(1) (2020), 62–67.

97.

Xiong ,

Zhang ,

Xu and

He , A modified chaotic binary particle swarm optimization scheme and its application in face-iris multimodal biometric identification, Electronics 10(2) (2021), 217. doi:10.3390/electronics10020217.

98.

Jagadiswary and

Saraswady , Biometric authentication using fused multimodal biometric, Procedia Computer Science 85 (2016), 109–116. doi:10.1016/j.procs.2016.05.187.

99.

Yang ,

Wang ,

Hu ,

Zheng and

Valli , A fingerprint and finger-vein based cancelable multi-biometric system, Pattern Recognition 78 (2018), 242–251. doi:10.1016/j.patcog.2018.01.026.

100.

G.S.

Walia ,

Rishi ,

Asthana ,

Kumar and

Gupta , Secure multimodal biometric system based on diffused graphs and optimal score fusion, IET Biometrics 8(4) (2019), 231–242. doi:10.1049/iet-bmt.2018.5018.

101.

Patil and

Jagtap , Multi-modal biometric system using finger knuckle image and retina image with template security using PolyU and DRIVE database, International Journal of Information Technology 12 (2020), 1043–1050. doi:10.1007/s41870-020-00501-0.

102.

Y.N.

Singh ,

S.K.

Singh and

Gupta , Fusion of electrocardiogram with unobtrusive biometrics: An efficient individual authentication system, Pattern Recognition Letters 33(14) (2012), 1932–1941. doi:10.1016/j.patrec.2012.03.010.

103.

Manjunathswamy ,

Appaji Abhishek ,

Thriveni and

Venugopal , Multimodel biometrics using ECG and fingerprint, International Journal of Computer Applications (2014), 777–784.

104.

Komeili ,

Armanfard and

Hatzinakos , Liveness detection and automatic template updating using fusion of ECG and fingerprint, IEEE Transactions on Information Forensics and Security 13(7) (2018), 1810–1822. doi:10.1109/TIFS.2018.2804890.

105.

Hammad and

Wang , Parallel score fusion of ECG and fingerprint for human authentication based on convolution neural network, Computers & Security 81 (2019), 107–122. doi:10.1016/j.cose.2018.11.003.

106.

Nafea ,

Ghouzali ,

Abdul and

E.-U.-H.

Qazi , Hybrid multi-biometric template protection using watermarking, The Computer Journal 59(9) (2016), 1392–1407. doi:10.1093/comjnl/bxv107.

107.

Ilankumaran and

Deisy , Multi-biometric authentication system using finger vein and iris in cloud computing, Cluster Computing 22(Suppl 1) (2019), 103–117. doi:10.1007/s10586-018-1824-9.

108.

Huang ,

Liu ,

Li ,

Yang and

Chen , A robust face and ear based multimodal biometric system using sparse representation, Pattern Recognition 46(8) (2013), 2156–2168. doi:10.1016/j.patcog.2013.01.022.

109.

Aboshosha ,

K.A.

El Dahshan ,

E.A.

Karam and

E.A.

Ebeid , Score level fusion for fingerprint, iris and face biometrics, International Journal of Computer Applications 111(4) (2015). doi:10.5120/19530-1171.

110.

A.M.

Siddiqui ,

R.L.

Telgad ,

S.A.

Lothe and

P.D.

Deshmukh , Development of secure multimodal biometric system for person identification using feature level fusion: Fingerprint and iris, in: Recent Trends in Image Processing and Pattern Recognition: Second International Conference, RTIP2R 2018, Solapur, India, December 21–22, 2018, Revised Selected Papers, Part II, Vol. 2, Springer, 2018, pp. 406–432.

111.

Hammad ,

Liu and

Wang , Multimodal biometric authentication systems using convolution neural network based on different level fusion of ECG and fingerprint, IEEE Access 7 (2018), 26527–26542. doi:10.1109/ACCESS.2018.2886573.

112.

Karthiga and

Mangai , Feature selection using multi-objective modified genetic algorithm in multimodal biometric system, Journal of medical systems 43 (2019), 1–11. doi:10.1007/s10916-018-1115-2.

113.

Supreetha Gowda ,

Hemantha Kumar and

Imran , Multimodal biometric recognition system based on nonparametric classifiers, in: Data Analytics and Learning: Proceedings of DAL 2018, Springer, 2019, pp. 269–278. doi:10.1007/978-981-13-2514-4_23.

114.

Alay and

H.H.

Al-Baity , Deep learning approach for multimodal biometric recognition system based on fusion of iris, face, and finger vein traits, Sensors 20(19) (2020), 5523. doi:10.3390/s20195523.

115.

Zhong ,

Shao and

Du , A hand-based multi-biometrics via deep hashing network and biometric graph matching, IEEE Transactions on Information Forensics and Security 14(12) (2019), 3140–3150. doi:10.1109/TIFS.2019.2912552.

116.

Yang and

Zhang , Feature-level fusion of fingerprint and finger-vein for personal identification, Pattern Recognition Letters 33(5) (2012), 623–628. doi:10.1016/j.patrec.2011.11.002.

117.

Sanches ,

Antunes and

P.L.

Correia , A single sensor hand biometric multimodal system, in: 2007 15th European Signal Processing Conference, IEEE, 2007, pp. 30–34.

118.

Peng ,

A.A.

Abd El-Latif ,

Li and

Niu , Multimodal biometric authentication based on score level fusion of finger biometrics, Optik 125(23) (2014), 6891–6897. doi:10.1016/j.ijleo.2014.07.027.

119.

B.A.

El-Rahiem ,

F.E.A.

El-Samie and

Amin , Multimodal biometric authentication based on deep fusion of electrocardiogram (ECG) and finger vein, Multimedia Systems 28(4) (2022), 1325–1337. doi:10.1007/s00530-021-00810-9.

120.

Yuan , Multimodal cryptosystem based on fuzzy commitment, in: 2014 IEEE 17th International Conference on Computational Science and Engineering, IEEE, 2014, pp. 1545–1549. doi:10.1109/CSE.2014.286.

121.

L.R.

Haddada ,

Dorizzi and

N.E.B.

Amara , Watermarking signal fusion in multimodal biometrics, in: International Image Processing, Applications and Systems Conference, IEEE, 2014, pp. 1–6.

122.

Sridevi and

Shobana , Multimodal security of iris and fingerprint with bloom filters, Aegaeum Journal 8(10) (2020), 402–410.

123.

Kant and

Chaudhary , A watermarking based approach for protection of templates in multimodal biometric system, Procedia Computer Science 167 (2020), 932–941. doi:10.1016/j.procs.2020.03.392.

124.

Paunwala and

Patnaik , Biometric template protection with DCT-based watermarking, Machine vision and applications 25(1) (2014), 263–275. doi:10.1007/s00138-013-0533-x.

125.

M.J.

Lee ,

A.B.J.

Teoh ,

Uhl ,

S.-N.

Liang and

Jin , A tokenless cancellable scheme for multimodal biometric systems, Computers & Security 108 (2021), 102350. doi:10.1016/j.cose.2021.102350.

126.

S.P.

Singh and

Tiwari , A dual multimodal biometric authentication system based on WOA-ANN and SSA-DBN techniques, Sci 5(1) (2023), 10. doi:10.3390/sci5010010.

127.

S.S.

Ali ,

V.S.

Baghel ,

I.I.

Ganapathi and

Prakash , Robust biometric authentication system with a secure user template, Image and Vision Computing 104 (2020), 104004. doi:10.1016/j.imavis.2020.104004.

128.

Yang ,

Wang ,

Shahzad and

Zhou , A cancelable biometric authentication system based on feature-adaptive random projection, Journal of Information Security and Applications 58 (2021), 102704. doi:10.1016/j.jisa.2020.102704.

129.

Singh and

Y.N.

Singh , Cancelable multibiometrics using random projection and fuzzy quantization for template protection, Journal of Intelligent & Fuzzy Systems 43(3) (2022), 3155–3171. doi:10.3233/JIFS-212814.

130.

Singh and

Y.N.

Singh , Cancelable multibiometrics template security using deep binarization and secure hashing, International Journal of Pattern Recognition and Artificial Intelligence 37(05) (2023), 2356007. doi:10.1142/S0218001423560074.

131.

Mekruksavanich and

Jitpattanakul , Biometric user identification based on human activity recognition using wearable sensors: An experiment using deep learning models, Electronics 10(3) (2021), 308. doi:10.3390/electronics10030308.

132.

Aquino ,

M.G.

Costa and

C.F.

Costa Filho , Explaining one-dimensional convolutional models in human activity recognition and biometric identification tasks, Sensors 22(15) (2022), 5644. doi:10.3390/s22155644.

133.

J.P.

Gupta ,

Singh ,

Dixit ,

V.B.

Semwal and

S.R.

Dubey , Human activity recognition using gait pattern, International Journal of Computer Vision and Image Processing (IJCVIP) 3(3) (2013), 31–53. doi:10.4018/ijcvip.2013070103.

134.

N.R.

Nair ,

Moya Rueda ,

Reining and

G.A.

Fink , Multi-channel time-series person and soft-biometric identification, in: International Conference on Pattern Recognition, Springer, 2022, pp. 256–272.

135.

M.S.

Islam ,

Iqbal ,

Naqvi ,

Bakhat ,

M.M.

Islam ,

Kanwal and

Ye , CAD: Concatenated action descriptor for one and two person (s), using silhouette and silhouette’s skeleton, IET Image Processing 14(3) (2020), 417–422. doi:10.1049/iet-ipr.2018.6437.

136.

M.S.

Islam ,

Bakhat ,

Khan ,

Iqbal ,

M.M.

Islam and

Ye , Action recognition using interrelationships of 3D joints and frames based on angle sine relation and distance features using interrelationships, Applied Intelligence 51 (2021), 6001–6013. doi:10.1007/s10489-020-02176-3.

137.

M.S.

Islam ,

Bakhat ,

Khan ,

Naqvi ,

M.M.

Islam and

Ye , Applied human action recognition network based on SNSP features, Neural Processing Letters 54(3) (2022), 1481–1494. doi:10.1007/s11063-021-10585-9.

138.

M.S.

Islam ,

Bakhat ,

Iqbal ,

Khan ,

Ye and

M.M.

Islam , Representation for action recognition with motion vector termed as: SDQIO, Expert Systems with Applications 212 (2023), 118406. doi:10.1016/j.eswa.2022.118406.

139.

S.N.

Garg ,

Vig and

Gupta , Multimodal biometric system based on decision level fusion, in: 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES), IEEE, 2016, pp. 753–758. doi:10.1109/SCOPES.2016.7955540.

140.

Abdolahi ,

Mohamadi and

Jafari , Multimodal biometric system fusion using fingerprint and iris with fuzzy logic, International Journal of soft computing and engineering 2(6) (2013), 504–510.

141.

Benaliouche ,

Touahria et al., Comparative study of multimodal biometric recognition by fusion of iris and fingerprint, The Scientific World Journal 2014 (2014).

142.

Elhoseny ,

Essa ,

Elkhateb ,

A.E.

Hassanien and

Hamad , Cascade multimodal biometric system using fingerprint and iris patterns, in: Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2017, Springer, 2018, pp. 590–599. doi:10.1007/978-3-319-64861-3_55.

143.

Gawande ,

S.R.

Nair ,

Balani ,

Pawar and

Kotpalliwar , A high speed frequency based multimodal biometric system using iris and fingerprint, International Journal on Advanced Computer Engineering and Communication Technology 1(2) (2012), 66–73.

Multimodal biometric authentication: A review

Abstract

Keywords

1. Introduction

2.1. Specific information-based authentication

2.2. Heterogeneous information-based authentication

2.3. Frictionless multitudinal information-based authentication

3. Biometric-based authentication

Table 2 Difference between unimodal and multimodal systems

3.3. Biometric identifiers

6. Performance comparison of existing multimodal biometric systems based on fingerprint and iris modalities

7. Conclusion

Footnotes

Acknowledgements

References

Table 2
Difference between unimodal and multimodal systems