Chatbots for online banking services

1. Introduction

“Digitalisation, the surge of mobile and internet connected devices has revolutionised the way people interact with one another and communicate with businesses” (Eeuwen [11]). Millennials are accepting and supporting new technology in their everyday life. Technology companies are streamlining Artificial Intelligence (AI) into the products they offer, such as Google Assistant, Google Home and Amazon Alexa. The new and upcoming generation are expected to be critical and game changing customers for businesses. They demand effortless experiences, answers within seconds, not minutes and more intelligent self-service options (Teller Vision [31]). The banking and the financial service industry was one of the first industries to adopt technology. This integration has grown massively, helping banks reach a wider customer base enabling them to perform their banking conveniently (Baptista and Oliveira [2]). Banks are becoming ever more competitive with each other to adopt the newest advancements in technology to provide an improved delivery service to satisfy customers. Ulster Bank, Deloitte, AIB and PTSB are wanting to focus on integrating new technology to improve the speed at which transactions are acknowledged (Global Banking News, 2017). With this in mind the relationship with the customer is always evolving due to the growth of technology.

Banks are now enabling the use of technology so customers can perform more tasks online, such as; cheque image clearing to allow the payment of cheques remotely and intelligent chatbots to increase customer service and assist employees. A chatbot is a “simple software program that can respond to customer prompts i.e. what’s my bank balance?” (Balkhi [1]). Mastercard has launched Kai an artificial intelligent chatbot and other bots for financial services. They can handle customer queries such as: ‘what is APR?’, requests, look at spending habits and solve problems. This in turn enables financial institutions to provide a new, engaging experience and strengthen their relationship with the customer, with the aid of natural language used by bots to establish a more personal and contextual conversation [24]. The focus of this project is to implement these new technologies to create an intelligent chatbot to enable banks to appeal to millennials and potentially gain a lifelong customer.

The banking industry has multiple electronic delivery channels in use to distribute technology assets and services for the benefit of their customers. Online banking is a commodity of commerce within financial services as well as banking industries [15]. Advancements in technology has transformed many of our services into the digital era and the banking industry is one of the primary industries to avail of these advancements to improve their services. Currently within the UK two paradigms are available for online banking. One of which is an integrated internet bank which still operates through the branch but has an online presence. The other, a stand-alone internet bank, that operates completely independently and its only existence is solely through the internet (MarketLine [23]). Banks implement technology to strengthen their processing capacity, acquire a larger customer base and expand the services they could offer (Consoli [6]). The use of internet banking has grown in demand enormously in the last decade. “15% of branch customers use online banking once a day, 59% once a week, 77% at least once a month and 53% were confident in carrying out the best part of their banking online” (Barty and Recketts [3]). Online banking has become more popular as it negates the need for customers to visit their local branch as they can manage their finances on the go to meet the demand of modern life. This is evident as branchless banks are now emerging from the industry such as Atom Bank and many banks now closing some of their branches. This is evident with the recent closure of 11 Ulster Bank Branches in NI due to the increased number of customers performing their banking online (The Belfast Telegraph). However, a recent study by Ling et al. [22], notes that most internet banking service providers struggle to get many of their customers to use their service. They identify lack of customer satisfaction when using online banking services to be a major cause. Service quality, web design and content, security, privacy, speed and convenience are stated as the top factors influencing customer satisfaction (Ling et al. [22]). This suggests that there is a lack of technology in place to enhance the customer online banking experience which could be improved by integrating a chatbot to provide an efficient, convenient and personal service.

This paper outlines a banking chatbot to facilitate users with queries and assist with personal banking. The application allows users to check their balance, account details & transactions. It also integrates SMS, currency conversions, two factor authentication and the ability to interact with the service through personal assistants like Google Assistant.

The advantages of our chatbot is that it allows users to interact with their bank through natural language interaction, granting users more convenient and efficient access to their banking information. We show out system performing at a very high standard and provide reliable and rapid responses to users compared to that of traditional methods. The average time spent interacting with the chatbot is very low as it provides an efficient way for users to manage their banking. The low interaction time reflects the high understanding and speech recognition rates, offered through the adoption of conversational user interfaces thus allowing users to freely interact with the chatbot to meet the demands of modern life. The chatbot has the advantage over traditional services in that it can fulfil the demand of users wanting instant access and availability information and services. We outline the design and implementation in following sections.

2. Chatbots

A chatbot is a software tool that utilises natural language processing (NLP) for human machine interaction (HMI) and Machine Learning (ML). “The complexity of a chatbot is directionally proportional to the scope of the domain”. An open domain requires a larger knowledge base, whereas, a closed domain has a more specific knowledge base that was developed to achieve a specific goal (Gregori [17]). Chatbot technology initially began in the 1960s to determine whether a chatbot could be portrayed as a human. Throughout the 1980s there was an elevated amount research carried out on natural language interfaces which lead to the development of sophisticated chatbot architectures such as A.L.I.C.E. This chatbot architecture is one of the earlier chatbots developed in 1995 by Dr Wallace which is now open-source, the acronym stands for Artificial Linguistic Internet Computer Entity. This is a chatbot you can create through interaction as it will learn from previous interactions to create its knowledge base. Its knowledge is saved in AIML (Artificial Intelligent Mark-up Language) files which evolved from the Extensible Mark-up Language (XML) (Shawar and Atwell [30]). Chatbots are developed using two approaches; a rule-based approach where chatbots operate by moving through branches of a tree diagram of an expert system. The second approach involves advanced artificial intelligence and machine learning, thus the chatbot can learn from the conversations, generating appropriate responses to continuously improve over time (Watson [32]). There are two modes in which chatbots can simulate a conversation with users which include system-initiated chatbots where they commence the conversation with the user and User-initiated chatbots where the user directs the conversation instead. Systems that incorporate the two methods of initiation are known as mixed initiative systems (Duijst [9]).

Most businesses and organisations are understanding the potential benefits of machine learning and artificial intelligence to have a positive change on how they perform business. Artificial intelligence has progressed to allow the development of more sophisticated chatbots. Organisations are focusing on specific areas of user engagement that take up a lot of time but can be replaced through the use of a chatbot. Chatbots can understand what the customer needs from a single text instead of the customer having to follow a process of multiple steps. Chatbots are used to automate customer service and reduce manual tedious tasks performed by employees so they can spend their time more productively on higher priority tasks. Establishments that regularly deal with its customers have discovered the potential of chatbots as a channel to distribute more efficient and immediate information to customers in comparison to a customer service representative regarding queries and issues (Onufreiv [26]). HDFC Bank has merged with Niki.ai an artificial intelligence company to develop a state of the art conversational banking chatbot (HDFC Bank [14]). The chatbot is accessible through the banks Facebook Messenger allowing users to utilise e-commerce and banking transactions all within the chatbot. There is also a chatbot integrated within the HDFC login page to assist in online banking. The chatbot was developed as a concierge service, this is definitely one approach that can increase customer satisfaction within online banking enabling banks to develop a better relationship with its customers. Chatbots will renew and modernise the customer service industry and the main sectors outlined including the banking and healthcare sector (Newswire [25]). Industries such as; retail, healthcare, e-commerce and banking, are expected to achieve considerable savings from integrating a chatbot. Between 2017 and 2020 chatbots are estimated to make savings of $8 billion per annum for businesses. The majority of savings will be made through customer service as customers can now ask queries about banking through the chatbot rather than having to call the bank, allowing banks to save on call centre staff. The study predicted that the integration of chatbots within the banking industry would rise from 12% to 90% by 2022.

OPEN IN VIEWER

Fig. 1.

Interest in chatbots within the Financial Industry (King [20]).

Most online banking services would benefit from having a chatbot integrated into their services. The use of bots helps internet banking service providers establish a better relationship with its customers. Customers can get answers to query’s immediately, conduct e-commerce and banking operations all from within the one bot conversation. This is another benefit of using an online banking system with and integrated chatbot as “87% customers banking online prefer to execute their personal finance operations within a single site” (Dauda et al. [7]). King, W.B. [20] records a survey conducted by Personetics which identified “40% of its clients are planning to integrate a bot into its services within the next 2 to 3 years and 70% said AI was an exciting opportunity” (see Fig. 1). This clearly identifies the demand and need for a chatbot within the financial services Industry. Chatbots provide more personal and immediate ways for customers to communicate with an organisation allowing them to provide consistent and appropriate customer service at scale (Conversational Business, 2017). Facebook has now released its own Messenger Bots, which allows organisations to develop their own Chatbot through Messenger to deliver their services. Facebook are also developing AI methods to obtain specific information related to users across relevant topics. The acquired information is used as a dataset to develop and improve their Artificial Intelligence so they can analyse, group and rate each user post (Rossmann and Zimmermann [29]). “44% of US consumers prefer to interact with a chatbot compared to a human in relation to customer queries” (Wellers and Pieper [33]). This is evident as Chatbots are also becoming more prevalent in other industries such as; insurance, recruitment, media and pharmaceuticals.

3. Banking chatbot

It is evident that modern financial services are constantly seeking to expand their technologies, both to improve customer service and increase delivery of services through the advancements in technology. This is to gain a competitive edge over other banks for financial benefits and to expand its customer base. A domain specific chatbot will be implemented to assist users with their banking. In order to overcome the user satisfaction issues associated with online banking services. The chatbot will provide personal and efficient communication between the user and their bank in order to manage their finances and get assistance when needed, such as; answering any queries and booking appointments. The chatbot will allow users to feel confident and comfortable when using this service regardless of the user’s computer literacy due to the natural language used in messages. It also provides a very accessible and efficient service as all interactions will take place within the one chat conversation negating the need for the user to navigate through a site. The proposed solution is to create a chatbot to simulate a human conversation to assist users with their banking needs and to provide a more personal experience. Advancements in Artificial Intelligence, machine learning techniques, improved aptitude for decision making, larger availability of domains and corpus, have increased the practicality of integrating a chat bot into applications (Dole et al. [8]). Users will be able to ask any banking related queries in natural language that they are comfortable using such as; view account information, transactions and check balance. The chatbot will identify and understand what the user is asking and generate an appropriate response based on the conversational context. Immediate responses will be provided by the chatbot to redeem the need for the user to have to call or visit their local banks branch and wait in queue in order to get through to an advisor for assistance. In order to make the application more secure Googles 2 Factor Authentication will be integrated to increase security ensuring only registered users can gain access to their account preventing the risk of fraud.

Figure 2 provides an abstract architectural design of the chatbot.

OPEN IN VIEWER

Fig. 2.

Chatbot architectural design.

Users interact with the chatbot from the web client and Google assistant app for Android devices. Users will carry out their interaction with the chatbot in the form natural language voice or text-based phrases. With the Google assistant integration, users will receive rich responses, such as; images, and cards thus improving ease of use and interaction experience for users as less typing and effort will be required when interacting with the chatbot mainly through the use of voice commands. The google assistant integration means the chatbot is easily be extended by Google Home. The client side of the application is developed using HTML5, CSS, JavaScript and Laravel Blade. This is a templating engine built into the Laravel framework which follows the MVC architectural pattern. Laravel Blade compiles and caches view files to improve the performance of the application. Blade template supports separation of view files to be split up into sections using the ‘@section’ annotation, which enhances the readability of frontend code, as HTML, PHP and JavaScript can be clearly separated into particular sections of the view and the ‘@yield’ annotation is utilised to render the content for sections of the view. Blade utilises template inheritance to allow views to derive the base layout of another view to modify and reuse UI components dynamically without replacing the code in the view it is inheriting from. Blade also has its own annotations for control structures which may be required in certain view files, for instance; the ‘@foreach’ annotation (Laravel [27]). The client side also avails of the chart.js CDN (Content Delivery Service) to display the user’s spending details in a graphical format so they can easily see where they spend the most of their money. The Laravel framework is also used to implement the Web client and Webhook which receives the JSON data from the NLU via HTTPS POST requests. The Webhook is implemented as a controller class so the corresponding web route can be defined as an endpoint, where the NLU posts real-time payload to the Webhook. The Webhook receives data from the NLU when it detects an intent has been triggered by the user from the client-side, Google Assistant or Home devices.

A webhook receives data posted to it from another service when an update or event occurs whereas an API follows a RESTful design approach that makes requests to retrieve the data (elastic.io, 2018). Dialogflow is utilised as the natural language understanding engine (NLU), which allows the chatbot to be trained to recognise entities and intents in a user utterance. The Chabot will be designed using intent mapping on the Dialogflow console to route user utterances to a collection of phrases. Figure 3 represents the flow of data as an intent is invoked by a user: Dialogflow recognises if an intent was triggered through its training using a collection of typical user phrases. This data is then posted to the fulfilment webhook. The required entities/parameters are extracted along with the action and a custom response is generated and returned to the user either visually or spoken from the webhook. The Webhook parses and validates the JSON from the payload it receives and encodes the JSON response generated to be sent back to the user for both text and voice interaction.

OPEN IN VIEWER

Fig. 3.

“Intent invoked”.

Once the NLU evaluates the user input and determines a context, the user utterance is passed to the Fulfilment webhook. The webhook utilises POST endpoints and accepts JSON returned from the chatbot NLU. The webhook then determines the associated action of an intent the user supplied utterance is mapped to; the action of an intent was defined in the Dialogflow console during training. If the action is present it will extract the entities from the given utterance and call the Fixer.io API and return the appropriate response encoded in JSON format. If the NLU cannot recognise the intent of the user supplied utterance it will prompt the user for more information to try and recognise the intent and context to provide fulfilment for the user. Figure 4 outlines the flow of data for registration/login. The NLU determines the context and presence of required entities in user input that was matched to an intent. The webhook then receives the user input in JSON format.

OPEN IN VIEWER

Fig. 4.

“Registration and login” UML activity diagram.

The webhook is utilised to deliver appropriate custom fulfilment to users by extracting the action of the intent it receives and checking whether the action is present in the intent. It then calls the appropriate function based on the current action. If the action is present the function getBalance() is called. This method calls the TrueLayer API which uses REST endpoints. The response is then encoded in JSON format and sent to the chatbot NLU. The Laravel framework comes with an authentication package to handle authentication of the application. Figure 4 outlines the process for login and registration. Users must first register on the application to set up an account. If the user attempts to login before registering they will not be allowed access the chatbot view. Upon requesting to login, a database check is performed to determine the validity of the user’s login details. If the details are valid the user will be redirected to the Two-Factor Authentication view. In this view the user will be required to scan a QR code displayed on screen to receive a unique code which must be entered to verify their account, if the code is correct the user is redirected to chatbot view.

The sequence diagram shown in Fig. 5 highlights the operation of the user requesting their banking information through the chatbot, outlining the fundamental process involved. The webhook receives the user input from the Chatbot NLU engine (Dialogflow) in the form of JSON. The webhook determines the action of the defined intent posted from the NLU by parsing the JSON it receives. If the action matches a defined banking operation, the webhook will make the appropriate method call to the TrueLayerClient Class, which calls the TrueLayer API and returns the data to the webhook to generate a response. Once the webhook receives the data returned by the API it encodes the data to JSON which is the response sent back to the NLU to be displayed to the user through the Chatbot view.

OPEN IN VIEWER

Fig. 5.

“Request banking information” sequence diagram.

In order to implement a successful chatbot the user group that will be interacting with it is carefully thought through. The target audience for the chatbot consists of different age groups with varying technical ability, as mobile banking is relevant across a broad range of age groups. To appeal to all age groups the chatbot will utilise simplistic and straightforward responses. Google assistant allows developers to determine a voice for their bot that matches the personality and utilise rich responses (Google [16]). However, the majority of users of an application like this already carry out the most part of their online interaction through some messaging platform, so the use of such an application should come with ease (Følstad & Brandtzæg [13]).

3.2. Dialog design

As main focus of interaction is between the user and the chatbot, which is carried out through natural language, the conversation design is a crucial aspect to consider when designing the chatbot. To achieve this; appropriate dialog will be designed through the Dialogflow console using follow-up and fall-back intents. A particular intent will be set, that once invoked by the user, the chatbot will end the conversation effectively regardless of the conversational context. The dialog is the collection of words and phrases used to respond to user input. The dialog will be designed with human like phrases to respond to users with in order to sound more natural. Another design aspect will include fall-back intents. If a user utterance cannot be matched to an intent, the chatbot will respond with a message stating it did not match the input to an intent and prompt the user to repeat the query or provide more details. The dialog is also designed using follow-up intents, these provide a more natural conversational flow when interacting with the chatbot. Follow-up intents extract more information from users as and when required, for instance; a user may say;

OPEN IN VIEWER

User: “Please Book me an appointment”

The chatbot is aware it needs more information from the user such as; date, time and phone number and will therefore use follow-up intents to extract the required information from the user. Dialogflow recommend designing a linear dialog for interactions where data present in the conversation is extracted to help the users achieve their goal, this is applied to the design specification is applied to the chatbot as it is a tasked-based interaction chatbot that consist of banking domain knowledge. The user utterance can be said in many different ways. In this example the chatbot will extract the required entity parameters ‘balance’ and ‘account’ in order to return a custom response to the user (Ozcelik [28]).

OPEN IN VIEWER

Fig. 6.

“Banking dialog”.

OPEN IN VIEWER

Fig. 7.

“Currency dialog”.

Figure 6 and Fig. 7 illustrate the design for the Google assistant application. It follows a simplistic layout so it is intuitive and efficient for users, in comparison to other forms of traditional UI (DZone, [10]). Users can directly ask a question to the chatbot using the input box or tapping the devices microphone. Once the icon is tapped it will activate the devices microphone and make the user aware it has started recording and the chatbot response will be displayed on screen instantly. Google assistant uses specific colours for each message. The use of different colours will be utilised to distinguish between chatbot response and user input.

OPEN IN VIEWER

Fig. 8.

“Web-based chatbot”.

Figure 8 outlines the design for the web-based chatbot view. Users can see how much they spend and what they spend their money on in a graphical format, this will help users identify over expenditures, for instance; they may realise they spend far too much on coffee each week. It is made visually clear to the user what the bot’s response is, as each individual message has an associated title and colour depending on whether or not it was sent by the user or the bot.

OPEN IN VIEWER

Fig. 9.

“Transaction email”.

OPEN IN VIEWER

Table 3

“User phrases: test results”

Simulated User Utterance (including misspelt phrases)	Confidence Score	Matched Intent (s)	Unrecognised Intent (s)	Response Time (ms)
Hello bankingbot, what is my balance?	1	✓	χ	3480
Hello bankingbot, please show me my balance?	0.9999	✓	χ	3414
Display my blance	1	χ	✓	N/A
Show me my account details please?	1	✓	χ	3446
“Accont dctails” (account details)	1	✓	χ	2181
Can I see my account details	0.8700	✓	χ	3480
How much money have I spent this week?	0.8299	✓	χ	3649
What hove I spunt	0.3000	✓	χ	3308
Wat have I spnt this weke	0.3300	✓	χ	2504
Can you send me my transactions for the last seven days?	1	✓	χ	7339
Tamsucyions	1	χ	✓	N/A
How much is 5000 Canadian dollars in Japanese yen	0.8900	✓	χ	3228
Convert 600 euro to Japanese yen	0.9499	✓	χ	3182
And 700 euro	1	✓	χ	1613
And 800 pounds	1	✓	χ	3381
What about 500 US dollar	1	✓	χ	3128
Can ypu please book me an appointment please?	0.6200	✓	χ	3380
Buuk me apptmnt	1	✓	χ	3345
Book me an appqentmttt for sqivings on the 23/04/2018	1	✓	χ	3322
Ok, my phone number is 089456787642	1	✓	χ	4212

OPEN IN VIEWER

Table 4

“Average test results”

Average Understanding Score	Average Response Time	Dialog Duration
17.7897 / 20 = 0.8894	60594 (ms) 60.594 (sec)/20 = 3.02 sec	3 minutes and 26 seconds

Users will be able to view their transactions on a regular basis without the need to navigate through various web-pages (see Fig. 9). Once the user requests to view their transactions an email will be sent to their email address that was provided at registration.

The dialog was tested to measure the efficiency of the chatbot which includes measuring how well the chatbot understood the users supplied utterance for both voice and text interactions, even if misspelt. Identifying if the intent was recognised, with average response times between text and voice interactions also included in said response. The chatbot was tested in a specific manner to record the performance metrics. A Dialogflow command line tool called dialogflow-cli,1

Table 4 shows average test results and Table 5 displays the results from “typical” interactions with the chatbot. The phrases were derived from the user questionnaires and observing the users interacting with the chatbot, the dialog for the chatbot was devised during the design stage. Every utterance is given a “Confidence Score”, rated on a scale of 0.0–1, with 1 being a complete match and 0 meaning the utterance was not successfully matched to any intent. However, the NLU has also given words or utterances, where no intent was recognised, a complete confidence score of 1 as it was matched to the default fall-back intent to handle unexpected input from the user with ease. It is clear that the response times do vary depending on the underlying functionality required for the user to achieve their goal, for instance: requesting their transactions via email or the chatbot sending text confirmation of their appointment.

Overall, the chatbot is able to match intents with the majority of the utterances with only 10% of the utterances failing to match. This is taking spelling mistakes into account also. The total number of successful matched intents out of 20 user phrases amounted to 80%. These results indicate that the chatbot can understand most phrases including spelling mistakes reflecting the quality of the dialog. The phrase “wat have I spnt this weke” (meaning “what have I spent this week”) obtained a low confidence score of 0.33, however the chatbot was able to understand the intent of the utterance and identify the action.

The average response time has been rounded to 3 seconds, which is quite a rapid response time. This suggests that users will be able to perform their necessary tasks through interacting with the chatbot relatively quickly which is also reflective on the overall dialog duration as shown in Table 6, this represents the overall interaction time. The results will be further elaborated upon in the Evaluation. Both the Google Assistant and Home agent were tested through the AoG Simulator on the Actions on Google console (AoG). The Google Assistant integration was tested through the simulator using text input while the Google Home was tested through voice interaction on the AoG simulator. The simulator returns a JSON object holding information regarding the chatbots understanding level identifying whether the utterance was matched to an intent. Similar phrases were used across devices to outline any variations in the NLU understanding.

Thorough testing was performed across multiple simulated devices. It was determined that the Google Home device could recognise the intent of an utterance even when colloquial phrases were produced. It was also identified that the Google Home device could recognise utterances where particular words were incomplete or mispronounced as shown in Table 7. It was found that input text provided to the Google Assistant on android phones, follows a more rigid syntax as it expects the intent present in the given input text to match the defined intent used within the training phrases, however this only applies when the use of input text is provided by the user, as the same speech recognition and machine learning algorithms are implemented with the Google Assistant regardless of the device or surface its embedded in. The speech recognition software in the Google Assistant transforms the user’s speech, even if incorrect, to the appropriate input text before it’s sent to the NLU. Consequently, the Google Home device was able to recognise and detect incomplete or mispronounced words, as interaction only occurred through voice commands. Based on these results, it is clear that voice interaction is the optimal method of engaging with the chatbot due to it being able to comprehend pronunciation errors and interpret them accordingly.

It was also found that the Google Assistant could only match an intent of a misspelt utterance when the trained intent or entity was typed correctly during text interaction. The Google Home device was the prime mode of interaction with the chatbot, as it was able to match the intent of each supplied utterance, achieving a perfect score of 100%. Establishing that spoken natural language conversational interfaces perform best, compared to that of text-based interactions as the Google Assistant achieved a result of 57.14%. There is a stark contrast between the two conversational interfaces, implying spoken dialog is a much better method of interacting with natural language conversational interfaces. The speech recognition on the Google Assistant has a higher recognition rate in comparison to the HTML5 Speech API as it had an average speech recognition of 85.71%, suggesting the Google platforms will perform better in real interactions with users.

Table 7 outlines the number of turns the chatbot took to try and understand the intent of an utterance corresponding to the misspelt phrase defined in Table 5 before exiting the conversation. The intents 1, 2 & 3 refer to checking balance, making a deposit and changing a pin on these occasions. On each turn the chatbot responded with fallback intents such as: “Sorry, I didn’t quite get that, could you repeat that please?” or “Sorry, I’m having difficulty understanding what you said, could you say that again?”. This feature handles unexpected input or when the chatbot cannot understand a given utterance. This also outlines the use of different fallback intents used even if the user utterance was the same on each turn. The chatbot exited the conversation due to repeatedly prompting the user for an appropriate utterance and lack thereof, its inability to understand the intent of the given utterance to successfully match it to a trained intent to carry out the appropriate action and fulfil the user’s goal.

Further user testing was conducted across all three communication channels of the chatbot; Google Assistant, Home and the web-based chatbot. This will give an insight into the true quality and overall usefulness of the chatbot which is measured by the user’s interaction experience. In order to conduct user testing, a set of questions were constructed to evaluate the chatbot. The results from the user questionnaires provided insight into the overall response and precision rate. The questionnaires were distributed across a user group consisting of 15 individuals with varying technical knowledge. The results from the user questionnaires were compared to the user interactions identifying subjective and objective metrics. During the questionnaire the users were observed to capture free standing information regarding the interaction experience which also qualify as subjective measurements. This allowed the identification of other quality metrics which were not initially considered and led to a deeper understanding of the chatbots performance. They completed a user questionnaire targeted to capture the quality of the chatbot. The subjective and objective metrics outlined in Table 8 were obtained using a range of question types including Likert scale and numerical scales to accurately assess the chatbots performance. Each metric outlined in Table 8 was measured in relation to a specific question to gather qualitative results. These questionnaires can be found in appendices.

The majority of banks struggle to get their customers to use the technology in place due to low user satisfaction when interacting with their services. From the results shown in Table 8, it can be established that integrating a chatbot into their banking services would greatly increase the rate of user satisfaction. This is evident as 73.33% of all users stated the chatbot was “extremely” quick compared to the current technology offered by their bank, for instance, online banking applications. From completing the user testing phase, it was found that 86.66% of all respondents found the chatbot was either “extremely” or “moderately” easy to talk to and it achieved an overall likability of 93.33% as outlined in Table 8. Reaffirming that chatbots can successfully engage users and increase satisfaction. The objective metrics obtained from the questionnaire results identify the chatbots performance and quality in a real interaction. Overall the chatbot received a very high speech recognition rate as 80.00% of users stated they were “very strongly” understood by the chatbot. The chatbot obtained an overall understanding score of 0.8894 (89.00%) in the range between 0.0–1 during the tests. This result is on the higher tier of the understanding scale.

Evidence from the results gathered during user-testing, through measuring the response accuracy to real user interaction, that the chatbot has a high competency level as both the results from the simulated and user tests have a very low variance of 2.33% in regards to the understanding score. These measurements reflect the accuracy and thoroughness of the emulated actions performed during testing.

OPEN IN VIEWER

Fig. 10.

“Users preference”.

The majority of users preferred to interact with the chatbot through voice commands, with 66.67% of user’s preference being voice interaction, as shown in Fig. 10. This supports that more users are wishing to be able to freely interact with applications and not be limited to traditional modes of interaction. This feature also offers users the ability to interact with chatbot without it requiring much attention on focusing on the task at hand, as users ask the chatbot a question and it will recognise the users voice negating the need for the user to type. The focus of any chatbot is to allow users to complete tasks with ease using natural language. It is important that the chatbot clearly understood the user and the user felt confident in interacting with the chatbot. Figure 11 shows this was achieved as only 13.33% of users stated they were “often” left in a state where they were unsure how to carry on the conversation. These results reflect low frustration rates among users as 60% of user never experienced this and 26.67% stating it occurred “Not often.” This clearly suggests the chatbot is able to sufficiently guide users through the conversation with ease providing appropriate responses to the majority of user phrases.

OPEN IN VIEWER

Fig. 11.

“Conversation flow”.

Enabling the chatbot to be context aware provides a great advantage for users as they can simply refer back to a previous conversation and still be understood by the chatbot. This is particularly useful in long conversational flows, reducing the amount in which the user has to repeat themselves thus shortening the time taken to achieve a task (see Fig. 12).

OPEN IN VIEWER

Fig. 12.

“Chatbot conversational context”.

The future of chatbots seems to be the desire to add more ‘naturalness’ so that users become more relaxed when dealing with a bot. Google have made some impressive leaps in this area with their Google Assistant service known as Google Duplex which can make calls to book restaurant reservations and hair salon appointments (Kinsella, [21]).

We show here that a chatbot is a suitable method for adopting technology as a means to distribute a service. The developed chatbot allows users to interact with their bank through natural language interaction, granting users more convenient and efficient access to their banking information. It was determined that chatbots perform at a very high standard and provide reliable and rapid responses to users compared to that of traditional methods. The average time spent interacting with the chatbot is very low as it provides an efficient way for users to manage their banking. The low interaction time reflects the high understanding and speech recognition rates, offered through the adoption of conversational user interfaces thus allowing users to freely interact with the chatbot to meet the demands of modern life. The chatbot has proven to fulfil the demand of users wanting instant access and availability information and services. Future work will concentrate on a knowledgebase specialized for online banking service or more powerful chatbot models. We will also integrate the chatbot across popular platforms such as Amazon Echo.