Chatbots for online banking services

Abstract

People are interacting increasingly with systems through voice assistants and chatbots sectors such as banking and financial are adopting them to create more responsive customer services. This project aimed to implement a web-based chatbot to assist with online banking, using tools that expose artificial intelligence methods such as natural language understanding. The chatbot allows users to view all their personal banking information from within the chatbot. The system was tested across a range of devices such as Google Home and Assistant on android devices to outline the key differences between the two modes of interaction, spoken and text dialog. These tests showed that chatbots can enhance accessibility, reaffirming that they are more than just a passing fad and have a viable use.

Keywords

Chatbots HCI home assistants mobile

1. Introduction

“Digitalisation, the surge of mobile and internet connected devices has revolutionised the way people interact with one another and communicate with businesses” (Eeuwen [11]). Millennials are accepting and supporting new technology in their everyday life. Technology companies are streamlining Artificial Intelligence (AI) into the products they offer, such as Google Assistant, Google Home and Amazon Alexa. The new and upcoming generation are expected to be critical and game changing customers for businesses. They demand effortless experiences, answers within seconds, not minutes and more intelligent self-service options (Teller Vision [31]). The banking and the financial service industry was one of the first industries to adopt technology. This integration has grown massively, helping banks reach a wider customer base enabling them to perform their banking conveniently (Baptista and Oliveira [2]). Banks are becoming ever more competitive with each other to adopt the newest advancements in technology to provide an improved delivery service to satisfy customers. Ulster Bank, Deloitte, AIB and PTSB are wanting to focus on integrating new technology to improve the speed at which transactions are acknowledged (Global Banking News, 2017). With this in mind the relationship with the customer is always evolving due to the growth of technology.

Banks are now enabling the use of technology so customers can perform more tasks online, such as; cheque image clearing to allow the payment of cheques remotely and intelligent chatbots to increase customer service and assist employees. A chatbot is a “simple software program that can respond to customer prompts i.e. what’s my bank balance?” (Balkhi [1]). Mastercard has launched Kai an artificial intelligent chatbot and other bots for financial services. They can handle customer queries such as: ‘what is APR?’, requests, look at spending habits and solve problems. This in turn enables financial institutions to provide a new, engaging experience and strengthen their relationship with the customer, with the aid of natural language used by bots to establish a more personal and contextual conversation [24]. The focus of this project is to implement these new technologies to create an intelligent chatbot to enable banks to appeal to millennials and potentially gain a lifelong customer.

The banking industry has multiple electronic delivery channels in use to distribute technology assets and services for the benefit of their customers. Online banking is a commodity of commerce within financial services as well as banking industries [15]. Advancements in technology has transformed many of our services into the digital era and the banking industry is one of the primary industries to avail of these advancements to improve their services. Currently within the UK two paradigms are available for online banking. One of which is an integrated internet bank which still operates through the branch but has an online presence. The other, a stand-alone internet bank, that operates completely independently and its only existence is solely through the internet (MarketLine [23]). Banks implement technology to strengthen their processing capacity, acquire a larger customer base and expand the services they could offer (Consoli [6]). The use of internet banking has grown in demand enormously in the last decade. “15% of branch customers use online banking once a day, 59% once a week, 77% at least once a month and 53% were confident in carrying out the best part of their banking online” (Barty and Recketts [3]). Online banking has become more popular as it negates the need for customers to visit their local branch as they can manage their finances on the go to meet the demand of modern life. This is evident as branchless banks are now emerging from the industry such as Atom Bank and many banks now closing some of their branches. This is evident with the recent closure of 11 Ulster Bank Branches in NI due to the increased number of customers performing their banking online (The Belfast Telegraph). However, a recent study by Ling et al. [22], notes that most internet banking service providers struggle to get many of their customers to use their service. They identify lack of customer satisfaction when using online banking services to be a major cause. Service quality, web design and content, security, privacy, speed and convenience are stated as the top factors influencing customer satisfaction (Ling et al. [22]). This suggests that there is a lack of technology in place to enhance the customer online banking experience which could be improved by integrating a chatbot to provide an efficient, convenient and personal service.

This paper outlines a banking chatbot to facilitate users with queries and assist with personal banking. The application allows users to check their balance, account details & transactions. It also integrates SMS, currency conversions, two factor authentication and the ability to interact with the service through personal assistants like Google Assistant.

The advantages of our chatbot is that it allows users to interact with their bank through natural language interaction, granting users more convenient and efficient access to their banking information. We show out system performing at a very high standard and provide reliable and rapid responses to users compared to that of traditional methods. The average time spent interacting with the chatbot is very low as it provides an efficient way for users to manage their banking. The low interaction time reflects the high understanding and speech recognition rates, offered through the adoption of conversational user interfaces thus allowing users to freely interact with the chatbot to meet the demands of modern life. The chatbot has the advantage over traditional services in that it can fulfil the demand of users wanting instant access and availability information and services. We outline the design and implementation in following sections.

2. Chatbots

A chatbot is a software tool that utilises natural language processing (NLP) for human machine interaction (HMI) and Machine Learning (ML). “The complexity of a chatbot is directionally proportional to the scope of the domain”. An open domain requires a larger knowledge base, whereas, a closed domain has a more specific knowledge base that was developed to achieve a specific goal (Gregori [17]). Chatbot technology initially began in the 1960s to determine whether a chatbot could be portrayed as a human. Throughout the 1980s there was an elevated amount research carried out on natural language interfaces which lead to the development of sophisticated chatbot architectures such as A.L.I.C.E. This chatbot architecture is one of the earlier chatbots developed in 1995 by Dr Wallace which is now open-source, the acronym stands for Artificial Linguistic Internet Computer Entity. This is a chatbot you can create through interaction as it will learn from previous interactions to create its knowledge base. Its knowledge is saved in AIML (Artificial Intelligent Mark-up Language) files which evolved from the Extensible Mark-up Language (XML) (Shawar and Atwell [30]). Chatbots are developed using two approaches; a rule-based approach where chatbots operate by moving through branches of a tree diagram of an expert system. The second approach involves advanced artificial intelligence and machine learning, thus the chatbot can learn from the conversations, generating appropriate responses to continuously improve over time (Watson [32]). There are two modes in which chatbots can simulate a conversation with users which include system-initiated chatbots where they commence the conversation with the user and User-initiated chatbots where the user directs the conversation instead. Systems that incorporate the two methods of initiation are known as mixed initiative systems (Duijst [9]).

Most businesses and organisations are understanding the potential benefits of machine learning and artificial intelligence to have a positive change on how they perform business. Artificial intelligence has progressed to allow the development of more sophisticated chatbots. Organisations are focusing on specific areas of user engagement that take up a lot of time but can be replaced through the use of a chatbot. Chatbots can understand what the customer needs from a single text instead of the customer having to follow a process of multiple steps. Chatbots are used to automate customer service and reduce manual tedious tasks performed by employees so they can spend their time more productively on higher priority tasks. Establishments that regularly deal with its customers have discovered the potential of chatbots as a channel to distribute more efficient and immediate information to customers in comparison to a customer service representative regarding queries and issues (Onufreiv [26]). HDFC Bank has merged with Niki.ai an artificial intelligence company to develop a state of the art conversational banking chatbot (HDFC Bank [14]). The chatbot is accessible through the banks Facebook Messenger allowing users to utilise e-commerce and banking transactions all within the chatbot. There is also a chatbot integrated within the HDFC login page to assist in online banking. The chatbot was developed as a concierge service, this is definitely one approach that can increase customer satisfaction within online banking enabling banks to develop a better relationship with its customers. Chatbots will renew and modernise the customer service industry and the main sectors outlined including the banking and healthcare sector (Newswire [25]). Industries such as; retail, healthcare, e-commerce and banking, are expected to achieve considerable savings from integrating a chatbot. Between 2017 and 2020 chatbots are estimated to make savings of $8 billion per annum for businesses. The majority of savings will be made through customer service as customers can now ask queries about banking through the chatbot rather than having to call the bank, allowing banks to save on call centre staff. The study predicted that the integration of chatbots within the banking industry would rise from 12% to 90% by 2022.

Fig. 1.

Interest in chatbots within the Financial Industry (King [20]).

Most online banking services would benefit from having a chatbot integrated into their services. The use of bots helps internet banking service providers establish a better relationship with its customers. Customers can get answers to query’s immediately, conduct e-commerce and banking operations all from within the one bot conversation. This is another benefit of using an online banking system with and integrated chatbot as “87% customers banking online prefer to execute their personal finance operations within a single site” (Dauda et al. [7]). King, W.B. [20] records a survey conducted by Personetics which identified “40% of its clients are planning to integrate a bot into its services within the next 2 to 3 years and 70% said AI was an exciting opportunity” (see Fig. 1). This clearly identifies the demand and need for a chatbot within the financial services Industry. Chatbots provide more personal and immediate ways for customers to communicate with an organisation allowing them to provide consistent and appropriate customer service at scale (Conversational Business, 2017). Facebook has now released its own Messenger Bots, which allows organisations to develop their own Chatbot through Messenger to deliver their services. Facebook are also developing AI methods to obtain specific information related to users across relevant topics. The acquired information is used as a dataset to develop and improve their Artificial Intelligence so they can analyse, group and rate each user post (Rossmann and Zimmermann [29]). “44% of US consumers prefer to interact with a chatbot compared to a human in relation to customer queries” (Wellers and Pieper [33]). This is evident as Chatbots are also becoming more prevalent in other industries such as; insurance, recruitment, media and pharmaceuticals.

2.1. Chatbot natural language understanding engine

The chatbot engine is thought of as one of the most critical elements of a chatbot, alias “Natural Language Understanding (NLU) engine” (Kar and Haldar [19]). The NLU holds liability for the translation of conversational dialogs to actions which are understood by the machine. NLU engines use a variety of artificial intelligence methods to understand the natural language used in conversational interfaces such as chatbots. These methods consist of: Natural Language Processing (NLP) and Machine Learning (ML) (Kar and Haldar [19]). Googles Dialogflow, previously known API.ai, is a natural language understanding engine that identifies the intent and context from the natural language in user supplied utterances. These concepts are used to develop the behaviour of the chatbot and how coherently it interacts with the user. Intents are used to establish a connection between the user input and the appropriate action to be executed by the chatbot in order for the user to achieve their goal. Contexts are utilised to differentiate and understand user input which may have an alternative meaning depending upon the current conversational context (Gregori [17]; Jurafsky and Martin [18]).

Establishing the context of user message is a vital feature that allows the chatbot to deal with situations that it may not be able to carry out a specific action for. This is due to the user input being very vague or may have an alternative meaning. The context is the capacity of a chatbot to sustain its state, also referred to as the number of user supplied input (utterances) when the context is extracted and the appropriate intent is paired to conduct the desired action for the user. Intents are the core backbone of conversational interfaces. The intents represent what the customer is wanting to achieve such as ‘show me my balance’. The text sent by the user in natural language is analysed to identify the corresponding intent of the text. This requires matching a specific user supplied phrase with an appropriate action to be executed by the system. The chat bot would then return appropriate dialog in order for the user to achieve their goals. Actions are the processes or steps executed by the system when the intent of a message is identified, they contain parameters which categorise and define its properties (Kar and Haldar [19]). Sentiment analysis incorporates multiple natural language processing techniques in order to extract meaning and polarity from text. Polarity detection classifies text to be either positive or negative and measures the intensity of the overall polarity detected. Sentiment analysis achieves a more in depth understanding of the contextual role of each concept within a given piece of text (Cambria and White [5]; Cahn [4]).

2.2. Chatbot architecture

Previously chatbots solely supported a single adjacency pair, also known as a one-shot conversation. However, modern chatbots can sustain multiple adjacency pairs, remembering states and contexts between conversations and have the capability to associate data in different adjacency pairs which is related. This is the bots ability to preserve the conversation. A chatbot consists of four main parts: front-end, knowledge-base, back-end and corpus which is the training data. The front end is accountable for enabling communication between the bot and the user. The NLU utilises Artificial intelligence methods to identify the intent and context of the user input. An appropriate response is generated from the users’ intent. The knowledge base defines the chatbots knowledge, which is created within the NLU and supported by the back-end, the back-end applies the domains corpus to produce the knowledge base (Gregori [17]). Input can be supplied to the chatbot in the form of text or speech. The Input is sent to the dialog management system which is the NLU in this case, which determines an appropriate response and amends the chatbots state accordingly to carry out the required action. The chatbot will produce text and speech responses in the form of both text and speech.

There is clear evidence that there will be a drastic increase in the number of chatbots being implemented within the financial service industry. The vast amount of research that has been carried out, and currently ongoing, within the artificial intelligence field has led to the rise of more sophisticated and intellectual chatbots. This will prove to be immensely beneficial in providing convenient and accessible customer service at a rapid scale.

3. Banking chatbot

It is evident that modern financial services are constantly seeking to expand their technologies, both to improve customer service and increase delivery of services through the advancements in technology. This is to gain a competitive edge over other banks for financial benefits and to expand its customer base. A domain specific chatbot will be implemented to assist users with their banking. In order to overcome the user satisfaction issues associated with online banking services. The chatbot will provide personal and efficient communication between the user and their bank in order to manage their finances and get assistance when needed, such as; answering any queries and booking appointments. The chatbot will allow users to feel confident and comfortable when using this service regardless of the user’s computer literacy due to the natural language used in messages. It also provides a very accessible and efficient service as all interactions will take place within the one chat conversation negating the need for the user to navigate through a site. The proposed solution is to create a chatbot to simulate a human conversation to assist users with their banking needs and to provide a more personal experience. Advancements in Artificial Intelligence, machine learning techniques, improved aptitude for decision making, larger availability of domains and corpus, have increased the practicality of integrating a chat bot into applications (Dole et al. [8]). Users will be able to ask any banking related queries in natural language that they are comfortable using such as; view account information, transactions and check balance. The chatbot will identify and understand what the user is asking and generate an appropriate response based on the conversational context. Immediate responses will be provided by the chatbot to redeem the need for the user to have to call or visit their local banks branch and wait in queue in order to get through to an advisor for assistance. In order to make the application more secure Googles 2 Factor Authentication will be integrated to increase security ensuring only registered users can gain access to their account preventing the risk of fraud.

Figure 2 provides an abstract architectural design of the chatbot.

Fig. 2.

Chatbot architectural design.

Users interact with the chatbot from the web client and Google assistant app for Android devices. Users will carry out their interaction with the chatbot in the form natural language voice or text-based phrases. With the Google assistant integration, users will receive rich responses, such as; images, and cards thus improving ease of use and interaction experience for users as less typing and effort will be required when interacting with the chatbot mainly through the use of voice commands. The google assistant integration means the chatbot is easily be extended by Google Home. The client side of the application is developed using HTML5, CSS, JavaScript and Laravel Blade. This is a templating engine built into the Laravel framework which follows the MVC architectural pattern. Laravel Blade compiles and caches view files to improve the performance of the application. Blade template supports separation of view files to be split up into sections using the ‘@section’ annotation, which enhances the readability of frontend code, as HTML, PHP and JavaScript can be clearly separated into particular sections of the view and the ‘@yield’ annotation is utilised to render the content for sections of the view. Blade utilises template inheritance to allow views to derive the base layout of another view to modify and reuse UI components dynamically without replacing the code in the view it is inheriting from. Blade also has its own annotations for control structures which may be required in certain view files, for instance; the ‘@foreach’ annotation (Laravel [27]). The client side also avails of the chart.js CDN (Content Delivery Service) to display the user’s spending details in a graphical format so they can easily see where they spend the most of their money. The Laravel framework is also used to implement the Web client and Webhook which receives the JSON data from the NLU via HTTPS POST requests. The Webhook is implemented as a controller class so the corresponding web route can be defined as an endpoint, where the NLU posts real-time payload to the Webhook. The Webhook receives data from the NLU when it detects an intent has been triggered by the user from the client-side, Google Assistant or Home devices.

A webhook receives data posted to it from another service when an update or event occurs whereas an API follows a RESTful design approach that makes requests to retrieve the data (elastic.io, 2018). Dialogflow is utilised as the natural language understanding engine (NLU), which allows the chatbot to be trained to recognise entities and intents in a user utterance. The Chabot will be designed using intent mapping on the Dialogflow console to route user utterances to a collection of phrases. Figure 3 represents the flow of data as an intent is invoked by a user: Dialogflow recognises if an intent was triggered through its training using a collection of typical user phrases. This data is then posted to the fulfilment webhook. The required entities/parameters are extracted along with the action and a custom response is generated and returned to the user either visually or spoken from the webhook. The Webhook parses and validates the JSON from the payload it receives and encodes the JSON response generated to be sent back to the user for both text and voice interaction.

Fig. 3.

“Intent invoked”.

Once the NLU evaluates the user input and determines a context, the user utterance is passed to the Fulfilment webhook. The webhook utilises POST endpoints and accepts JSON returned from the chatbot NLU. The webhook then determines the associated action of an intent the user supplied utterance is mapped to; the action of an intent was defined in the Dialogflow console during training. If the action is present it will extract the entities from the given utterance and call the Fixer.io API and return the appropriate response encoded in JSON format. If the NLU cannot recognise the intent of the user supplied utterance it will prompt the user for more information to try and recognise the intent and context to provide fulfilment for the user. Figure 4 outlines the flow of data for registration/login. The NLU determines the context and presence of required entities in user input that was matched to an intent. The webhook then receives the user input in JSON format.

Fig. 4.

“Registration and login” UML activity diagram.

The webhook is utilised to deliver appropriate custom fulfilment to users by extracting the action of the intent it receives and checking whether the action is present in the intent. It then calls the appropriate function based on the current action. If the action is present the function getBalance() is called. This method calls the TrueLayer API which uses REST endpoints. The response is then encoded in JSON format and sent to the chatbot NLU. The Laravel framework comes with an authentication package to handle authentication of the application. Figure 4 outlines the process for login and registration. Users must first register on the application to set up an account. If the user attempts to login before registering they will not be allowed access the chatbot view. Upon requesting to login, a database check is performed to determine the validity of the user’s login details. If the details are valid the user will be redirected to the Two-Factor Authentication view. In this view the user will be required to scan a QR code displayed on screen to receive a unique code which must be entered to verify their account, if the code is correct the user is redirected to chatbot view.

The sequence diagram shown in Fig. 5 highlights the operation of the user requesting their banking information through the chatbot, outlining the fundamental process involved. The webhook receives the user input from the Chatbot NLU engine (Dialogflow) in the form of JSON. The webhook determines the action of the defined intent posted from the NLU by parsing the JSON it receives. If the action matches a defined banking operation, the webhook will make the appropriate method call to the TrueLayerClient Class, which calls the TrueLayer API and returns the data to the webhook to generate a response. Once the webhook receives the data returned by the API it encodes the data to JSON which is the response sent back to the NLU to be displayed to the user through the Chatbot view.

Fig. 5.

“Request banking information” sequence diagram.

In order to implement a successful chatbot the user group that will be interacting with it is carefully thought through. The target audience for the chatbot consists of different age groups with varying technical ability, as mobile banking is relevant across a broad range of age groups. To appeal to all age groups the chatbot will utilise simplistic and straightforward responses. Google assistant allows developers to determine a voice for their bot that matches the personality and utilise rich responses (Google [16]). However, the majority of users of an application like this already carry out the most part of their online interaction through some messaging platform, so the use of such an application should come with ease (Følstad & Brandtzæg [13]).

3.1. Conversational user interface design

Interaction will take place within the chatbot in the form of natural language and this will be the focus of design in regards to conversational user interfaces (CUI). Conversation as a method of interaction is a modern User Interface (Fadhil et al. [12]) defines a set of design principles to adhere to when designing Conversational User Interfaces (see Table 1).

A set of design patterns were outlined for domain specific chatbots including those utilised within a finance industry (Table 2).

Table 1
CUI design principles

Principles Description

1. Design by personality The bot personality should provide a natural and unique interaction experience for users. The personality should be designed to guide the users through the interaction which may include being aware of what the user may say or want. The personality is the style and manner the bot holds throughout the interaction.

2. Flexibility in response Flexible responses should be provided to user input, the bot should have various/random responses to the same user input.

3. Text vs Custom Buttons It should be clear to the user what input is expected to carry on in the converastion. A fallback response should be used in situations where the input is not recognised such as suggesting other operations that are available. Instead of solely relying on natural text commands from the user, a good UI should render structured options to the user in form of buttons or text.

4. Simplicity in Interaction Be concise and clear in response to questions. The conversation flow should be straight and not branch out to complex paths.

5. Conversation Flow The conversation flow is important to ensure that the user is not irritated in situations where the bot may not be able to provide an appropriate answer immediately. The user would then have to undergo a conversation in order to determine the answer to their query.

6. Tasks and duty specifications The bot should deliver explicit tasks for a particular domain.

7. Rigid syntax, then NLP People are prone to spelling mistakes or using colloquial phrases which may not be understood by the bot.

8. Empathy and emotional state It is important that the bot builds a rapport with the user and convey emotion in responses based on certain user input.

9. Keep conversation short Users want to interact with the bot to receive answers or reach a goal quickly. Avoid long winded conversations as it will make the interaction feel burdensome. This helps avoid ambiguity.

10. Triggers and actions The bot needs to persuade users and seek ways to encourage users to carry out specific actions through the bot.

11. Predict and personalise Bots can analyse previous input from users for important parameters, in order to anticipate a user’s predisposition. It will then provide an answer that would be unique to that particular user.

12. Fully/partially automated NLP should be used for automated tasks.

13. Providing a way out Allow user to begin the conversation again or exit the conversation.

14. User boredom The conversation should engage the user throughout the interaction.

	Principles	Description
1.	Design by personality	The bot personality should provide a natural and unique interaction experience for users. The personality should be designed to guide the users through the interaction which may include being aware of what the user may say or want. The personality is the style and manner the bot holds throughout the interaction.
2.	Flexibility in response	Flexible responses should be provided to user input, the bot should have various/random responses to the same user input.
3.	Text vs Custom Buttons	It should be clear to the user what input is expected to carry on in the converastion. A fallback response should be used in situations where the input is not recognised such as suggesting other operations that are available. Instead of solely relying on natural text commands from the user, a good UI should render structured options to the user in form of buttons or text.
4.	Simplicity in Interaction	Be concise and clear in response to questions. The conversation flow should be straight and not branch out to complex paths.
5.	Conversation Flow	The conversation flow is important to ensure that the user is not irritated in situations where the bot may not be able to provide an appropriate answer immediately. The user would then have to undergo a conversation in order to determine the answer to their query.
6.	Tasks and duty specifications	The bot should deliver explicit tasks for a particular domain.
7.	Rigid syntax, then NLP	People are prone to spelling mistakes or using colloquial phrases which may not be understood by the bot.
8.	Empathy and emotional state	It is important that the bot builds a rapport with the user and convey emotion in responses based on certain user input.
9.	Keep conversation short	Users want to interact with the bot to receive answers or reach a goal quickly. Avoid long winded conversations as it will make the interaction feel burdensome. This helps avoid ambiguity.
10.	Triggers and actions	The bot needs to persuade users and seek ways to encourage users to carry out specific actions through the bot.
11.	Predict and personalise	Bots can analyse previous input from users for important parameters, in order to anticipate a user’s predisposition. It will then provide an answer that would be unique to that particular user.
12.	Fully/partially automated	NLP should be used for automated tasks.
13.	Providing a way out	Allow user to begin the conversation again or exit the conversation.
14.	User boredom	The conversation should engage the user throughout the interaction.

Table 2

CUI design patterns

	Design Patterns	Description
1.	Follow-up questions	Follow-up questions should be specific in the context of the conversation. An example of this is when the user wishes to request a currency conversion. The chatbot would ask; “what is the amount and currency you wish to convert to?”
2.	Cognitive load	Chatbots should present coherent choices to users to reduce the amount of effort needed for the user to interact with it. This can be achieved through the use of buttons and images.

3.2. Dialog design

As main focus of interaction is between the user and the chatbot, which is carried out through natural language, the conversation design is a crucial aspect to consider when designing the chatbot. To achieve this; appropriate dialog will be designed through the Dialogflow console using follow-up and fall-back intents. A particular intent will be set, that once invoked by the user, the chatbot will end the conversation effectively regardless of the conversational context. The dialog is the collection of words and phrases used to respond to user input. The dialog will be designed with human like phrases to respond to users with in order to sound more natural. Another design aspect will include fall-back intents. If a user utterance cannot be matched to an intent, the chatbot will respond with a message stating it did not match the input to an intent and prompt the user to repeat the query or provide more details. The dialog is also designed using follow-up intents, these provide a more natural conversational flow when interacting with the chatbot. Follow-up intents extract more information from users as and when required, for instance; a user may say;

User: “Please Book me an appointment”

The chatbot is aware it needs more information from the user such as; date, time and phone number and will therefore use follow-up intents to extract the required information from the user. Dialogflow recommend designing a linear dialog for interactions where data present in the conversation is extracted to help the users achieve their goal, this is applied to the design specification is applied to the chatbot as it is a tasked-based interaction chatbot that consist of banking domain knowledge. The user utterance can be said in many different ways. In this example the chatbot will extract the required entity parameters ‘balance’ and ‘account’ in order to return a custom response to the user (Ozcelik [28]).

Fig. 6.

“Banking dialog”.

Fig. 7.

“Currency dialog”.

Figure 6 and Fig. 7 illustrate the design for the Google assistant application. It follows a simplistic layout so it is intuitive and efficient for users, in comparison to other forms of traditional UI (DZone, [10]). Users can directly ask a question to the chatbot using the input box or tapping the devices microphone. Once the icon is tapped it will activate the devices microphone and make the user aware it has started recording and the chatbot response will be displayed on screen instantly. Google assistant uses specific colours for each message. The use of different colours will be utilised to distinguish between chatbot response and user input.

Fig. 8.

“Web-based chatbot”.

Figure 8 outlines the design for the web-based chatbot view. Users can see how much they spend and what they spend their money on in a graphical format, this will help users identify over expenditures, for instance; they may realise they spend far too much on coffee each week. It is made visually clear to the user what the bot’s response is, as each individual message has an associated title and colour depending on whether or not it was sent by the user or the bot.

Fig. 9.

“Transaction email”.

Table 3

“User phrases: test results”

Simulated User Utterance (including misspelt phrases)	Confidence Score	Matched Intent (s)	Unrecognised Intent (s)	Response Time (ms)
Hello bankingbot, what is my balance?	1	✓	χ	3480
Hello bankingbot, please show me my balance?	0.9999	✓	χ	3414
Display my blance	1	χ	✓	N/A
Show me my account details please?	1	✓	χ	3446
“Accont dctails” (account details)	1	✓	χ	2181
Can I see my account details	0.8700	✓	χ	3480
How much money have I spent this week?	0.8299	✓	χ	3649
What hove I spunt	0.3000	✓	χ	3308
Wat have I spnt this weke	0.3300	✓	χ	2504
Can you send me my transactions for the last seven days?	1	✓	χ	7339
Tamsucyions	1	χ	✓	N/A
How much is 5000 Canadian dollars in Japanese yen	0.8900	✓	χ	3228
Convert 600 euro to Japanese yen	0.9499	✓	χ	3182
And 700 euro	1	✓	χ	1613
And 800 pounds	1	✓	χ	3381
What about 500 US dollar	1	✓	χ	3128
Can ypu please book me an appointment please?	0.6200	✓	χ	3380
Buuk me apptmnt	1	✓	χ	3345
Book me an appqentmttt for sqivings on the 23/04/2018	1	✓	χ	3322
Ok, my phone number is 089456787642	1	✓	χ	4212

Table 4

“Average test results”

Average Understanding Score	Average Response Time	Dialog Duration
$17.7897 / 20 = 0.8894$	60594 (ms) 60.594 (sec)/20 = 3.02 sec	3 minutes and 26 seconds

Users will be able to view their transactions on a regular basis without the need to navigate through various web-pages (see Fig. 9). Once the user requests to view their transactions an email will be sent to their email address that was provided at registration.

4. Evaluation

The dialog was tested to measure the efficiency of the chatbot which includes measuring how well the chatbot understood the users supplied utterance for both voice and text interactions, even if misspelt. Identifying if the intent was recognised, with average response times between text and voice interactions also included in said response. The chatbot was tested in a specific manner to record the performance metrics. A Dialogflow command line tool called dialogflow-cli,1

¹
https://www.npmjs.com/package/dialogflow-cli

written in JavaScript, was installed into the project directory for the purpose of undertaking these tests. This was employed to replicate how users would interact with the chatbot and metrics were recorded for each simulated interaction. To initialise the mock interaction for testing the tool fires a welcome event to the Dialogflow API once instantiated in the command line, it then displays the response returned from the API as the result. Each type of use case was tested through the command line and the response was returned in JSON format. A total of 20 simulated utterances were tested and the results are shown in Table 3. Each utterance tested was uniquely phrased including variations of the same utterance to accumulate a range of results so the chatbots understanding can be thoroughly classified. From the results shown, it can be determined how well the chatbot understood the sentence and typical response times are displayed to measure the overall performance of the chatbot. This also outlines how it handles unexpected input, such as misspelt words. The confidence score is returned along with the intent the test user phrase was matched to.

Table 5

“Simulated device testing”

Intent	Google assistant (Text)	Matched Intent (s)	Google Home (voice)	Matched Intent (s)	Web APP (Voice)	Matched Intent (s)
1.	Correctly spelt phrase:	✓	Correctly pronounced phrase:	✓	Correctly pronounced phrase:
	1. What is my balance	✓	1. What is my balance	✓	1. What is my balance	✓
	Incorrectly spelt phrases:		Incorrectly pronounced phrases:		Incorrectly pronounced phrases:
	1. What is my balan	χ	1. What is my balan		1. What is my balan
	2. Wat is my bolonce		2. Wat is my bolonce		2. Wat is my bolonce
		✓	Detected as:	✓	Detected as:	✓
			1. What is my balance	✓	1. What is my balance	χ
			2. What is my balance		2. What is my bowl
2.	Correctly spelt phrase:	✓	Correctly pronounced phrase:	✓	Correctly pronounced phrase:	✓
	1. Show me my transactions		1. Show me my transactions		1. Show me my transactions
	Incorrectly spelt phrases:		Incorrectly pronounced phrases:		Incorrectly pronounced phrases:
	1. Shw me mi tronsoctions	χ	1. Show me my Tronsoctions		1. Show me my tronsoctions
			Detected as:		Detected as:
			1. Show me my Transactions	✓	1. Show me my Transactions	✓
3.	Correctly spelt phrase:		Correctly pronounced phrase:		Correctly pronounced phrase:
	1. Convert 200 Australian dollars to pounds	✓	1. Convert 200 Australian dollars to pounds	✓	1. Convert 200 Australian dollars to pounds	✓
	Incorrectly spelt phrases:		Incorrectly pronounced phrases:		Incorrectly pronounced phrases:
	1. Convert 200 pound to yoyo	χ	Convert 200 pound to yoyo		Convert 200 pound to yoyo
			Detected as:		Detected as:
			Convert 200 pond to euro	✓	Convert 200 pound to euro	✓

Table 6

“Overall accuracy comparison of devices”

	Google Assistant	Google Home	Web App (HTML5 Speech API)
% Matched Intent	57.14%	100%	85.71%

Table 4 shows average test results and Table 5 displays the results from “typical” interactions with the chatbot. The phrases were derived from the user questionnaires and observing the users interacting with the chatbot, the dialog for the chatbot was devised during the design stage. Every utterance is given a “Confidence Score”, rated on a scale of 0.0–1, with 1 being a complete match and 0 meaning the utterance was not successfully matched to any intent. However, the NLU has also given words or utterances, where no intent was recognised, a complete confidence score of 1 as it was matched to the default fall-back intent to handle unexpected input from the user with ease. It is clear that the response times do vary depending on the underlying functionality required for the user to achieve their goal, for instance: requesting their transactions via email or the chatbot sending text confirmation of their appointment.

Overall, the chatbot is able to match intents with the majority of the utterances with only 10% of the utterances failing to match. This is taking spelling mistakes into account also. The total number of successful matched intents out of 20 user phrases amounted to 80%. These results indicate that the chatbot can understand most phrases including spelling mistakes reflecting the quality of the dialog. The phrase “wat have I spnt this weke” (meaning “what have I spent this week”) obtained a low confidence score of 0.33, however the chatbot was able to understand the intent of the utterance and identify the action.

The average response time has been rounded to 3 seconds, which is quite a rapid response time. This suggests that users will be able to perform their necessary tasks through interacting with the chatbot relatively quickly which is also reflective on the overall dialog duration as shown in Table 6, this represents the overall interaction time. The results will be further elaborated upon in the Evaluation. Both the Google Assistant and Home agent were tested through the AoG Simulator on the Actions on Google console (AoG). The Google Assistant integration was tested through the simulator using text input while the Google Home was tested through voice interaction on the AoG simulator. The simulator returns a JSON object holding information regarding the chatbots understanding level identifying whether the utterance was matched to an intent. Similar phrases were used across devices to outline any variations in the NLU understanding.

Table 7

“Conversation exit rate”

Intent	(n) Turns before exit
1.	3
2.	4
3.	4

Thorough testing was performed across multiple simulated devices. It was determined that the Google Home device could recognise the intent of an utterance even when colloquial phrases were produced. It was also identified that the Google Home device could recognise utterances where particular words were incomplete or mispronounced as shown in Table 7. It was found that input text provided to the Google Assistant on android phones, follows a more rigid syntax as it expects the intent present in the given input text to match the defined intent used within the training phrases, however this only applies when the use of input text is provided by the user, as the same speech recognition and machine learning algorithms are implemented with the Google Assistant regardless of the device or surface its embedded in. The speech recognition software in the Google Assistant transforms the user’s speech, even if incorrect, to the appropriate input text before it’s sent to the NLU. Consequently, the Google Home device was able to recognise and detect incomplete or mispronounced words, as interaction only occurred through voice commands. Based on these results, it is clear that voice interaction is the optimal method of engaging with the chatbot due to it being able to comprehend pronunciation errors and interpret them accordingly.

It was also found that the Google Assistant could only match an intent of a misspelt utterance when the trained intent or entity was typed correctly during text interaction. The Google Home device was the prime mode of interaction with the chatbot, as it was able to match the intent of each supplied utterance, achieving a perfect score of 100%. Establishing that spoken natural language conversational interfaces perform best, compared to that of text-based interactions as the Google Assistant achieved a result of 57.14%. There is a stark contrast between the two conversational interfaces, implying spoken dialog is a much better method of interacting with natural language conversational interfaces. The speech recognition on the Google Assistant has a higher recognition rate in comparison to the HTML5 Speech API as it had an average speech recognition of 85.71%, suggesting the Google platforms will perform better in real interactions with users.

Table 7 outlines the number of turns the chatbot took to try and understand the intent of an utterance corresponding to the misspelt phrase defined in Table 5 before exiting the conversation. The intents 1, 2 & 3 refer to checking balance, making a deposit and changing a pin on these occasions. On each turn the chatbot responded with fallback intents such as: “Sorry, I didn’t quite get that, could you repeat that please?” or “Sorry, I’m having difficulty understanding what you said, could you say that again?”. This feature handles unexpected input or when the chatbot cannot understand a given utterance. This also outlines the use of different fallback intents used even if the user utterance was the same on each turn. The chatbot exited the conversation due to repeatedly prompting the user for an appropriate utterance and lack thereof, its inability to understand the intent of the given utterance to successfully match it to a trained intent to carry out the appropriate action and fulfil the user’s goal.

Further user testing was conducted across all three communication channels of the chatbot; Google Assistant, Home and the web-based chatbot. This will give an insight into the true quality and overall usefulness of the chatbot which is measured by the user’s interaction experience. In order to conduct user testing, a set of questions were constructed to evaluate the chatbot. The results from the user questionnaires provided insight into the overall response and precision rate. The questionnaires were distributed across a user group consisting of 15 individuals with varying technical knowledge. The results from the user questionnaires were compared to the user interactions identifying subjective and objective metrics. During the questionnaire the users were observed to capture free standing information regarding the interaction experience which also qualify as subjective measurements. This allowed the identification of other quality metrics which were not initially considered and led to a deeper understanding of the chatbots performance. They completed a user questionnaire targeted to capture the quality of the chatbot. The subjective and objective metrics outlined in Table 8 were obtained using a range of question types including Likert scale and numerical scales to accurately assess the chatbots performance. Each metric outlined in Table 8 was measured in relation to a specific question to gather qualitative results. These questionnaires can be found in appendices.

Table 8

“Subjective & objective measurements obtained from user questionnaire”

Subjective metrics	(n) Users	Objective metrics	(n) Users
Naturalness	44.00%	Speech Recognition	86.67%
Likeability	93.33%	Response accuracy	86.67%
Ease of use	86.66%	Speed of Interaction	73.33%

The majority of banks struggle to get their customers to use the technology in place due to low user satisfaction when interacting with their services. From the results shown in Table 8, it can be established that integrating a chatbot into their banking services would greatly increase the rate of user satisfaction. This is evident as 73.33% of all users stated the chatbot was “extremely” quick compared to the current technology offered by their bank, for instance, online banking applications. From completing the user testing phase, it was found that 86.66% of all respondents found the chatbot was either “extremely” or “moderately” easy to talk to and it achieved an overall likability of 93.33% as outlined in Table 8. Reaffirming that chatbots can successfully engage users and increase satisfaction. The objective metrics obtained from the questionnaire results identify the chatbots performance and quality in a real interaction. Overall the chatbot received a very high speech recognition rate as 80.00% of users stated they were “very strongly” understood by the chatbot. The chatbot obtained an overall understanding score of 0.8894 (89.00%) in the range between 0.0–1 during the tests. This result is on the higher tier of the understanding scale.

Evidence from the results gathered during user-testing, through measuring the response accuracy to real user interaction, that the chatbot has a high competency level as both the results from the simulated and user tests have a very low variance of 2.33% in regards to the understanding score. These measurements reflect the accuracy and thoroughness of the emulated actions performed during testing.

Fig. 10.

“Users preference”.

The majority of users preferred to interact with the chatbot through voice commands, with 66.67% of user’s preference being voice interaction, as shown in Fig. 10. This supports that more users are wishing to be able to freely interact with applications and not be limited to traditional modes of interaction. This feature also offers users the ability to interact with chatbot without it requiring much attention on focusing on the task at hand, as users ask the chatbot a question and it will recognise the users voice negating the need for the user to type. The focus of any chatbot is to allow users to complete tasks with ease using natural language. It is important that the chatbot clearly understood the user and the user felt confident in interacting with the chatbot. Figure 11 shows this was achieved as only 13.33% of users stated they were “often” left in a state where they were unsure how to carry on the conversation. These results reflect low frustration rates among users as 60% of user never experienced this and 26.67% stating it occurred “Not often.” This clearly suggests the chatbot is able to sufficiently guide users through the conversation with ease providing appropriate responses to the majority of user phrases.

Fig. 11.

“Conversation flow”.

Enabling the chatbot to be context aware provides a great advantage for users as they can simply refer back to a previous conversation and still be understood by the chatbot. This is particularly useful in long conversational flows, reducing the amount in which the user has to repeat themselves thus shortening the time taken to achieve a task (see Fig. 12).

Fig. 12.

“Chatbot conversational context”.

The future of chatbots seems to be the desire to add more ‘naturalness’ so that users become more relaxed when dealing with a bot. Google have made some impressive leaps in this area with their Google Assistant service known as Google Duplex which can make calls to book restaurant reservations and hair salon appointments (Kinsella, [21]).

5. Conclusion

We show here that a chatbot is a suitable method for adopting technology as a means to distribute a service. The developed chatbot allows users to interact with their bank through natural language interaction, granting users more convenient and efficient access to their banking information. It was determined that chatbots perform at a very high standard and provide reliable and rapid responses to users compared to that of traditional methods. The average time spent interacting with the chatbot is very low as it provides an efficient way for users to manage their banking. The low interaction time reflects the high understanding and speech recognition rates, offered through the adoption of conversational user interfaces thus allowing users to freely interact with the chatbot to meet the demands of modern life. The chatbot has proven to fulfil the demand of users wanting instant access and availability information and services. Future work will concentrate on a knowledgebase specialized for online banking service or more powerful chatbot models. We will also integrate the chatbot across popular platforms such as Amazon Echo.

References

Balkhi, How to Increase Your Chatbot Engagement, Entrepreneur Magazine (2019), https://www.entrepreneur.com/article/337430.

Bapista and

Oliveira, Understanding Mobile Banking: The Unified Theory of Acceptance and Use of Technology Combined with Cultural Moderators, Elsevier Ltd., 2015.

Barty and

Recketts, Promoting Competition in the UK banking Industry, Promoting Competition in the UK banking Industry. BBA Competition Report (2014), 4. Available at: https://www.bba.org.uk/wpcontent/uploads/2014/06/BBA_Competition_Report_23.06_WEB_2.0.pdf (Accessed 10/10/2017).

Cahn, CHATBOT: Architecture, design, and development, Ph. D. Dissertation, University of Pennsylvania, 2017. https://s3-amazonaws-com-443.web.bisu.edu.cn/academia.edu.documents/57035006/CHATBOT_thesis_final.pdf.

Cambria and

White, Jumping NLP Curves: A Review of Natural Language Processing Research, IEEE Computational Intelligence MAgazine (2014). Available at: http://sentic.net/jumping-nlp-curves.pdf (Accessed: 21/10/2017).

Consoli, The dynamics of technological change in UK retail banking services: An evolutionary perspective, Research Policy, Elsevier 34(4) (2005), 461–480. doi:10.1016/j.respol.2005.02.001.

S.Y.

Dauda and

Lee, A conjoint analysis of consumers’ preference on future online banking services, Information Systems 53 (2015), 1–15, https://www-sciencedirect-com-443.web.bisu.edu.cn/science/article/pii/S0306437915000721.

Dole,

Sansare,

Harekar and

Athalye, Towards user interactive chatbots, International journal of emerging trends and technology in computer science 4(5) (2015), 45–56. Available at:http://www.ijettca.org/Volume4Issue5(2)/IJETTCS-2015-10-09-16.pdf [Accessed 11 Oct. 2017].

Duijst, Can we Improve the User Experience of Chatbots with Personalisation?, University Of Amsterdam (2017). Available: 10.13140/RG.2.2.36112.92165 (Accessed: 11/10/2017).

10.

DZone, ‘A Writer’s Guide to Conversational Interfaces’, 2018. Available at: https://dzone.com/articles/a-writers-guide-to-conversational-interfaces?fromrel=true (Accessed: 20/03/2018).

11.

V.M.

Eeuwen, Mobile conversational commerce: Messenger chatbots as the next interface between businesses and consumers, University of Twente 2017. Available at: http://essay.utwente.nl/71706/1/van%20Eeuwen_MA_BMS.pdf (Accessed: 20/11/2017).

12.

Fadhil,

Matteotti,

Armellin,

Villafiorita and

Betti, Coach me: a platform for promoting healthy lifestyle, in: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, ACM (2016), 2016, pp. 1077–1080.

13.

Følstad and

Brandtzæg, Chatbots and the new world of HCI, Interactions 24(4) (2017), 38–42. doi:10.1145/3085558.

14.

FRPT, HDFC Bank, Niki.ai tie up chatbot for banking, 2017. FRPT Research-Finance, 2017, pp. 24–25. Available at: http://web.b.ebscohost.com/bsi/pdfviewer/pdfviewer?vid=8&sid=c34ab15e-2256-41a5-8757-f8ef87bd08e8%40sessionmgr104 (Accessed 11/10/2017).

15.

George and

Kumar, Antecedents of Customer Satisfaction In Internet Banking: Technology Acceptance Model (TAM) Redefined, Global Business Review 14(4) (2013), 627–638. Available at: http://journals.sagepub.com/doi/10.1177/0972150913501602. doi:10.1177/0972150913501602.

16.

Google, Design Principles and Methodology, 2018. Available at: https://developers.google.com/actions/design/principles (Accessed: 27/03/2018), (Accessed 04/03/2018).

17.

Gregori, Evaluation of Modern Tools for an OMSCS Advisor Chatbot, 2017. Available at: https://smartech.gatech.edu/bitstream/handle/1853/58516/evaluation_of_modern_tools_for_an_omscs_advisor_chatbot%281%29.pdf?sequence=1&isAllowed (Accessed: 21/10/2017).

18.

Jurafsky and

J.H.

Martin, An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, 3rd edn, McMillan, NY, NUSA, 2017.

19.

Kar and

Haldar, Applying Chatbots to the Internet of Things: Opportunities and Architectural Elements, (2016). Available at: https://arxiv.org/abs/1611.03799 (Accessed: 06/11/2017).

20.

W.B.

King, Year of the chatbot: Credit unions gearing up for artificial intelligence, Credit Union Journal 21(4) (2017), 18. (Accessed 11/10/2017).

21.

Kinsella, New Google Assistant Ad Shows Google Duplex Making a Restaurant Reservation and People Are Using it Today, 2018, voicebot.ai, June 27 2018, https://voicebot.ai/2018/06/27/new-google-assistant-ad-shows-google-duplex-making-a-restaurant-reservation-and-people-are-using-it-today/.

22.

G.M.

Ling,

Y.S.

Fern,

L.K.

Boon and

T.S.

Huat, Understanding customer satisfaction of Internet banking: A case study in malacca, Procedia Economics and Finance 37 (2016), 80–85, Available at: doi:10.1016/S2212-5671(16)30096-X.

23.

Marketline , Kindre AI – Putting robots in human shoes, MarketLine (2017), https://www.marketline.com/blog/22mk-kindred-ai-putting-robots-in-human-shoes/ (29 Nov 2017).

24.

Metz, New Chatbot Would Like to Help You With Your Bank Account. Wired, 2016, https://www.wired.com/2016/06/new-banking-ai-now-chatbots/.

25.

Newswire , European Chatbots Market Report 2017 – Forecasts to 2023, Newswire (2017), https://www.prnewswire.com/news-releases/european-chatbots-market-report-2017—forecasts-to-2023-300503863.html, (15 August 2017).

26.

Y.Y.

Onufreiv, The Rise in Chatbots Trends, Appinventic, November 21 2017 (2017). https://appinventiv.com/blog/rise-trend-chatbots-2017 (Accessed 16/10/2017).

27.

Otwell, Laravel, 2018. Available at: https://laravel.com/docs/5.5/blade (Accessed: 05/02/2018).

28.

Ozcelik, DialogFlow – Create your succeeding chatbot. Chatbotslife (2017). https://chatbotslife.com/dialogflow-create-your-succeeding-chatbot-10655c6ebfd.

29.

Rossmann and

Zimmermann, Digital Enterprise Computing, Vol. 2, Gesellschaft für Informatik, Bonn, 2007, pp. 61–65.

30.

B.A.

Shawar and

Atwell, Chatbots: Are they Really Useful? LDV-Forum 2007. Available at: http://eprints.whiterose.ac.uk/81930/1/AComparisonBetweenAliceElizabeth.pdf (Accessed:22/11/2017).

31.

Teller Vision, HGS Identifies Top 10 Customer Experience Trends, Issues and Trends (2017). Available at: http://eds.a.ebscohost.com/eds/pdfviewer/pdfviewer?vid=14&sid=1fd6a005-b85b-430e-b3c8-30a3f70d947e%40sessionmgr4007 (Accessed: 21/11/2017).

32.

Watson, Preparing for the Cognitive Generation of Decision Support, MIS Quarterly Executive 16(3) (2017). Available at: http://www.misqe.org/ojs2/index.php/misqe/article/viewFile/705/465.

33.

Wellers and

Pieper, A New Computing Paradigm: Conversational AI For Consumers And In The Enterprise, Digitalist Magazine, 2017. https://www.digitalistmag.com/digital-economy/digital-futures/2017/01/10/conversational-ai-for-consumers-and-enterprise-04830223.