Application of artificial neural network model in predicting profitability of Indian banks

2.1 Bank specific internal factors

2.2 External factors

Some of the prominent macro-economic factors commonly used by researchers are as follows: i) Inflation [5, 14, 16, 25, 51, 53, 86, 87], and ii) GDP growth rate [15, 25, 51, 87, 88]. In addition, some researchers used growth rate of personal income [15], unemployment rate , exchange rate [88], and private consumption [51] as macroeconomic factors. It is observed that inflation and consumer price index are related. We have included inflation and GDP growth in our study. The ability of the banker to recover dues through legal action is constrained by the enforceability of the contract which depends upon the ‘rule of law’ in the country. Thus, the level of NPAs and hence the profitability of banks is related to the rule of law [89]. The factor ‘corruption’ has been included in view of India’s low rank (78) as per Corruption Perceptions Index, 2018 [90] as it negatively influences profitability of banks [54].

The real data for all the indicators is available for the financial years (financial year in India pertains to the period from 1 st April to 31 st March) 2010–11 to 2017–18. We have interpolated the data between the years from the year 2010–11 to 2015–16 i.e., for six years. In each year, 100 data points were identified by using the formula as shown below:

X i ⁢ j = X i + X i + 1 - X i 100 × n i ⁢ j

where i pertains to the year viz. 2010–11, 2012–13, …, 2015–16, and j pertains to numbers 1 to 100. Thus, in all we have 600 data points between the years 2010–11 to 2016–2017.

3.2 Data normalisation

OPEN IN VIEWER

As the data are measured in different units, it is important to perform data normalisation. It is computed to obtain and improve the subsequent numeric computation and the resulting output. Similar to the previous studies, we implemented min-max scaling technique which scales the data from 0 to 1 [113, 114]. The benefit of it is exactly preserving every relationship in the data without introducing any bias. The derivation of normalisation is shown below:

X i ′ = ( X i - X min X max - X min ) ⁢ ( X min ′ - X max ′ ) + X min ′

where X , X min and X max are the actual input data, minimum and maximum values of the input data. X min ′ and X max ′ are the minimum and maximum target values.

ANN is a dynamic flexible and non-parametric modelling tool. With regards to the mapping capabilities, ANN can perform any function mapping with arbitrarily desired accuracy [30, 115]. One of the most widely used neural network architectures is referred to as Multi-Layer Perceptron (MLP) model. MLP consists of several layers of computing nodes. Each node in the MLP receives an input signal from the other nodes or external inputs after which processed signals through transfer functions are relayed to the other nodes or final output. In MLP, all nodes and layers are arranged in a feed-forward manner. The first layer in MLP is called the input layer in which the external information is fed into. In between, there can be one or many layers called hidden layers which are important for ANN to identify the complex patterns in the data. The last layer is called the output layer where the ANN offers the model solution [30, 32]. The MLP with two hidden layers, reflecting one of the ANN structures used in this study is shown in Fig. 1. This ANN structure includes the first layer with 19 inputs, two hidden layers (first hidden layer with 55 nodes and the second with 25 nodes), and one output.

The ANN is trained using back propagation algorithm with gradient descent and momentum terms [31, 115, 116]. The main purpose of training is estimation of weights so that the errors measured by mean squared errors (MSE), mean absolute errors, sum of squared errors, etc. are minimized [30]. Consistent with the previous studies [37, 115], we use root mean squared error (RMSE) to measure the accuracy of the ANN model.

RMSE = ∑ i = 1 n ( y ´ i - y i ) 2 n

where y ´ i is the predicted performance of banks, y i is the performance of banks and n is the total number of banks or observations.

RMSE and Mean Absolute Error (MAE) are the two error measures widely used not only in machine learning and data mining discipline but also in other disciplines such as time series, bio sciences, atmospheric sciences, etc. Several authors have argued that MAE is a better error measure in comparison to RMSE. RMSE and other sum of square measures are considered to be ambiguous and sensitive to sample size as compared to MAE. However, [117], based on large set of simulated data, showed that RMSE is not ambiguous in its meaning. In addition, RMSE is more appropriate in contrast to MAE when the errors are normally distributed. Both MAE/RMSE reaches a constant when number of observations, n , is large [118].

In ANN there exists no formal method to derive a network configuration in arriving at a solution. A lot of factors such as hidden layers, number of hidden nodes, normalisation of the data, methodology applied in training the data set, among others can influence the performance of the ANN model. As such, the optimal or the best model is arrived through experiments [30]. For this study we use both a single and two hidden layers. The number of hidden nodes is not easy to determine a priori. Even though, there are several rules of thumb suggested for ascertaining the number of hidden nodes, e.g., n / 2, n , $n+$ 1, 2 $n+$ 1, etc., where n is the number of inputs, none of these work in all the situations [30, 113]. In this study, we use different variations of hidden nodes ranging from 9 to 200. We use Rectified Linear Units (ReLU) transfer function for the hidden units and linear function for the output. It has strong biological and mathematical underpinning.

Most of the modern neural networks apply ReLU functions. It is also the default activation function recommended to be applied in modern feed-forward neural networks [119, 120, 121, 122]. Application of ReLU to the output of a linear transformation yields a nonlinear transformation. This activation function has been found in improving the training of deep neural networks [123]. ReLU preserves many of the properties that make linear models easy to optimize with gradient-based methods and also this activation function helps to preserve many of the properties that make linear models generalize well [122]. ReLU is considered to perform better than other activation functions (e.g., sigmoid, tanh, linear activation function, softmax, etc.) in most cases as all the neurons are not activated at the same time. In some cases, the weights and biases are also not updated during backpropagation step of training the neural network, as the value of gradient is zero [124]. In the modern feed-forward neural networks, ReLU has replaced classical activation functions such as sigmoid, Tanh, etc. as those activation functions suffer from vanishing gradient problem [120, 121, 125]. ReLU is defined by the function:

g ⁢ ( x ) = max ⁡ { 0 , z }

With regards, to the choosing of optimizer for the neural network, at present, there is no consensus on the use of single optimizer. Currently, the most popular and widely used optimizers are SGD, SGD with momentum, RMSProp, RMSProp with momentum, AdaDelta and Adam. The optimization algorithm used for this study is “Adam”. The name “Adam” is derived from the phrase “Adaptive moments”. This optimizer was proposed by [126]. It is arguably the most used optimizer algorithm in the literature [122, 127]. It takes advantage of two other popular methods, i.e., AdaGrad [128] which works well with sparse gradients and RMSProp [129], which works well in on-line and non-stationary settings. Some advantages of Adam are that “the magnitudes of parameter updates are invariant to rescaling of the gradient, its stepsizes are approximately bounded by the stepsize hyperparameter, it does not require a stationary objective, it works with sparse gradients, and it naturally performs a form of step size annealing” [126].

OPEN IN VIEWER

In order to avoid overfitting, we applied early stopping and dropout of nodes in the network. Early stopping is used for regularization in deep learning. Its popularity can be attributed to its effectiveness and simplicity [122]. We also used dropout technique which refers to dropping of the units both hidden and visible in the neural network. It has been shown that dropout technique can prevent overfitting problem while at the same time enhances the performance of the network models. It randomly chooses the units to be dropped out and temporarily removes them from the layer [130, 131]. We applied uniform dropout rate of 0.2 for both the input and hidden layers. The MLP with a drop out is depicted in Fig. 2. The epoch refers to the number of training samples offered to the network between weight updates [132]. We performed training of the model at epoch size of 1000. We also used the fixed learning rate of 0.01. Traditionally, learning rate are kept fixed during training the model and optimal learning rate are generally determined by trial and error [122, 132, 133].

In order to achieve more reliable accuracy and robustness of the ANN model, we employed cross-validation technique [134, 135]. Cross-validation is a resampling technique whereby a sample is randomly is split into training and test subsamples. The major benefit of cross-validation is that the entire sample is used for testing and most of them are also employed for training the model. It not only improves the robustness of the model with sampling variation, but also offers simple method for examining the predictive accuracy of the model [30, 32]. For this study, we used ten-fold cross-validation which carries out the simple cross-validation ten-times. In it, the sample is split in ten equal and mutually exclusive parts. Training is performed on any of the nine of the ten portions and testing is done on the remaining. As such, ten overlapping training samples are created and testing is also performed ten times. The average test score of the entire ten partitions is a good indicator for the out-of-sample performance of the regressor [30].

In this study, we trained the model using the interpolated data of Bank 1 for six years from 2010-11 to 2015-16 and validated the models on Bank 1 for 2016–17 and 2017–18. Further, using the same model/s we predicted the profitability of rest of the banks for the same period, i.e., 2016–17 and 2017–18.

The main reason for training the ANN model on a single bank dataset for several years rather than training it on the dataset of several banks for several years is for the simplification of the problem for the machine learning algorithm. This will make the ANN model learn the pattern from the historical time-series data of a single bank to predict the future performance of the same bank or different banks better. The summary of the proposed ANN architecture consisting of parameters for this study is shown in Table 2.

4.1 Results using ANN

We perform several ANN models with a single as well as two hidden layers having different neurons. Table 3 shows these ANN models with RMSEs of both the training set and validation of the models. As ANN is performed applying 10 fold cross-validation to increase the robustness of the model, the train error in terms of RMSE along with the mean RMSE of the 10 sub-samples is given for the training set. It can be observed that the errors of the models are very minimal in the training set as compared to the validation set. Of all the ANN models, RMSE in the validation of the model is lowest in model 7 followed by 6, indicating that these are the most accurate models. Thus, we show the prediction of bank profitability of these two ANN models only (see Tables 3 and 4). From Table 4, it can be seen that both the ANN models are capable of predicting the profitability of banks very accurately.

4.2 Multiple linear regression

We also perform multiple linear regression to make a comparison with the performance of ANN models. The RMSE of multiple linear regressions is 0.605 and it can be seen than the regression model performs well in predicting the performance of banks. On comparison with ANN models it can be observed that the best ANN models, i.e. ANN models 6 and 7 offer marginally better solution as compared to the regression model. In fact, majority of the ANN models, except for model 8 and 9, offer marginally better solution in comparison to regression model (refer to Tables 3 and 4).

5.1 Managerial implications

In India, the individual banks and the Reserve Bank of India are monitoring the profitability on quarterly basis, as the financial statements of the banks are available only at the end of the quarters i.e. March, June, September and December. However, frequent monitoring of the profitability of the individual banks as well as that of the banking system is desirable. It will help individual banks to arrest declining trends and to take corrective actions in time. The frequent monitoring will help RBI to frame effective policy responses for managing business cycles and for promoting growth of the economy with financial stability. The internal factors are under the supervision and control of managements of individual banks and the government in coordination with RBI can influence external input factors. As such, input variables can be used as control factors for monitoring of individual banks and banking system as a whole.

The present study will enable individual banks to monitor their profitability on near-continuous basis and reduce the monitoring gap substantially. Besides, central bank can consider entire banking system as a single bank and predict and monitor profitability of the entire system. This will help the central bank to monitor the economy more closely. In addition, the findings of this study will enable the central bank to observe the trends in profitability more closely. The trends in profitability of banks have implications on the risk taking ability of banks which in turn affect supply of credit and economic growth. Furthermore, the model presented in the study can be used as a tool for handling economic crisis and ensuring economic stability. Thus, the use of current study can improve the efficiency of the central bank.

5.2 Limitations and future directions

As per the authors’ knowledge, this is the first work in approaching the panel data and ANN wherein the training of the model was done on a single bank rather than on multiple banks, thus, reducing the panel data to time series data. We also used interpolation of the training set to enlarge the data points. It will be worthwhile to further explore this approach in predictive analytics whether in the use of regression, ANN or other techniques. This study is faced with the difficulty in selecting the most appropriate ANN structure, particularly, number of hidden layer/s and number of neurons in those hidden layers. Selection of the most appropriate ANN model was done on trial and error basis. Future studies can explore/apply methods that help in automatically selecting the most appropriate ANN structure/model based on the characteristics of the dataset, e.g., Automatic Model Selection (AMS) by [137]. Metaheuristics which are nature inspired artificial intelligence (AI)-based nature inspired solution methodologies or algorithms can also be explored in predicting bank profitability [138].