An Evaluation of the Tracking Performance of Exchange Traded Funds (ETFs): The Case of Indian Index ETFs

Abstract

The present study investigates how efficiently India-domiciled exchange traded funds (ETFs) replicate the returns of their underlying indices and analyses the factors that determine the tracking performance. We use a three-pronged approach involving Capital Asset Pricing Model (CAPM) regression, cointegration-Vector Error Correction Model methodology and tracking errors (TEs) to assess tracking efficiency. Random-effects panel regression is employed to evaluate how fund-specific factors influence tracking ability. We find that ETFs carry significantly lower exposure towards their indices than what their objective would suggest. Long-run linkages with benchmarks exist for most ETFs, but the price deviations from the indices are fairly persistent. The TEs for the majority of the funds are large and non-trivial. Bid-ask spreads, price-net asset value deviations, fund’s age and, to some extent, its size are the primary factors that influence tracking performance. ETFs in developed markets such as the USA and Europe have been found to exhibit superior benchmarking abilities. The study is expected to assist investors in developing a more efficient ETF portfolio and to help fund providers improve the quality of their offerings.

Keywords

Exchange Traded Funds (ETFs)Tracking Efficiency India Passive Investing Emerging Markets

Executive Summary

The phenomenal expansion of the exchange traded funds (ETF) markets worldwide has attracted the attention of researchers and academics. Consistent underperformance by active funds, especially in the large-cap space, is strengthening the case for passive investing and thus ETFs. The ETF industry in India grew at a compounded annual growth rate (CAGR) of 50.84 per cent in the last decade, which is among the fastest in the world. The literature on Indian ETF markets is limited, moreover, with very few exceptions; most studies are based on small samples and employ primitive analytical techniques. The present article attempts to fulfil these research gaps. The objective of this article is to assess the tracking performance of equity ETFs listed on the Indian stocks exchanges and to identify the factors that determine how well these ETFs replicate the returns of their corresponding benchmarks. The sample includes 18 funds which have been analysed over a five-year period from 1 January 2014 to 31 December 2018. The tracking performance has been assessed on the basis of three approaches. First, a Capital Asset Pricing Model (CAPM) regression is run using daily ETF and index returns to understand the extent of under or over performance by the fund and to determine if the ETF managers follow a full-replication strategy. Second, using Johansen’s method we test whether the price and index series are cointegrated, that is, whether they exhibit a long-run equilibrium, additionally, we employ the Vector Error Correction Model (VECM) to investigate how persistent are the deviations from such an equilibrium. Finally, we compute three variants of tracking errors (TEs) to quantify the magnitude of the tracking inefficiency. The second part of the analysis pertains to the identification of fund-specific factors that have an impact on the fund’s tracking performance. We use the random effects method with tracking error as the dependent variable, the explanatory variables include an estimate of fund’s spread, asset size, age, volume, premium and the number of constituents in its index.

The empirical findings reveal that ETFs in India exhibit sub-optimal tracking performance; the fund managers fail to follow a full replication strategy. However, there is no indication of under or over performance. Most of the sample funds share a long-term equilibrium relationship with their corresponding indices but the deviations from this equilibrium are persistent; for the average fund, it takes approximately eight days for half of the deviation to disappear. The TEs are non-trivial and are larger than those reported for ETFs in developed regions. The panel regression shows that funds with higher spreads and premiums are likely to be poor trackers while older and to some extent, larger funds are expected to track their indices more closely. The study is expected to assist investors in developing an ETF portfolio with superior benchmarking abilities and the fund providers in improving the quality of their offerings.

Introduction

Ever since their launch in 1993 with the S&P 500 Depository Receipts, ETFs have become one of the most successful financial innovations in recent times. The primary objective of a traditional ETF is to closely mirror the returns of its underlying benchmark which may be a stock or bond index or a commodity.

An ETF is primarily a vehicle of passive investment which is continuously tradeable throughout the day and which can be created or redeemed in-kind by exchanging it with its underlying securities.¹ Their growth has been phenomenal in developed financial markets such as the US and Europe where active funds find it increasingly difficult to consistently generate above-average returns that justify their high expenses and commissions.

The objective of ETFs is similar to that of index mutual funds, that is, to provide the investors with an indexing option that allows them to hold a diversified portfolio of assets at low cost (Kostovetsky, 2003); however, ETFs are structured differently which allow them to offer some key advantages. Unlike index funds, ETFs provide intraday liquidity, they may be short sold and traded on margin. Moreover, ETF units are convertible and redeemable in-kind which allows the fund to manage its inflows and outflows without incurring liquidity costs (Gastineau, 2010). The in-kind creation and redemption mechanism also allows the fund to defer its capital gains thereby making it more tax efficient (Poterba & Shoven, 2002).

ETFs have been shown to have a significant impact on the attributes of their stocks and the financial markets in general (Ben-David et al., 2018; Da & Shive, 2018; Madura & Ngo, 2008); moreover, such impact is not just limited to country’s domestic funds but also applies to foreign funds which are traded outside the country’s boundaries but are benchmarked to its domestic stock and indices (Narend & Thenmozhi, 2019).

By May 2019, the assets invested in the Global ETF/ETP industry had touched a record high of USD 5.57 trillion (ETFGI, 2019). According to Kealy et al. (2017), if the current growth rates continue then the passive ETF industry is expected to surpass the actively managed funds by 2027. The United States is by far the biggest market for ETFs in the world. Out of the USD 4.8 trillion of assets invested in ETFs at the end of 2017, USD 3.4 trillion or about 72 per cent are in the US markets alone.

The size of the ETF market in India has historically been small but is now growing rapidly. The ETF industry registered a CAGR of 50.84 per cent in its assets under management (AUM) during the last decade which is among the highest in the world (Business Standard, 2017). Especially now with the Central Government choosing the ETF route for meeting its divestment targets, the Employees’ Provident Fund Organisation opting for them as the preferred mode of investing in the equity segment of the market and with the launch of India’s first corporate bond ETF, it is clear that ETFs are here to stay.

The key question that the present work seeks to answer is how successful are ETFs in their primary objective of tracking their underlying index closely. Further, we examine if a long-term equilibrium relationship exists between ETF price and index series and how persistent are the short-term deviations from such an equilibrium. The final objective is the identification of elements that may be driving TEs in funds and measuring their impact on tracking efficiency.

We find that ETFs in India do not track the movements of their underlying indices perfectly and generate non-trivial TEs, such a finding is expected as the index returns do not take into consideration the cost of trading and holding stocks. However, the TEs are larger in comparison with the funds operating in developed regions such as the US, Europe and Australia. Cointegration and VECM analysis reveal that the deviation between ETF prices and its index can be quite persistent. The panel regression indicates that market transaction costs such as the bid-ask spreads, deviations between ETF prices and its net asset value (NAV), the age of the fund and the size of its asset corpus are the determinants of TEs in funds.

Literature Survey

The phenomenal growth of the ETFs in the last two decades has led to an increased interest in this segment of the financial market. The literature on ETFs, especially from the developed markets, has expanded significantly. However, ETF-related research in emerging markets is still limited.

Studies from around the world have examined how effectively ETFs replicate the returns of their corresponding financial assets. Shin and Soydemir (2010) reported significant and persistent TEs and underperformance vis-à-vis the benchmark in a sample of 26 US-listed ETFs, in addition, the tracking performance of ETFs which were based on Asian indices were found to be substantially more inefficient. Similarly, Chen et al. (2017) in their analysis of New Zealand ETFs observed non-trivial TEs and low benchmark exposures. The volume and spread of the funds and the index constituents, as well as the volatility of the index, were found to be the main factors contributing to TEs. However, Johnson et al. (2013) found that Europe domiciled funds are efficient at tracking their indices and generate small TEs. Gallagher and Segara (2006) arrived at a similar conclusion for Australian ETFs.

In a multi-country analysis, Wong and Shum (2010) examined 15 ETFs from seven markets and found that ETFs always produced higher risk-adjusted returns. The authors also posit that market conditions may also impact the fund’s tracking performance; Jensen’s alpha and risk premiums were found to be higher when market expectations were bullish. Rompotis (2011) made a similar observation wherein a sample of 50 iShares ETFs was found to exceed the returns of their corresponding benchmarks. The TEs were substantial and persistent and were found to be affected by the fund’s expenses, volatility and age. Singh and Kaur (2016) who undertook a similar study for India-listed ETFs observed the presence of significant TEs; fund’s size, volume and volatility were found to be the primary determinants of tracking efficiency.

Another related area of research relates to pricing efficiency and arbitrage. Pricing efficiency implies minimization of the difference between the fund’s trading price and its NAV, that is, premium or discount. In theory, the arbitrage mechanism based on in-kind creation and redemption is expected to ensure that ETFs remain price efficient. Ackert and Tian (2008) who studied domestic and country ETFs trading in the US market found the premiums to be small but varying over time. The deviations were relatively larger and more persistent for the country funds. Kayali (2007) examined the performance of Istanbul’s first ETF, the Dow Jones Istanbul 20 (DJIST). He observed that price and NAV series are highly linked and move close together. Marshall et al. (2013) found that arbitrage opportunities in US and Swiss ETFs are exploited fairly quickly, with the median length of five minutes. Sethi and Tripathi (2019) showed that ETF returns in India are more volatile than their underlying assets and the majority of the excess volatility is attributable to factors that inhibit arbitrage.

Overall, we observe that ETFs often fail to track their benchmarks closely and generate substantial TEs, this is especially true for ETFs which track foreign and/or illiquid securities. Moreover, ETFs in developing markets are more prone to tracking inefficiencies than funds from developed regions. ETF markets in developed regions such as US and Europe have an effective arbitrage mechanism which ensures that ETF prices do not exhibit large and persistent deviations from their underlying assets. TEs are a popular metric used to ascertain the tracking effectiveness and the performance of ETFs. However, using TEs as the sole measure of an ETF’s performance can often produce misleading results (Lin & Mackintosh, 2010). More recently, authors have also employed CAPM methodology, cointegration analyses and long-term return performance to analyse ETFs. Furthermore, examination of contemporary literature reveals funds’ volume, age, expenses, volatility and size as well as the liquidity of its constituents to be the chief determinants of their tracking efficiency.

ETF related literature based on emerging markets in general and the Indian markets, in particular, is fairly limited. Few studies such as Shanmugham and Zabiulla (2012), Purohit et al. (2014), Singh and Kaur (2016) and Saji (2017) have analysed tracking efficiency of Indian ETFs. However, the analyses have mostly been based on TEs only. Moreover, most studies could examine only small samples as the market has started expanding during the last few years only.

We aim to fill this research gap by employing more standardized econometric techniques such as cointegration and error correction models, in addition to CAPM regression and TEs to examine tracking performance. The methodology adopted here allows us to investigate not just the extent of tracking inefficiency but also its persistence. The present work also attempts to identify factors that contribute to TEs; an area that has remained under-explored. Furthermore, the rapid growth of ETFs permits us to examine a larger sample than those of the above-mentioned studies which would improve the generalizability of our results.

Research Methodology

Sample and Data Description

To arrive at the sample for the study, we began by identifying all equity-index ETFs which are listed on either the National Stock Exchange or the Bombay Stock Exchange. Since the focus of the present work is on ETFs which consist of only stocks in their underlying portfolio, we exclude commodity-based funds (primarily Gold ETFs) and funds whose underlying portfolios include fixed-income or money-market securities. To ensure that all the ETFs have at least five years of trading data, we consider only those funds which had been listed before 1 January 2014 and were still in operation at the end of 2018. A few funds had to be dropped due to lack of continuous data. The final sample consists of 18 index ETFs, including sector, broad-market as well as foreign-index ETFs. This sample is one of the largest amongst studies focused on Indian ETF markets. The study period extends from 1 January 2014 to 31 December 2018. Table 1 presents a snapshot of all ETFs which form part of the sample.

Table 1.

Profile of Sample ETFs

Symbol	Fund Name	Underlying Benchmark	Date of Inception	AUM (as on 30 December 2018) (in ₹ Thousands)	Expense Ratio in % (as on 30 December 2018)
BSLNIFTY	Aditya Birla Sun Life Nifty ETF Fund	NIFTY 50 TRI	22 July 2011	14,739	0.05
ICICINIFTY	ICICI Prudential Nifty ETF	NIFTY 50 TRI	20 March 2013	96,559.48	0.05
KOTAKNIFTY	Kotak Nifty ETF Fund	NIFTY 50 TRI	02 February 2010	56,903.24	0.10
NIFTYBEES	Nippon India ETF Nifty BeES	NIFTY 50 TRI	28 December 2001	94,508.47	0.11
QNIFTY	Quantum Nifty ETF	NIFTY 50 TRI	10 July 2008	506.15	0.09
KTKSENSEX	Kotak Sensex ETF	S&P BSE Sensex TRI	06 June 2008	1,177.67	0.27
SBISENSEX	SBI Sensex ETF	S&P BSE Sensex TRI	08 March 2013	1,286,283.88	0.07
ICICINF100	ICICI Prudential Nifty 100 ETF	Nifty 100 TRI	20 August 2013	407.88	0.45
RELCNX100	Nippon India ETF Nifty 100	Nifty 100 TRI	22 March 2013	541.5	1.04
JUNIORBEES	Nippon India ETF Junior BeES	Nifty Next 50 TRI	21 February 2003	74,550.28	0.23
BANKBEES	Nippon India ETF Bank BeES	Nifty Bank TRI	27 May 2004	382,966.7	0.19
KOTAKPSUBK	Kotak PSU Bank ETF	NIFTY PSU Bank TRI	08 November 2007	6,217.63	0.49
PSUBNKBEES	Nippon India ETF PSU Bank BeES	NIFTY PSU Bank TRI	25 October 2007	10,308.97	0.54
INFRABEES	Nippon India ETF Infra BeES	NIFTY Infrastructure TRI	29 September 2010	1,270.03	1.14
M100	Motilal Oswal Midcap 100 ETF	NIFTY Midcap 100 TRI	31 January 2011	2,178.69	0.20
SHARIABEES	Nippon India ETF Shariah BeES	NIFTY 50 Shariah TRI	18 March 2009	309.61	1.16
N100	Motilal Oswal NASDAQ 100 ETF	Nasdaq 100 TRI	29 March 2011	9,599.3	0.54
HNGSNGBEES	Nippon India ETF Hang Seng BeES	Hang Seng TRI	09 March 2010	695.89	1.13

Source: Funds’ monthly factsheets.

The data for the study has been obtained from multiple sources. Basic characteristics such as underlying benchmark, date of inception, fund size and expense ratios have been sourced from reports of the asset management companies which operate the fund. The pricing and volume data has been obtained from the exchange where the fund is listed while reports of Association of Mutual Funds of India provide historical AUMs for the sample ETFs.

Methodology

The methodology section is divided into two parts. First, we explain the approaches employed to evaluate how efficiently ETFs replicate the movements and returns of their corresponding benchmarks. The second section discusses the possible sources that may cause the fund’s performance to deviate from that of its target index.

Evaluating Tracking Performance

We use the following three approaches to assess tracking efficiency:

CAPM Regression Model

We run the following linear regression equation based on CAPM:

r_{t, E T F} - r_{f} = α + β (r_{t, i n d e x} - r_{f}) + ε_{t}

(1)

where r_t,ETF and r_t,index are the returns on the ETF and on the underlying index, respectively, on day t while r_f is a proxy for the daily risk-free rate based on the yields of three-month government bonds. If the funds track their benchmarks closely then we should expect to see α, which measures the extent to which an ETF outperforms or underperforms its index, to be close to zero. Similarly, β should be close to one, as it captures ETF’s exposure towards its index.

Cointegration and Error Correction Model

Cointegration between two variables implies that they share a common stochastic trend (Hanck et al., 2019). Cointegrated variables are characterized by a long-term equilibrium and the time paths of such variables are influenced by the extent of deviation from this equilibrium (Enders, 2008). We use the cointegration methodology to determine the existence of a long-term equilibrium relationship between the prices of ETFs and their underlying indices. Johansen’s (1991) procedure is employed as a formal test of cointegration between the two series. As per this method, the existence of a cointegrating relationship is determined on the basis of the maximum eigenvalue (λmax) and the trace statistic (λtrace) which are expressed as follows:

λ_{\max} = - T \ln (1 - λ_{r + 1})

(2)

λ_{t r a c e} = - T \sum_{i = r + 1}^{k} \ln (1 - λ_{i})

(3)

where λ_i is the value of the characteristic root or the eigenvalue obtained from the estimated π matrix. ‘T’ refers to the numbers of usable observations available.

Furthermore, the existence of a cointegrating relationship would allow us to model the ETF and index prices in the form of an error correction model. Specifically, we estimate the following VECM:

\begin{matrix} Δ \ln E T F_{t} = α_{e, 0} + \sum_{j = 1}^{p - 1} α_{e, i} Δ \ln E T F_{t - i} + \\ \sum_{i = 1}^{p - 1} b_{e, i} Δ \ln I n d e x_{t - i} + δ Z_{t - 1} + ε_{e, t} \end{matrix}

(4)

where α_e_,0 is the intercept and α_e_,i and b_e_,i are the lag operator coefficients. Of primary interest, here is Z_t_–1, that is, the error-correction term (ECT) which measures the rate at which ETF prices adjust to correct any imbalance from the long-run equilibrium. The speed of adjustment coefficient (δ) indicates the rate at which ETF prices adjust to correct any imbalance from the long-run equilibrium between the ETF and the index. Following Chen et al. (2017), we also compute the half-life² of the price deviation which is an estimate of the number of days required to eliminate half of the ETF-index deviation. The optimal number of lags have been chosen on the basis of a combination of lag-length criterions, namely the Final Prediction Error (FPE), Schwartz Information Criterion, Akaike Information Criterion and the Hannan–Quinn Information Criterion.

Tracking Errors

The final method used to assess the tracking performance of ETFs is TEs. We follow Frino and Gallagher (2001) and Shin and Soydemir (2010) to compute three variants of TEs.

TE₁ is computed as the average of the absolute difference between the daily returns on the fund and the index.

T E_{1} = \frac{\sum_{t = 1}^{n} | r_{t, E T F} - r_{t, i n d e x} |}{n}

(5)

TE₂ is based on the standard deviation of the differences between the ETF and index returns.

T E_{2} = \sqrt{\frac{1}{n - 1} \sum_{t = 1}^{n} {(e_{p t} - \overset{─}{e_{p}})}^{2}}

(6)

where e_pt = r_t,ETF – r_t,index

Finally, TE₃ may be calculated as the standard error of the residuals from the following regression:

r_{t, E T F} = α_{i} + β_{i} r_{t, i n d e x} + ε_{i t}

(7)

T E_{3} = S . E . (ε_{i t})

(8)

Determinants of Tracking Efficiency

In this section, we identify factors that may be affecting the TEs and thus leading to sub-optimal replication. These factors have been selected on the basis of general economic reasoning and existing literature.

The following panel regression model captures both cross-sectional and time effects in TEs:

\begin{matrix} T E_{i, t} = β_{0} + β_{1} s p r d_{i, t} + β_{2} a b s p r e m_{i, t} + \\ β_{3} \ln ​ a u m_{i, t} + β_{4} \ln ​ v o l_{i, t} + \\ β_{5} \ln ​ a g e_{i, t} + β_{6} \ln ​ i n d e x c o n s_{i} + ε_{i, t} \end{matrix}

(9)

sprd_i,t is the estimated bid-ask spread of ETFs which has been computed using daily high, low and close prices using the formula developed by Abdi and Ranaldo (2017), absprem_i,t is the absolute difference between ETF price and NAV expressed in percentage terms. lnaum_i,t represents the AUM of the fund while lnvol_i,t is average daily traded volume, lnage_i,t indicates the number of years since the fund’s inception. Lastly, lnindexcons_i is the number of constituents in the fund’s index. The last four variables have been expressed in natural log terms to adjust for skewness. The time unit for the model is a year.

The spread of the fund is expected to have a positive relationship with TEs as funds with higher spreads are costlier and hence more difficult to trade. Consequently, the fund’s price may not always closely represent the value of the underlying index. Similarly, a higher premium indicates that fund prices vary widely from the value of its underlying securities and is likely to be associated with higher TEs. Daily traded volume measures liquidity; liquid funds are expected to track their indices more closely. Likewise, higher AUM is likely to enhance tracking performance as the fund’s expenses are borne by a larger asset base. We expect the tracking performance to improve with the age of the fund as the manager and authorized participants become more efficient in managing inflows and redemptions; moreover, the initial time period in the fund’s life is usually characterized by demand and supply imbalances which worsens tracking ability. Finally, lnindexcons_i is expected to be positively related to TEs as a higher number of underlying securities may translate to higher trading, holding and rebalancing costs. The description and the relevant literature for the variables used in the regression analysis is presented in Table 2.

Table 2.

Description of Variables Used in the Tracking Error Regression

Panel A: Dependent Variables.
Variable	Description	Relevant Literature
Tracking Error 1 (TE₁)	The first specification of the ETF tracking error. TE₁ is computed as the average of the absolute difference between the daily returns on the fund and the index.	Pope and Yadav (1994), Frino and Gallagher (2001), Shin and Soydemir (2010)
Tracking Error 2 (TE₂)	The second specification of the ETF tracking error. TE₂ is the standard deviation of the differences between the returns of the ETF and the index.
Tracking Error 3 (TE₃)	The third specification of the ETF tracking error. TE₃ is calculated as the standard error of the residuals from the regression of ETFs’ returns on the returns of their benchmark indices.
Panel B: Explanatory Variables
Variable	Description	Relevant Literature
Bid-Ask Spread ( sprd )	Estimated bid-ask spread of ETFs computed as per Abdi and Ranaldo (2017)	Chen et al. (2017)
Absolute Premium ( absprem )	The average absolute difference between ETF’s trading price and its NAV expressed in percentage	Purohit and Malhotra (2017)
Assets Under Management ( lnaum )	The natural log of assets under management of the ETFs. It is a proxy for the fund’s size.	Chu (2011), Singh and Kaur (2016)
Trading Volume ( lnvol )	The natural log of average daily trading volume of the ETF. It is a proxy for fund’s liquidity.	Singh and Kaur (2016), Chen et al. (2017), Shin and Soydemir (2010), Buetow and Henderson (2012)
Fund’s Age ( lnage )	The natural log of the number of years since the fund’s inception.	Rompotis (2011), Singh and Kaur (2016)
Number of Index Constituents ( lnindexcons )	The natural log of the number of constituents in the fund’s benchmark index. It may act as proxy for holding and rebalancing costs.	Gastineau (2010), Johnson et al. (2013)

Source: Authors’ own research.

Note: The table provides detailed descriptions of dependent and independent variables used in the panel regression. The relevant literature for each variable is also reported alongside.

The dependent variable is the average TE. There are three specifications of the model: one with each variant of TE. We impose the random effect assumption on the model on the basis of the results suggested by the Sargan–Hansen test of overidentifying restrictions, which is a robust Hausman type test used for choosing between fixed and random effects when cluster-robust standard errors are employed.

Findings

The descriptive statistics for the main variables used in the analysis are given in Table 3. We observe that the value of TE₂ and TE₃ is greater than one (in percentage terms) while TE₁ is slightly lower at 0.93 per cent. The high values of TEs provide preliminary evidence which indicates that non-trivial tracking inefficiencies exist in the sample funds.

Table 3.

Summary Statistics of Main Variables

Variable	Mean	Std. Dev.	Min	Max	p25	p50	p75
Tracking Error 1 (TE₁)	0.0093	0.0070	0.0010	0.0307	0.0043	0.0063	0.0159
Tracking Error 2 (TE₂)	0.0140	0.0111	0.0014	0.0574	0.0058	0.0103	0.0224
Tracking Error 3 (TE₃)	0.0133	0.0108	0.0014	0.0575	0.0056	0.0096	0.0220
Bid-Ask Spread ( sprd )	0.0090	0.0055	0.0021	0.0258	0.0047	0.0077	0.0130
Absolute Premium ( absprem )	1.8282	2.7659	0.1321	16.1379	0.3227	0.5569	2.1816
Assets Under Management ( lnaum )	8.4640	2.1269	4.5868	13.9354	6.5664	8.1606	9.6780
Trading Volume ( lnvol )	7.7403	2.1389	3.7802	11.6648	6.0508	7.5349	9.5524
Fund’s Age ( lnage )	1.7766	0.6375	0.0000	2.8332	1.3863	1.8688	2.1972
Number of Index Constituents ( lnindexcons )	3.6730	0.7177	2.4849	4.6052	3.2189	3.9120	3.9120

Source: Authors’ own computations.

Next, we regress the excess returns of the ETFs on the excess returns of their underlying benchmarks. The regression is based on daily data; the results are presented in Table 4.

Table 4.

Regression of ETF and Index Return

Fund	Intercept		Coefficient		R²
Fund	α (×100)	t-stat	β	t-stat	R²
BSLNIFTY	0.0539	1.43	0.1468	1.59	0.0023
ICICINIFTY	0.0330	1.18	0.6096	7.59***	0.0955
KOTAKNIFTY	−0.0708	−0.94	0.8592	49.07***	0.0725
NIFTYBEES	0.0061	1.87*	0.9190	107.75***	0.9379
QNIFTY	0.0139	0.93	0.7114	31.74***	0.5294
KTKSENSEX	0.0133	1.08	0.6830	22.99***	0.3324
SBISENSEX	0.0330	0.97	0.6665	8.25***	0.0819
ICICINF100	0.0565	1.52	0.3349	3.30***	0.0152
RELCNX100	0.0273	0.94	0.6052	9.32***	0.0849
JUNIORBEES	0.0058	0.74	0.8845	39.56***	0.7456
BANKBEES	0.0035	0.69	0.9441	118.21***	0.9250
KOTAKPSUBK	0.0044	0.27	0.8000	14.74***	0.7635
PSUBNKBEES	0.0031	0.20	0.8412	16.13***	0.7804
INFRABEES	0.0131	0.64	0.8085	17.78***	0.3279
M100	0.0207	1.11	0.6630	18.84***	0.3443
SHARIABEES	0.0735	1.94*	0.2723	2.14**	0.0088
N100	0.0487	1.18	0.1098	2.57**	0.0059
HNGSNGBEES	0.0480	1.01	0.2940	4.46***	0.0197

Source: Authors’ own computations.

Notes: The table presents results of the following regression: r_t,ETF – r_f = α + β(r_t,index – r_f) + ε_t. r_t,ETF and r_t,index are the returns on the ETF and on the underlying index, respectively, on day t while r_f is a proxy for the daily risk-free rate based on the yields of three-month government bonds. The t-statistics are based on Newey-West adjusted standard errors.

*, ** and *** denote significance at 10%, 5% and 1% levels, respectively.

We find that the intercept term for all the funds is very close to zero and is statistically insignificant for all funds except two (where it is weakly significant). Thus, the funds do not appear to underperform or outperform their underlying indices. This finding is in line with the passive nature of ETFs whose primary objective is simply to produce returns that are commensurate with the returns of their benchmarks. The beta coefficients of all funds exhibit statistical significance; however, all the values are smaller than one. With a mean value of 0.6196, the coefficient of the average fund is substantially lower than unity which suggests that ETFs fail to follow a full replication strategy (Rompotis, 2012) and have far lower exposure to their benchmarks than their objectives would suggest (Chen et al., 2017). The average R² of the regression is 0.3374, implying that only about a third of the variation in ETF prices is explained by the index movements. Overall, the low values of the coefficient and the R² are suggestive of poor tracking efficiency. These results may be attributable to low levels of trading and liquidity in the ETFs as well as a possible cash drag as fund managers are required to maintain some cash balance for dividend payouts (Chen et al., 2017). Purohit and Malhotra (2015) suggest that a demand and supply imbalance in the ETF market may also be responsible for such findings.

Next, using the ADF and the KPSS tests we find the log ETF prices and log index series to be integrated of order one, hence a test of cointegration may be suitably applied.³ We present the results of the Johansen’s test of cointegration between log ETF prices and log index values in Table 5. The results of the VECM and the half-life measure are also reported alongside.

The trace and max-eigenvalue test statistics from the Johansen’s test reveal that for all but two ETFs the price and index series are cointegrated. Hence, nearly all the ETFs in our sample share a long-term equilibrium relationship with their underlying benchmarks. In order to examine the error correction mechanism between the ETF and its index in case of a deviation, we employed the VECM to those funds whose price and index series are cointegrated. Table 5 reports the ECTs from VECM. Eleven of the funds generate ECTs which are negative and statistically significant indicating that the price-index system is stable and exhibits long-run convergence. The average ECT (based on significant values) is −0.1711 implying that about 17 per cent of the disturbance in the ETF-index equilibrium is corrected by the movement in ETF prices.

Table 5.

Cointegration in ETF and Benchmark Prices and the VECM

ETF	Johansen’s Test (H₀: r = 0)			VECM			Half-life (in days)
ETF	λ-max	λ-trace	γ	δ	t-statistic	Adjusted R²	Half-life (in days)
BSLNIFTY	14.41	22.20	−1.0269
ICICINIFTY	47.31***	54.30***	−1.0554	−0.1414	–6.22***	0.2107	4.90
KOTAKNIFTY	21.67**	28.34**	−1.0064	−0.0503	−1.25	0.0693	13.78
NIFTYBEES	19.13*	23.45*	−1.0436	0.0604	1.38	0.0340	11.48
QNIFTY	7.67	10.24	−0.6304
KTKSENSEX	19.93**	25.44*	−0.9702	−0.1088	−3.27***	0.3018	6.37
SBISENSEX	25.74***	32.11***	−1.1190	−0.0800	−4.92***	0.2353	8.66
ICICINF100	19.17*	26.99**	−0.6681	−0.0984	−4.61***	0.2819	7.04
RELCNX100	66.57***	74.37***	−0.9973	−0.2560	−7.57***	0.2942	2.71
JUNIORBEES	40.44***	49.35***	−0.9849	−0.3240	−4.55***	0.0975	2.14
BANKBEES	29.62***	36.97***	−1.0058	−0.0784	−0.89	0.0187	8.85
KOTAKPSUBK	15.05	23.93*	−0.9816	0.0134	0.27	0.0716	51.75
PSUBNKBEES	46.82***	53.93***	−1.0025	−0.1216	−1.42	0.0666	5.70
INFRABEES	184.43***	191.75***	−0.9927	−0.4020	−8.31***	0.2157	1.72
M100	24.91***	33.27***	−0.9603	−0.1005	−4.79***	0.1172	6.90
SHARIABEES	68.32***	74.02***	−0.9805	−0.3090	−8.31***	0.3780	2.24
N100	18.58*	23.43*	−1.4474	−0.0255	−4.08***	0.0942	27.23
HNGSNGBEES	20.25**	26.23**	−1.0343	−0.0372	−4.40***	0.1339	18.64

Source: Authors’ own computations.

Notes: The table presents the results from the Johansen’s test of cointegration and VECM between the ETF and index. λ-max and λ-trace are the max-eigenvalues and the trace statistics, respectively, from the Johansen’s test between log ETF and log index prices. γ is the normalized cointegrating coefficient. The equation of the VECM is as follows: $Δ Ln E T F_{t} = α_{e, 0} + \sum_{j - 1}^{p - 1} α_{e, i} Δ Ln E T F_{t - i} + \sum_{j = 1}^{p - 1} b_{e, i} Δ {LnIndex}_{t - i} + δ Z_{t - i} + ε_{e, i}$ . δ is the speed of adjustment coefficient or the ECT from the VECM.

*, ** and *** denote significance at 10%, 5% and 1% levels, respectively.

The mean half-life of ETFs with significant ECTs is 8.05 days; hence, it takes about eight days for half of the disequilibrium between fund prices and its index to be corrected. The half-life measures of the ETFs which track foreign indices (N100 and HNGSNGBEES) are much higher than average which is a likely consequence of asynchronous trade timings and higher costs of arbitrage associated with such funds. However, the maximum half-life value of 51 days is generated by the KOTAKPSUBNK which is surprising and points towards critical issues in the tracking ability of the fund. On the whole, the tracking inefficiencies appear to be quite persistent.

In Table 6, we present the mean TEs based on daily data. TEs are a commonly reported statistic as they help in quantifying tracking inefficiencies. We report the values for all the three measures of TEs. There are slight differences in the values of the variants. The standard deviation based TE₂ produces the highest values while TE₁ which is based on the absolute differences between the ETF and index returns generates the lowest estimates. TE₃, which has been computed as the standard error of the regression between ETF and index returns lies between TE₁ and TE₂. Based on TE₃, the TEs range from 0.20−2.73 per cent; with half of the funds producing a mean TE in excess of 1 per cent. For four funds, the TEs cross the 2 per cent mark. At 1.36 per cent, the TE of the average fund in the sample is fairly substantial.

Table 6.

Daily Average Tracking Errors (TEs) of Sample ETFs (in%)

Fund	No. of Observations	TE₁	TE₂	TE₃
BSLNIFTY	1,111	2.0278	2.8264	2.7270
ICICINIFTY	1,223	1.1546	2.3965	1.7204
KOTAKNIFTY	1,232	0.2370	0.3220	0.2952
NIFTYBEES	1,233	0.1532	0.2126	0.2021
QNIFTY	907	0.5384	0.7378	0.6765
KTKSENSEX	1,205	0.5600	0.8669	0.8243
SBISENSEX	1,153	1.0913	2.0001	1.9797
ICICINF100	1,171	1.3010	2.5736	2.5050
RELCNX100	1,106	1.1393	1.8864	1.8520
JUNIORBEES	1,226	0.3900	0.5606	0.5480
BANKBEES	1,225	0.2199	0.3233	0.3167
KOTAKPSUBK	1,230	0.7120	1.0578	0.9648
PSUBNKBEES	1,225	0.6454	1.0286	0.9707
INFRABEES	1,222	0.8295	1.3847	1.3659
M100	1,227	0.8019	1.0631	0.9998
SHARIABEES	1,074	1.9244	2.7272	2.6410
N100	1,218	1.2879	1.8003	1.5242
HNGSNGBEES	1,086	1.8299	2.5546	2.4180

Source: Authors’ own computations.

Note: The table presents the average of daily tracking errors of the sample funds during the study period. TE₁ is based on the absolute difference between ETF and its benchmark returns, TE₂ is computed as the standard deviation of the difference between ETF and benchmark returns. TE₃ is the standard error from following regression: r_t,ETF = α_i + β_ir_t,index + ε_it.

Rompotis (2011) in his analysis of 50 iShare ETFs domiciled in the US reported a mean TE of 0.63 per cent, Johnson et al. (2013) found the European ETFs to be generating an annualized TE between 0.04−0.21 per cent while Chen et al. (2017) in their study on New Zealand-based ETFs report TEs in the range of 0.65−0.94 per cent. Thus, the tracking performance of India domiciled ETFs appears to be less efficient as compared to the funds from more developed regions. The higher TEs may be attributable to the emerging nature of this market and consequently the low levels of trading and liquidity. We discuss these issues in more detail now.

Next, we use the panel regression methodology to examine the role of fund-specific factors, which were discussed earlier, in generating TEs. Table 7 reports the results of the random-effects panel regression of TE determinants. The spread variable is highly significant and positive in all the specifications which suggest that the cost involved in trading ETFs is a major factor that contributes to TEs. As the ETF market in India is still characterized by sparse trading and low levels of liquidity, the bid-ask spreads tend to be relatively high which in turn magnifies the TEs of the funds. The absolute premium variable is also of the expected sign and exhibits statistical significance for all variants of tracking error, thus funds whose price diverges widely from their NAVs are poor trackers of their underlying benchmarks. This finding implies that pricing efficiency and tracking efficiency are inter-connected. The AUM variable which is an indicator of the fund’s size is significant and negative in the case of TE₁ and insignificant (and negative) for the other two measures; thus, larger funds do seem to generate lower TEs but the effect might be spurious. Fund’s age appears to be negatively related to tracking inefficiency, that is, older funds track their indices more closely. This finding may be attributable to large fluctuations in prices caused by a demand–supply imbalance in the early phase of the fund’s life. lnvol and lnindexcons variables were insignificant in all the specifications.

Table 7.

Determinants of Tracking Errors

Independent Variable	TE₁		TE₂		TE₃
Independent Variable	Coefficient (× 10²)	Robust Standard Error	Coefficient (× 10²)	Robust Standard Error	Coefficient (× 10²)	Robust Standard Error
Sprd	86.6302***	0.0423	111.2275***	0.0843	111.5282***	0.0716
Absprem	0.0655***	0.0001	0.0828***	0.0001	0.0662***	0.0001
Lnaum	−0.0374***	0.0001	−0.0294	0.0003	−0.0244	0.0003
Lnvol	−0.0119	0.0001	−0.0057	0.0004	−0.0066	0.0004
Lnage	−0.0961*	0.0005	−0.2878***	0.0008	−0.2925***	0.0008
Lnindexcons	−0.0017	0.0004	0.0153	0.0006	0.0142	0.0006
Intercept	0.6156***	0.0021	0.9966**	0.0044	0.9317**	0.0044
Wald Chi	1256.28***		380.03***		485.77***
R -squared	91.99		66.18		64.90

Source: Authors’ own computations.

Notes: The table presents the results from the following random-effects panel regression: TE_i,t = β₀ + β₁ sprd_i,t + β₂absprem_i,t + β₃lnaum_i,t + β₄lnvol_i,t + β₅lnage_i,t + β₆lnindexcons_i + ε_i,t where TE_i,t represents average tracking error. sprd_i,t is the estimated bid-ask spread of ETFs computed using the formula developed by Abdi and Ranaldo (2017), absprem_i,t is the absolute difference between ETF price and NAV. lnaum_i,t represents the log of the fund’s AUM, lnvol_i,t is log of average daily traded volume, lnage_i,t is the log of fund’s age. lnindexcons_i is the log of number of constituents in the fund’s benchmark. The coefficients are based on robust standard errors.

*Significant at 10%, **significant at 5% and ***Significant at 1%.

The model is most successful in the case of TE₁, all the factors taken together account for about 92 per cent of the variation in it. It explains about 65–66 per cent of the variation for the other two TE measures.

Conclusion

The present work studies the effectiveness of India-domiciled ETFs in achieving their primary objective of benchmark replication. Low-benchmark exposure and high TEs suggest that the funds exhibit serious inefficiencies in their tracking mechanism. The tracking performance of the sample ETFs is significantly poorer in comparison with their counterparts in more developed regions.

The ETF-index disequilibrium has been found to be quite persistent in the short run, implying that tracking inefficiency is not only large but also non-transitory. As evidenced by low beta values in the CAPM regression, the funds do not seem to be employing full-replication strategies. Panel regression analysis shows that TEs are positively affected by the fund’s spread and premium. Moreover, older funds and, to some extent, larger funds are expected to be associated with better tracking efficiency.

Implications and Recommendations

The sub-optimal tracking performance is a cause of concern for the ETF markets as well as the investors who hold such products as they may earn returns which are significantly different from the benchmark which the fund is expected to mimic. Since the index represents only a paper portfolio, thus TEs can only be minimized and not eliminated. We believe that investors who base their investment decisions on factors identified in this study should be able to minimize the TEs of the ETFs in their portfolio. At the same time, fund providers should take these factors into account and adopt measures that would improve the tracking performance of their products.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

Notes

References

Abdi

, & Ranaldo

(2017). A simple estimation of bid-ask spreads from daily close, high, and low prices. The Review of Financial Studies, 30(12), 4437–4480.

Ackert

L. F.

, & Tian

Y. S.

(2008). Arbitrage, liquidity, and the valuation of exchange traded funds. Financial Markets, Institutions & Instruments, 17(5), 331–362.

Ben-David

, Franzoni

, & Moussawi

(2018). Do ETFs increase volatility? The Journal of Finance, 73(6), 2471–2535.

Buetow

G. W.

, & Henderson

B. J.

(2012). An empirical analysis of exchange-traded funds. The Journal of Portfolio Management, 38(4), 112–127.

Business Standard (2017). How India’s ETF market became the second fastest growing in the world. https://www.business-standard.com/article/markets/how-india-s-etf-market-became-the-second-fastest-growing-in-the-world-117030900426_1.html

Chen

, Chen

, & Frijns

(2017). Evaluating the tracking performance and tracking error of New Zealand exchange-traded funds. Pacific Accounting Review, 29(3), 443–462.

Chu

P. K. K.

(2011). Study on the tracking errors and their determinants: Evidence from Hong Kong exchange traded funds. Applied Financial Economics, 21(5), 309–315.

, & Shive

(2018). Exchange traded funds and asset return correlations. European Financial Management, 24(1), 136–168.

Enders

(2008). Applied econometric time series. John Wiley & Sons.

10.

ETFGI (2019). ETFGI global ETF and ETP insights. https://etfgi.com/research

11.

Frino

, & Gallagher

D. R.

(2001). Tracking S&P 500 index funds. The Journal of Portfolio Management, 28(1), 44–55.

12.

Gallagher

D. R.

, & Segara

(2006). The performance and trading characteristics of exchange-traded funds. Journal of Investment Strategy, 1(2), 49–60.

13.

Gastineau

G. L.

(2010). The exchange-traded funds manual (Vol. 186). John Wiley & Sons.

14.

Hanck

, Arnold

, Gerber

, & Schmelzer

(2019). Introduction to econometrics with R. University of Duisburg-Essen.

15.

Johansen

(1991). Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models. Econometrica: Journal of the Econometric Society, 59(6), 1551–1580.

16.

Johnson

, Bioy

, Kellett

, & Davidson

(2013). On the right track: Measuring tracking efficiency in ETFs. The Journal of Index Investing, 4(3), 35–41.

17.

Kayali

M. M.

(2007). Pricing efficiency of exchange traded funds in Turkey: Early evidence from the Dow Jones Istanbul 20. International Research Journal of Finance and Economics, 10, 14–23.

18.

Kealy

, Daly

, Melville

, Kempeneer

, Forstenhausler

, Michel

, & Kerr

(2017). Reshaping around the investor-global ETF research 2017. Ernst & Young Global.

19.

Kostovetsky

(2003). Index mutual funds and exchange traded funds. The Journal of Portfolio Management, 29(4), 80–92.

20.

Lin

, & Mackintosh

(2010). ETF mythbuster: Tracking down the truth. The Journal of Index Investing, 1(1), 95–106.

21.

Madura

, & Ngo

(2008). Impact of ETF inception on the valuation and trading of component stocks. Applied Financial Economics, 18(12), 995–1007.

22.

Marshall

B. R.

, Nguyen

N. H.

, & Visaltanachoti

(2013). ETF arbitrage: Intraday evidence. Journal of Banking & Finance, 37(9), 3486–3498.

23.

Narend

, & Thenmozhi

(2019). Do country ETFs influence foreign stock market index? Evidence from India ETFs. Journal of Emerging Market Finance, 18(1), S59–S86.

24.

Pope

P. F.

, & Yadav

P. K.

(1994). Discovering errors in tracking error. The Journal of Portfolio Management, 20(2), 27.

25.

Poterba

J. M.

, & Shoven

J. B.

(2002). Exchange-traded funds: A new investment option for taxable investors. American Economic Review, 92(2), 422–427.

26.

Purohit

, Choudhary

, & Tyagi

(2014). An evaluation of tracking error on world indices ETFs traded in India. IUP Journal of Applied Finance, 20(3), 41.

27.

Purohit

P. H.

, & Malhotra

(2015). Pricing efficiency & performance of exchange traded funds in India. IUP Journal of Applied Finance, 21(3), 16–35.

28.

Rompotis

G. G.

(2011). Predictable patterns in ETFs’ return and tracking error. Studies in Economics and Finance, 28(1), 14–35.

29.

Rompotis

G. G.

(2012). An empirical look at Swiss exchange traded funds market. Aestimatio: The IEB International Journal of Finance, 4, 52–81.

30.

Saji

T. G.

(2017). Performance and tracking efficiency of exchange traded funds: An analysis of Indian equity index funds. Indian Journal of Research in Capital Markets, 4(2), 7–14.

31.

Sethi

, & Tripathi

(2019). Excess volatility and costly arbitrage in exchange traded funds (ETFs): Evidence from India. Global Business Review. https://doi.org/10.1177/0020294019857485.

32.

Shanmugham

, & Zabiulla (2012). Pricing efficiency of Nifty BeES in bullish and bearish markets. Global Business Review, 13(1), 109–121.

33.

Shin

, & Soydemir

(2010). Exchange-traded funds, persistence in tracking errors and information dissemination. Journal of Multinational Financial Management, 20(4–5), 214–234.

34.

Singh

, & Kaur

(2016). Tracking efficiency of exchange traded funds (ETFs) empirical evidence from Indian equity ETFs. Paradigm, 20(2), 176–190.

35.

Wong

K. H.

, & Shum

W. C.

(2010). Exchange-traded funds in bullish and bearish markets. Applied Economics Letters, 17(16), 1615–1624.