A complementary webometric ranking system based on the website quality and traffic measures: A study focusing on top-ranked world universities

Abstract

Ranking of universities regarding their web-based activities plays a pivotal role in promoting scientific advancement since it motivates the open access accessibility to scientific results. In this study, a new ranking system based on the website quality factors and traffic evaluation was proposed. Since top-ranked universities are usually considered as the standard models for lower ranked ones, the focus of this study was on top-ranked universities. The proposed ranking was compared with well-known Webometrics ranking system. The website traffic and quality assessment were acquired for websites of top-ranked world universities and the correlation between these indices and the Webometrics ranking was evaluated. The summation of the weighted value of obtained measures according to an optimal weight vector obtained by a genetic algorithm framework was used for ranking purposes. The results showed that the website total traffic size was correlated with Webometrics rank ( $R\approx-$ 0.6, $p<$ 0.01). Also, using the weighted value of website quality and traffic measures, the proposed ranking system could predict Webometrics ranking by the accuracy of up to 69%. Even though the method was proposed for universities, it could be applied for ranking other types of centers or companies, provided that the suitable cost function for the genetics algorithm framework was defined.

Keywords

Webometrics ranking website traffic website quality measures correlation analysis genetics algorithm

1. Introduction

The world wide web has become the most important tool for information exchange and communication between researchers, scientific centers and universities all over the world (Park & Thelwall, 2006). In this regard, the quality and quantity of information presented by a center at web space has been the center of attention during the past decade. So far, several ranking systems were proposed in order to motivate publishing the scientific contents on the Web and in this way help the scientific development (Wimaladharma & Herath, 2016).

The international webometric ranking systems were suggested since the mid of 20 century, focusing on the quantity and quality of the content of websites (Almind & Ingwersen, 1997). The criteria such as the size of website, collaboration with other websites and the extent of rich files such as documents, slides and multimedia were considered for ranking purposes. According to the Björneborn and Ingwersen, webometric is the quantitative aspect of the construction of website and the application of information resources based on the informatics approaches (Björneborn & Ingwersen, 2004). Webometric measures the aspects related to websites, webpages and hyperlinks using search engines (Michael Thelwall, 2009). The webometric index plays a pivotal role in information and also organization management (Longqing & Qingfeng, 2011). Such an index aims to exhibit a clear picture of the scientific and educational status of an organization and helps the less-developed organizations to resolve their weaknesses toward improving the quality and quantity of information (Taheri et al., 2015).

The most popular webometric ranking system is Webometrics that is developed by cybermetrics lab, a group of the Spanish National Research Council, which takes into account the quantity and quality of Web content. Four measures of Visibility/Impact, Size/Presence, Openness/Richness and Excellence/Scholar (refer to Table 1 for detail description) were considered for ranking thousands of academic centers around the world according to their Web-based and scientific activities. So far, several researches have used the Webometrics criteria for comparing the scientific status of independent institutions. The comparison between the webometric status of world universities (Nissom & Kulathuramaiyer, 2012), the comparison between website impact factor of Arab-world universities (Elgohary, 2008), the study of Webometrics rank of open access repositories (Aguillo et al., 2010), the comparison between top-ranked Iranian medical universities with top-ranked world universities (Farashi et al., 2020) and the webometric status of Iranian hospitals (Shadpour et al., 2013) are some examples.

Despite the advantages of Webometrics ranking, it has several drawbacks (Mike Thelwall, 2008). The methodology for ranking and the period of reports are two controversial issues for Webometrics. This ranking system releases two reports each year, therefore continuous tracking of the quality and quantity of Web-based activities is difficult based on Webometrics. On the other hand, the criteria used by Webometrics are all quantitative measures (i.e. the number of backlinks, number of webpages, number of citations from top authors and number of high-quality papers), while it seems that the quality of the website is a missing concept for university evaluation by Webometrics. Therefore, the Webometrics ranking can exhibit only a part of the research and educational activities of a university (Mike Thelwall, 2008). In this regard, proposing new terms and systems for the assessment of Web-based activities of universities has been the subject of some studies.

Jati et al. proposed a ranking system that used entropy measure for dynamically weighing the visibility, size, openness and excellence values (Jati & Dominic, 2017). The method provided the same result as Webometrics which uses fixed weight value for each measure. Kats and Rokach presented an innovative concept named Wikiometrics that used the extracted metrics from Wikipedia for ranking purposes and obtained comparable results with standard ranking systems (Singh et al., 2013). Patidar and Vishwakarma proposed a new ranking system for considering different kinds of research organizations in the same package using a clustered domain ranking strategy (Patidar & Vishwakarma, 2014).

Here, it is hypothesized that the traffic indicators of a website and the criteria for evaluating the quality of website design and content might be suitable measures for webometric assessment. In this regard, investigating the relationship between Webometrics ranking and website traffic indices is a necessary practice. The results may reveal the applicability of website related information to predict the Webometrics rank or help to suggest more completed webometric metrics for a more accurate ranking.

So far several studies have been proposed for using traffic measures to evaluate Web-based activities. Guskov et al. investigated the relationship between website traffic (number of user audiences) and Webometrics ranking for 10 scientific organizations. The obtained results showed no significant relationship between Webometrics ranking and website traffic (Guskov et al., 2015). However, Vaughan et al. calculated the website traffic of several universities and business centers of USA and China and showed that there was a significant correlation between website traffic and performance of such centers (Vaughan & Yang, 2013). Sarkar et al. investigated the websites associated with tourism industry companies in India considering measures such as traffic size, webpage loading speed, number of webpages and so on (Sarkar et al., 2018). Jun et al. studied the capability of website traffic for forecasting by analogy. They showed that website traffic could be used as a potential tool for forecasting purposes (Jun et al., 2017).

The present study checked out the hypothesis if there was any correlation between Webometrics ranking and website traffic and quality assessment measures. Furthermore, it was investigated if such measures could be used for proposing a new ranking system, as an independent ranking system or for anticipating future ranking.

2. Material and methods

2.1 Sample population

For choosing the sample population for this study, the website traffic size of several Webometrics top-ranked universities were obtained from “similarweb” website, then an exponential curve was fitted to the traffic values. Where the fitted exponential curve crossed a threshold equal to 10% of maximum value, the addition of further universities to the sample population was terminated. In this way 30 universities were selected for further analysis. This guarantees the inclusion of the most successful universities from webometric perspective to the sample population, which are considered as the reference for other universities.

These 30 top-ranked world universities contained 25 universities from USA, three from England and two from Canada. Top-ranked universities are usually followed by lower-ranked universities from scientometrics methodologies and website design strategies. In this regard, we focused on top-ranked world universities instead of a broader range of universities. The information concerned the total website traffic size, traffic sources, average visit duration, average pages per visit, bounce rate (the last three ones were considered for the quality assessment of the website design and content) were acquired from “similarweb” website (https://www.similarweb.com). The description of the above-mentioned terms was given in Table 1. The number of broken links was investigated by checking out the internal and external URLs contained in the website. For this purpose, a free online broken link checker (accessible at: https://www.brokenlinkcheck.com) was used. The Webometrics ranks of universities were obtained using the Webometrics website (http://www.webometrics.info/en/world). The acquired information was presented in Table 2. Since the academic semester initiation and period that affect the website’s visit might be different for each university, for mitigating such bias on obtained results, the average value of measures for a long period (3 months, i.e. January, February and March 2019) was considered and reported in Table 2. In this regard, the average contribution of traffic sources was rescaled to cover 100% of total traffic size.

Table 1
Description of terms. The last five terms were averaged for three successive months

Term	Description
Visibility/impact	Describes the facility for a search engine crawler to reach a given Web page (Weideman, 2013). This index is calculated using the number of backlinks.
Presence/size	The total number of webpages hold by the website main domain and its subdomains (Asekun-Olarinmoye, 2015).
Openness/richness	Number of citations from top authors.
Excellence/scholar	The number of academic papers published as 10% most cited papers with affiliated personnel (Asekun-Olarinmoye, 2015).
Total website traffic	Number of user audiences or visitors.
Average visit duration(s)	The average time that visitors spend for website view.
Average pages per visit	Average number of pages that website visitors click through one session.
Bounce rate (%)	Percentage of website visitors who leave the site after viewing the first page.
Traffic sources	The way that website audience access the website which contains Direct access, Access using search engines, Referrals, Social networks and Emails.

2.2 Methodology

The block diagram for the procedures performed in this study was depicted in Fig. 1.

2.2.1 Feature set for ranking

The features considered in this study were divided into two categories: features from website traffic assessment and features from evaluating the quality of design

Table 2
Webometrics ranking and website assessment measures acquired for 30 top-ranked world universities

Webometrics rank	University	Visibility rank	Size rank	Openness rank	Excellence rank	Total traffic (millions visitors)	Average visit duration(s)	Average pages per visit	Bounce rate (%)	Traffic source (%)					Broken links
										Direct	Referrals	Search	Social networks	Emails
1	Harvard	2	38	1	1	34.28	124	2.86	66.01	33.5	5.33	56.74	2.69	1.69	12
2	Stanford	3	61	2	3	28.22	168	3.13	62.51	33.1	5.18	57.69	2.48	1.49	3
3	MIT	1	42	4	11	44.29	153	5.48	49.23	42.36	4.08	48.81	3.7	0.96	1
4	Oxford	14	1	10	4	11.65	228	4.5	53.25	37.15	5.93	50.58	4.96	1.09	25
5	California (Berkeley)	4	109	3	16	19.41	249	3.8	55.14	42	4.77	47.12	1.97	4.07	32
6	Michigan	7	68	13	5	14.69	230	4.18	53.44	36.49	7.82	48.3	1.83	5.54	7
7	Washington	6	139	43	7	11.88	165	3.17	60.36	26.23	13.84	55.42	1.65	2.84	75
8	Cornell	5	77	35	22	17.6	216	3.66	60.63	34.59	5.51	56.83	1.79	1.26	62
9	Colombia	9	136	16	13	14.6	224	4.07	59.45	37.23	3.22	49.13	3.39	6.94	139
10	Pennsylvania	12	130	32	15	10.63	268	4.42	52.15	43.28	4.29	48.42	2	1.95	50
11	Cambridge	18	102	6	10	9.52	256	4.77	50.3	44.24	5.28	45.8	3.33	1.31	175
12	John Hopkins	39	278	42	2	7.05	260	4.4	57.17	37.29	7.87	51.27	1.62	1.51	12
13	UCLA	16	176	23	12	13.25	289	5.17	50.57	39.36	3.31	50.78	2.2	4.34	20
14	Yale	10	208	19	20	8.73	165	2.97	59.98	33.86	5.36	55.77	2.57	2.4	25
15	Wisconsin	13	114	44	31	12.62	236	3.88	56.24	40.97	5.84	50.49	1.64	1.03	35
16	California San Diego	31	211	5	18	10.54	338	5.38	41.57	50.77	3.85	40.92	2.18	2.27	27
17	Minnesota	17	147	38	33	0.442	94	1.55	71.95	64.29	4.86	28.98	1.73	0.13	33
18	Pennsylvania	11	92	80	47	21.07	157	3.14	67.47	25.68	3.75	68.4	1.19	0.9	199
19	Toronto	34	123	47	9	13.09	357	6.29	35.77	52.65	3.07	38.43	3.17	2.61	204
20	Duke	27	151	12	21	7.21	193	4.33	56.15	30.62	8.98	56.42	2.16	1.75	42
21	College London	2	38	1	1	8.76	291	4.65	47.59	43.01	6.13	45.28	3.76	1.8	2
22	Texas Austin	3	61	2	3	11.6	240	4.32	52.55	44.48	5.59	47.33	1.43	1.15	1
23	Princeton	1	42	4	11	8.81	194	3.1	63.47	32.26	5.25	59.19	2.01	1.24	207
24	Chicago	14	1	10	4	8.3	257	4.2	51.72	37.53	5.59	51.38	2.21	3.27	168
25	New York	4	109	3	16	11.72	283	5.28	50.61	38.87	3.02	47.42	3.05	7.56	198
26	North Carolina Chapel Hill	7	68	13	5	7.55	272	4.72	52.9	42.6	4.45	48.76	2.63	1.49	205
27	California Irvine	6	139	43	7	9.65	349	5.86	37.67	55.74	6.48	33.09	2.4	2.24	26
28	British Columbia	5	77	35	22	10.98	377	5.52	43.61	47.19	2.88	44.88	2.76	2.24	184
29	Illinois Urbana Champaign	9	136	16	13	11.93	359	5.73	52.11	42.29	7.87	45.33	1.83	2.65	79
30	Northwestern	12	130	32	15	7.15	277	4.48	52.33	42.71	3.88	48.56	2.04	2.75	38

Figure 1.

Block diagram of the proposed ranking system.

and contents. The first category included features such as total traffic size and the contribution of different sources for attracting visitors to the website, while the second category included the measures such as average visit duration, average pages per visit, bounce rate and the number of broken links. It is worth noting that the total website traffic size was evaluated as the number of visits during a specified period of time and should not be interpreted as the volume of information entering the website (the common definition in information technology). The way that total traffic size is created is another important issue that reflects the Web-based activity of a university. Different sources that create the total traffic size of a website consist of direct access to the website through web browsers (Direct access), access to the website by referrals, access to the website through the search engines, access to the website via social networks and access to the website by available links embedded at emails. For universities that the staffs, researchers and students are well trained for using the web space facilities, the contribution of direct access to the website is high. On the other hand, considering SEO strategies helps universities to highlight the contribution of search engines in the total traffic size of their website. More activity of affiliated persons of the university at the Web space (in other websites or social networks) can also promote the website’s traffic size. Altogether, it seems that the contribution of traffic sources on total traffic size may suggest how universities use the Web space facilities for improving the university presence on the Web.

Besides the website’s traffic measures, there are some other factors which may motivate a visitor for using the published information on the website or contribute to the web-based activities of a university. These factors are directly or indirectly related to the quality of website design and also the quality and quantity of published information at the website. It is a difficult task to find the optimized website quality assessment tools since the universal accepted are not available. Even though there are different criteria for evaluating the website quality, here we selected some limited numbers (average visit duration, average pages per visit, bounce rate and the number of broken links) according to the available tools that obtain an assessment for these measures.

For each university, a vector of numbers contains total traffic size, the contribution of five different sources in total traffic size, average visit duration, average pages per visit, bounce rate and the number of broken links, i.e. a ten-element feature vector was created for further analysis.

2.2.2 Correlation analysis

In order to investigate the relationship between website traffic indices and Webometrics ranking, Spearman’s rank correlation coefficient was used. The Spearman’s rank correlation coefficient ( $R,-1<R<+1$ ) searches the monotonic relation between the rank of two random variables and its output is $+$ 1 if a perfect monotonic correlation exists. If $X$ and $Y$ be two independent variables and $\textit{RG}_{x}$ and $\textit{RG}_{Y}$ be the rank of these two variables, the Spearman’s rank correlation is calculated by Eq. (1) (Hatch & Farhady, 1982).

$\displaystyle R=\rho_{\textit{RG}_{X},\textit{RG}_{Y}}=\textit{cov}({\textit{% RG}_{X},\textit{RG}_{Y}})/\sigma_{\textit{RG}_{X}}\sigma_{\textit{RG}_{Y}}$ (1)

In which, $R$ is the regular Pearson’s correlation, cov is the covariance operator and ${\sigma}$ shows the standard deviation. The statistical significance of the correlation coefficient, $R$ , was checked using $t$ -test statistics.

Furthermore, the adjusted coefficient of determination ( $R^{2}$ ) that shows the proportionate amount of variation in the outcomes explained by the regression model was calculated according to Eq. (2).

$\displaystyle R^{2}=1-\left({\frac{n-1}{n-p}}\right)\frac{\textit{SSE}}{% \textit{SST}}$ (2)

In which, $n$ is the number of observations, SSE is the sum of squared error, SST is the sum of squared total and $p$ is the number of regression coefficients. Larger $R^{2}$ values show more variability of the obtained regression model.

2.2.3 Proposed ranking system

Information obtained from website’s traffic measures (i.e. total traffic size, the contribution of search engines, direct access, social networks, referrals and emails in total traffic size) and quality measures (i.e. average pages per visit, average visit duration, bounce rate and number of broken links) were considered as feature vector (consist of 10 elements) for checking if these factors were potent to predict the Webometrics ranking of top-ranked universities. For investigating the weight of each factor in total ranking score, a genetic algorithm strategy was used as follows.

1.
The sample population (feature vectors) was divided into train and test sets by a K-fold cross-validation strategy (K $=$ 5). The training samples were used for finding the optimal feature weight vector using a genetic algorithm framework, while the test set was used for testing the performance of the trained ranking system.
2.
The initial weight vector (initial chromosome) was specified by the correlation between each factor and the total Webometrics rank of universities in the study population. In order to consider the statistical significance, the initial weight for each factor was multiplied by 1-pvalue of the correlation between that factor and total Webometrics ranks.
3.
The weight vector was multiplied by the feature vector of each university and summed up to obtain a score for that university. Such a score was calculated for all universities in the population. The obtained scores were sorted in a descending manner. The ideal was that the sorted order was similar to the Webometrics rank, i.e. the first university of Webometrics rank should have the highest score and the second university in Webometrics ranking should have the second-highest rank and so on. However, due to differences between Webometrics measures and the considered features in this study, it was difficult to have the identical ranks, therefore another approach was used for evaluating the proposed method compared with Webometrics ranking. This strategy was to consider all pairs of universities, check if the rank in Webometrics was better for one of them whether the same was correct in the obtained rank by our proposed method. A similarity measure was calculated by dividing the positive cases by all possible pairs.
4.
Using genetic algorithm operators i.e. crossover and mutation and selecting the winner parents with a higher probability (weight vectors with higher similarity measures), new weight vectors were generated and the procedure explained in step 2 was repeated until the average change in fitness value was lower than a predefined threshold.
5.
Finally, 10 weight vectors with the highest similarity measure were chosen as the winner chromosomes and their average value was calculated as the final weight vector. The averaging was performed for reducing the effect of outlier weight values.
6.
The method performance was evaluated by test set. In other words, when the proposed ranking system was trained, the feature matrix that each row corresponded to feature vector of each test sample, was applied and the ranking procedure was performed between test samples. The obtained rank was compared with Webometrics rank of test sample in a pair-wise manner to obtain a similarity value. The above-mentioned procedure was repeated 10 times and an average similarity value was reported.

3. Results

Figure 2.

Correlation analysis between website traffic size and Webometrics ranks for (A) size, (B) visibility, (C) openness and (D) excellence. Each point showed the Webometrics rank against the website total traffic size. For thirty top-ranked world universities, the total traffic size was reported in terms of million visitors (M visitor) obtained from “similarweb” website. The solid line showed the fitted line to the data points by regression analysis. Spearma’s rank correlation coefficient ( $R$ ), coefficient of determination ( $R^{2}$ ) and $p$ -value were reported above each panel.

Figure 3.

Correlation between website total traffic size and overall Webometrics rank. For 30 top-ranked world universities, the traffic was reported in terms of million visitors (M visitor) obtained from “similarweb” website. The solid line showed the fitted line to the data points according to the regression analysis. The Spearman’s rank correlation coefficient ( $R$ ), coefficient of determination ( $R^{2}$ ) and $p$ -value were reported above each panel.

Figure 2A–D showed the relationship between website total traffic size and its Webometrics rank for size, visibility, openness and excellence measures, respectively. Based on the obtained results, there was a statistically significant ( $p<$ 0.001) correlation between total traffic size and size rank ( $R=-$ 0.708), while the correlation with visibility rank was $R=-$ 0.782. According to Fig. 2C and D there was no significant correlation between total traffic size and openness/excellence ranks. Figure 3 showed the relationship between the total traffic size for website of top-ranked world universities and their overall rank in the Webometrics system. The results indicated a statistically significant correlation of $-$ 0.59 ( $p<$ 0.01).

The average visit duration and average pages per visit indicate the traffic quality of a website (Subhani, 2014). For longer visit duration, the probability of taking an action is higher. The action could be purchasing a product or service or subscribing the website which are in line with the purposes of the website owner (Prasetio et al., 2016). In the present study, besides average visit duration and average pages per visit measures, their ratio was also considered as a criterion for website quality assessment. The correlation between these measures and size, visibility, openness and excellence rank was calculated using Spearman’s rank correlation coefficient. The result was reported in Table 3.

Table 3

Correlation between average visit duration, average pages per visit and their ratio with Webometrics ranks and bounce rate

	Bounce rate (%)	Size rank	Visibility rank	Openness rank	Excellence rank
Average visit duration(s)	$R=-$ 0.86, $p<$ 0.0001	$R=$ 0.17, $p=$ 0.365	$R=$ 0.21, $p=$ 0.263	$R=$ 0.1, $p=$ 0.61	$R=-$ 0.08, $p=$ 0.67
Average pages per visit	$R=-$ 0.94, $p<$ 0.0001	$R=$ 0.11, $p=$ 0.57	$R=$ 0.19, $p=$ 0.30	$R=$ 0.06, $p=$ 0.76	$R=-$ 0.09, $p=$ 0.61
Average visit duration/average pages per visit	$R=-$ 0.18, $p=$ 0.34	$R=$ 0.15, $p=$ 0.43	$R=$ 0.6, $p=$ 0.75	$R=$ 107, $p=$ 0.57	$R=$ 0.20, $p=$ 0.289
Bounce rate (%)	$R=$ 1, $p=$ 0	$R=-$ 0.02, $p=$ 0.90	$R=-$ 0.14, $p=$ 0.46	$R=$ 0.07, $p=$ 0.72	$R=$ 0.14, $P=$ 0.46
Number of broken links	$R=-$ 0.001, $p=$ 0.99	$R=$ 0.05, $p=$ 0.81	$R=$ 0.19, $p=$ 0.32	$R=$ 0.35, $p=$ 0.06	$R=$ 0.39, $p=$ 0.29

Since the result of Figs 2 and 3 indicated a statistically significant correlation between Webometrics ranking and website traffic size, those traffic sources which were responsible for such a correlation were investigated. For this purpose, the correlation between overall Webometrics rank and the contribution of each traffic sources (i.e. direct, search engines, referrals, social networks and emails) in total traffic source was calculated using Spearman’s rank correlation and the results were reported in Table 4. The result of Table 4 implied that the statistically significant effect ( $p<$ 0.05) was only observed for direct access and access by search engines, however, the correlation was smaller compared with total traffic size ( $R<$ 0.5).

Table 4

Correlation between overall Webometrics rank and the contribution of different traffic sources in total traffic size of the website

Rank/traffic source	Direct	Referral	Search	Social networks	Email
Total rank	$R=$ 0.40	$R=-$ 0.10	$R=-$ 0.36	$R=-$ 0.07	$R=$ 0.13
	( $p=$ 0.029)	( $p=$ 0.590)	( $p=$ 0.048)	( $p=$ 0.70)	( $p=$ 0.48)

In order to investigate whether the feature set introduced in this work was potent to predict Webometrics ranking of top-ranked world universities, a new ranking system

Table 5

Evaluation of the proposed method for predicting the next Webometrics ranking. The Webometrics ranking (January 2020 edition) and traffic information acquired for 5 top-ranked world universities were reported in this table. The system was trained according to the Webometrics report for January 2019

University	Total traffic (millions visitors)	Average visit duration(s)	Average pages per visit	Bounce rate (%)	Traffic source (%)					Broken link	Rank in current Webometrics report	Rank in past Webometrics report	Predicted rank based on proposed method
					Direct	Referrals	Search	Social	Email
Harvard University	50.69	139	2.51	69.91	28.87	4.45	62.25	3.09	1.29	36	1	1	1
Stanford University	21.56	195	3.50	61.76	31.16	4.45	59.75	2.86	1.73	11	2	2	3
MIT	4.22	70	1.63	80.80	25.38	3.65	69.41	1.40	0.16		3	3	2
California Berkeley	16.98	323	5.08	47.22	43.49	3.93	45.96	2.56	4.03	72	4	5	5
Washington	10.75	182	3.38	61.77	27.08	12.89	54.42	2.15	3.42	96	5	7	4

was proposed based on these features. According to the methodology section, a genetic algorithm-based ranking system was trained by train samples (24 feature vectors out of 30 feature vectors) and evaluated using test samples in a K-fold (K $=$ 5) cross-validation framework. The learning procedure was performed using train data with the average similarity of 78.30 $\pm$ 2.90% (min $=$ 72.83%, max $=$ 81.16%) with Webometrics ranking, while testing was performed with the average similarity of 80.00 $\pm$ 7.10% (min $=$ 70%, max $=$ 90%) with Webometrics ranking (10 iterations). The similarity value was calculated based on a pair-wise comparison strategy that was described in step 3 of the “proposed ranking system” section.

In order to check the ability of the proposed method for Webometrics rank prediction, the website traffic and quality assessment measures of five top-ranked world universities were extracted based on the January 2020 edition of Webometrics report (i.e. a newer and different edition from the one that had been used as train sample for this study). The values were shown in Table 5. It should be noted that in this analysis the proposed ranking framework was trained according to the last ranking reports and then was used for predicting the next Webometrics ranking.

Table 5 indicatesd that even though the output ranks of the proposed method were not similar to Webometrics, in a pairwise comparison mode, 8 out of 10 pairs were correctly ranked. For example, while Stanford University had greater rank compared with MIT, California Berkeley and Washington in Webometrics, it was assigned a higher rank compared with only California Berkeley and Washington by the proposed ranking system i.e. in two out of three possible pairs the ranking position of Stanford University were similar between these two ranking systems (Webometrics and the proposed one). Repeating this procedure for five randomly selected sets ( $n=$ 5) obtained the prediction accuracy of 69.44 $\pm$ 8.22%.

This study was focused on top-ranked world universities, however, it was speculated if the proposed ranking system could obtain the reliable outcomes for middle-ranked or low-ranked universities, when it was trained according to the information of top-ranked universities. For this purpose from the January 2020 report of Webometrics, three sets of samples ( $n=$ 5) from middle-ranked universities (100 $<$ rank $<$ 200) and also three sets of samples from low-ranked universities (500 $<$ rank $<$ 600) were extracted randomly and each was fed to the trained system. The average similarity value between the output of the proposed method and Webometrics ranking systems (in a pair-wise comparison mode) was 58 $\pm$ 9.68% for middle-ranked universities while for lower-ranked universities the similarity dropped to 55 $\pm$ 15.67%.

4. Discussion

Figure 2A indicated that by increasing website pages (lower size rank) the total traffic size (or total visits) was larger. Furthermore, websites with higher number of backlinks (lower visibility rank) were more connected to other websites and in this way shared more information contents which increased total traffic size (Fig. 2B). However, the analysis (Fig. 2C and D) showed that there was no significant correlation between total website traffic size and openness/excellence ranks. Since only limited numbers of published papers by affiliated persons of the university are specified as 10% most cited papers, which describes the excellence ranking, it is reasonable to suppose that excellence has limited effect on website traffic size.

The weighted summation of impact, presence, openness and excellence scores is used by Webometrics to rank universities. The correlation analysis presented by Fig. 3 showed that there was a statistically significant correlation between overall Webometrics rank and total traffic size of a website ( $R=-$ 0.59, $p<$ 0.01). Such a correlation was reported by other studies (Vaughan & Yang, 2013). This result suggested that the traffic of the website could be used as a potential candidate for predicting the website ranking, provided that other factors related to traffic were considered.

The results of Table 3 showed no statistically significant correlation between the quality of website content and Webometrics ranking, while the longer average visit duration and larger average pages per visit were correlated with bounce rate. Since bounce rate indicated the percentage of website visitors who left the website after observing the first page, lower bounce rate indicated the higher quality of the webpage content, which convinced the visitors to stay longer on the website and reviewed more webpages. The results of Table 3 emphasized that the Webometrics ranking system did not consider the quality of website pages (Mike Thelwall, 2008).

The website total traffic size for 30 top-ranked world universities showed that direct access contributed for 40.41% (min $=$ 25.68%, max $=$ 64.29%, Median $=$ 40.16%) of total traffic size, while the contribution for referrals was 5.44% (min $=$ 2.88%, max $=$ 13.84%, Median $=$ 5.26%). Furthermore, the search engines were responsible for 49.25% (min $=$ 28.98%, max $=$ 68.4%, Median $=$ 48.78%) of website total traffic size, while the contribution of social networks and email for total traffic size was 2.412% (min $=$ 1.19%, max $=$ 4.96%, Median $=$ 2.19) and 2.415% (min $=$ 0.13%, max $=$ 7.56%, Median $=$ 1.87% ), respectively. This result indicated that search engines and direct access were the most important ways for referring to the website content, when the population under study was top-ranked world universities.

Where we calculated the correlation between traffic size and ranks (Fig. 2), the negative correlation indicated that top-ranked universities with better rank (lower ranks) had higher traffic size. Also, it was reasonable that for a high quality website, higher average pages per visit and average visit duration was accompanied by substantially lower bounce rate (i.e. a big negative correlation reported in Table 3). Furthermore, Table 4 indicated that when the Webometrics rank of a university was better, more users accessed to its website content by searching strategies (i.e. negative correlation between search and total rank in Table 4), however, perhaps due to the advanced SEO strategies that was considered by top-ranked universities which facilitated the access to their websites by search engines, majority of users preferred to reach website content by search engines instead of direct access. While for lower ranked universities direct access probably obtained better results which caused the reported positive correlation in Table 4.

Among the features discussed here, the contribution of direct access and search engines in total traffic size and also the total traffic size of the website showed a statistically significant correlation with Webometrics total rank (see Fig. 3 and Table 4, $p<$ 0.01). The interesting point about the obtained results reported by Table 4 was that whatever the website ranking was closer to 1, the contribution of direct access on website traffic was reduced, while the search engines showed the higher impact on total traffic size, possibly due to more optimized search engine optimization. This result emphasized that universities should gain special attention to the search engine optimization process (Weideman, 2013).

Even though the proposed method could predict the correct position of universities in Webometric ranking (with a prediction accuracy of up to 69%), the result for forecasting the future rank of middle and low-ranked universities by the proposed method (while the ranking system was trained by the information of top-ranked universities) indicated that the trained system was not able to predict the ranking of universities with low or middle rank in Webometrics with satisfactory accuracy. However, training the system with related information of the corresponding universities (i.e. low rank or middle-ranked universities) might increase the performance of the system.

It is worth noting that, due to the random nature of the genetics algorithm, the weight vector obtained for each repeat (distinct set of train samples) was completely different from other repeats. In this regard, it was not possible to declare a distinct and unique weight for each feature to show its importance on ranking.

4.1 How to use the proposed ranking system?

In this study, a new strategy was proposed for ranking universities according to the website traffic and quality measures. The simulation results showed that the prediction accuracy was about 69% when Webometrics ranking was used as the standard. Since the required information for the proposed ranking system is obtained in a quicker and easier way compared with the required information for Webometrics, it is worth using it for continuously tracking the webometric status of a university when compared with some other targeted universities. Upon the release of the Webometrics report, our proposed system can be retrained by website traffic information and quality measures. The trained system can be used for predicting the possible position of a university among some targeted universities by applying the updated information of website traffic and quality measures. This procedure can be performed for example monthly, therefore, a good estimate about the future Webometrics rank of the university can be obtained. The output of the proposed ranking system provides an evaluation of university Web-based activities toward improving Webometrics ranking. Even though the focus of this study was on universities and their webometric ranking, the proposed method can simply be used as a ranking prediction tool for other types of websites, provided that it is trained by suitable information.

4.2 A note on study sample size

Sample size has substantial effect on machine learning procedure. Small sample size may lead to a biased performance estimate and a low statistical power. However, this study was restricted to top-ranked universities according to their website traffic measure. This limited the sample population to only 30 samples. Regarding our regression analysis i.e. Figs 2 and 3, even though the study sample size is quite small, Van der Ploeg et al. found the optimum sample size for linear regression model should be at least 20 to 50 samples for each candidate feature (van der Ploeg et al., 2014). Furthermore, in the current study, a regression/prediction model related to several predictors was proposed. The accuracy of the prediction method is relied on sample size and the squared multiple correlation coefficient ( $R^{2}$ which is equal to $\sim$ 0.34 in our study). Algina et al. (Algina et al., 2002) calculated the dependence between prediction/regression accuracy and $R^{2}$ . With a 0.05 bound error and our fixed sample size (30 samples), considering the perfect correlation accuracy of 1, the prediction accuracy may perturbed by $\sim\pm$ 0.15 (Refer to Table 8 in (Algina et al., 2002)). Therefore, regarding our machine learning approach, it seems that with the selected sample size a $\sim\pm$ 15% perturbation should be considered for reported prediction results.

4.3 Study limitation and future studies

In this study, a complementary webometric strategy was proposed which might enrich the evaluation of web-based activities of different types of organizations. However, here the proposed method was only tested for webometric evaluation of top-ranked universities. Testing the methodology for other types of organizations by future studies might be useful. Furthermore, it is useful to test the generality of the method by inclusion lower rank universities. Also, in this study the traffic size was measured based on the number of visitors, however, other definitions of traffic measures such as the size of transmitted and received bytes can be considered for website usage and the results be compared with this study. Another shortcoming of the current study is that the included universities were from English-language countries (USA, Canada and England) with the near identical culture. This might affect obtained results since studies showed that cultural characteristics might influence website design, structure (Fletcher, 2006) and website’s usage (Singh et al., 2013). Inclusion of samples from different cultures might obtained more generalized results. Finally, in this study a limited number of website quality assessment measures were used. Inclusion of new quality-specific assessment measures such as ease of use, reliability, availability of the needed information, speed of downloading measures and so on can enhance the interpretation.

5. Conclusion

In order to promote competition between scientific centers and improving the web-based publication of scientific information, different types of webometric ranking systems have been developed during the last decades. The best known is Webometrics, despite its benefits, it has many drawbacks including ignoring factors such as the quality of website design and content updating. These drawbacks limit the potential of Webometrics ranking for an accurate ranking. In the current study which was focused on the webometric evaluation of top-ranked world universities, the relationship of website traffic and quality measures was investigated with Webometrics ranking. Furthermore, a complementary ranking system based on extracted features of website traffic information and some quality measures was proposed that incorporated the genetics algorithm in the training step. The results showed that there is a correlation between Webometrics ranking and website traffic ( $R\approx-$ 0.6, $p<$ 0.01). Furthermore, among different traffic sources, the traffic produced by search engines was the most important one for promoting the Webometrics ranking. This result emphasized the importance of enhancing search engine optimization. Also, the results showed no significant correlation between Webometrics ranking and the factor related to the quality of website design which probably showed the necessity for more comprehensive criteria for webometric evaluation than Webometrics. Using the website traffic and quality measures, it was possible to predict the ongoing ranking with accuracy up to 69%, therefore, the proposed system could be used for continuously evaluating Web-based activities of universities. It should be noted that even though several measures are available for website quality assessment, here we tried to use some simple measures which their evaluation could be easily performed using free available tools and the usage of complex measures were prevented.

Funding

This work was supported by deputy of research and technology, (Grant No. 980321217).

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Not applicable.

Footnotes

Conflict of interest

The authors declare that they have no conflict of interest.

References

Aguillo

I.F.

Ortega

J.L.

Fernández

, & Utrilla

A.M.

(2010). Indicators for a webometric ranking of open access repositories. Scientometrics, 82(3), 477-486. doi: 10.1007/s11192-010-0183-y.

Algina

Moulder

B.C.

, & Moser

B.K.

(2002). Sample size requirements for accurate estimation of squared semi-partial correlation coefficients. Multivariate Behavioral Research, 37(1), 37-57. doi: 10.1207/S15327906MBR3701_02.

Almind

T.C.

, & Ingwersen

(1997). Informetric analyses on the world wide web: methodological approaches to ‘webometrics’. J DOC, 53(4), 404-426.

Asekun-Olarinmoye

(2015). The importance of research in university’s webometric ranking: UNIOSUN case study. Res. J. Health Sci, 3(3), 184-195.

Björneborn

, & Ingwersen

(2004). Toward a basic framework for webometrics. J Am Soc Inf Sci Tec, 55(14), 1216-1227.

Elgohary

(2008). Arab universities on the web: a webometric study. Electron Libr, 26(3), 374-386.

Farashi

Bashirian

, & Zareian

(2020). Comparison between top-ranked iranian medical universities and top-ranked world universities based on the website analysis. International Journal of Information Science and Management, 18(2), 203-214.

Fletcher

(2006). The impact of culture on web site content, design, and structure: an international and a multicultural perspective. Journal of Communication Management, 10(3), 259-273. doi: 10.1108/13632540610681158.

Guskov

A.E.

Bykhovtsev

E.S.

, & Kosyakov

D.V.

(2015). Alternative webometrics: study of the traffic of the websites of scientific organizations. Scientific and Technical Information Processing, 42(4), 274-289. doi: 10.3103/S0147688215040139.

10.

Hatch

, & Farhady

(1982). Research design and statistics for applied linguistics.

11.

Jati

, & Dominic

D.D.

(2017). A new approach of indonesian university webometrics ranking using entropy and promethee II. Procedia Comput Sci, 124, 444-451. doi: 10.1016/j.procs.2017.12.176.

12.

Jun

S.-P.

Sung

T.-E.

, & Park

H.-W.

(2017). Forecasting by analogy using the web search traffic. Technol Forecast Soc, 115, 37-51. doi: 10.1016/j.techfore.2016.09.014.

13.

Longqing

, & Qingfeng

(2011). Data sources of webometrics. Paper presented at the 2011 Seventh International Conference on Computational Intelligence and Security.

14.

Nissom

, & Kulathuramaiyer

(2012). The study of Webometrics ranking of World Universities.

15.

Park

H.W.

, & Thelwall

(2006). Web-science communication in the age of globalization. New Media & Society, 8(4), 629-650.

16.

Patidar

, & Vishwakarma

R.G.

(2014, 9–9 Oct. 2014). An implementation of generalized webometrics for all research organizations. Paper presented at the 2014 International Conference on Smart Structures and Systems (ICSSS).

17.

Prasetio

Sharif

O.O.

Perdana

, & Alamanda

D.T.

(2016). Analyzing the impact of traffic source on visit duration. Int J Bus, 21(3), 243.

18.

Sarkar

Pal

, & Kar

(2018). Webometric Mapping of Tourism Sites in India. Webometric Mapping of Tourism Sites in India.

19.

Shadpour

Teimourpour

, & Asadi

(2013). Webometrics-based analysis and ranking of Iranian hospital websites. Int J Hosp Res, 2(2), 77-84.

20.

Singh

Zhao

, & Hu

(2013). Cultural adaptation on the web: a study of american companies’domestic and chinese websites. Journal of Global Information Management, 11(3), 63-80.

21.

Subhani

A.B.

(2014). Intro to WWW Marketing: Abbott Press.

22.

Taheri

Ghazavi

Zahed

Otroj

Mazaheri

, & Soleimanzade-Najafi

N.-S.

(2015). The webometric status of isfahan university of medical sciences, Iran. Acta Inform Med, 23(4), 215-219. doi: 10.5455/aim.2015.23.215-219.

23.

Thelwall

(2008). Bibliometrics to webometrics. J Inf Sci, 34(4), 605-621.

24.

Thelwall

(2009). Introduction to webometrics: quantitative web research for the social sciences. Synthesis Lectures on Information Concepts, Retrieval, and Services, 1(1), 1-116.

25.

van der Ploeg

Austin

P.C.

, & Steyerberg

E.W.

(2014). Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Medical Research Methodology, 14(1), 137. doi: 10.1186/1471-2288-14-137.

26.

Vaughan

, & Yang

(2013). Web traffic and organization performance measures: relationships and data sources examined. J Informetr, 7(3), 699-711.

27.

Weideman

(2013). Comparative analysis of homepage website visibility and academic rankings for UK universities. Information Research: An International Electronic Journal, 18(4), n4.

28.

Wimaladharma

, & Herath

H.M.

(2016). An analysis of significance of Revised Web Impact Factor for ranking the websites of state universities in Sri Lanka. Paper presented at the 2016 IEEE International Conference on Information and Automation for Sustainability (ICIAfS).