An Assessment of the Structural Validity and Measurement Invariance of the Web-Use Skills Scale for Aging Internet Users

Abstract

Research on digital inequality has found that aging adults are often at risk of digital exclusion. Understanding the validity of survey measures assessing Internet skills in this population is critical to providing the high-quality data needed for effective digital inclusion policy interventions. This cross-validation study examines the structural validity and measurement invariance (across age, gender, and education groups) of the Web-Use Skills scale (WUS), which is commonly used as a proxy measure of Internet skills. We tested the 14-item version of the WUS. The scale was translated into the Slovenian language and pretested with older Internet users. Data were collected from two independent samples of Internet users aged 50+ years (N₁ = 259 and N₂ = 256) drawn from an online opt-in panel in Slovenia. The examination of structural validity confirmed that the WUS adequately reflects the one-factor structure of the web-use skills construct, although in a shorter six-item form. Moreover, the analysis confirmed strict measurement invariance between the two samples and, at least, scalar invariance between age, gender, and education groups. The results support the applicability of WUS in cross-group comparisons of Internet skills in the population of aging Internet users and point to several opportunities for future work.

Introduction

Arecent increase in Internet access and use in the general population is mainly due to the growth of users in older age groups,¹ but research indicates that the risk of digital exclusion in the aging population remains high.² Thus, it is vital to understand the validity of measures of Internet skills for this population because digital inclusion policies and interventions should not only be conceptually grounded but also empirically supported by valid, large-scale survey data.

In this sense, the Web-Use Skills scale (WUS)^3–5 seems useful for self-assessment of Internet skills among Internet users in later life because of its wide use in prior research and favorable properties related to scale comprehension and interpretation (e.g., scale brevity, item complexity, response option format).

The WUS is defined as a proxy measure of Internet skills; it was designed to assess a person's self-reported understanding of various Internet-related terms. It assumes that the more knowledgeable a person is about various Internet-related technologies, the better the person's Internet skills will be.³ Importantly, the WUS tackles only one aspect of Internet skills, namely web-use skills, a unidimensional construct that represents a user's ability to locate content on the web effectively and efficiently.³

Hargittai³ first proposed a 7-item scale (derived from a 38-item list) and confirmed its criterion validity by demonstrating the hypothesized correlations between responses to the WUS and an individual's observed online browsing behavior. She⁴ extended the original scale to 27 items in 2009, while Hargittai and Hsieh⁵ showed that 6-, 10-, and 15-item versions have adequate reliability and can be administered to the general population and—in adapted forms—to low-skilled Internet users.

Despite noting that longer scales have higher reliability, they suggested that scholars should always consider the trade-off between scale length and respondent burden when using the WUS. They also argued that due to the fast-changing nature of Internet tools and services, it is important to update such an instrument over time.⁵

In later years, many Internet researchers have followed their suggestion. In fact, the importance of the WUS is illustrated by the large number of articles in Google Scholar citing Hargittai and Hsieh's adaptations of the WUS.⁵ A September 2021 search retrieved 60 peer-reviewed articles that empirically used the WUS, with the number of items ranging from 4 to 38. Ninety-four different items formed the inventories in various combinations.

Most articles included 6 (14 articles) or 27 items (7 articles). In 40 articles, respondents answered on a scale from 1 (do not understand) to 5 (fully understand), but other response scales were also found (1–4, 0–11, and dichotomous). Ten articles applied the WUS to older adults.^6–15

While a handful of studies addressed the measurement properties of the WUS, only 1¹⁰ of the 60 identified articles assessed its structural validity. Structural validity refers to the extent to which a scale's score adequately reflects the dimensionality of the construct being measured.¹⁶ Although Hofer et al.¹⁰ confirmed the one-factor structure of the 6-item WUS among older adults, they did not validate the 10- and 15-item scales developed by Hargittai and Hsieh.⁵

In addition, none of the retrieved articles reported on the assessment of measurement invariance of the WUS in the aging population. Measurement invariance assumes that scale items have the same meaning and are used in the same way by different respondent groups.¹⁷ For instance, invariance is required when researchers want to obtain unbiased comparisons of correlations between Internet skills and covariates (e.g., Internet outcomes) for middle-aged and older adults or when their aim is to compare latent means of Internet skills between age groups.¹⁸

Hence, the original contribution of this study is not only in testing the structural validity of the longer version of the WUS using a cross-validation design (two samples) but also in validation of its measurement invariance on the population of aging adults.

The assessments conducted here are important for two reasons. First, an assessment of structural validity is needed to determine whether the scale items truly reflect the dimensionality of the construct. Second, an assessment of measurement invariance is warranted because the aging population is highly heterogeneous^7,9,19–23 and generally has lower levels of Internet skills.^6,9,24,25

Indeed, there is a large difference between the lifestyles of those aged 50–64 years (largely employed) and those aged 65+ years (largely retired), leading to an increasing risk of social and digital exclusion in the older group.^21,26–28 Differences in Internet skills also relate to gender, with women reporting lower levels of Internet skills,^6,29–32 and education; here, Internet skills are shown to increase with the level of education.^1,30,31

Materials and Methods

Data collection and procedure

As in many recent studies,^10,31,33,34 respondents were recruited from an online opt-in panel, operated by a leading professional market research agency, using a quota sampling design based on region, education, and age and gender combined. The target population was Slovenian Internet users aged 50+ years. According to the rules of the University of Ljubljana and the funding agency, the methods and subjects involved in this study were classified under research categories that are considered exempt from the Ethics Committee's oversight.

For the purpose of cross-validation, in July 2021, two independent samples were obtained as Sample 1 (N₁ = 259, participation rate: 65 percent) and Sample 2 (N₂ = 256, participation rate: 62 percent). Demographic characteristics are shown in Table 1. The results of chi-square tests showed no significant differences in the demographic structure of the two samples.

Table 1.

Characteristics of Data Samples

Variables	Sample 1		Sample 2		χ^2,^a	p
Variables	N	%	N	%	χ^2,^a	p
Gender
Male	123	47.5	123	48.0	0.016	0.899
Female	136	52.5	133	52.0	0.016	0.899
Age (years)^b
50–64	119	45.9	124	48.4	0.321	0.571
65–89	140	54.1	132	51.6	0.321	0.571
Education^c
Primary school (ISCED levels 0–2)	8	3.1	7	2.7	0.238	0.888
Secondary school (ISCED level 3)	139	53.7	132	52.0
Higher education (ISCED levels 5–8)	112	43.2	115	45.3
Employment status
Employed, self-employed, or farmer	85	32.9	88	35.1	0.316	0.382
Retired	159	61.6	151	60.1
Homemaker, unable to work, or unemployed	14	5.5	12	4.8
Region
East Slovenia	116	44.8	117	45.7	0.044	0.835
West Slovenia	143	55.2	139	54.3	0.044	0.835
Frequency of Internet use
Every day	223	86.4	218	85.2	0.172	0.678
Less often than every day	35	13.6	38	14.8	0.172	0.678

The chi-square test included only units with no missing values per variable. The share of missing values per variable was very small (≤2%).

M₁ = 64.3 and M₂ = 63.6.

Educational level reported according to the ISCED 2011.³⁸

ISCED, International Standard Classification of Education.

The Web-Use Skills scale

We tested a 14-item WUS (Table 2), with 10 items taken from Hargittai and Hsieh's scale for the general population⁵ and 4 items added from previous adaptations of the WUS in the target population to improve its content validity.^{4–6,9,35,36} Notably, Hashtag was included because it is commonly used in social media,³⁶ while Operating System, Cookie, and Application embrace the contemporary experience of everyday (mobile) Internet use.⁶

Table 2.

Items Included in Validation of the Web-Use Skills scale

Source	Items
Hargittai and Hsieh⁵	Advanced Search Tagging PDF Computer Virus (modified from Spyware) Wiki JPG Blog (modified from Weblog) Cache Malware Phishing
Nguyen et al.³⁶	Hashtag
Hargittai and Dobransky⁶	Operating System Cookie Application (modified from Widget)

Because the scale had never been used in Slovenia before, we prepared a Slovenian translation of the instrument following the translation, review, adjudication, pretesting, and documentation procedure.³⁷ The translated scale was pretested on three older Internet users with the use of cognitive interviews.

In accordance with pretest results, we modified the translation of several terms and decided to keep the English words alongside Slovenian translations for five items (i.e., Cache, Tagging, Malware, Phishing, and Hashtag) because these original English terms are widely used in Slovenian everyday communication.

Respondents answered the following question: “Listed below are some items related to the Internet. How well would you say you understand each item, on a scale from 1 to 5 (where 1 means ‘I don't understand’ and 5 means ‘I fully understand’)?”

Analytic strategy

The analyses consisted of several sequential steps. First, the structural validity of the one-factor 14-item WUS was assessed through confirmatory factor analysis (CFA) using data from Sample 1. Because of the model's poor fit, we conducted the CFA with 10 items from the study by Hargittai and Hsieh.⁵ Since this model also exhibited a bad fit, we ran an exploratory factor analysis (EFA) on all 14 items. The EFA suggested a meaningful one-factor solution with six items. We cross-validated the six-item one-factor model with CFA using data from Sample 2.

Since the model had an adequate fit, its measurement invariance was assessed between the two samples. After confirming strict invariance between them, we merged both samples and assessed the measurement invariance of the six-item WUS across age (50–64 vs. 65+ years), gender (male vs. female), and education (secondary school or lower vs. higher education) groups. As (at least) scalar invariance was established in all cases, we compared latent mean differences between group members.

For EFA, we used maximum likelihood (ML) estimation with oblimin rotation.¹⁷ The number of factors to retain was determined based on the scree plot and parallel analysis.¹⁷ We retained items that had a high standardized loading on only one factor (>0.50) and communality >0.50.¹⁷ Since we found deviations from normality for some WUS items, we used the robust maximum likelihood (MLR) estimator in the CFA.^39,40 Standardized loadings had to be >0.50.¹⁷

The model fit was assessed with the Satorra–Bentler scaled chi-square test statistic (SBχ²), which was interpreted with caution because of its sensitivity to sample size.¹⁷ We also observed the root mean square error of approximation (RMSEA), where a value <0.08 indicates a reasonable fit and <0.10 indicates acceptable fit.^41,42 The values of the comparative fit index (CFI) >0.95 and standardized root mean residual (SRMR) <0.08 were considered acceptable.^17,41

Composite reliability (CR; also known as McDonald's omega) and Cronbach's alpha (α) were used to assess internal consistency, with values >0.70 defined as acceptable.¹⁷ Convergent validity was evaluated by the average variance extracted (AVE), with AVE >0.50 considered as acceptable.¹⁷

Measurement invariance was assessed with a multigroup CFA using a series of increasingly stringent models in which between-group restrictions constrain different elements of the measurement model.¹⁷ We tested configural (equivalence of model form), metric (equivalence of factor loadings), scalar (equivalence of item intercepts), and strict (equivalence of items' residuals) invariance.⁴³ In such testing, if the fit of the more constrained model is not worse than the fit of the less constrained model, then measurement invariance is supported.

To assess the change in model fit, we conducted the ΔSBχ² difference test. However, because this test is sensitive to sample size and violations of the normality assumption,⁴⁴ we primarily observed the ΔCFI to be <−0.010, supplemented by ΔSRMR <0.030 for metric and <0.010 for scalar and strict invariance.⁴³ All analyses were conducted in R⁴⁵ using the psych⁴⁶ and lavaan⁴⁷ packages.

Results

Sample 1

Before conducting the CFA, we analyzed correlations among the items in Sample 1. They ranged from 0.331 to 0.784 and were all significant at p < 0.05 (Appendix Table A1). Results of the CFA showed that all items had acceptable standardized factor loadings (0.604–0.850). However, the structural validity of the model was not confirmed because of its poor fit [SBχ²₍₇₇₎ = 482.214, RMSEA = 0.155, SRMR = 0.086, and CFI = 0.816].

The model–data fit of the 10-item WUS⁵ was not acceptable either [SBχ²₍₃₅₎ = 237.397, RMSEA = 0.167, SRMR = 0.071, and CFI = 0.841]. Thus, we performed an EFA on the 14 items. The scree plot suggested a one-factor solution, whereas the parallel analysis indicated a two-factor solution.

We tested a two-factor model, and results of the EFA showed that six items (i.e., PDF, Computer Virus, Blog, Application, Cookie, and Operating System) loaded highly on the first factor (standardized loadings between 0.615 and 0.911), whereas the remaining items (i.e., Cache, Tagging, Wiki, Malware, Phishing, and Hashtag) loaded highly on the second factor (standardized loadings between 0.558 and 0.895), except for Advanced Search and JPG, which cross-loaded (Table 3).

Table 3.

Results of Exploratory and Confirmatory Factor Analyses on Samples 1 and 2

Item^c	CFA, Sample 1^a	CFA, Sample 1^a	EFA, Sample 1^a			EFA, Sample 1^a		CFA, Sample 2^b
Item^c	Standardized loadings	Standardized loadings	Communalities	Standardized loadings: Factor 1	Standardized loadings: Factor 2	Communalities	Standardized loadings	Standardized loadings
Advanced Search	0.604	0.600	0.378	0.344	0.340	—	—	—
Cache	0.588	0.699	0.667	−0.120	0.885	—	—	—
Tagging	0.646	0.738	0.734	−0.065	0.895	—	—	—
PDF	0.734	0.703	0.531	0.615	0.166	0.508	0.713	0.597
Computer Virus	0.808	0.727	0.678	0.774	0.077	0.677	0.823	0.775
Wiki	0.638	0.706	0.544	0.104	0.669	—	—	—
JPG	0.713	0.750	0.517	0.378	0.422	—	—	—
Blog	0.850	0.766	0.742	0.782	0.120	0.735	0.857	0.681
Malware	0.752	0.808	0.633	0.292	0.582	—	—	—
Phishing	0.750	0.797	0.620	0.310	0.558	—	—	—
Application	0.819	—	0.766	0.898	−0.037	0.773	0.879	0.902
Cookie	0.765	—	0.686	0.873	−0.076	0.684	0.827	0.815
Operating System	0.833	—	0.790	0.911	−0.037	0.779	0.883	0.833
Hashtag	0.687	—	0.540	0.248	0.556	—	—	—

N₁ = 259.

N₂ = 256.

Items in bold are included in the proposed one-factor 6-item Web-Use Skills scale.

CFA, confirmatory factor analysis; EFA, exploratory factor analysis.

We found that items that load on the first factor relate to general computer and Internet use, while items loading highly on the second factor relate to specific and varied Internet domains.

In the next analyses, we focused on six items of the first factor for three reasons. First, web-use skills have been conceptually and empirically defined as a unidimensional construct.^3–5 Second, it has been suggested that context-specific items referring to specific digital platforms or online activities should be avoided when measuring Internet skills. In fact, Hargittai and Hsieh⁵ note that some individuals might be more familiar with certain terms not because they possess higher skills, but because these terms are related to their specific online interests or activities. In addition, van Deursen et al.³¹ underscored that to ensure applicability of a set of items for a longer period of time, items that do not depend on currently popular Internet activities or platforms should be used in Internet skills survey inventories.

Third, the meaning of the second factor could not be unambiguously interpreted since the items with the highest standardized loading on this factor (>0.85; Cache and Tagging) have little common content. DeVellis⁴⁸ recommends that such factors should not be considered as indicators of a latent variable. When we ran the EFA with only six items of the first factor, the results were satisfactory. The items had standardized loadings between 0.713 and 0.883, with all items having communalities >0.50 (Table 3) and α = 0.923.

Sample 2

For Sample 2, we first analyzed correlations between the six items. They ranged from 0.409 to 0.753 and were all significant at p < 0.05 (Appendix Table A2). Next, the cross-validation based on the CFA showed that the six-item one-factor structure was acceptable. The standardized factor loadings ranged from 0.597 to 0.902, with SBχ²₍₉₎ = 20.402, RMSEA = 0.085, SRMR = 0.028, and CFI = 0.981 (Table 3). Internal consistency and convergent validity were also adequate, with the following values: AVE = 0.599, CR = 0.898, and α = 0.878.

Measurement invariance

To assess the measurement invariance of the six-item WUS between samples 1 and 2, we first fitted the model into each sample. The results indicated acceptable fit for both samples [Sample 1: SBχ²₍₉₎ = 15.815, RMSEA = 0.071, SRMR = 0.021, and CFI = 0.990; and Sample 2: SBχ²₍₉₎ = 20.402, RMSEA = 0.085, SRMR = 0.028, and CFI = 0.981]. Furthermore, we tested for configural invariance, and the model exhibited an acceptable fit (Table 4).

Table 4.

Results of Measurement Invariance Testing of the Six-Item Web-Use Skills Scale in Samples 1 and 2

Model	Goodness of fit				Model comparison
Model	SBχ²_(df)	RMSEA (90% CI)	SRMR	CFI	ΔSBχ²_(df)	p	ΔCFI
Configural model	35.861₍₁₈₎	0.078 (0.040–0.116)	0.025	0.986	—	—	—
Metric model	49.429₍₂₃₎	0.083 (0.051–0.115)	0.056	0.980	14.161₍₅₎	0.015	−0.006
Scalar model	60.691₍₂₈₎	0.081 (0.053–0.108)	0.058	0.977	11.518₍₅₎	0.042	−0.003
Strict model	74.419₍₃₄₎	0.085 (0.059–0.111)	0.063	0.969	13.533₍₆₎	0.035	−0.008

Note: N_total = 515 (N₁ = 259 and N₂ = 256).

CFI, comparative fit index; CI, confidence interval; RMSEA, root mean square error of approximation; SBχ², Satorra–Bentler scaled chi-square test statistic; SRMR, standardized root mean residual.

We then introduced constraints across samples to test for metric, scalar, and strict invariance. At each step, we evaluated the decrease in fit. While the ΔSBχ² was significant at p < 0.05 in all cases, the changes in CFI and SRMR were small enough to provide support for invariance: ΔCFI was <−0.01 for all models, and SRMR increased by 0.031 in the case of the metric model, which is just at the threshold, whereas the increase for the scalar and strict models was <0.01. Thus, we concluded that the six-item WUS demonstrated full, strict measurement invariance between samples.

After confirming measurement invariance across the two samples, we could merge them to have a larger sample for analysis of measurement invariance across selected sociodemographic characteristics. Applying the same procedure to the merged dataset (N = 515), the results showed full scalar measurement invariance across gender (Table 5) and age (Table 6), as well as full strict invariance across educational groups (Table 7).

Table 5.

Results of Measurement Invariance Testing of the Six-Item Web-Use Skills Scale by Gender (Male vs. Female)

Model	Goodness of fit				Model comparison
Model	SBχ²_(df)	RMSEA (90% CI)	SRMR	CFI	ΔSBχ²_(df)	p	ΔCFI
Configural model	32.307₍₁₈₎	0.072 (0.028–0.111)	0.023	0.989	—	—	—
Metric model	36.529₍₂₃₎	0.060 (0.015–0.095)	0.032	0.990	3.231₍₅₎	0.664	0.001
Scalar model	56.050₍₂₈₎	0.076 (0.046–0.104)	0.041	0.981	25.378₍₅₎	0.000	−0.009
Strict model	89.600₍₃₄₎	0.102 (0.077–0.128)	0.058	0.957	26.406₍₆₎	0.000	−0.024

Note: N_total = 515 (N_male = 246 and N_female = 269).

Table 6.

Results of Measurement Invariance Testing of the Six-Item Web-Use Skills Scale by Age Group (50–64 and 65+ years)

Model	Goodness of fit				Model comparison
Model	SBχ²_(df)	RMSEA (90% CI)	SRMR	CFI	ΔSBχ²_(df)	p	ΔCFI
Configural model	37.655₍₁₈₎	0.085 (0.046–0.123)	0.025	0.984	—	—	—
Metric model	46.288₍₂₃₎	0.079 (0.045–0.112)	0.042	0.982	8.003₍₅₎	0.156	−0.002
Scalar model	62.937₍₂₈₎	0.085 (0.057–0.113)	0.052	0.975	19.832₍₅₎	0.001	−0.007
Strict model	100.303₍₃₄₎	0.112 (0.087–0.137)	0.061	0.947	29.538₍₆₎	0.000	−0.028

Note: N_total = 515 (N_50–64 = 243 and N₆₅₊ = 272).

Table 7.

Results of Measurement Invariance Testing of the Six-Item Web-Use Skills Scale by Education Level Groups (Secondary School or Lower vs. Higher Education)

Model	Goodness of fit				Model comparison
Model	SBχ²_(df)	RMSEA (90% CI)	SRMR	CFI	ΔSBχ²_(df)	p	ΔCFI
Configural model	38.382₍₁₈₎	0.084 (0.047–0.120)	0.025	0.984	—	—	—
Metric model	44.230₍₂₃₎	0.074 (0.040–0.107)	0.038	0.985	4.938₍₅₎	0.424	0.001
Scalar model	48.678₍₂₈₎	0.064 (0.032–0.094)	0.038	0.986	2.194₍₅₎	0.822	0.001
Strict model	54.016₍₃₄₎	0.060 (0.027–0.090)	0.041	0.985	6.992₍₆₎	0.322	−0.001

Note: N_total = 513 (N_upToSecEdu = 286 and N_highEdu = 227).

As scalar invariance was confirmed in all cases, we compared the WUS latent mean values across age, gender, and educational groups. We expected that females, older individuals, and less educated individuals have lower Internet skills than males, younger individuals, and more educated individuals.^32,49 If these differences are confirmed, then this provides additional evidence of scale validity (i.e., known-group validity).⁵⁰

We constrained the latent means to be zero for females and older (65+ years) and less educated individuals (secondary school or lower), while freely estimating them for males, younger individuals (50–64 years), and individuals with higher education. The analysis showed that the WUS latent mean values were significantly higher for males and younger individuals (p ≤ 0.010 in both cases).

However, the difference was not significant in the case of education (p = 0.697), although more educated individuals had slightly higher WUS scores.

Discussion

This is the first study to comprehensively investigate the structural validity and measurement invariance of the WUS in a population of aging Internet users. Results show that the WUS—although in a short six-item form—is a valid and reliable proxy measure of the ability of Internet users, aged 50+ years, to find information efficiently and effectively on the web. Yet, while our findings provide convincing evidence for the future use of the six-item WUS in this population group, they also raise several conceptual and methodological issues for discussion.

First, we could not confirm the one-factor structure of either the 14-item or the original 10-item WUS.⁵ On the one hand, this could be due to rapid changes in digital technologies, in that some Internet-related terms from the original scale could be outdated.⁴⁹ On the other hand, the 14- and 10-item scales included terms related to various computer and Internet domains,⁵¹ but there is no guarantee that a person knowledgeable in one domain will be knowledgeable in another.⁵

Hence, our finding that only the six-item scale demonstrates adequate structural validity is consistent with Hargittai and Hsieh's definition of Internet skills and their goal of proposing scales that have fewer components, but still optimally capture differences among respondents' web-use skills.^3,5 Moreover, this is consistent with current research²⁰ showing that Internet proficiency decreases with age, which might explain why the general nature of the six-item WUS fits well in this population group.

In fact, Hargittai and Hsieh⁵ proposed (shorter) scales with more general items for low-skilled groups of Internet users. However, our EFA results highlight the possibility that the concept of web-use skills could be extended to capture not only general but also more conceptually distinct and domain-specific dimensions of Internet use in later life.

Second, compared with the original 10-item WUS,⁵ our proposed 6-item scale includes 3 of the originally proposed items (PDF, Computer Virus, and Blog), whereas 3 items come from adapted versions of the WUS (Table 2). This supports our decision to add items to the scale, confirming prior work arguing that Hargittai and Hsieh's inventory needs to be adapted after almost 10 years, in the sense of what comprises general Internet skills.^4,5,35,52,53

The newly added items relate to basic understanding of computer software (i.e., Operating System), important contemporary web privacy issues (i.e., Cookie), and widespread use of mobile Internet (i.e., Application). They are also general enough not to be platform specific, which is important because Internet skills scales should be cognizant of large-scale changes in Internet technology while avoiding context-specific items related to short-lived services and/or transitory online activities.³¹

In this sense, the proposed set of items provides a good balance between generalizability and temporality, which are needed in general Internet skills scales.⁵

Third, the concise scale used here is particularly welcome in general social survey research. Measures of Internet skills are included as covariates in almost every study of digital engagement of any population.⁴⁹ A shorter inventory is also appropriate for use in large-scale studies^5,49 because it reduces the cognitive load, which is critical in research with aging adults. It may also help reduce measurement bias related to a high response burden.⁵ This is likely to be the case for self-administered questionnaires where respondents have very limited or no external memory support.⁵⁴

Fourth, given the heterogeneity of the aging population,^{9,19–23,27,28} assessment of measurement invariance is essential. Confirmation of (at least) scalar measurement invariance across age, gender, and educational groups suggests that the six-item WUS measures the same concept in the same way within these groups and that users with the same level of the latent trait have the same scores on the manifest (observed) variables.

These results also imply that both comparisons of structural relationships between the WUS and other constructs and comparisons of the latent means across age, gender, and education groups are valid and reliable only when the WUS contains items that are only related to proficiency in general Internet use. In fact, the WUS latent mean comparisons confirmed the assumption that older users (65+ years) and females have lower WUS scores.

The nonsignificant difference with respect to education could be a direct consequence of highly understood items (i.e., items with high averages) and might suggest that both educational groups share similar experiences with the fairly general Internet use related to the terms in the six-item list.

Limitations and future work

While a quota sample can be considered representative of aging Internet users in Slovenia in terms of sociodemographic characteristics, future research is invited to use probability-based sampling designs. This could further eliminate potential biases related to the fact that participants in opt-in Internet panels may have higher Internet proficiency when compared with the general population.⁵⁵ Accordingly, their mean scores on the WUS may be higher than those of the general population of aging Internet users.

In addition, further examination of the multidimensionality of the WUS is warranted. Future research could use more or even all of the original WUS items⁵ to examine to what extent they fit multiple factor or bifactor models.⁵⁶ As a next step, research focusing on measurement invariance of the WUS could highlight other factors of digital inequality, such as occupation and income,² and investigate whether the WUS is also invariant in cross-country settings.³⁰ Moreover, it may be interesting to test the convergent validity of the WUS with other survey measures of Internet skills. In particular, this scale could be compared with those based on multidimensional measurement models to assess which theoretical dimensions of Internet skills besides web-use skills are captured by the WUS.

Footnotes

Author Disclosure Statement

No competing interests exist.

Funding Information

This research received public financial support from research grants (nos. L5-9337, Z5-8234, J5-2558, and P5-0399) administered through the Slovenian Research Agency and a Young Researcher Fellowship, cofounded by the Slovenian Research Agency from the national budget, awarded to the fourth author.

Appendix

Appendix Table A2.

Descriptive Statistics and Correlations of Web-Use Skills Scale Items in Sample 2

Item	M	SD	1	2	3	4	5	6
PDF (1)	3.91	1.311	—
Computer Virus (2)	4.33	0.808	0.497	—
Blog (3)	4.11	0.966	0.447	0.538	—
Application (4)	4.30	0.831	0.506	0.686	0.638	—
Cookie (5)	4.33	0.727	0.409	0.628	0.542	0.753	—
Operating System (6)	4.18	1.018	0.579	0.656	0.514	0.744	0.680	—

Note: N₂ = 256. All correlations are significant at p < 0.05.

References

Blank

, Dutton

, Lefkowitz

. Oxis 2019: Digital divides in Britain are narrowing but deepening. SSRN Electron J, 2020; doi: 10.2139/ssrn.3522083.

Van Deursen

AJAM

, Helsper

, Eynon

, et al. The compoundness and sequentiality of digital inequality. Int J Commun, 2017; 11:452–473. Available from: https://ijoc.org/index.php/ijoc/article/view/5739 [Last accessed: September 17, 2020].

Hargittai

Survey measures of web-oriented digital literacy. Soc Sci Comput Rev, 2005; 23(3):371–379; doi: 10.1177/0894439305275911.

Hargittai

An update on survey measures of web-oriented digital literacy. Soc Sci Comput Rev, 2009; 27(1):130–137; doi: 10.1177/0894439308318213.

Hargittai

, Hsieh

. Succinct survey measures of web-use skills. Soc Sci Comput Rev, 2012; 30(1):95–107; doi: 10.1177/0894439310397146.

Hargittai

, Dobransky

. Old dogs, new clicks: Digital inequality in skills and uses among older adults. Can J Commun, 2017; 42(2):195–212; doi: 10.22230/cjc.2017v42n2a3176.

Nguyen

, Hunsaker

, Hargittai

. Older adults' online social engagement and social capital: The moderating role of internet skills. Inf Commun Soc, 2020; 25(7):942–958; doi: 10.1080/1369118X.2020.1804980.

Hunsaker

, Hargittai

, Piper

. Online social connectedness and anxiety among older adults. Int J Commun, 2020; 14(29):697–725. Available from: https://ijoc.org/index.php/ijoc/article/view/11841 [Last accessed: September 20, 2021].

Hargittai

, Piper

, Morris

. From internet access to internet skills: Digital inequality among older adults. Univers Access Inf Soc, 2019; 18(4):881–890; doi: 10.1007/s10209-018-0617-5.

10.

Hofer

, Hargittai

, Buchi

, et al. Older adults' online information seeking and subjective well-being: The moderating role of internet skills. Int J Commun, 2019; 13:4426–4443; doi: 10.5167/UZH-175837.

11.

Munger

, Gopal

, Nagler

, et al. Accessibility and generalizability: Are social media effects moderated by age or digital literacy?. Res Polit, 2021; 8(2):205316802110169; doi: 10.1177/20531680211016968.

12.

, Ellison

, Lampe

. Facebook use and its role in shaping access to social benefits among older adults. J Broadcast Electron Media, 2018; 62(1):71–90; doi: 10.1080/08838151.2017.1402905.

13.

Schreurs

, Quan-Haase

, Martin

. Problematizing the digital literacy paradox in the context of older adults' ICT use: Aging, media discourse, and self-determination. Can J Commun, 2017; 42(2):359–377; doi: 10.22230/cjc.2017v42n2a3130.

14.

Hunsaker

, Nguyen

, Fuchs

, et al. “He explained it to me and I also did it myself”: How older adults get support with their technology uses. Socius, 2019; 5; doi: 10.1177/2378023119887866.

15.

Karaoglu

, Hargittai

, Hunsaker

, et al. Changing technologies, changing lives: Older adults' perspectives on the benefits of using new technologies. Int J Commun, 2021; 15(21):3887–3907. Available from: https://ijoc.org/index.php/ijoc/article/view/16047 [Last accessed: September 20, 2021].

16.

Mokkink

, Terwee

, Patrick

, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol, 2010; 63(7):737–745; doi: 10.1016/j.jclinepi.2010.02.006.

17.

Hair

JFJ

, Black

, Babin

, et al. Multivariate Data Analysis. Cengage Learning EMEA: Hampshire, UK; 2019.

18.

Grošelj

, van Deursen

AJM

, Dolničar

, et al. Measuring internet skills in a general population: A large-scale validation of the short Internet Skills Scale in Slovenia. Inf Soc, 2020; 37(2):63–81; doi: 10.13039/501100004329.

19.

Czaja

, Sharit

, Hernandez

, et al. Variability among older adults in internet health information-seeking performance. Gerontechnology, 2010; 9(1):46–55; doi: 10.4017/gt.2010.09.01.004.00.

20.

Quan-Haase

, Williams

, Kicevski

, et al. Dividing the grey divide: Deconstructing myths about older adults' online activities, skills, and attitudes. Am Behav Sci, 2018; 62(9):1207–1228; doi: 10.1177/0002764218777572.

21.

Vuori

, Holmlund-Rytkönen

. 55+ people as internet users. Mark Intell Plan, 2005; 23(1):58–76; doi: 10.1108/02634500510577474.

22.

Van Boekel

, Peek

, Luijkx

. Diversity in older adults' use of the internet: Identifying subgroups through latent class analysis. J Med Internet Res, 2017; 19(5):e180; doi: 10.2196/jmir.6853.

23.

Vulpe

, Crăciun

. Silver surfers from a European perspective: Technology communication usage among European seniors. Eur J Ageing, 2020; 17(1):125–134; doi: 10.1007/s10433-019-00520-2.

24.

Blank

, Dutton

, Lefkowitz

. Perceived threats to privacy online: The internet in Britain, the Oxford Internet Survey, 2019. SSRN Electron J, 2020; doi: 10.2139/ssrn.3522106.

25.

Friemel

TN.

The digital divide has grown old: Determinants of a digital divide among seniors. New Media Soc, 2016; 18(2):313–331; doi: 10.1177/1461444814538648.

26.

Lee

, Chen

, Hewitt

. Age differences in constraints encountered by seniors in their use of computers and the internet. Comput Hum Behav, 2011; 27(3):1231–1237; doi: 10.1016/j.chb.2011.01.003.

27.

Tsatsou

Aging: The two faces of Janus in digital inclusion?. Int J Commun, 2021; 15:1309–1329. Available from: https://ijoc.org/index.php/ijoc/article/view/14220 [Last accessed: November 2, 2021].

28.

Kania-Lundholm

, Torres

. The divide within: Older active ICT users position themselves against different ‘others’. J Aging Stud, 2015; 35:26–36; doi: 10.1016/j.jaging.2015.07.008.

29.

Helsper

EJ.

Gendered internet use across generations and life stages. Commun Res, 2010; 37(3):352–374; doi: 10.1177/0093650209356439.

30.

Festic

, Büchi

, Latzer

. It's still a thing: Digital inequalities and their evolution in the information society. Stud Commun Media, 2021; 10(3):326–361; doi: 10.5771/2192-4007-2021-3-326.

31.

Van Deursen

AJAM

, Helsper

, Eynon

. Development and validation of the Internet Skills Scale (ISS). Inf Commun Soc, 2016; 19(6):804–823; doi: 10.1080/1369118X.2015.1078834.

32.

Van Dijk

JAGM

, van Deursen

AJAM

. Digital Skills: Unlocking the Information Society. Palgrave Macmillan: New York, NY; 2014.

33.

Surian

, Sciandra

. Digital divide: Addressing Internet skills. Educational implications in the validation of a scale. Res Learn Technol, 2019; 27: doi: 10.25304/rlt.v27.2155.

34.

Van Deursen

AJAM

, Helsper

, Eynon

Measuring Digital Skills. From Digital Skills to Tangible Outcomes Project Report. Oxford Internet Institute: Oxford, UK; 2014.

35.

Yen

, Chen

, Cheng

, et al. Joint media engagement between parents and preschoolers in the U.S., China, and Taiwan. Proc ACM Hum Comput Interact, 2018; 2(CSCW):1–19; doi: 10.1145/3274461.

36.

Nguyen

, Gruber

, Marler

, et al. Staying connected while physically apart: Digital communication when face-to-face interactions are limited. New Media Soc, 2022; 24(9):2046–2067; doi: 10.1177/1461444820985442.

37.

Harkness

, de Vijver

, Mohler

Cross-Cultural Survey Methods. Wiley-Interscience: Hoboken, NJ; 2003.

38.

UNESCO. International Standard Classification of Education (ISCED) 2011. UNESCO Institute for Statistics: Montreal, Canada; 2012. doi: 10.15220/978-92-9189-123-8-en.

39.

Mardia

KV.

Test of univariate and multivariate normality. In: Cognitive Computing: Theory and Applications. (Gudivada VV, Raghavan VV, Govindaraju V, Rao CR, eds.) Elsevier: Amsterdam, Netherlands; 2016; pp. 279–320.

40.

Beaujean

AA.

Latent Variable Modeling Using R: A Step-by-Step Guide. Routledge: New York, NY; 2014. doi: 10.4324/9781315869780.

41.

Browne

, Cudeck

. Alternative ways of assessing model fit. Sociol Methods Res, 1992; 21(2):230–258; doi: 10.1177/0049124192021002005.

42.

Kline

RB.

Principles and Practice of Structural Equation Modeling (4th ed.). The Guilford Press; New York, NY; 2016.

43.

Putnick

, Bornstein

. Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev Rev, 2016; 41:71–90; doi: 10.1016/j.dr.2016.06.004.

44.

Chen

FF.

Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equ Modeling, 2007; 14(3):464–504; doi: 10.1080/10705510701301834.

45.

R Core Team. R: A Language and Environment for Statistical Computing [version 4.0.3]. R Foundation for Statistical Computing: Vienna, Austria; 2020.

46.

Revelle

Psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University: Evanston, IL; 2021.

47.

Rosseel

Lavaan: An R package for structural equation modeling. J Stat Softw, 2012; 48(2):1–36; doi: 10.18637/jss.v048.i02.

48.

DeVellis

RF.

Scale Development: Theory and Applications. Sage Publications: Los Angeles, CA; 2016.

49.

Litt

Measuring users' internet skills: A review of past assessments and a look toward the future. New Media Soc, 2013; 15(4):612–630; doi: 10.1177/1461444813475424.

50.

De Gruijter

DNM

, van der Kamp

LJT

. Statistical Test Theory for the Behavioral Sciences. Taylor & Francis Group: Boca Raton, FL; 2008.

51.

Leukel

, Schehl

, Sugumaran

. Digital inequality among older adults: Explaining differences in the breadth of internet use. Inform Commun Soc, 2021; doi: 10.1080/1369118X.2021.1942951.

52.

Hargittai

, Litt

. New strategies for employment? Internet skills and online privacy practices during people's job search. IEEE Secur Priv, 2013; 11(3):38–45; doi: 10.1109/MSP.2013.64.

53.

Liu

, Xu

. Adoption of ride-sharing apps by Chinese taxi drivers and its implication for the equality and wellbeing in the sharing economy. Chin J Commun, 2019; 12(1):7–24; doi: 10.1080/17544750.2018.1524392.

54.

Knäuper

, Carrière

, Chamandy

, et al. How aging affects self-reports. Eur J Ageing, 2016; 13(2):185–193; doi: 10.1007/s10433-016-0369-0.

55.

Callegaro

, Villar

, Yeager

, et al. A critical review of studies investigating the quality of data obtained with online panels based on probability and nonprobability samples. In: Online Panel Research: A Data Quality Perspective, vol. 2. (Callegaro M, Baker R, Bethlehem J, Göritz AS, Krosnick JA, Lavrakas PJ, eds.) John Wiley: New York, NY; 2014; pp. 23–55.

56.

Chen

, Stephen

, Karen

. A comparison of bifactor and second-order models of quality of life. Multivar Behav Res, 2006; 41(2):189–225; doi: 10.1207/s15327906mbr4102_5.