Abstract
Job search quality is important for unemployed individuals pursuing reemployment. To comprehensively measure job search quality, we develop and test a 20-item Job Search Quality Scale (JSQS), using four samples of unemployed individuals (pilot sample, N=218; exploration sample, N=3372; confirmation sample, N=3372; and replication sample, N=434). Results show a four-dimensional structure, composed of (a) goal establishment and planning, (b) preparation and alignment, (c) emotion regulation and persistence, and (d) learning and improvement. Substantial evidence was found for its reliability, convergent and discriminant validity. Building job search quality’s nomological net, conscientiousness, learning goal orientation, self-efficacy, employment commitment, autonomous job search motivation, and social support emerged as positive correlates. Supporting its criterion-related validity, the JSQS predicted key job search and employment outcomes. Moreover, usefulness analyses supported its incremental validity beyond extant job search measures. Our findings have important implications for studying and measuring job search quality in future research and career counseling practice.
The COVID-19 pandemic has caused unprecedented job losses and an immense rise in unemployment worldwide (OECD, 2020). Job loss and subsequent unemployment have severe negative consequences for the well-being of individuals themselves, their families, and broader society (Klehe & Van Hooft, 2018). Getting back to work is important because reemployment positively affects people’s mental and physical health (McKee-Ryan et al., 2005). Finding reemployment, however, often is not easy. Indeed, there is a myriad of self-help books and websites with tips and advice on how to conduct a job search effectively, but the evidence base for such recommendations is often unclear.
Job search has been theorized as a multidimensional construct referring to a goal-directed, motivational, and self-regulatory process involving cognition, affect, and behavior aimed at preparing for, identifying, and/or applying for job opportunities (Kanfer et al., 2001; Van Hooft et al., 2021). Consistent with a motivational interpretation of job search, prior research has mostly focused on the time and effort that people devote to job search activities (Boswell et al., 2012; Van Hoye, 2014; Wanberg, 2012). Although theory suggests that motivational intensity facilitates reemployment and meta-analyses found that measures of job search effort and intensity positively predict finding a job, effect sizes are typically small (Kanfer et al., 2001; Liu et al., 2014; Van Hooft et al., 2021). Furthermore, meta-analytical evidence demonstrates that the intensity of people’s job search is unrelated to employment quality in the newfound job (Van Hooft et al., 2021). These findings can partially be explained by the fact that finding (high-quality) employment depends on many factors beyond the job seeker’s control (e.g., labor demand and hiring discrimination) or outside the job search space (e.g., qualifications and social capital). Additionally, current job search measures covering motivational intensity are to some degree deficient. That is, considering the definition of job search as a goal-directed, motivational, and self-regulatory process, these measures lack content on goal-directedness and self-regulation of cognition, affect, and behavior.
This has led to calls for research on other job search components beyond effort and intensity. Both early and more recent work has theorized or examined concepts that tap into goal-directedness or self-regulatory aspects of job search. For example, Stumpf et al. (1983) and Stevens and Beach (1996) described various job search strategies, suggesting that a systematic strategy is more effective than a fortuitous/haphazard strategy. Others noted the importance of social and background preparation (Caldwell & Burger, 1998), assertiveness, proactivity, and feedback-seeking (Schmit et al., 1993), and learning and improvement (Barber et al., 1994) for effective job search. Various scholars introduced goal-related and self-regulatory aspects of job search such as job search clarity, emotion and motivation control, and metacognitive activities (Turban et al., 2009; Wanberg et al., 1999, 2002). Lastly, scholars coined the notion of job search quality (e.g., Saks, 2005; Stremersch & Van Hoye, 2020; Van Hooft & Noordzij, 2009; Vuori & Vinokur, 2005; Wanberg et al., 2002).
Unifying these notions, Van Hooft et al. (2013) proposed a self-regulatory process model describing the regulation of cognitions, affect, and behavior during job search. They drew on Total Quality Management (Dean & Bowen, 1994) and self-regulation theories to describe job search quality as a self-regulation process that includes goal establishment (i.e., selecting clear goals, goal commitment, and developing subgoals), planning the goal pursuit (i.e., strategy selection, intention formation, prioritizing, and preparation), goal striving (i.e., self-control to initiate and maintain the goal pursuit), and reflection (i.e., analysis of performed activities and learning). They further reviewed previous findings on links between elements of job search process quality and employment outcomes. Integrating theory and previous findings, they propose that a high-quality job search process enhances reemployment probabilities because it evokes a learning process, leading job seekers to learn what employers want such that they can prepare and align their behaviors (e.g., networking and interview behavior) and products (e.g., application letters and resumes) to the employers’ expectations.
Research is starting to demonstrate the importance of job search quality; however, this research has been hindered by two issues. First, the limited research on job search quality is scattered, using different measures representing different parts of the construct (e.g., Koen et al., 2016; Song et al., 2020; Turban et al., 2009; Wanberg et al., 2002; 2020). Second, there is limited understanding of which dimensions underly the job search quality construct, and to what extent each of these dimensions are important for finding a job. Although previous work has proposed directions regarding the structure and dimensionality of job search quality (e.g., Van Hooft et al., 2013), these ideas have yet to be empirically tested. Job search quality is becoming more crucial given the increasing complexity of job search as well as the ambiguity and heterogeneity of the labor market in which job search occurs. This raises important questions as to what job search quality exactly entails, its antecedents, and its relation with employment success outcomes. Our ability to answer such questions is severely limited by the absence of a reliable and valid job search quality measure. Theoretically, without such a measure we cannot examine the distinctiveness of job search quality from job search effort and intensity, and we lack the ability to build its nomological network and test proposed relationships and models. Practically, conceptual clarity and adequate measurement of job search quality would offer key insights for how to advance job search most effectively, benefiting both job seekers and career counselors. Also, a validated job search quality scale enables counseling organizations to diagnose and target job seekers in need of help, and to evaluate the effectiveness of job search training programs (Liu et al., 2014).
The present study was designed to fill these gaps by developing a comprehensive job search quality measure, and assessing its dimensionality, its position in the nomological net of job search constructs, and its value in predicting job search and employment outcomes among unemployed job seekers. Drawing upon extant job search theory and frameworks we gain a thorough understanding of the structure of the job search quality construct, and we test theory on antecedents, correlates, and outcomes of job search quality. To develop a reliable and valid measure of job search quality, we conducted a series of studies following procedures and recommendations outlined in the scale construction literature (Hinkin, 1998; Worthington & Whittaker, 2006; Wright et al., 2017). To ensure that the psychometric properties of our job search quality measure hold across samples and contexts, we conducted pilot studies and administered the items in two subsequent independent samples of unemployed job seekers (Wright et al., 2017).
This paper describes the procedures and results of these studies in five phases. Phase 1 details the item generation process and steps taken to warrant the content validity of the initial item pool. Phase 2 describes exploratory analyses using data from a qualitative pilot (N = 8), a quantitative pilot (Sample 1, N = 218), and an exploration sample (Sample 2a, N = 3372) to reduce the item pool and establish the factor structure, resulting in a 20-item four-dimensional job search quality scale (JSQS). In Phase 3, we report confirmatory analyses to test the factor structure and reliability in a confirmation sample (Sample 2b, N = 3372), and replicate this in an independent sample (Sample 3, N = 434). Phase 4 presents the examination of the convergent, divergent, and concurrent validity of the JSQS (using Samples 2b and 3), building the nomological network of job search quality. In Phase 5, we extend the nomological network by examining the criterion-related validity of the JSQS for job search outcomes (i.e., number of interviews and fit with interviewed jobs) and employment outcomes (i.e., employment status, duration till reemployment, fit with the new job, and job satisfaction), using Sample 2b and Sample 3 follow-up data (n = 322). In addition, we conducted usefulness analyses to test the incremental validity of the JSQS.
Samples 1 and 2 were collected as part of a larger project to evaluate the effectiveness of a training program of the Dutch national employee insurance agency for older unemployed job seekers (aged ≥50 years). Older job seekers are especially likely to benefit from understanding and advancing job search quality, given their precarious labor market position (Wanberg et al., 2016). To test the generalizability of our findings, we collected additional data among English-speaking unemployed job seekers aged 18–65 years (Sample 3).
Phase 1: Item Generation
The Phase 1 purpose was to develop an initial item pool covering the content domain of job search quality. We combined a deductive and inductive approach for item development (Hinkin, 1998). Given the research available, we started deductively. Specifically, we first generated a definition of job search quality to delineate its content domain. Based on previous theory, we conceptualize job search quality as part of the multidimensional job search construct that can be described as “a pattern of thinking, affect, and behavior” (Kanfer et al., 2001 p. 838). The job search quality component of job search concerns the regulation and optimization of the job search process, which involves self-regulation (Van Hooft et al., 2013; Van Hoye, 2014). Combining these perspectives we define job search quality as a pattern of thinking, affect, and behavior aimed at regulating and optimizing the job search process, which manifests itself in goal development, planning, preparation of goal-directed activities, goal maintenance, and analysis and improvement of goal-directed activities. Rather than taking an external perspective to evaluate job search quality (which would require assessment of job search products [e.g., application letters and interview performance] by recruiting organizations), with this definition we take an internal perspective to evaluate job search quality by focusing on the quality of the job search process (cf. Van Hooft et al., 2013).
Second, guided by this definition, we culled the literature to collect existing items and scales that measure aspects of job search quality or resemble aspects of job search quality. This resulted in an extensive item base of items on goal commitment (Klein et al., 2001), job search clarity (Wanberg et al., 2002), prioritizing (Wanberg et al., 2020), planning and deadline setting (Saks & Ashforth, 2002; Van Hooft et al., 2005), strategy development (Crossley & Highhouse, 2005), preparation (Caldwell & Burger, 1998), emotion and motivation control (Wanberg et al., 1999), emotion regulation (Gross & John, 2003), self-control (De Boer et al., 2011), feedback-seeking (Ashford, 1986), strategy awareness and learning from failure (Noordzij et al., 2013), and metacognition (Turban et al., 2009). We reviewed items whether they fit the theoretical content domain of job search quality, and if necessary rephrased items to fit the job search context. We wrote new items for aspects of job search quality that were inadequately covered (e.g., proximal goals, systematic strategy use, deadline setting, emotion control and persistence, and reflection, learning, and improvement).
Next, we used an inductive approach to supplement the deductive approach. To gather examples of high and low job search quality, we (a) screened the training manual of a Dutch reemployment training for unemployed job seekers, (b) conducted interviews with three subject matter experts (SMEs), who worked as employment counselor and trainer for the Dutch national employee insurance agency, and (c) observed two training meetings of unemployed job seekers. This led to an expansion of the item pool with items on adequate self-presentation and careful preparation of job search activities (e.g., preparing examples for one’s strengths; using one’s network to gather information on jobs/organization before applying; alignment of application efforts to the job requirements and employer demands).
Based on the deductive and inductive approach we generated 113 initial items, covering the aspects of job search quality as delineated in our definition. Items were written using basic guidelines for item formulation (i.e., clear wording; consistent perspective; both indicative and counter-indicative items; avoiding double-barreled statements, intensifiers in the items, and statements most will (dis-)agree with; Hinkin, 1998; Hofstee, 1999). Similar to standard job search intensity and effort measures (Blau, 1993, 1994), items were written in past tense and preceded by a heading referring to a specific time interval in the past. The time interval should be carefully chosen depending on the sample, study design, and context, balancing the needs for generating sufficient response variance while minimizing retrospective bias (i.e., three months in Samples 1 and 2; one month in Sample 3). We used a 5-point Likert response scale (cf. Hinkin, 1998), ranging from never to very often for items asking frequencies, and from not at all applicable to me to fully applicable to me for items describing statements.
Guided by our multidimensional job search quality definition and the literature review, the 113 items were generated to cover several aspects of job search quality. For each aspect, we developed multiple (similar) items. To adequately but at the same time parsimoniously sample the theoretical content domain of job search quality (cf. Cronbach & Meehl, 1955; Wright et al., 2017), for each of the aspects, the first and third author indicated the core/best items, which after consultation resulted in a selection of 40 items. For example, from the five goal commitment items that we adapted from Klein et al. (2001), we selected two core items to parsimoniously cover the commitment to finding employment. Subsequently, four SMEs (i.e., knowledge and research advisor, client satisfaction advisor, policy advisor, and project manager) from the Dutch national employee insurance agency carefully screened the initial item selection to identify redundant items, and poorly worded, ambiguous, or difficult/lengthy items. Based on their feedback, eight redundant items were removed (e.g., “I had a clear idea of what qualities I want in a job.”), and 13 items were revised to simplify and clarify the wording (e.g., “…to see if they can provide me with information…” was simplified to “…to see if they had information…”), resulting in a final initial item pool of 32 items.
Phase 2: Exploration and Item Selection
The purpose of the second phase was to evaluate the initial item pool with unemployed job seekers, to examine the need for further item refinement and reduction, and to explore the factor structure and reliability. To this purpose, we conducted a small-scale qualitative pilot test and a quantitative pilot study (Sample 1), made some small revisions, and subsequently administered the items to a large sample of unemployed job seekers (Sample 2).
Pilot Tests
We first pilot tested the job search quality items among eight unemployed job seekers, who were asked to fill in an online survey including the 32 job search quality items, and instructed to indicate whether items were unclear to them. Responses were positive, although some mentioned that the survey contained many repetitive items, suggesting the potential for further item reduction. Second, we conducted a quantitative pilot in a sample of Dutch unemployed job seekers (Sample 1; N = 218; 41.7% female; Mage = 57.4 years, SD = 3.92; 25.7% Bachelor/Master degree; Mwork experience = 35.47 years, SD = 7.42). Inspection of the descriptives (i.e., M, SD, skewness, and kurtosis) to verify whether items showed sufficient response variance and were normally distributed led to the rewording of one item. Principal component analyses tentatively indicated a distinction between four factors. Because the suggested factors differed somewhat from the four phases of job search quality in Van Hooft et al.’s (2013) self-regulatory model, we collected a new sample to thoroughly explore and confirm the underlying factor structure of job search quality.
Exploratory Factor Analyses
To establish the factor structure and develop the final job search quality scale (JSQS), we used a two-step procedure. As recommended by Worthington and Whittaker (2006) and common in the scale development literature (e.g., Bauer et al., 2001; Kinicki et al., 2013; Wanberg, Zhang, & Diehn, 2010; Yu, 2019), we conducted exploratory factor analyses (EFAs) and reliability analyses, followed by a cross-validation of the item selection and factor structure using CFA in a separate sample (see Phase 3). The EFA and reliability analyses served to explore the factor structure and select the items that provide the best representation of the job search quality dimensions, in order to secure adequate domain sampling and suggest items for deletion to minimize unnecessary repetitiveness (Hinkin, 1998; Hofstee, 1999).
Method
We prepared an online survey, which included an informed consent form, the 32 job search quality items, and several other items (see Phases 4 and 5). Respondents were asked to indicate to what extent the job search quality items applied to them considering the past three months (response options were 1 = Never to 5 = Very often for 4 items, and 1 = Not at all applicable to me to 5 = Fully applicable to me for 28 items). An invitation to participate was sent to 30,649 job seekers aged ≥50 after three months of unemployment, of whom 14,904 completed the survey. For the present study, we only selected those who completed none of the follow-up surveys that were part of a larger research project on training effectiveness (N = 6744; Sample 2). We randomly split the sample in an exploration half (n = 3372; Sample 2a) and a confirmation half (n = 3372; Sample 2b; cf. Hinkin, 1998). In Sample 2a, Mage was 56.3 years (SD = 3.89), 43.5% were female, 8.8% finished only primary education, 60.7% secondary education, and 30.4% had a Bachelor/Master degree, Mwork experience was 34.3 years (SD = 7.54), and Munemployment duration was 12.3 weeks (SD = 0.78).
Analyses and results
To explore the factor structure and establish the final item set, we conducted principal component analyses (PCAs) on the 32 items in Sample 2a. PCA was chosen because our main purpose in this phase was exploration and item reduction, and we wanted to retain original item variance because the job search quality construct is theorized as composed of a series of behaviors that need not be exact parallel items (so uniqueness is not necessarily error). Supporting the factorability of the data, the Kaiser–Meyer–Olkin measure of sampling adequacy was .935. Oblique rotation was used because the underlying job search quality dimensions are theoretically related. To determine the optimal number of factors, we used the Kaiser criterion (eigenvalues >1), parallel analysis (cf. O’Connor, 2000 syntax), the item loadings, and the interpretability of the factors (cf. Hinkin, 1998; Worthington & Whittaker, 2006; Zwick & Velicer, 1986). An initial PCA using the Kaiser criterion indicated a 7-factor solution. Various items loaded on multiple factors, and some factors seemed to represent methodological artifacts (i.e., negatively keyed items) rather than a substantive job search quality dimension. Subsequent parallel analysis (with 100 repetitions and using the 95th percentile of generated eigenvalues) suggested a 4-factor solution. Because the Kaiser criterion overestimates the number of factors (Zwick & Velicer, 1986), we proceeded with a 4-factor solution as indicated by the parallel analysis.
We subsequently ran a series of PCAs extracting four factors and using oblique rotation. We combined the quantitative results with inductive reasoning to come to a conceptually interpretable solution (cf. Hinkin, 1998; Worthington & Whittaker, 2006). As such, we retained items that loaded on a single substantively interpretable factor and deleted inappropriately loading items, and repeated the analyses until we obtained a clear and interpretable factor structure. To adequately represent the content domain, we aimed for at least four items per dimension, but not more than six or seven to preserve parsimony (Hinkin, 1998). Specifically, the first PCA showed that six all negatively keyed items loaded together on a substantively uninterpretable factor. Because this factor seemed to represent a methodological artifact rather than a substantive factor, we deleted these items. Although reverse-scored items may attenuate response-pattern bias, they also introduce systematic error (Hinkin, 1998; Wright et al., 2017). A second PCA again showed a factor with four negatively keyed items that was substantively uninterpretable. We removed these items and ran a third PCA. One item did not clearly load on a single factor (i.e., loadings of .353 and .338 on the first two factors). After removal of this item, the fourth PCA resulted in a clear and interpretable 4-factor structure. Based on the item loadings, reliability analyses, and number of items per dimension, we removed one more item that was substantively redundant. A fifth PCA on the selected 20-item set resulted in four factors (all eigenvalues >1), accounting for 55.8% of the variance. All items loaded >.48 on one of the factors. In line with suggested values (Worthington & Whittaker, 2006), cross-loadings were <.32 with two exceptions (i.e., .355 for Item 3; .351 for Item 20), and cross-loadings differed >.15 with the item’s highest loading for all but one item (i.e., .13 for Item 20). As these cross-loadings are substantively justifiable and because the job search quality dimensions are conceptually interrelated, we retained these items.
The first factor refers to goal establishment and planning, including seven items on goal clarity, job search clarity, systematic search strategy, deadline setting, job search prioritizing, job search planning, and goal commitment. The second factor refers to preparation and alignment, consisting of five items on the preparation of job search activities and alignment of application efforts (i.e., the presentation of one’s qualifications and strengths) to what the organization is looking for in applicants. The third factor refers to emotion regulation and persistence, including four items on self-control of emotions, thoughts, and behavior as to facilitate and persist in job seeking. The last factor can be labelled learning and improvement, with four items referring to reflection on job search activities and active feedback-seeking as to learn and improve the job search. The four subscales showed adequate reliabilities for a new scale (i.e., >.70), which could not be increased by item removal. Specifically, Cronbach’s alphas (incl. 95% confidence intervals [CI]; cf. Wright et al., 2017) were .826 (CI [.817; .835]) for goal establishment and planning, .817 (CI [.807; .827]) for preparation and alignment, .706 (CI [.689; .722]) for emotion regulation and persistence, and .751 (CI [.737; .764]) for learning and improvement.
Phase 3: Confirmation of the Factor Structure
The purpose of the third phase was to cross-validate the item selection and four-factor structure of the JSQS as determined in Phase 2. To this purpose, we conducted confirmatory factor analyses (CFAs) in the validation half of Sample 2 (i.e., Sample 2b). As recommended by Hinkin (1998) and Wright et al. (2017) to enhance the generalizability, we examined the factor structure in a new sample (Sample 3). This served as a replication in an independent sample and examination of the psychometric properties of an English version of the JSQS.
Methods
As noted in Phase 2, Sample 2b is the confirmation random half of the sample (n = 3372; Mage = 56.2 years, SD = 3.91; 42.6% female; 9.1% primary education, 59.5% secondary education, 31.3% Bachelor/Master degree; Mwork experience = 34.1 years, SD = 7.65; Munemployment duration = 12.3 weeks, SD = 0.76).
Sample 3 consisted of unemployed job seekers registered with the online research portal Prolific Academic. The study consisted of a T1 survey and a three month follow-up (T2; see Phase 5). The T1 survey included an informed consent form, the JSQS items, and several other items on job seeking (see Phase 4). Respondents received £1.10 for T1 participation. The JSQS items were independently translated from Dutch into English by the first two authors, reaching consensus on the English version after discussion. A job search scholar fluent in Dutch and English independently back-translated the items. Comparison of original and back-translated items revealed one slight difference, which was resolved by adjusting the English item. Respondents indicated the extent that the statements applied to them in the past month (1 = Not at all applicable to me, 5 = Fully applicable to me). The study was published to individuals registered at Prolific Academic as unemployed, actively looking for a job, aged 18–65 years, English as first language, living in the U.S., Canada, U.K., Ireland, Australia, or New Zealand, and with a 90–100% Prolific Academic approval rate. Of the eligible individuals, 714 approved the informed consent and completed the survey. We excluded 212 respondents who indicated that they were not unemployed or not currently actively looking for a job, and 25 respondents who were unemployed <1 month (because our items refer to the previous month). We also excluded those who did not complete all JSQS items (n = 4), and those who did not correctly respond to a careless responding item (n = 39), resulting in a final sample of 434 (Mage = 28.06 years, SD = 10.09; 50.7% female; 1.4% primary education, 24.9% secondary education, 36.4% some continued education, 37.4% Bachelor/Master degree; Mwork experience = 6.80 years, SD = 8.69; median unemployment duration = 11 months).
Analyses and Results
Model Fit Statistics of the Confirmatory Factor Analyses in Samples 2b and 3.
Note. GE&P = Goal establishment and planning; P&A = Preparation and alignment; ER&P = Emotion regulation and persistence; L&I = Learning and improvement; SRMR = Standardized root mean squared residual; CFI = Comparative fit index; RMSEA = Root mean squared error of approximation; AIC = Akaike information criterion; BIC = Bayesian information criterion.
Factor Loadings from the Confirmatory Factor Analyses in Samples 2b and 3.
Note. N = 3370 in Sample 2b and N = 434 in Sample 3.
We compared the hypothesized Model 1 with two likely alternative two-factor models that combine two theoretically adjacent dimensions based on Van Hooft et al.’s (2013) model (Models 2 and 3), and with a single factor model (Model 4). As Δχ2 is sample size dependent, we used ΔCFI for model comparison, with a value >.01 indicating better fit (cf. Cheung & Rensvold, 2002). We also consulted ΔAIC and ΔBIC. Table 1 shows that the alternative Models 2–4 fit worse than Model 1 (ΔCFIs >.01; ΔAICs and ΔBICs between −233.932 and −2691.365). Lastly, given that the four dimensions purport to represent underlying dimensions of the same job search quality construct, we tested a four-factor model with an overarching second-order factor (Model 5). This model also resulted in a reasonable to good fit (SRMRs <.08, CFIs >.90, RMSEAs = .06). The ΔAICs and ΔBICs indicated in Sample 2b that Model 5 fit worse than Model 1, but in Sample 3 that it fit similarly or better. ΔCFI was <.01 in both samples, suggesting that Model 5 did not fit worse than Model 1. In subsequent analyses, we therefore present findings for the four separate dimensions of job search quality, as well as for the composite job search quality score (calculated as the average of the 20 items).
Alphas, Descriptives, and Correlations for the JSQS, Nomological Network Variables, and Criterion-Related Variables in Sample 2b.
Note. GE&P = Goal establishment and planning; P&A = Preparation and alignment; ER&P = Emotion regulation and persistence; L&I = Learning and improvement. Gender was coded as 0 = male and 1 = female. Educational level was coded as 1 = primary education, 2 = lower secondary education, 3 = secondary education/intermediate vocational education, 4 = bachelor degree, 5 = master degree. Response scales for Variables 3–18 ranged from 1 to 5. Number of job interviews ranged from 1 to 10. Employment status was determined six months after the survey and coded as 0 = still unemployed and 1 = reemployed. Due to incidental missing values n ranges between 3367 and 3372. Cronbach’s alphas are on the diagonal. Correlations > |.053| are significant at p < .01.
a 95% CI [.823; .840].
b 95% CI [.811; .831].
c 95% CI [.685; .718].
d 95% CI [.729; .757].
e 95% CI [.900; .909].
Alphas, Descriptives, and Correlations for the JSQS, Nomological Network Variables, and Criterion-Related Variables in Sample 3.
Note. GE&P = Goal establishment and planning; P&A = Preparation and alignment; ER&P = Emotion regulation and persistence; L&I = Learning and improvement. Gender was coded as 0 = male and 1 = female. Educational level was coded as 1 = primary education, 2 = secondary education, 3 = some college or vocational education, 4 = vocational degree, 5 = undergraduate degree, 6 = graduate degree. Response scales for Variables 3–7, 10–14, and 16 ranged from 1 to 5. Response scales for Variables 8–9 and 19–20 ranged from 1 to 7. Number of job interviews ranged from 1 to 30. Employment status was coded as 0 = still unemployed and 1 = reemployed. Unemployment duration until reemployment was measured in months. N = 434 for T1 variables. N is 320 for Variable 15, 206 for Variable 16, 322 for Variable 17, and N varies between 83 and 92 for correlations with Variables 18–20. Cronbach’s alphas are on the diagonal.
† p < .10. * p < .05. ** p < .01.
a 95% CI [.807; .855].
b 95% CI [.729; .799].
c 95% CI [.722; .796].
d 95% CI [.670; .758].
e 95% CI [.897; .920].
Phase 4: Convergent, Discriminant, and Concurrent Validity
The purpose of the fourth phase was to further establish the construct validity of the JSQS by examining the convergent and discriminant validity (cf. Hinkin, 1998). Convergent validity refers to the extent to which the JSQS correlates with other scales aimed to measure similar constructs, and discriminant validity refers to the extent to which the JSQS does not strongly correlate to conceptually different constructs (Hinkin, 1998). Wright et al. (2017) note, however, that this distinction represents an oversimplified dichotomy. Rather, the JSQS should correlate with a variety of scales aimed to measure theoretically related constructs, ranging from totally similar (i.e., expected correlation near 1 or −1) to totally dissimilar (i.e., expected correlation near zero). Therefore, similar to other scale development studies (e.g., Bauer et al., 2001; McCarthy & Goffin, 2004), we examined not only the convergent and discriminant validity of the JSQS, but also started to explore its nomological network by examining relationships with other theoretically related constructs (i.e., concurrent validity).
Overview of Predictions, Measures, and Results on the Convergent, Discriminant, and Concurrent Validity of the JSQS (Phase 4).
a Measured in Sample 2b.
b Measured in Sample 3.
In terms of concurrent validity, we begin exploring the nomological network of the JSQS by examining its associations with other theoretically related constructs. Kanfer et al. (2001) identified six categories of individual difference variables that affect self-regulatory mechanisms during job search: Personality traits, generalized expectancies, self-evaluations, motives, social context, and biographical variables. Combining Kanfer et al.’s (2001) theorizing and framework with Van Hooft et al.’s (2013) job search quality theory, we selected relevant constructs in each of these six categories and developed expectations regarding their relationships with the JSQS. Table 5 specifies the constructs and rationales.
Methods, Analyses, and Results
To explore the convergent, discriminant, and concurrent validity of the JSQS, we used the Samples 2b and 3. See Phase 3 for sample descriptions, Table 5 for the measures that were collected for this purpose in these samples, and Tables 3 and 4 for the Cronbach’s alphas.
Correlations between the study variables are presented in Table 3 for Sample 2b and in Table 4 for Sample 3. Table 5 presents an overview of the findings. Supporting the JSQS’s convergent validity, career self-exploration correlated strongly with the JSQS composite, and moderately to strongly with its dimensions (see Table 3). Further supporting the convergent validity, Table 4 shows strong correlations of job search clarity, haphazard job search, and metacognitive activities with the JSQS composite, and moderate to strong correlations with its dimensions. Regarding the discriminant validity of the JSQS, negligible to small correlations were found of gender with the JSQS composite and its dimensions in Sample 2b (see Table 3). In Sample 3, correlations were small to moderate (see Table 4), such that females display somewhat higher job search quality than males. For social desirability, correlations with the JSQS composite and its dimensions were small to moderate, suggesting that the JSQS is not overly sensitive to socially desirable responding. Further, as expected, the JSQS composite and its dimensions showed strong positive correlations with measures of job search intensity (see Tables 3 and 4) and job search effort (see Table 4). To assess whether job search quality is empirically distinct from job search intensity and effort, we conducted a series of CFAs using Samples 2b and 3 (see online supplemental materials), which supported the distinctiveness of the JSQS composite and its dimensions from job search intensity and effort.
Supporting the concurrent validity of the JSQS, correlations in Sample 2b (see Table 3) show that both personality variables (i.e., conscientiousness and learning goal orientation) related moderately to strongly to the JSQS composite and its dimensions. Regarding generalized expectancies, correlations show support for our expectations on reemployment efficacy, with a moderate positive correlation with the JSQS composite and small to moderate correlations with its dimensions. However, in contrast to our expectations, job search locus of control was not or weakly negatively related to the JSQS (which may be explained by the measure used; see Discussion). For self-evaluations, job search self-efficacy related moderately to strongly to the JSQS composite and its dimensions, supporting the concurrent validity of the JSQS. In the category motives, autonomous motives (i.e., autonomous job search motivation and employment commitment) correlated moderately to strongly with the JSQS composite and its dimensions, supporting the concurrent validity. For financial need as a controlled motive, we formulated opposing predictions. The results indicate small positive relations with the JSQS composite and its dimensions, suggesting that financial need may slightly promote rather than hinder (elements of) a high-quality job search. As an indicator of job seekers’ social context, social support related strongly positively to the JSQS composite and moderately to strongly to its dimensions, as expected. Lastly, we expected positive relations with the educational level, which was supported for the JSQS composite and for the first two dimensions. However, the emotion regulation and persistence and the learning and improvement dimensions showed weak or non-significant correlations, suggesting education is less important for these.
Phase 5: Criterion-Related and Incremental Validity
The purpose of the fifth and final phase was to examine the criterion-related validity of the JSQS by examining its relation with important outcomes, and investigate its incremental validity in predicting these outcomes beyond related job search measures. Based on a process-oriented perspective, we distinguish between (a) job search outcomes (i.e., proximal outcomes that occur during the job search process), and (b) employment outcomes (i.e., distal outcomes that occur after completion of the job search process; Saks, 2005; Van Hooft et al., 2013). Following recommendations (Boswell et al., 2012; Saks, 2005), we focus on both quantitative and qualitative job search outcomes (i.e., number of job interviews that job seekers generate while searching for reemployment vs. fit with the interviewed jobs), and both quantitative and qualitative employment outcomes (i.e., employment status and duration until reemployment vs. fit and satisfaction with the newfound job). Because high job search quality implies that job seekers learn what employers want such that they can attune their behaviors and products (e.g., application letter and interview behavior) to potential employers, job seekers engaging in a high-quality job search more likely meet/exceed the expectations of demanding parties at the labor market, are invited for job interviews, and obtain a job faster. Further, because high job search quality involves learning what employers want, it likely also increases job seekers’ knowledge about jobs/organizations in their field. Consequently, a higher quality job search more likely results in identifying suitable and fitting job options, and increased chances to obtain higher quality jobs. Altogether, we thus expect that job search quality positively relates to quantitative and qualitative job search and employment outcomes.
In addition, we conduct usefulness analyses to examine the incremental validity of the JSQS beyond related job search constructs such as career self-exploration, job search effort, job search intensity, job search clarity, haphazard job search strategy, and metacognitive activities. Given that the content domain of the JSQS is conceptually broader and/or different from these constructs, we expect the JSQS to explain unique variance in the outcomes.
Methods
To explore the criterion-related and incremental validity of the JSQS, we used Samples 2b and 3 (see Phase 3 for descriptions of these samples; see Table 4 for Cronbach’s alphas).
Sample 2b
In addition to the JSQS and the Phase 4 variables, the survey contained an item measuring number of job interviews (i.e., “How many times have you gone on a job interview in the past three months?”; response options 0–100). To reduce effects of outliers, we trimmed scores >10 (4.1%) to equal 10. Further, we assessed participants’ employment status six months after the survey, using data from the Dutch employee insurance agency.
Sample 3
Three months after T1, we approached participants with a follow-up survey through Prolific Academic. The T2 survey included an informed consent and measures for job search outcomes and employment outcomes. Respondents received £1.00 for T2 participation. Of the final T1 sample, 322 participated at T2 (74.2%). Logistic regression with the T1 variables as predictors and T2 participation as dependent showed some signs of non-random attrition, χ2(10) = 18.849, p = .042. T1 respondents higher on job search intensity, Exp(B) = .557, p = .015, and job search effort, Exp(B) = .710, p = .046, less likely participated at T2.
Number of job interviews was asked with the item “How many times have you gone on a job interview in the past three months or up till you obtained a job?” (response options from 0 to 30). If the number of interviews was >1 (n = 206), respondents were asked to complete four items on perceived fit with the interviewed jobs (adapted from Saks & Ashforth, 2002). Respondents were asked to think of the interviewed jobs in the past three months or up till they obtained a job, and report the perceived fit (e.g., “To what extent were these jobs a good match for you?”; 1 = to a very little extent, 5 = to a very large extent). Employment status was measured with the question “Three months ago you participated in our first survey. Have you obtained a new job since then?” (1 = Yes, 0 = No). Among those reemployed (n = 92) we further measured the duration in months from becoming unemployed to finding a job (cf. Wanberg et al., 2002), by asking: “How long [in months] were you unemployed before you found this job? Please estimate as accurately as possible” (response options from 0 to 120 months). We further assessed two indicators of employment quality (response options 1 = strongly disagree to 7 = strongly agree). Perceived fit was measured with Cable and DeRue’s (2002) 9-item scale assessing needs-supplies, demands-abilities, and person-organization fit (e.g., “My personal values match this organization’s values and culture”), and job satisfaction with three items of Cammann et al., 1983; e.g., “All in all, I am satisfied with this job”).
Analyses and Results
The criterion-related validity was examined with Samples 2b and 3 correlations as shown in Tables 3 and 4. Given that unemployment duration until reemployment, perceived fit of the new job, and job satisfaction could only be assessed for Sample 3 reemployment participants (with ns between 83 and 92), we interpret correlations for these variables with ps < .10.
Job search outcomes
The JSQS composite and its dimensions demonstrated small to moderate positive correlations with number of job interviews assessed in a simultaneous survey in Sample 2b (Table 3). In Sample 3, the correlation of the JSQS composite and its dimensions with number of job interviews assessed three months later were positive and of medium to strong size (Table 4), supporting the criterion-related validity of the JSQS. In further support, Sample 3 results show small to moderate positive correlations of the JSQS composite and three of its dimensions with perceived fit with the interviewed jobs. Only the correlation of the dimension learning and improvement was not significant.
Employment outcomes
In Sample 2b, the JSQS composite related positively to database employment status after six months, with a small effect size. The dimensions showed small positive correlations, with the highest correlation for goal establishment and planning (Table 3). In Sample 3, the JSQS composite had a small to moderate correlation with self-reported employment status after three months. Of the dimensions, emotion regulation and persistence had the highest correlation, while the correlation of learning and improvement was not significant (Table 4). Table 4 further shows a small to moderate negative correlation of the JSQS composite with unemployment duration, indicating that higher job search quality predicts faster reemployment. Of the dimensions, preparation and alignment displayed the strongest correlation, while the correlation for emotion regulation and persistence was not significant. Lastly, we examined the relations of the JSQS with two indicators of employment quality in Sample 3. The JSQS composite correlated positively with both perceived fit with the new job and job satisfaction (small to medium effect size). For both indicators, goal establishment and planning showed the highest correlation. Of the other dimensions, only the correlation of emotion regulation and persistence with perceived fit was significant.
Usefulness analyses
Usefulness Analyses of the JSQS Compared to Career Self-Exploration, Job Search Clarity, Haphazard Job Search Strategy, Metacognitive Activities, Job Search Effort, and Job Search Intensity.
Note. JSQS represents the four job search quality dimensions which were entered as separate constructs.
*** p < .001. ** p < .01. * p < .05. † p < .10.
a Hierarchical linear regression analysis.
b Hierarchical logistic regression.
c Hierarchical Cox regression.
General Discussion
Job loss and unemployment are detrimental for individuals, their families, and society as a whole. With rising unemployment rates as a consequence of the COVID-19 crisis (OECD, 2020), understanding how unemployed individuals can effectively search for reemployment is of utmost importance. We sought to enhance the understanding of what a high-quality job search process entails, by developing and validating the Job Search Quality Scale (JSQS). Following established procedures and recommendations for scale construction, in five phases we generated items to cover the job search quality content domain (Phases 1–2), confirmed and replicated a four-factor structure of the JSQS (Phase 3), found support for its convergent validity and distinctiveness from job search effort and intensity measures and identified key correlates (Phase 4), and demonstrated its predictive value for job search and employment outcomes and incremental validity above extant job search measures (Phase 5).
Theoretical Implications and Suggestions for Future Research
Our findings have important implications for theory and research on job seeking and reemployment. Below we discuss these implications and offer directions for future research.
Dimensionality of job search quality
This study details the underlying structure of job search quality by uncovering four dimensions: (a) goal establishment and planning (i.e., clarity, specificity, and systematicity of job search goals, strategies, and planning), (b) preparation and alignment (i.e., careful preparation of job search activities and alignment of application efforts to what organizations look for in applicants), (c) emotion regulation and persistence (i.e., self-control of emotions, thoughts, and behavior to facilitate and persist in job seeking), and (d) learning and improvement (i.e., reflection on job search activities and feedback-seeking to learn and improve the job search). These dimensions include cognitive, affective, and behavioral elements, which are core to the broader multidimensional job search behavior construct (Kanfer et al., 2001). Supporting Van Hooft et al.’s (2013) theorizing, the four dimensions illustrate the self-regulatory nature of job search quality. However, the four JSQS dimensions also differ somewhat from Van Hooft et al.’s (2013) four self-regulatory job search process quality components. For example, goal establishment and planning converged into one dimension (instead of two separate components), while preparation and alignment emerged as a separate dimension (instead of as subcomponent of the planning component). This structure can be understood such that the goal establishment and planning dimension is more self-focused (i.e., referring to one’s own job search goals, strategies, and planning), while the preparation and alignment dimension is more outward-focused (i.e., referring to what organizations seek and thinking from the perspective of the employer when preparing applications). The emergence of preparation and alignment as a separate dimension illustrates the importance of these activities, and shows the added value of an inductive approach to item generation (as these items mostly arose from interviews with practitioners).
Although the four JSQS dimensions displayed moderate to strong interrelations, factor analyses pointed toward a four-factor model, suggesting the distinctiveness of the four dimensions. Future research could examine the relative importance of each dimension in predicting outcomes. Are high levels on all four dimensions needed to increase employment success (e.g., in an interactive fashion)? Or are some dimensions of general importance and some dimensions important in specific situations? For example, possibly goal establishment and planning is of more generic importance, while emotion regulation and persistence is especially important when finding employment is difficult and extends over a longer period. Future research could adopt cluster analytic or latent class modeling approaches to examine how the job search quality dimensions act in concert. In addition, building on theory and research outlining the dynamic nature of job search (e.g., Barber et al., 1994; Da Motta Veiga & Gabriel, 2016; Song et al., 2018; Van Hooft et al., 2013; Wanberg, Zhu, & Van Hooft, 2010), future research should examine how the job search quality dimensions evolve and mutually affect one another over time. When research questions pertain to a more general level of job search quality, future research may focus on a composite job search quality score, as our analyses also found support for a four-factor model with a single overarching second-order factor.
Job search intensity and job search quality
Commonly used measures of job search behavior refer to the intensity or effort with which job seekers engage in job search activities (e.g., Blau, 1994). Such measures do not touch upon the quality-related component of the broader job search behavior construct, which was identified as an important dimension of job search behavior (Kanfer et al., 2001; Van Hooft et al., 2013; Van Hoye, 2014). The present study supports Kanfer et al.’s (2001) conceptualization of job search behavior, showing that job search quality and its dimensions are relatively strongly related to measures of job search effort and intensity. However, CFAs in two independent samples also demonstrated that job search quality and its dimensions are empirically distinct from job search effort and intensity. This suggests that the JSQS taps into a theoretically different part of the job search construct space. Usefulness analyses further indicated the value of job search quality beyond job search effort and intensity, showing that the JSQS explains unique variance in outcomes such as number of job interviews, chances to obtain employment, and speed of reemployment.
Future research is needed to examine in more detail if and how job search intensity and job search quality jointly relate to employment success. For example, do intensity and quality have only additive effects or do they interact in predicting employment success (e.g., chances to obtain employment increase when both intensity and quality are high; or high job search quality may compensate for low job search intensity)? And does the predictive value of job search quality versus intensity depend on personal and situational characteristics (e.g., job seeker human capital; blue-collar vs. white collar jobs; and tightness of the labor market)?
Correlates of job search quality
Based on Kanfer et al.’s (2001) framework we examined the JSQS’s links with six categories of individual difference variables. Especially conscientiousness, learning goal orientation, job search self-efficacy, autonomous job search motivation, employment commitment, and social support stood out as moderate to strong (i.e., rs > .30) positive correlates of job search quality. Reemployment efficacy, perceived financial need, and educational level showed small to moderate positive links (i.e., .10 < rs < .30). Although we measured these variables at the same time as job search quality, based on motivation and self-regulation theories (Kanfer et al., 2001; Van Hooft, 2018a; Van Hooft et al., 2013; Vansteenkiste & Van den Broeck, 2014), these form likely antecedents of job search quality. However, future longitudinal and intervention research needs to further establish the underlying (causal) mechanisms between job search quality and its antecedents.
The potential antecedents of job search quality appear similar to antecedents of job search intensity. This is no surprise as both are components of the broader job search behavior construct, and both are characterized by their self-regulatory nature (Kanfer et al., 2001; Van Hooft et al., 2021). Nevertheless, our data suggests that of the stronger correlates especially learning goal orientation, autonomous job search motivation, and employment commitment are more strongly related to job search quality than to intensity (i.e., difference in rs of >.09, ts > 7.60, ps < .001). These findings point toward the importance of theoretical perspectives such as self-determination theory (Deci & Ryan, 2000; Vansteenkiste & Van den Broeck, 2018) and achievement goal theory (Payne et al., 2007; Van Hooft, 2018b) to understand job search quality, suggesting that high-quality self-regulation is especially fostered by autonomous motives and an orientation toward learning and developing competence.
Job search locus of control failed to relate positively to job search quality. This is opposite to our predictions because for a high-quality job search, people need to believe that their chances at the labor market are changeable and under their control (Van Hooft et al., 2013). Possibly our measure did not capture attributions comprehensively, as it only focused on (external) causality attributions. Future research should therefore use measures that include all three dimensions of attributions (i.e., locus of causality, stability, and controllability).
Outcomes of job search quality
Various scholars proposed that job search quality may be of importance in the process of obtaining employment (e.g., Saks, 2005; Van Hooft et al., 2013; Van Hoye, 2014; Vuori & Vinokur, 2005; Wanberg et al., 2002). Our findings support these propositions by demonstrating that the JSQS positively relates to quantitative outcomes such as the number of job interviews, employment status, and reemployment speed. In addition, usefulness analyses showed that the JSQS explained unique variance in these outcomes beyond extant job search measures such as job search effort and intensity, career self-exploration, job search clarity, haphazard search strategy, and metacognitive activities. Theoretically, our findings suggest that to improve the prediction of employment success, job search behavior should be measured in its full breadth, including job search quality.
Further, preliminary support was found for the value of the JSQS in predicting qualitative outcomes, with small to medium-sized relations of the JSQS with perceived fit with the interviewed jobs, and perceived fit and job satisfaction with the new job. These findings are promising, as previous research typically found null effects for job search intensity in predicting employment quality (Van Hooft et al., 2021). Future research is needed to further examine job search quality’s relations with employment quality outcomes in larger samples, and using additional employment quality indicators (e.g., job improvement and salary).
Additionally, future research could examine the underlying mechanisms that explain how job search quality relates to employment success. For example, does job search quality lead to enhanced reemployment probabilities via improved networking behavior, higher quality applications, and better interview behavior? And how do job seekers change and improve their behaviors as a consequence of what they learn along the way? Building on findings by Chawla and colleagues (2019), future research is also needed to establish what type of feedback helps to improve job search quality. Lastly, future research is needed to establish the causality of the job search quality—employment success relation, for example, by conducting intervention studies in which job search quality is trained.
Practical Implications
The present findings have important implications for job seekers and career counseling. For job seekers, the JSQS and its items may serve as a guideline or checklist for how to conduct a high-quality job search. Job seeking oftentimes is a rather lengthy process filled with obstacles and setbacks, which negatively impacts job seekers’ emotions and persistence (e.g., Kreemers et al., 2018; Song et al., 2009; Wanberg et al., 2012). The JSQS provides detailed information on how to navigate this difficult process more effectively. Counseling organizations often use inventories for diagnosing and profiling job seekers, in order to identify those in need of help (e.g., Wanberg, Zhang, & Diehn, 2010; Wijnhoven & Havinga, 2014). Such inventories may benefit from including a comprehensive assessment of job search quality. In addition, profiling may benefit from a focus on important correlates of job search quality such as learning goal orientation, autonomous job search motivation, and employment commitment to identify those job seekers that likely may or may not conduct a high-quality job search. Previous research indicated that job search interventions increase reemployment success, especially when these include aspects such as promoting goal-setting, teaching job search skills, encouraging proactivity, and improving self-presentation (Liu et al., 2014). The JSQS dimensions and items add to these findings by offering specific directions what to address in training programs and interventions to optimize the job search process.
Limitations and Conclusion
Although our findings provide support for the validity and added value of the JSQS, some limitations must be taken into account. First, the JSQS is a self-report measure, and it can be debated whether job seekers are able to adequately self-report their job search quality. However, ultimately job seekers themselves have the most complete view on their thoughts, affect, and behavior. Also, our findings support the criterion validity of the JSQS by showing positive relations with outcomes such as number of interviews, employment status, and reemployment speed, which are relatively objective in nature (and in Sample 2b obtained via agency data). Nevertheless, future research is needed to further validate the JSQS by examining its relationships with, for example, counselor reports of job search quality, ratings of resume quality, and recruiter evaluations of interview behavior.
Second, in examining the criterion-related validity in Sample 3, some non-random attrition occurred between T1 and T2, although the response rate at T2 was relatively high. This non-random attrition may have affected the estimates. However, given that the criterion-related validity findings in Sample 3 converge with those in Sample 2, we may conclude that the attrition does not threaten the validity of our conclusions. Further, regarding employment quality, our conclusions should be interpreted as preliminary given the small sample size these were based on. Sample size is often an issue when studying employment quality in samples of unemployed job seekers, as usually only a small portion finds reemployment within the study period. Thus, future research needs to further test the predictive validity of the JSQS, targeting large samples of unemployed job seekers or samples that more likely obtain employment.
Third, we focused on Dutch samples of older short-term unemployed job seekers and an international English-speaking sample of longer unemployed job seekers (i.e., median of 11 months). Although the reliability, validity, and usefulness of the JSQS was supported in these different samples, future research needs to examine the generalizability of our findings to unemployed samples in other contexts and to other job seeker types (i.e., graduating students and employed job seekers). Also, our studies were conducted prior to the COVID-19 pandemic. Given the dramatic consequences of this crisis for unemployment rates, vacancy rates, and methods of job search and recruitment, future research should use the JSQS to investigate how the altered circumstances affect job seekers’ intensity and quality of search in relation to job search and employment outcomes.
In conclusion, this study presents the JSQS as a comprehensive measure of job search quality, developed based on previous research on motivation, self-regulation, and job seeking combined with input from practice. The four-dimensional JSQS details the content of job search quality, and was found to predict important employment success outcomes. With the JSQS we provide a validated measure to be used in research on job search and reemployment to further test and extend theory on job search quality. Practically, the JSQS informs job seekers how to conduct a high-quality job search, enables counseling organizations to diagnose job seekers, and provides input what to focus on in job search training programs.
Supplemental Material
sj-pdf-1-jca-10.1177_10690727211052812 – Supplemental Material for How to Optimize the Job Search Process: Development and Validation of the Job Search Quality Scale
Supplemental Material, sj-pdf-1-jca-10.1177_10690727211052812 for How to Optimize the Job Search Process: Development and Validation of the Job Search Quality Scale by Edwin A. J. van Hooft, Greet Van Hoye and Sarah M. van den Hee in Journal of Career Assessment
Footnotes
Author Note
A previous version of this paper was presented at the 1st Careers Division Community Conference of the Academy of Management (February 2020), Vienna, Austria.
Acknowledgments
We thank the Dutch national employee insurance agency (UWV), and especially Han van der Heul, for the help in conducting this research and Jolien Stremersch for her suggestions to an earlier draft of this paper.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a visiting foreign scholar grant of Ghent University.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
