A Rasch and Factor Analysis of a Paramedic Graduate Attribute Scale

Abstract

This study examined the construct validity of the Paramedic Graduate Attribute scale (PGAS) using factor analysis and Rasch Analysis. A convenience sample was used in the study involving paramedics from all states and territories in Australia. Participants were asked to rate the importance of 47 graduate attribute items. Principal components analysis (PCA) was undertaken on the 47 items followed by Oblique Oblimin rotation. For the Rasch analysis item fit, item invariance and dimensionality were examined. A total of 872 paramedics participated in the study (23% response rate). PCA of the 47 items revealed seven factors with eigenvalues greater than 1, accounting for 40.6% of the total variance. The subsequent Rasch analyses based on the seven factors produced seven misfitting items and confirmed a 7-factor solution. The 7-factor PGAS produced a good fit to the Rasch Model and exhibited good reliability and unidimensionality, offering the Australian paramedic discipline a set of empirically based graduate attributes.

Keywords

higher education graduate attributes paramedics factor analysis Rasch analysis

Background: Paramedic Education and Training

The Australian paramedic discipline has undergone important transformations over the past 30 years in its education, training, and health care identity, essentially moving from a stretcher-bearer to ambulance driver to autonomous clinical practitioner. Significantly, these transformations are currently taking place when the health care system is searching for alternative service delivery models as it continues to be placed under growing pressures to its health delivery and changing workforce patterns. Additionally, given the changes in paramedic education and training and health care reforms, this will likely change future clinical practice and health care identity, indicating that that current models of paramedic service delivery must change (O’Meara et al., 2006). Therefore, preparing future health care graduates for these expected changes requires careful alignment of graduate attributes to core curriculum taught to paramedic students. Current health care reforms, such as increased workforce mobility, greater emphasis on interprofessional collaboration, and a single national registration and accreditation governance structure (Australian Health Practitioner Regulation Agency, 2010), highlight that the paramedic discipline is “professionally” vulnerable given its “semi-professional” status in Australia. Without formal professional recognition, exclusion from national professional registration and lack of a formal accreditation scheme for paramedic education programs, this places the discipline in a tenuous situation with occupational takeover a real possibility.

Currently, entry-level paramedic qualifications are now offered at a bachelor undergraduate degree or graduate diploma (nursing conversion) level. While this progression to higher education has been an important step in the professionalization of the paramedic sector in Australia, it is also complicated by the numbers of programs now being offered around the country. Since the first undergraduate bachelor program was offered in 1994, there are now 11 universities (9 since the year 2000) offering pre-entry programs into the paramedic health care industry. However, national curriculum standards do not exist in Australia, and none of these education programs have obtained formal academic accreditation or have created graduate attributes (Williams, Onsman, & Brown, 2010a). It is argued that one of the main reasons why the paramedic discipline is yet to be formally recognized as a profession is largely due to its inability to standardize its curriculum, provide consistent, enforceable academic standards, and be certain whether graduates meet industry and health service needs (Williams et al., 2010a).

One solution toward protecting its occupational territory and promoting its chances of obtaining national registration, formal education program accreditation, and recognition as a full-fledged profession is the establishment of national paramedic curriculum standards. In developing these standards, establishing graduate attributes (often used interchangeably with competencies and/or capabilities) becomes a central part of creating national standards. The establishment of paramedic graduate attributes provides the capacity to benchmark and map curriculum for ongoing accreditation as well as internal and external audits. Currently, no national graduate attributes formally exist that inform Australian paramedic undergraduate curriculum. Although a recent pilot study involving paramedic educators and academics undertook the first attempt at generating empirically developed graduate attributes (Williams, Onsman, & Brown, 2010b). This article will build upon this earlier work, using a combination of several multivariate statistical approaches: Factor analysis and Rasch Analysis.

Background: Exploratory Factor Analysis (EFA)

Factor analysis is commonly used in the fields of psychology, health science, and education (Spearman, 1904) and is considered the method of choice for interpreting self-reporting questionnaires. (Henson & Roberts, 2006) Factor analysis is a multivariate statistical procedure that has many uses (Gorsuch, 1983; Hair, Anderson, Tatham, & Black, 1995; Hogarty, Hines, Kromrey, Ferron, & Mumford, 2005), three of which will be briefly noted here. First, factor analysis reduces a large number of scale items into a smaller set of composite variables (also referred to as factors). Second, it establishes underlying dimensions between measured variables and latent constructs, thereby allowing the formation and refinement of theory. And third, it provides construct validity evidence of self-reporting scales.

There are two major classes of factor analysis: Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Broadly speaking EFA is heuristic. In EFA, the investigator has no expectations of the number or nature of the variables and as the title suggests, is exploratory in nature. That is, it allows the researcher to generate a theory or model from a relatively large set of latent constructs (Swisher, Beckstead, & Bebeau, 2004; Thompson, 2004). Whereas in CFA (a form of structural equation modeling), the researcher uses this approach to test a proposed theory or model and in contrast to EFA, has assumptions and expectations based on priori theory regarding the number of factors and which factor theories or models best fit (Thompson, 2004). In this study however, we will use Rasch Analysis instead of CFA.

Background: Rasch Analysis

The Rasch Model, named after Danish physicist and mathematician Georg Rasch utilizes a mathematical model in the assessment and testing of construct validity within instruments and scales (Tennant & Conaghan, 2007). The Rasch model has been widely used by educational researchers for the past 40 years. However, its application is gaining popularity among researchers in the health and human science disciplines; particularly over the past decade (Andrich, 2004; Tennant & Conaghan, 2007).

Researchers have a number of ways in which to apply the Rasch Model. First, the Rasch Model can be utilized in the application of scale development where items may form part of subscale totals or overall scale scoring. In addition to this, the Rasch Model can be used to formulate a hierarchy map of items to person’s ability to further strengthen dimensionality. This process is referred to as calculating person logit and item logit measures. Second, the model allows for psychometric testing of scale unidimensionality, differential item functioning ([DIF] item bias), construct dimensions of ordinal scale data, and conversion of ordinal data into interval-level data.

The purpose of this article was to explore the construct validity of the Paramedic Graduate Attribute scale (PGAS) a factor analysis, using principal component analysis (PCA) with an Oblique Oblimin rotation (a Classical Test Theory approach). Following the factor analysis, further exploration of the properties of the scale was examined using Rasch Model analysis (a type of Item Response Theory).

Participants

Participants who were invited to take part in this study were financial members of the Australian College of Ambulance Professionals ([ACAP] n = 3800). Distribution of the PGAS coincided with the Response Magazine (Spring Edition, 2008), which is the national paramedic magazine circulated to all ACAP members on a quarterly basis. The PGAS was inserted inside the magazine along with a replied paid envelope. The latest salaried-paramedic employment figures are quoted at 9,491 (Council of Ambulance Authorities, 2008), suggesting that approximately 40% of all Australian paramedic membership were sampled. Ethical approval was sought and received from the Monash University Standing Committee on Ethics in Research Involving Humans (SCERH).

Instrumentation

A self-reporting instrument designed to assess which graduate attributes were best suited to the paramedic sector was used. This design was based on an earlier pilot study that generated a 10-factor construct solution. These constructs were used to inform this larger study. Participants were provided with an explanatory statement and were informed that participation was voluntary and anonymous. Participants were asked to rate 53 items using a 5-point Likert scale (1 = strongly disagree and 5 = strongly agree). Section 1 asked participants about their personal and professional characteristics, while Section 2 asked participants about their views on the most desirable graduate attributes. The graduate attributes were sourced from national and international literature (Kilner, 2004a, 2004b; O’Neil, 1993, 1998; Sivamalai, 2008; Spielman, Fulmer, Eisenberg, & Alfano, 2005; Verma, Paterson, & Medves, 2006; Walker, Osborne, Behrens, & Binet, 2007), including both paramedic and other cognate health care sources (e.g., nursing, medicine, dentistry, occupational therapy, physiotherapy, pharmacy, and physician assistants). This ensured that the attributes were sourced from a variety of aspects within the health care system, providing a widespread and relevant set of graduate attributes.

Data Analysis

The Statistical Package for the Social Sciences (SPSS; Version 17.0) was used for data storage, tabulation, and the generation of statistics. The data were also analyzed using PCA followed by Oblique Oblimin rotation. WINSTEPS (version 3.64.2) was used to complete the Rasch analysis, using the Rasch Rating Scale Model (Bond & Fox, 2007). The factor analysis was first performed and item loadings on viable factors were investigated. The identified viable factors were then subjected to analysis using the Rasch Model using Rating Scale Model (Andrich, 1978). While Rasch’s initial work surrounded the use of dichotomous data (e.g., yes and no responses), others such as Andrich (1978; The Rating Scale Model) and Masters (1982; The Partial Credit Model) have extended Rasch’s earlier work into analyzing polytomous data such as Likert-type rating and ordinal data (Bond & Fox, 2007; McAllister, 2008).

The Rasch analyses investigated three aspects of the PGAS: (a) item fit, (b) unidimensionality, and (c) DIF.

Item Fit

Item fit evaluates how the items fit the Rasch Model expectations. This is commonly measured using the mean-square residual fit statistic (MnSq). Item fit was evaluated according to whether the MnSq were within the range 0.70–1.30 and infit standardized values (ZSTD) were within the range −2.0 to 2.0. Misfitting items were discarded following an iterative process, as the removal of one item alters where the other items sit within the model (e.g., the removal of a difficult item makes, by comparison, the easier items appear less easy in the model). This process was continued until the best Rasch Model fit was achieved. High- or low-fit statistics represent abnormalities in the response pattern to the item that may be related to a lack of unidimensionality, DIF, poorly placed items in terms of developmental sequencing, or poorly worded items (Wright & Linacre, 1989). Item fit is also one step in the confirmation of unidimensionality.

Dimensionality

A scale is unidimensional if its items fit together to measure one latent attribute or dimension (Bond & Fox, 2007). Unidimensionality is important to understand as it means that other, unexpected latent dimensions are not confounding the results. To further explore the unidimensionality of the PGAS, a Rasch-residual-based PCA was completed. These PCAs are indicative, not definitive about secondary conditions and the results vary from conventional factor analysis (Linacre, 2010). At a minimum, 50% of the variation should be explained by the measures. Moreover, first contrasts that explain more than 5% of the variance or have eigenvalues greater than 2 typically indicate the presence of multiple dimensions and associations between data (Wuang, Lin, & Su, 2009).

DIF

DIF means the items that comprise a scale are not dependent on the population that the scale has been administered to (Embretson & Reise, 2000; Tennant & Conaghan, 2007). That is, if one group of participants do not understand the question in the same way as another group of participants (Embretson & Reise, 2000). DIF exists if contrast >0.50 and p < .001. In this study, item invariance was tested using the age, gender, and length of service variables using WINSTEPS (version 3.64.2). Where items display DIF, these items can be amended or discarded by researchers, since items are generating different success rates despite groups being at the same ability (Bond & Fox, 2007; Tennant & Pallant, 2007).

Procedures

Participants were informed about the study via an explanatory letter that was attached to the PGAS. There were no exclusion criteria. Participants were advised of the anonymous and confidential nature of the study and that they could withdraw from the study prior to submitting the questionnaire. No incentives were offered and one questionnaire format was used for all participants who agreed to take part. Questionnaires were returned by replied paid envelopes addressed to the researcher.

Results

Participant Demographics

The characteristics of the participants are described in relation to age, gender, employment status, current professional role, and length of current professional role.

The response rate was 23%. All missing data were excluded from the analysis (Peat & Barton, 2005). Of the 872 participants, almost 40% were aged between 35 and 44 years, while only 5% were less than 25 years. The ages of participants ranged from 18 to 75 years. The overall mean age was 40.71 years, with a standard deviation of 9.92. As expected the number of male participants was higher than their female counterparts, given the traditional male dominance of the paramedic workforce. Nearly three quarters of the participants were male (n = 650; 74.5%) with the remaining (n = 222; 25.5%) being female.

The majority of participants lived in New South Wales (n = 342; 39.2%), Victoria (n = 160; 18.3%), or Queensland (n =135; 15.5%). Only two participants lived outside of Australia. All states and territories were represented in the national study. Regarding where participants were employed, the vast majority of participants worked for an ambulance service (n = 807; 92.5%), followed by university undergraduates (n = 24; 2.8%) and academic staff members (n = 22; 2.5%). The majority of participants were paramedics (n = 701; 80.4%), followed by paramedic clinical instructors (n = 62; 7.1%), paramedic managers (n = 45; 5.2%). Participants were employed in their current professional role ranging from 3 months to 45 years, with a mean of 9.41 (SD = 8.48). Demographic distribution is illustrated in Table 1 .

Table 1.

Distribution of Demographic Data (n = 872)

	Number	Percent
Location
Victoria	160	18.3
Tasmania	32	3.7
New Zealand	1	0.1
ACT	26	3.0
Queensland	135	15.5
South Australia	87	10
Western Australia	72	8.3
New South Wales	342	39.2
Northern Territory	16	1.8
South Africa	1	0.1
Total	872	100
Employment status
Ambulance service	807	392.5
University	22	2.5
Hospital	7	0.8
University student	24	2.8
Defence	8	0.9
Other	4	0.5
Total	872	100
Professional role
Paramedic	701	80.4
Paramedic Manager	45	5.2
Paramedic clinical instructor	62	7.1
Academic	21	2.4
Student	32	3.7
Nursing	4	0.5
Other	7	0.8
Total	872	100

Factor Analysis Results

The data were considered suitable for factor analysis following the multiple “rule of thumbs” that included the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (0.962) and Bartlett’s Test of Sphericity, χ² = 24708.80, df = 1,081, p = .000, the sample size-to-variable ratio (N:p ratio was 18.5:1), and inspection of the correlation matrix for loadings > 0.30. Each of these was adequately met, demonstrating the data was suitable to undertake a factor analysis. Potential factors were extracted by PCA followed by Oblique Oblimin rotation using four criteria, which included Kaiser’s criteria (eigenvalue > 1 rule; Kaiser, 1960), scree test (Cattell, 1966), and cumulative percentage of variance extracted and parallel analysis (Horn, 1965). While the parallel analysis confirmed a 3-factor structure, the other three criteria all produced a 7-factor structure.

As described by Gorsuch (1983), further evaluation of the adequacy of the number of factors was performed. This was undertaken by calculating and inspecting the correlation residual matrix for high covariances. Residuals are computed between observed and reproduced correlations. Residuals of between 0.5 and 1.0 are considered moderate, while values over 1.0 are considered large, suggestive that further factors may exist (Pett, Lackey, & Sullivan, 2003). Examination of the matrix produced 4 nonredundant residuals (19.0%), with absolute values greater than 0.05 (Table 2 ). Further attempts at different factor structures did not significantly change the number of residuals, therefore a 7-factor structure was considered best fit for these data.

Table 2.

Reproduced Residual Correlation Matrix

Reproduced Correlations		Factor_1	Factor_2	Factor_3	Factor_4	Factor_5	Factor_6
Residual^b	Factor_1
	Factor_2	−.015
	Factor_3	−.045	.006
	Factor_4	−.041	.030	−.103
	Factor_5	−.049	−.032	−.035	−.042
	Factor_6	−.057	−.052	−.030	−.021	.006
	Factor_7	−.046	−.090	−.111	−.025	−.037	.022

The decision to keep the 7-factor structure was based on three aspects. First, and most importantly, the 7-factor structure were interpretable and could be theoretically and conceptually justified by the researchers. Second, the study was exploratory in nature and the 7-factor structure would undergo further examination using Rasch analysis. Third, as suggested by Gorsuch (1983), if there is doubt on the number of factors, researchers should extract more, rather than too few factors.

Analysis of the 47 items from Section 2 of the questionnaire revealed seven factors with eigenvalues above 1, accounting for 59.8% of the total variance. Items with loadings greater than ±.40, with the factor in question, were used to characterize the factor solutions. Inspection of the scree plot and eigenvalues produced a departure from linearity coinciding with a 7-factor construct. The best fit from the national data generated a 7-factor solution (see Table 3 ). Following the Oblique Oblimin rotation, the items were examined and assessed for factorial suitability. Those not loading or unable to be assigned to a factor using the above guides were discarded. The seven factors were labeled as follows:

Table 3.

Pattern and Structure Matrix for PCA With Oblimin Rotation of 7-Factor Solution of the PGAS

Item	Pattern Coefficients							Structure Coefficients							h ²
	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7
Independent team member (40)	.554							.703							.608
Oral and written communication (43)	.515							.671							.563
Use reflection (47)	.499							.663							.554
Clinical reasoning (42)	.480							.655							.660
Recognize limitations (45)	.412							.639							.595
Trustworthy (5)		.761							.783						.663
Nonjudgmental (1)		.651							.703						.572
Respectful (3)		.560							.693						.629
Accountability (7)		.549							.688						.571
Responsible for actions (11)		.530							.682						.555
Ethical and legal boundaries (26)		.527							.681						.607
Code of ethics (25)		.461							.635						.582
Clinical competence (29)		.434							.612						.573
Critical thinking (4)			.637							.757					.696
Think critically (13)			.582							.709					.650
Enquiry and research (2)			.581							.668					.522
Broader health system (23)				.452							.610				.623
Patients point of view (8)				.450							.609				.568
Culturally sensitive (12)				.445							.612				.622
Understand their role (9)				.412							.570				.580
Self development (17)					.763							.817			.714
Interdisciplinary team (18)					.664							.783			.669
Self-directed (28)					.660							.740			.598
Evidence-based practice (10)					.623							.710			.674
Responsibility for quality care (41)					.518							.817			.597
High clinical status						.733							.750		.596
Continuous improvement (34)						.702							.802		.698
Specialist knowledge (46)						.662							.660		.469
Population-based care (31)						.595							.721		.623
Social responsibility (35)						.485							.674		.632
Committed to discipline (32)						.485							.668		.627
Basic sciences (37)						.472							.665		.593
Leadership and mentoring (20)							.737							.794	.725
Good listener (16)							.584							.714	.596
Accept guidance (21)							.538							.694	.609
Counseling skills (14)							.482							.647	.529
Learn to help others (36)							.421							.626	.639
Communication ICT (19)							.400							.611	.578

Note. Boldface loadings highlight item allocation for each factor.

h ² = communality (the estimated correlation of item communality); PCA = principal components analysis; PGAS = Paramedic Graduate Attribute scale.

The seven resultant factors were descriptively labeled as indicated below. Factor 1 was given the label Personal Behavior and Attitudes. There were five items that loaded on this factor, with loadings ranging from 0.41 to 0.70 (explained variance 40.6%). The top item within the factor was “A professional paramedic should have the ability to work independently as well as a member of a team” (loading 0.70). Factor 1 included characteristics such as demonstrating clinical reasoning skills, ability to work independently as well as a member of a team, an ability to learn including the ability to use reflection and learn from experience, recognized limitations, and demonstrating effective written and oral communication skills. These components illustrated the factor labeling as Personal Behavior and Attitudes.

Factor 2 was labeled Patient Interaction and Welfare. There were eight items that loaded on this factor, with loadings ranging from 0.43 to 0.78 (explained variance 5.6%). The top item within the factor was “A professional paramedic should be trustworthy” (loading 0.78). Factor 2 included characteristics such as being trustworthy, accountable, nonjudgmental, and nondiscriminatory, adhering to a code of ethics, operating within appropriate ethical and legal boundaries, and being respectful considering moral, ethical, social, and religious aspects of health care. These characteristics emphasized the factor labeling as Patient Interaction and Welfare.

Factor 3 was given the label Scientific Approach to Patient Care. There were three items that loaded on this factor, with loadings ranging from 0.58 to 0.75 (explained variance 3.6%). The top item within the factor was “A professional paramedic should demonstrate critical thinking” (loading 0.75). Factor 3 included characteristics such as the having capacity for enquiry and research, demonstrating critical thinking, and being able to think critically. These characteristics reinforced the factor labeling as Scientific Approach to Patient Care.

Factor 4 was labeled Paramedic and Society. There were four items that loaded on this factor, with loadings ranging from 0.41 to 0.61 (explained variance 2.8%). The top item within the factor was “A professional paramedic should be culturally sensitive and have an inclusive approach to differences” (loading 0.61). Factor 4 included characteristics such as seeing things from the patients’ point of view, understanding their role in providing a primary care health service, being culturally sensitive and have an inclusive approach to differences, and understanding the role of paramedic care within the broader health care system. These features emphasized the factor labeling as Paramedic and Society.

Factor 5 was labeled Commitment to Professional and Health Care Outcomes. There were five items that loaded on this factor, with loadings ranging from 0.51 to 0.81 (explained variance 2.4%). The top item within the factor was “A professional paramedic should be able to take responsibility for quality of care and health outcomes” (loading 0.81). Factor 5 included characteristics such as following evidence-based practice, capacity to undertake self-directed approaches to learning, and being committed to self-development. These items reinforced the factor labeling as Commitment to Professional and Health Care Outcomes.

Factor 6 was labeled Professional Behavior. There were seven items that loaded on this factor, with loadings ranging from 0.47 to 0.80 (explained variance 2.4%). The top item within the factor was “A professional paramedic should contribute to continuous improvement within the healthcare system” (loading 0.80). Factor 6 included characteristics such as committed to the discipline, attaining high clinical status within the health care system, and contributing to continuous improvement within the health care system. These characteristics reinforced the factor labeling as Professional Behavior.

Factor 7 was labeled Interaction Skills. There were six items that loaded on this factor, with loadings ranging from 0.40 to 0.79 (explained variance 2.1%). The top item within the factor was “A professional paramedic should be able to provide leadership, mentoring, and supervision skills” (loading 0.79). Factor 7 included characteristics such as providing leadership, mentoring, and supervision skills, being a good listener, and accepting guidance from colleagues. These characteristics reinforced the factor labeling as Interaction Skills.

Cronbach’s ά coefficients were used to assess the internal consistency of each of the seven factors identified from the PCA. Five of the seven factors produced Cronbach’s ά coefficients above .7, indicating good internal consistency, particularly for educational and psychological instruments in the context of exploratory research (Hair et al., 1995; factor 1: 0.83; factor 2: 0.87; factor 3: 0.74; factor 4: 0.78; factor 5: 0.82; factor 6: 0.86; and factor 7: 0.83).

Pearson reliability coefficient was used to assess correlations between each of the 7 factors and ranged between r = .51 and r = .72. Although the size of the correlation coefficient should be analyzed with care (Anthony, 1999), these findings suggested that the inter-factor correlation coefficients ranged from moderate to strong (Pett et al., 2003). The subsequent Rasch analyses will be based on these seven factors.

Rasch Analysis Results

The complete item fit analysis for each of the seven factors is presented in Table 4 . In analyzing Factor 1, Item 45 “A professional paramedic should recognizing their limitations” showed misfit (MnSq = 1.38, ZSTD = 6.2 and age and gender DIF) and was ultimately removed. No further item misfit was identified following removal of item 45, resulting in good fit to the model. Analysis of Factor 2, Item 29 “A professional paramedic should have clinical competence” demonstrated misfit (ZSTD=2.2 and age and gender DIF) and also Item 7 “A professional paramedic should have accountability and confidentiality” (gender and length of service DIF). Rasch analyses of Factors 3–5 did not produce any item misfit or item redundancy, with all remaining items falling between the MnSq value of 0.7 and 1.30. Factor 6 demonstrated misfit in three items: Item 46 “A professional paramedic should have specialist knowledge” (MnSq = 1.51, ZSTD = 8.5, and gender and length of service DIF), Item 30 (high clinical status; MnSq = 1.40, ZSTD = 6.6), and Item 37 “A professional paramedic should be able to apply knowledge of the basic sciences” (ZSTD = 2.1 and age DIF) each of these items were ultimately removed, resulting in good fit to the model. Lastly, in analyzing Factor 7, Item 19 “A professional paramedic should be able to communicate using ICT” showed misfit (MnSq = 1.48, ZSTD = 8.6, and gender DIF) and was ultimately removed. These findings suggest the 7-factor structure met the requirements of the Rasch Model and add support that the newly developed sub-scales demonstrate construct validity.

Table 4.

Rasch Analysis (Factors 1–7)

Item	Measure	InfitMnSq	Infit ZSTD	Outfit MnSq	Outfit ZSTD	DIF Age	DIF Gender	DIF Length of Service
Factor 1
Recognize limitations (45)	−.48	1.38	6.2	1.23	4.7	√	√
Independent team member (40)	.66	1.12	2.2	1.01	.1
Oral and written communication (43)	−1.06	1.12	1.8	1.05	.4
Use reflection (47)	.61	1.04	.7	.95	−.3
Clinical reasoning (42)	−.20	.74	−5.2	.58	−3.3
Factor 2
Clinical competence (29)	−.85	1.16	2.2	1.18	1.5	√		√
Accountability (7)	−.09	1.12	1.7	1.01	.1		√	√
Responsible for actions (11)	−3.1	1.09	1.3	1.05	.5
Nonjudgmental (1)	.99	1.05	.9	1.05	.6
Code of ethics (25)	.49	1.00	.0	.90	−1.0
Respectful (3)	.36	.99	−.2	.94	−.6
Trustworthy (5)	−1.59	.88	−1.4	.93	−.4		√
Ethical and legal boundaries (26)	.07	.83	−2.8	.72	−3.0
Factor 3
Critical thinking (4)	1.75	.95	−.7	1.14	3.7
Think critically (13)	−.63	1.00	.1	.87	−1.1
Enquiry and research (2)	−1.12	.81	−2.3	.80	−1.7
Factor 4
Broader health system (23)	.08	1.07	1.2	1.01	.2
Patients point of view (8)	.55	1.07	1.1	.93	−.9
Culturally sensitive (12)	.41	.99	−.2	.92	−.9
Understand their role (9)	−1.04	.80	−3.4	.75	−3.1
Factor 5
Evidence-based practice (10)	.10	1.33	4.4	1.34	4.4			√
Self-directed (28)	.44	1.17	1.9	1.34	3.5			√
Responsibility for quality care (41)	.04	1.05	.7	.95	−.5		√
Self-development (17)	−.47	.89	−1.5	.80	−2.4		√
Interdisciplinary team (18)	−0.1	.85	−2.1	.84	−1.8
Factor 6
Specialist knowledge (46)	.91	1.51	8.5	1.60	9.9		√	√
High clinical status (30)	−.52	1.40	6.6	1.30	4.7
Basic sciences (37)	−.90	1.12	2.1	1.07	1.1	√
Social responsibility (35)	−.04	1.05	.9	.99	−.1
Continuous improvement (34)	−.59	1.03	.5	.95	−.8
Committed to discipline (32)	−.69	1.00	.1	.92	−1.2
Population-based care (31)	1.33	.95	−1.0	.98	−.3
Factor 7
Communication ICT (19)	.87	1.48	86	1.47	8/7		√
Leadership and mentoring (20)	−5.3	1.16	2.2	1.11	1.0
Good listener (16)	.89	.97	−.4	1.04	.5
Accept guidance (21)	−3.6	.82	−2.7	.73	−2.9
Counseling skills (14)	−3.7	.93	−1.2	.92	−1.1
Learn to help others (36)	−1.33	.86	−2.4	.83	−2.0

Note. Underlined values indicate that the MnSq or ZSTD values were beyond the misfitting criteria, e.g., MnSq > 1.3 and ZSTD > 2.0. DIF exists if contrast >0.50 and P < .001. Items in boldface met the misfitting criteria and were subsequently removed. √ demonstrates DIF if contrast <0.50 and P < 0.01.

Dimensionality

Following the removal of the 7 items, a PCA was performed on the residuals of the remaining 7 factors (31 items) to assess that other additional structures did not exist (local dependence). Item reliability for each factor ranged from 0.97 and variance explained by items ranged from 54.8% to 66.8%with first contrast eigenvalues ranging from 1.3 to 1.8. Overall, each factor demonstrated excellent reliability and factor unidimensionality.

Discussion

Results from the graduate attribute section provided important information regarding which graduate attributes were considered most desirable for the Australian paramedic membership. Moreover, the PGAS demonstrated a factor structure that made conceptual sense evidence of construct validity and good internal consistency; offering the Australian paramedic discipline with its first set of empirically based graduate attributes. Since formal education program accreditation and national curricula standards do not currently exist among the 11 university paramedic programs, exploring which attributes are most desirable or suitable, will provide important curriculum blueprinting and mapping structure in achieving educational consistency.

Following the initial extraction using factor analysis, the data were then examined using Oblique Oblimin rotation providing further clarity on the number of factors and evidence that the best-fit model was seven factors. A total of nine items failed to load on any factor and hence were deleted from the PGAS. While one of main objectives of factor analysis is to reduce the number of latent variables, three surprise omissions nevertheless occurred. In particular, items such as “A professional paramedic should be caring and empathic,” “A professional paramedic should be literate and numerate,” and “A professional paramedic should have theoretical knowledge” did not load on any of the seven factors. Each of these items could be considered core paramedic attributes. The only explanation for the items relating to literacy and numeracy, and having theoretical knowledge, could be the assumption that all graduates now complete undergraduate degrees, and therefore they must possess theoretical knowledge and demonstrate competence in literacy and numeracy prior to graduation. No explanation can be given for the items relating to care and empathy, since one would assume that caring and empathy would be a core personal or professional attribute for paramedics. This notion may change with a small body of work being undertaken, where investigators are examining undergraduate students’ levels of empathy across a range of health conditions and clinical situations (Boyle et al., 2010).

Least surprising was Item 44 “A professional paramedic should practice preventative health care,” since the term is often associated with social models of care, general health models, and more specifically primary health care, which paramedics have played very little part of in the past. Although as alternative models of health care are being examined, this may change as extended scope roles, such as paramedic practitioners, are being considered and trialed in some parts of Australia, such as the Isolated Practice Paramedic (Queensland Ambulance Service), and Extended Care Paramedics (New South Wales and South Australian Ambulance Services; The Council of Ambulance Authorities, 2008).

Following Rasch analysis, Item 46 “A professional paramedic should have specialist knowledge e.g. practitioner or extended scope roles,” Item 30 “A professional paramedic should have high clinical status within the healthcare system,” and Item 37 “A professional paramedic should be able to apply knowledge of the basic sciences” each misfitted the Rasch model. In addition, Item 46 also produced DIF in gender and length of service groups, and Item 37 produced DIF in the age group. Possible explanation for the poor model fit of Item 46 could be explained by the ambiguous statement offering participants two options in the item stem, thus potentially confusing or misleading participants. Given the current discourse surrounding alternative models of service delivery in the health sector, terms such as paramedic practitioner, generic worker, interprofessional collaborative practitioner, and physician assistants, it is likely that many participants do not understand what their core roles and functions are.

Regarding Item 37, one possible explanation for the DIF could be related to previous paramedic education and training, particularly related to those older than 40 years of age, who completed courses where basic sciences were not routinely taught in their paramedic curriculum. It was not until 1990–2000s when basic sciences were included in all paramedic programs during the transition to university-based education. This might suggest that those participants older than 40 years, those who were likely to have undertaken their formative education prior to 1990, do not see the importance or need to learn the basic sciences in order to improve paramedic clinical practice.

Item 29 “A professional paramedic should have clinical competence” did not fit the Rasch Model and also produced DIF in the age and length of service groups, while Item 7 “A professional paramedic should be aware of professional issues such as; accountability and confidentiality” produced DIF in both gender and length of service groups. Potential explanations for this DIF occurring are not clear, although one possible explanation could be the term competence itself, which may have connotations with the former vocational roots and links with competency-based training. Participants may have felt this terminology did not reflect the current educational state of play or future education and training needs.

Regarding Item 7, one possible explanation for the DIF finding could be connected to the alternate terms in the item stem (i.e., accountability and confidentiality), possibly misleading participants. Both of these terms have different meanings and clinical implications that are likely to have caused misfitting. One potential cause for the DIF in the length of service group could be related to when participants completed their training programs. Items such as accountability and confidentiality were not routinely included in paramedic programs prior to the transition to university-based education. This may suggest that participants do not see these items as being important graduate attributes. No clear explanation can be given for the gender DIF.

Following the factor analysis of the national study, which produced a best-fit model of 38 items and 7 subscales, its data were further analyzed using the Rasch model. A total of 7 items did not fit the Rasch Model (one in Factor 1, two in Factor 2, three in Factor 6, and one in Factor 7), reducing the total items in the PGAS to 31, while also retaining the 7 subscale constructs demonstrated by the earlier PCA.

Future Research and Implications for Practice

Based on these results, several possibilities exist for future investigation and interpretation. First, further examination of the PGAS and additional graduate attributes should be considered. Particular emphasis should be placed on item construction. Second, additional sociological research opportunities exist in measuring attitudes and perceptions from participants who have been employed for substantial periods of time in the paramedic discipline. Third, how the paramedic graduate attributes might be integrated and translated into a curriculum design framework that allows national educational consistency as well as offering institutional autonomy across the Australian paramedic university programs is warranted.

The study has limitations. The use of convenience sampling limits the generalizability of the study. This method, while easier to recruit participants, is less likely to recruit a representative sample. Those participants who do volunteer to respond may bias the results. Despite this limitation, the results between different groups have raised some interesting questions. External validity is also limited by the low response rate to the survey. Further research is required to include another sample with an alternative sampling technique, reducing the likelihood of self-selection bias.

Conclusion

The factor analysis and Rasch analysis demonstrated the PGAS was a unidimensional, reliable, and valid instrument reducing the total items to 31, underlined by 7 factors, and offers the Australian paramedic discipline with a set of empirically based graduate attributes. This instrument provides the discipline with a potential graduate attribute and curriculum blueprint for paramedic course accreditation and curriculum designers. It importantly offers the discipline with an opportunity to standardize its curriculum and maintains academic standards, thus strengthening its claim to become a legitimate health care profession.

Footnotes

BW, AO, and TB conceived the study, developed the questionnaire, compiled the data file, and participated in writing the manuscript. BW assisted in developing the questionnaire and undertook the data analysis.

Acknowledgment

We wish to thank the participants who contributed to this study.

The author(s) declared no conflicts of interest with respect to the authorship and/or publication of this article.

The author(s) received no financial support for the research and/or authorship of this article.

References

Andrich

(1978). Scaling attitude items constructed and scored in the Likert tradition. Educational and Psychological Measurement, 38(3):665–680.

Andrich

(2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(suppl 1):1–16.

Anthony

(1999). Understanding advanced statistics: A guide for nurses and health care researchers London: Churchill Livingstone.

Australian Health Practitioner Regulation Agency. (2010). National registration and accreditation scheme update Melbourne: Australian Health Practitioner Regulation Agency.

Bond

Fox

(2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). New Jersey: Lawrence Erlbaum Associates:

Boyle

Williams

Brown

Molloy

McKenna

Molloy

Lewis

(2010). Levels of empathy in undergraduate health science students. Internet Journal of Medical Education, 1(1):71.

Cattell

R. B.

(1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2):245–276.

Embretson

S. E.

Reise

S. P.

(2000). Item response theory for psychologists New Jersey: Lawrence Erlbaum Associates Inc.

Gorsuch

R. L.

(1983). Factor analysis Hillsdale, NJ: Erlbaum.

10.

Hair

Anderson

R. E.

Tatham

R. L.

Black

W. C.

(1995). Multivariate data analysis (4th ed.). New Jersey: Prentice-Hall Inc.

11.

Henson

R. K.

Roberts

J. K.

(2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3):393–416.

12.

Hogarty

Hines

Kromrey

Ferron

Mumford

(2005). The quality of factor solutions in exploratory factor analysis: The influence of sample size, communality, and overdetermination. Educational and Psychological Measurement, 65(2):202–226.

13.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2):179–185.

14.

Kaiser

H. F.

(1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.

15.

Kilner

(2004a). Desirable attributes of the ambulance technician, paramedic, and clinical supervisor: findings from a Delphi study. Emergency Medicine Journal, 21(4):374–378.

16.

Kilner

(2004b). Educating the ambulance technician, paramedic, and clinical supervisor: using factor analysis to inform the curriculum. Emergency Medicine Journal, 21(4):379–385.

17.

Linacre

J. M.

(2010). User’s guide to Winsteps Ministep Rasch-Model Computer Programs. Retrieved 5 August 2010, from http://www.winsteps.com/winman/

18.

Masters

G. N.

(1982). A Rasch model for partial credit scoring. Psychometrika, 47(2):149–174.

19.

McAllister

(2008). Introduction to the use of Rasch analysis to assess patient performance. International Journal of Therapy and Rehabilitation, 15(11):482–490.

20.

O’Meara

Walker

Stirling

Pedler

Tourle

Davis

Jennings

(2006). The rural and regional paramedic: Moving beyond emergency response Bathurst: The Council of Ambulance Authorities.

21.

O’Neil

E. H.

(1993). Health professions education for the future: Schools in service to the nation San Francisco, CA: The Pew Health Professions Commission.

22.

O’Neil

E. H.

(1998). Recreating health professional practice for a new century San Francisco, CA: The Pew Health Professions Commission.

23.

Peat

Barton

(2005). Medical statistics: A guide to data analysis and critical appraisal Oxford: Blackwell Publishing.

24.

Pett

M. A.

Lackey

N. R.

Sullivan

J. J.

(2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research California: SAGE.

25.

Sivamalai

(2008). Desired attributes of new graduate nurses as identified by the rural community. Rural and Remote Health, 8, 938.

26.

Spearman

(1904). “General intelligence”, objectively determined and measured. American Journal of Psychology, 15, 201–293.

27.

Spielman

A. I.

Fulmer

Eisenberg

E. S.

Alfano

M. C.

(2005). Dentistry, Nursing, and Medicine: A Comparison of Core Competencies. Journal of Dental Education, 69(11):1257–1271.

28.

Swisher

L. L.

Beckstead

J. W.

Bebeau

M. J.

(2004). Factor analysis as a tool for survey analysis using a professional role orientation inventory as an example. Physical Therapy, 84(9):784–799.

29.

Tennant

Conaghan

(2007). The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis & Rheumatism, 57(8):1358–1362.

30.

Tennant

Pallant

(2007). DIF matters: A practical approach to test if differential item functioning makes a difference. Rasch Measurement Transactions, 20(4):1082–1084.

31.

The Council of Ambulance Authorities. (2008). Expanding roles: An Australasian overview of emerging paramedic models of care Flinders Park: The Council of Ambulance Authorities.

32.

Thompson

(2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications Washington, DC: American Psychological Association.

33.

Verma

Paterson

Medves

(2006). Core competencies for health care professionals. Journal of Allied Health, 35(2):109–115.

34.

Walker

Osborne

Behrens

Binet

(2007). Guidelines for the assessment and accreditation of university paramedic programs (version 2.0) Adelaide: Council of Ambulance Authorities.

35.

Williams

Onsman

Brown

(2010a). Is the Australian paramedic discipline a full profession? Journal of Primary Emergency Health Care, 8(1):23.

36.

Williams

Onsman

Brown

(2010b). Australian paramedic graduate attributes: A pilot study using exploratory factor analysis. Emergency Medicine Journal, 27(10):794–799.

37.

Wright

B. D.

Linacre

J. M.

(1989). Observations are always ordinal; measurements, however, must be interval. Archives of Physical Medicine and Rehabilitation, 70(12):857–860.

38.

Wuang

Lin

(2009). Rasch analysis of the Bruininks-Oseretsky test of motor proficiency-second edition in intellecutal disabilities. Research in Developmental Disabilities, 30(6):1132–1144.