Cross-cultural validation of the Prosthesis Evaluation Questionnaire in vascular amputees fitted with prostheses in Spain

Abstract

Background:

The lack of specific prosthetic-related outcome instruments for Spanish amputees must be addressed.

Objective:

To elaborate a culturally equivalent version of the Prosthesis Evaluation Questionnaire in the Spanish language.

Study design:

Cross-cultural questionnaire validation.

Methods:

Two-step process for cultural adaptation: forward and backward translations of English original and Spanish translated versions; assessment of both construct and criterion validity and reliability in a group of vascular amputees.

Results:

A total of 61 patients were recruited, 44 men (72.1%) and 17 women (27.9%), with a median age of 71.1 years (standard deviation: 7.7 years; range: 51–87 years). In the Prosthesis Evaluation Questionnaire–Spanish, the lowest scores were for gait and frustration, and the highest scores were for noise and stump health. Internal consistency of the questionnaire was acceptable (>0.70) for four of the scales used in the Prosthesis Evaluation Questionnaire but poor (<0.50) for the scales relating to appearance and stump health. Correlations with the quality-of-life levels as measured by the Short Form–36 were positive and mostly significant.

Conclusion:

Prosthesis Evaluation Questionnaire–Spanish could assess the quality of life in patients who have undergone vascular amputations and then been fitted with a prosthetic limb. The questionnaire shows adequate criteria validity when compared with other instruments for measuring quality of life.

Clinical relevance

The Prosthesis Evaluation Questionnaire–Spanish could be a valid and reliable instrument for assessing adaptation to prostheses in vascular amputees. The questionnaire adds information relevant to the patient and the physician and may identify cases with poor expected adaptation to the prosthesis.

Keywords

Rehabilitation of prostheses users rehabilitation psychology

Background

Amputation is associated with greatly increased morbidity and mortality rates.^1,2 The majority of amputations are carried out on patients with peripheral arterial disease, a chronic illness whose most important risk factors include male sex, advanced age, smoking, coronary disease, high blood pressure, dyslipidemia, and diabetes.³ In the developed world, amputations with a vascular origin constitute approximately 80% of all amputation procedures performed.⁴

In total, 15% of amputees die within the first year after the procedure. With regard to survivors, although 70% reportedly return home and are living independently within 1 year of the procedure, their functional level is low^5,6 since very few patients display the clinical conditions necessary for regaining a normal level of autonomy.^4,5 In this process, the fitting of a prosthetic limb and patient acceptance of the prosthesis both play an important role.

The most common barriers to fitting a patient with a prosthesis have been identified as death of the patient, reamputation, cerebrovascular diseases, and cognitive deficiencies,⁷ along with perceived pain in the ghost limb.⁸

To date, most of the tools developed for assessing prosthesis use have been written in English for use in countries within the cultural context of the English-speaking world.⁹ Given the universal nature of the problem, however, these tools need to be adapted for use in diverse countries with different languages and cultural norms.¹⁰

Recently, two questionnaires have been developed for assessing both prosthesis use and the mobility of patients who have undergone lower limb amputation, namely, the Prosthetic Profile of the Amputee (PPA)^11–13 and the Prosthesis Evaluation Questionnaire (PEQ).^14,15

The PPA contemplates most of the disability levels established by the World Health Organization, with an additional, exhaustive section on prosthetic and orthotic categories. Its major disadvantage is that it is cumbersome to administer. Together with the Day Scale, it alone includes how frequently the prosthesis is actually used by the patient.

The PEQ was developed by Legro et al.¹⁵ in 1998 to evaluate patient prosthesis use, the quality of life related to having a prosthesis, and other associated categories. It was specifically designed for use in patients who had undergone lower limb amputation. The PEQ is a self-assessment tool composed of nine validated scales: ambulation, appearance, frustration, perceived response, residual limb, social burden, noise, usefulness, and ability to bear weight.

The goal of this study is to elaborate a culturally equivalent version of the PEQ in the Spanish language for use with Spanish amputees who have been fitted with a prosthesis. To do so, we have evaluated the internal consistency, the criteria validity, and the reliability of the translated PEQ.

Methods

This is a wholly descriptive and exploratory study in that it marks the first time that the PEQ has been used on a representative sample of vascular amputees in Spain. The stages used in adapting the English version to Spanish were as follows: translation and back-translation stages. We followed the recommendations of Guillemin et al.¹⁶ for carrying out a cross-cultural adaptation of the dimensions related to quality of life.

Translator selection

First phase (translation from English to Spanish):

Two bilingual translators (Spanish and English), both native speakers of Spanish, were chosen.

The translators were unaware of the objective of this study. Although they were familiar with medical and health-related terminology, they were not part of the research group.

Second phase (back-translation from Spanish to English):

Two bilingual translators (English and Spanish), native speakers of US English, were chosen;

The translators were unaware of the goal of this study and were unfamiliar with medical and health-related terminology.

The translators worked independently, aware that they were to translate the text freely rather than literally in order to guarantee a semantic and conceptual equivalent to the original text.

Objectives and constitution of the review committee for the various translations

A committee comprising four experts in the field was established, with special importance given to bilingual members. The items which had been rated more difficult to translate and those for which no consensus was reached with regard to their semantic and conceptual equivalence were discussed with the original author of the PEQ. After this revision phase, a first draft of the PEQ-Spanish was obtained and labeled “version 1.0.”

Assessment of the validity and reliability on a sample group of patients

The final version of the questionnaire was administered to a group of patients selected on the basis of the following criteria:

Patients who had undergone a vascular amputation and who had been fitted with a prosthesis;

Patients over 18 years of age;

Patients who had given their informed consent to participate in the study;

Patients with their cognitive faculties intact;

Patients with no severe visual impairments;

Patients for whom there was a minimum amount of administrative data for categorizing each case.

Recruitment centers

Participants were recruited from the Rehabilitaion Unit of the Xativa and Ontinyent Hospitals, both in the province of Valencia.

The sample comprised 61 patients (44 men and 17 women). The procedure was carried out as follows:

Recruitment appointment:

Explanation of the project and obtaining of informed consent;

Collection of sociodemographic data as well as data on the type of pathology, stage, and so on.

First physical therapy (PT) appointment:

Administration of the PEQ-Spanish version 1.0. Most questions on the PEQ use a visual analog scale (VAS) format, which is scored as a continuous numeric variable that measures the distance in millimeters from the left end point of the line. To rate an item, the patient must make a mark on any point of the line. This format is neither common nor easily understandable for most patients, especially the elderly and those with a lower educational background. This is a clear disadvantage since a questionnaire should be easy to complete without help. For these reasons, EVA of version 1.0 was replaced with a numerical rating from 0 to 10, where 0 was “is not able” and 10 was “no problem”;

Administration of other scales to assess the construct validity and convergences (the Short Form–36 (SF-36) and Yesavage questionnaires);

Collection of data from the physical examination and psychological testing carried out on the patient.

Assessment of reliability or internal consistency

The internal reliability of the questionnaire was assessed by calculating Cronbach’s alpha coefficient, with a coefficient above 0.8 indicating an adequate level of internal consistency.

Ethical approval

This study was approved by the Clinical Research Ethics Committee at our hospital prior to initiating data collection.

Analysis strategy

The associations between the various scales and dimensions (PEQ, SF-36, and Yesavage) were assessed with Spearman’s ρ coefficient for non-parametric correlations. The level of significance was established as p < 0.05. The data obtained were analyzed with the aid of the PASW 18.0 software package (SPSS, Inc.).

Results

In all, 61 patients were recruited, 44 men (72.1%) and 17 women (27.9%), with a median age of 71.1 years (standard deviation (SD): 7.7 years; range: 51–87 years).

Of the total number of study participants, two-thirds lived in rural areas (an urban area was considered to be a town with more than 30,000 inhabitants). Most of the recruited patients were retired and had only received a primary school education. Moreover, although more than half had not been born in their current town of residence, the majority said that they both understood and spoke the regional language, Valencian (Table 1).

Table 1.

Description of sociodemographic variables.

	N (%)
Residence
Urban	20 (32.8%)
Rural	41 (67.2%)
Educational level
No formal schooling	13 (21.3%)
Primary school	40 (65.6%)
Secondary school	8 (13.1%)
Work situation
Working	4 (6.6%)
Homemaker	12 (19.7%)
Retired	45 (73.8%)
Valencian speaker
No	16 (26.2%)
Yes	45 (73.8%)
Residence same as place of birth
No	34 (55.7%)
Yes	27 (44.3%)

The predominant underlying disease was diabetes mellitus (78%), followed by arterial ischemia (20.3%). The level of amputation was mostly transfemoral (91.4%). The SF-36 questionnaire showed generally high levels, especially in Role Physical, Social Function, and Role Emotional (Table 2).

Table 2.

Descriptive statistics of the dimensions of the SF-36 questionnaire.

Dimension	Minimum	25th percentile	Median	75th percentile	Maximum
Role Physical	0	0	50	100	100
General Health	5	30	45	65	90
Vitality	15	40	50	65	85
Social Function	0	50	63	75	100
Role Emotional	0	0	67	100	100
Mental Health	16	52	64	76	96

SF-36: Short Form–36.

The average and mean scores obtained with the scale for geriatric depression were 12 points, which falls within the category of “probable depression,” with values ranging between 1 and 22 points. Table 3 summarizes the nine scales included in the PEQ; the lowest scores were for the scales referring to gait and frustration, whereas the highest scores were for noise and residual limb. The internal consistency of the questionnaire was acceptable (>0.70) for four of the scales used in the PEQ but poor (<0.50) for the scales relating to appearance and residual limb (Table 4).

Table 3.

Descriptive statistics of PEQ scales.

PEQ scales	Minimum	25th percentile	Median	75th percentile	Maximum
Utility	1.0	5.4	6.5	7.8	8.8
Appearance	5.4	7.6	8.6	9.1	10.0
Sounds	4.5	6.8	10.0	10.0	10.0
Residual limb	4.3	8.2	9.2	9.7	10.0
Perceived appearance	0.0	6.8	8.3	9.5	10.0
Weight bearing	0.0	4.6	6.3	7.5	10.0
Frustration	0.0	4.3	6.0	9.0	10.0
Social burden	0.0	4.6	7.0	8.3	10.0
Ambulation	0.0	2.9	5.5	7.8	9.8

PEQ: Prosthesis Evaluation Questionnaire.

Table 4.

Analysis of the internal consistency of PEQ scales.

PEQ scales	Items	Cronbach’s alpha
Utility	8	0.807
Appearance	5	0.371
Sounds	2	0.552
Residual limb	6	0.152
Perceived appearance	4	0.885
Weight bearing	2	0.579
Frustration	2	0.867
Social burden	3	0.590
Ambulation	8	0.932

PEQ: Prosthesis Evaluation Questionnaire.

The correlations of the quality-of-life levels according to the SF-36 and the corresponding scales in the PEQ were positive and mostly significant, especially with regard to the scales of utility, ability to bear weight, frustration, social burden, and ambulation (Table 5).

Table 5.

Non-parametric correlations between dimensions of the PEQ and the SF-36 questionnaire.

PEQ scales	SF-36
PEQ scales	Role Physical	General Health	Vitality	Social Function	Role Emotional	Mental Health
Utility	0.547**	0.657**	0.625**	0.668**	0.475**	0.654**
Appearance	0.204	−0.008	0.076	0.202	0.103	0.069
Sounds	0.323*	0.333*	0.373*	0.319*	0.206	0.271
Residual limb	0.326*	0.303	0.261	0.375*	0.092	0.237
Perceived appearance	0.158	0.228	0.257	0.227	0.212	0.194
Weight bearing	0.423**	0.818**	0.832**	0.791**	0.693**	0.782**
Frustration	0.460**	0.601**	0.544**	0.662**	0.579**	0.481**
Social burden	0.495**	0.619**	0.546**	0.730**	0.472**	0.593**
Ambulation	0.544**	0.713**	0.727**	0.782**	0.571**	0.671**

PEQ: Prosthesis Evaluation Questionnaire; SF-36: Short Form–36.

p < 0.05; **p < 0.001; Bold indicate correlations with a ρ > 0.50.

Negative correlations were obtained between the SF-36/PEQ and the Yesavage scale, that is, the higher the scores for quality of life and adaptation to the prosthetic limb, the lower the levels of depression (Table 6). Unfortunately, the test–retest reliability of the questionnaire could not be assessed as it was impossible to re-administer the questionnaire after 3 months.

Table 6.

Non-parametric correlations between dimensions of the PEQ and the SF-36 questionnaires with the Yesavage geriatric depression scale.

		Yesavage
SF-36	Role Physical	−0.499**
	General Health	−0.885**
	Vitality	−0.873**
	Social Function	−0.791**
	Role Emotional	−0.767**
	Mental Health	−0.874**
PEQ	Utility	−0.689**
	Appearance	−0.132
	Sounds	−0.413**
	Residual limb	−0.353*
	Perceived appearance	−0.223
	Weight bearing	−0.841**
	Frustration	−0.642**
	Social burden	−0.612**
	Ambulation	−0.757**

PEQ: Prosthesis Evaluation Questionnaire; SF-36: Short Form–36.

p < 0.05; **p < 0.001.

Discussion

With this study, we have elaborated a culturally equivalent questionnaire to the PEQ to assess the quality of life in patients who have undergone vascular amputations and then been fitted with a prosthetic limb. The questionnaire shows adequate criteria validity when compared with other instruments for measuring quality of life, such as the SF-36 and the Yesavage scale for geriatric depression.

This is the first study to validate a questionnaire of this type in Spanish with the author’s permission. To date, similar validation studies have only been carried out in two European countries—Italy¹⁷ and the Netherlands¹⁸—and recently on an Arabic-speaking population.¹⁹

All four studies used the same questionnaire, either for its derivation or creation¹⁵ or for its validation.^17–19 Nevertheless, some methodological differences between the studies deserve mention, especially with regard to the type of patients included, the tools or scales used to compare patients, the number of patients recruited, and the manner in which data were collected and field work was carried out.

Selection criteria used

Apart from choosing adult patients with sufficient cognitive capacity to answer the items on the questionnaire, there are more differences than similarities between the four studies with regard to subject selection. For example, our study and the study conducted in the Netherlands both focused on vascular amputees. The American and Italian studies were carried out with substantially younger populations, with the latter focusing on patients between 20 and 28 years of age. It is thus difficult to compare the results of the various studies, as they represent different study populations and underlying diseases.

The pathologies or comorbidities associated with the need for amputation were similar in all studies, with a predominance of diabetes mellitus and peripheral vascular disease. These types of pathologies were also the main reason for performing a transfemoral amputation in our study. In contrast, in the study carried out in the United States, almost two-thirds of the amputations were transtibial, due to the fact that most of the cases were trauma-related. The validation of the Arabic version of the questionnaire was also carried out on amputees, although the reason for the amputation was not specified.

With regard to distribution by sex, all four studies showed a clear predominance of men over women, with the difference ranging from 65.2% in the Dutch cohort up to 85.9% in Legro and colleagues’ initial study.

Sample sizes

The Italian study was by far the largest, with 123 patients, followed by Legro and colleagues’ study, which included 92 subjects. Our study, with 61 patients, had the third largest sample size, while the Dutch and Arabic studies only examined 23 and 7 patients, respectively. With the exception of the Arabic study, all these sample sizes are sufficiently large to test the principal hypotheses put forth in each study; however, they lose statistical power when subgroups are compared. In our study, this may explain the observed lack of significant differences in the levels of adaptation to the prosthetic limb between various sociodemographic and cultural variables.

PEQ scales used/validated

The original study validated nine scales: four on prosthesis function (utility, residual limb, appearance, and noise), one on mobility (ambulation), three on psychosocial issues (perceived response, frustration, and social burden), and one on weight bearing. The Dutch study only used, but did not validate, the mobility scale, while the Italian study only used the scales concerning ambulation and transference questions (PEQ-MS12). Our study and the Arabic version were the only ones to fully adapt and validate the PEQ in a language other than English.

Instruments used in criteria validation

In their original study, Legro et al. used several instruments for validating the PEQ, including the SF-36, currently one of the most well-established scales for assessing a patient’s general state of health. She also used the Sickness Impact Profile (SIP) as a tool to measure the general state of health as one of its domains concerns social interaction, as well as the Profile of Mood States–short form (POMS-sf.), which evaluates patient mood.

The authors of the Dutch study used the Locomotor Capabilities Index (LCI; a subscale of the PPA) and the Timed Up and Go test (TUGT). In a similar fashion, the Italian study compared the values from the PEQ with those from the PPA and the LCI, which is actually a scale of the PPA. In our study, we preferred to utilize validation instruments that were as similar as possible to those used in the original study in order to facilitate comparisons. The Yesavage scale was added because our study population consisted almost solely of geriatric subjects, and this questionnaire has demonstrated its usefulness in assessing elderly patients.^20–22

Data collection

The time between carrying out the amputation and conducting the study was fairly heterogeneous among the four studies, from a mean of 12 years in the Dutch study to 38 years in the Italian study and with a wide range of between 1 and 53 years in the American study. In our study, the time between amputation and performing the study was much less; in 50% of the cases, the mean time elapsed was 21 months, while the range was between 6 and 42 months. There were also more subtle reasons for the differences in the various study results, commented on in greater detail below.

In the Dutch study, the functional use of a prosthetic limb 1 year after transtibial amputation was found to be associated with stump health and quality. A tibia length of 12–15 cm from the joint line of the knee correlated with better functionality than shorter tibia lengths. These results underscore the need for meticulous surgical technique and special attention to stump length to ensure good functional results after a transtibial amputation.

The Italian study showed high internal reliability in the test–retest process as well as convergent validity. The results indicate that the new PEQ-MS12/5 scale has psychometric properties which compare favorably with those included in the original PEQ-MS, despite the removal of one item and the simplification of the assessment scale. As expected, several of the threshold values of the 11 categories included in the PEQ-MS showed some alterations and had to be restructured in order to meet the pre-established quality criteria for the evaluation categories. The altered threshold values suggest that the questionnaire is unable to distinguish the subjects’ capabilities as precisely as suggested by the 11 response categories. Proper restructuring typically improves the mean, eliminating the repetition of underutilized assessment classification categories and ensuring that each category represents a different level of capability as compared with the next adjacent level. Thus, one item on the PEQ-MS (“Shower/Bath”) was found to be “unsuitable,” that is, inadequately related to the underlying measurement performed. This mismatch could be due, among other reasons, to the fact that it formed part of another, poorly defined construct or one that was overly sensitive to confounding variables. It was, therefore, proposed that this item be eliminated, both for statistical reasons and for content, since it is a construct that is sensitive to both cultural and environmental factors.

Finally, the Arabic validation, performed on seven patients, showed a high concordance with original English version. Unfortunately, the authors did not test either the reliability or validity of the scale.

Our study has several limitations that deserve comment. Because there were significant difficulties in administering the questionnaire in its original format, we had to make several adjustments, including changing the analog format of the scale to a numerical one and opting for an oral administration of the questionnaire. These changes were driven by the fact that most patients in our study population suffered from poor eyesight and/or did not have the necessary reading comprehension skills. There were also difficulties in having the patients fill out the questionnaires as some patients were functionally illiterate or had serious difficulty in reading the forms.

Of course, oral administration also presents several problems. Thus, in order to minimize bias, we used specially trained interviewers who made certain to use the same methodology for all patients when administering the questionnaire.

The internal consistency was moderate for several dimensions of the questionnaire. A low value for a given index may be due to the small number of items included (some dimensions only included two or three items) or to a true interrelation between items in a more heterogeneous construct.²³ Our results should thus serve as a guide to simplify and condense subsequent versions of this particular questionnaire. Although our results provide valuable information, we feel that improvements can be made in future studies in this field.

First, it would be useful to take into account a wider array of motives and underlying diseases leading to amputation and the subsequent fitting of a prosthesis. In this way, the impact of other explanatory variables could be assessed, including age (extending the range), sex (including pathologies that are not so prevalent in men), and level of amputation (which is surely a determining factor in adaptation to the prosthesis), among others.

Such studies should also be extended to other health areas and cultural environments within Spain by carrying out multicenter studies to include patients from other regions of the country. This would increase the statistical power of the results and rule out chance as a plausible explanation for the associations observed.

Conclusion

The PEQ-Spanish could be useful in assessing the quality of life in Spanish-speaking patients who have undergone amputations, above all of vascular origin, and then been fitted with a prosthetic limb. The questionnaire shows adequate criteria validity when compared with other instruments for measuring quality of life.

Footnotes

Author contribution

All authors contributed equally in the preparation of this manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Ministerio de Sanidad. Servicios Sociales e Igualdad—Portal Estadístico del SNS—Sistema de Información Sanitaria: Portal Estadístico del SNS—Portal Estadístico del SNS, http://www.msps.es/estadEstudios/estadisticas/sisInfSanSNS/home.htm (accessed 31 January 2013).

Haimovici

. Failed grafts and level of amputation. J Vasc Surg 1985; 2(3): 371–374.

Veith

Gupta

Samson

. Progress in limb salvage by reconstructive arterial surgery combined with new or improved adjunctive procedures. Ann Surg 1981; 194(4): 386–401.

Riol Blanco

. Apuntes sobre la rehabilitación del amputado por afección vascular. Rev Cubana Cir 1990; 29(2): 350–356.

Schoppen

Boonstra

Groothoff

. Physical, mental, and social predictors of functional outcome in unilateral lower-limb amputees. Arch Phys Med Rehab 2003; 84(6): 803–811.

Pernot

Winnubst

Cluitmans

. Amputees in Limburg: incidence, morbidity and mortality, prosthetic supply, care utilization and functional level after one year. Prosthet Orthot Int 2000; 24(2): 90–96.

Fletcher

Andrews

Butters

. Rehabilitation of the geriatric vascular amputee patient: a population-based study. Arch Phys Med Rehab 2001; 82(6): 776–779.

Finsen

Persen

Løvlien

. Transcutaneous electrical nerve stimulation after major amputation. J Bone Joint Surg Br 1988; 70(1): 109–112.

García Obrero

Echevarría

Sánchez Navarro

. Escalas de valoración funcional en el paciente amputado. Rehabilitación 1998; 32(2): 113–125.

10.

Echevarría Ruiz de Vargas

García Díaz

Zarco Periñan

. Adaptación de los instrumentos lingüísticos de medida de salud en rehabilitación. Rehabilitación 2000; 34(1): 91–95.

11.

Miller

Deathe

Speechley

. Lower extremity prosthetic mobility: a comparison of 3 self-report scales. Arch Phys Med Rehab 2001; 82(10): 1432–1440.

12.

Miller

Deathe

Speechley

. Psychometric properties of the Activities-Specific Balance Confidence Scale among individuals with a lower-limb amputation. Arch Phys Med Rehab 2003; 84(5): 656–661.

13.

Miller

Deathe

Harris

. Measurement properties of the Frenchay Activities Index among individuals with a lower limb amputation. Clin Rehab 2004; 18(4): 414–422.

14.

Gauthier-Gagnon

Grisé

. Prosthetic profile of the amputee questionnaire: validity and reliability. Arch Phys Med Rehab 1994; 75(12): 1309–1314.

15.

Legro

Reiber

Smith

. Prosthesis evaluation questionnaire for persons with lower limb amputations: assessing prosthesis-related quality of life. Arch Phys Med Rehab 1998; 79(8): 931–938.

16.

Guillemin

Bombardier

Beaton

. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993; 46(12): 1417–1432.

17.

Franchignoni

Giordano

Ferriero

. Measuring mobility in people with lower limb amputation: Rasch analysis of the mobility section of the prosthesis evaluation questionnaire. J Rehabil Med 2007; 39(2): 138–144.

18.

Arwert

Van Doorn-Loogman

Koning

. Residual-limb quality and functional mobility 1 year after transtibial amputation caused by vascular insufficiency. J Rehabil Res Dev 2007; 44(5): 717–722.

19.

Day

Buis

. Cross cultural equivalence testing of the Prosthetic Evaluation Questionnaire (PEQ) for an Arabic speaking population. Prosthet Orthot Int 2012; 36(2): 173–180.

20.

Ramos Brieva

Montejo Iglesias

Lafuente López

. Validación de la Escala-Criba Geriátrica para la Depresión. Actas Luso-Esp Neurol Psiquiatr Cienc Afine 1991; 9(3): 174–177.

21.

Salamero

Marcos

. Factor study of the Geriatric Depression Scale. Acta Psychiat Scand 1992; 86(4): 283–286.

22.

Yesavage

Brink

Rose

. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiat Res 1983; 17(1): 37–49.

23.

Tavakol

Dennick

. Making sense of Cronbach’s alpha. Int J Med Educ 2011; 2: 53–55.