Malay Translation and Validation of the Oslo Sports Trauma Research Center Questionnaire on Health Problems

Abstract

Background:

The Oslo Sports Trauma Research Center Questionnaire on Health Problems (OSTRC-H) is a proven surveillance method to register acute injury, overuse injury, or illness in athletes. However, it has not yet been translated into Malay.

Purpose:

To translate, culturally adapt, and validate the OSTRC-H into Malay using internationally established guidelines.

Study Design:

Cohort study (diagnosis); Level of evidence, 3.

Methods:

The Malay translation of the OSTRC-H (OSTRC-H.M) was distributed to 54 elite (ie, professional) and recreational athletes. Face validity, content validity, and construct validity were assessed for the validity measurement properties, while test-retest and internal consistency were used for reliability measurement properties.

Results:

The content validity index was satisfactory at 0.93. The OSTRC-H.M showed good internal consistency, with a Cronbach α value of .84. Confirmatory factor analysis showed a good model for construct validity, with values of 0.99 and 0.04 for comparative fit index and root mean square error of approximation, respectively. Per-item intraclass correlation coefficients (ICCs) for test-retest reliability ranged from 0.55 to 0.80. The overall test-retest reliability of the questionnaire was good (ICC, 0.8; 95% CI, 0.63-0.88; P < .001).

Conclusion:

The OSTRC-H.M was found to be valid and reliable and therefore acceptable for application among Malay-speaking athletes.

Keywords

translation patient-reported outcome measures athlete health problem monitoring

Health problems in athletes are defined as any physical or psychological complaint or manifestation, regardless of the need for medical attention or time lost from athletics.¹⁵ It encompasses acute injury, overuse injury, and illness. Such conditions can occur in and out of the competition period. In 2014, the Oslo Sports Trauma Research Center developed a method for capturing the occurrence and severity of overuse injuries in sport, known as the Oslo Sports Trauma Research Center Questionnaire on Health Problems (OSTRC-H).⁵ The questionnaire consists of 4 core items that are administered from the period before a competition, at regular intervals (weekly), and throughout the competition itself (Table 1). If at any given time the scores are less than minimal, the responding health personnel then follow up to determine a diagnosis and administer any intervention accordingly. This vigilant monitoring allows early detection and intervention, which in turn can lead to the condition's being treated optimally.⁵

Table 1

The 4 Core Items of the Oslo Sports Trauma Research Center Questionnaire on Health Problems⁵

Question 1: Have you had any difficulties participating in normal training and competition due to injury, illness, or other health problems during the past week?

○ Full participation without health problems

○ Full participation, but with injury/illness

○ Reduced participation due to injury/illness

○ Cannot participate due to injury/illness

Question 2: To what extent have you reduced your training volume due to injury, illness, or other health problems during the past week?

○ No reduction

○ To a minor extent

○ To a moderate extent

○ To a major extent

○ Cannot participate at all

Question 3: To what extent have injury, illness, or other health problems affected your performance during the past week?

○ No effect

○ To a minor extent

○ To a moderate extent

○ To a major extent

○ Cannot participate at all

Question 4: To what extent have you experienced symptoms/health complaints during the past week?

○ No symptoms/health complaints

○ To a mild extent

○ To a moderate extent

○ To a severe extent

The OSTRC-H has been translated and validated into Spanish,¹ German,⁶ Danish,⁷ Thai,⁸ Japanese,¹⁰ and Brazilian.¹³ However, there are no comparable monitoring tools available for Malay-speaking countries. Thus, in this study, we aimed to validate the Malay translation of the OSTRC-H (OSTRC-H.M) using internationally recognized guidelines. We hypothesized that the OSTRC-H.M would show good reliability and validity measurement properties and can therefore be used as a health problem monitoring instrument among Malay-speaking athletes.

Methods

This study started with the development of the study protocol by the research team. It was conducted in 2 phases; a linguistic translation and cultural adaptation of the OSTRC-H from English to Malay was performed in phase 1, followed by a prospective instrument validation and reliability study in phase 2. Ethics committee approval was received for this study.

Athlete Recruitment

Participants were recruited from various types of sports, including football, futsal, sepak takraw (also known as foot volleyball), and badminton, at either the elite (professional) or recreational level. This was done after confirmation with the original authors of the OSTRC-H⁵ that the questionnaire is applicable to all different levels of athletes and not confined to professionals. The eligibility criteria included (1) age >18 years and (2) ability to understand Malay. Those who agreed were briefed thoroughly regarding the questionnaire and then asked to complete the OSTRC-H.M via the Google Forms platform. They were then asked to complete the questionnaire again 1 week later with a Google Forms link sent via email. They were reminded the day after if the questionnaire had not been completed.

Sample Size

The sample size was calculated from a statistical aspect based on the guidelines for the respondent-to-item ratio of 10:1¹⁶ and recommendation from the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) checklist,¹¹ which suggested that between 50 and 100 respondents were considered adequate to test for the relevant measurement properties. A final sample of 54 athletes was included in this study using the convenience sampling method.

Translation Process

Permission to translate the English version of the OSTRC-H to Malay was obtained from the original developers.⁵ The entire process of translation and cross-cultural adaptation of the self-reported measures was conducted as per the guidelines of Beaton et al.³ There were 6 stages in total, as follows:

Initial translation: Two forward-translations of the instrument from English to Bahasa Melayu (Malay language) were performed by 2 independent bilingual translators (T1 and T2). Both were Malaysian residents with Bahasa Melayu as their native language. T1 was aware of the concept being examined (had medical background) while T2 was not.

Synthesis of the translation: Discrepancies between the translations (T1 and T2) were addressed between the 2 translators and a recording observer; this process eventually guided refinement of the translation. Further queries were also clarified with the original developer. Ultimately, a consensus on a common translation (T12) was established.

Back-translation (BT): The common translation (T12) was subsequently translated back into the English language by 2 different Malaysian residents (BT1 and BT2). Both were English teachers with no medical background to avoid information bias.

Evaluation by expert committee: The expert committee consisted of the forward and back translators, 2 sports medicine physicians, 1 exercise physiologist, 1 physical therapist, 2 family medicine specialists, and 1 public health specialist. The project manager communicated with the committee members via email for logistical reasons. All feedback from the committee was taken into account and discussed thoroughly. Semantic, idiomatic, experiential, conceptual, and functional equivalences were reached for all differences. Eventually, the team approved and agreed upon a prefinal version of the OSTRC-H.M, which was considered ready for field testing.

Field testing of the prefinal version: As part of phase 1, pilot testing of the OSTRC-H.M was conducted on 30 athletes from local football and futsal clubs, and these athletes completed cognitive interviews¹⁹ on 2 separate occasions to assess the relevance, comprehensiveness, and comprehensibility of the questionnaire. All the feedback and comments were taken and reviewed. Content validity was then determined by a panel of 7 experts (3 sports physicians, 2 family medicine specialists, and 2 physical therapists). This process provided a validity check and allowed further amendments to continue improving the quality of the questionnaire.

Submission of documentation for appraisal of the adaptation process: The results from the cognitive reviews were used to produce the final translated version. Written documentation on issues relating to the interpretation was carried out. The committee verified that the recommended stages were followed and that the reports accurately reflected the process. This in turn indicated that a reasonable translation had been achieved.

The final translated version was proofread and checked for spelling and grammatical errors.

Data Analysis: Validity and Reliability of the OSTRC-H.M

The basic information of the study participants was reported as means and standard deviations or as counts. Face validity, content validity, and construct validity were assessed for the validity measurement properties.

Some components of phase 2 were done during phase 1 of the study, as suggested by the guidelines used.³ Face validity and content validity were assessed during the pilot testing (phase 1). For face validity, the 30 local participants were asked what they thought was meant by each questionnaire item and its chosen response. They were also evaluated regarding the relevance, comprehensiveness, and comprehensibility of the items, response options, and instructions.¹² The distribution of responses was also explored to look for a high proportion of missing items or single responses. Content validity was assessed by the panel of 7 experts using the content validity ratio and content validity index (CVI). An acceptable value for CVI is ≥0.83.^9,20

The rest of the analysis was performed in phase 2 on the 54 study athletes. Construct validity was assessed using confirmatory factor analysis (CFA), with comparative fit index (CFI) and root mean square error of approximation (RMSEA) calculated. Values of >0.95 for CFI and <0.05 for RMSEA were considered a good model.² Test-retest reliability and internal consistency were used for reliability measurement properties. The retest was taken 1 week after the initial test, with numbering of the questions purposely changed to minimize recall bias. As the original OSTRC-H did not include a scoring system, we created the following system for the purposes of measuring reliability: responses (in order from top to bottom) to questions 1 and 4 were scored 4-3-2-1 points, and responses to questions 2 and 3 were scored 5-4-3-2-1 points. Test-retest reliability was calculated with the intraclass correlation coefficient (ICC), with a value of ≥0.9 considered high reliability. Internal consistency was determined using the Cronbach α, with 0 indicating no internal consistency and 1 indicating perfect internal consistency.¹⁸

All statistical analyses were performed using the SPSS Statistics (IBM; SPSS Statistic 22.0) as well as the lavaan, semPlot, and semptools packages in the Rstudio software (Posit BC; RStudio 2022.12).

Results

Participant Characteristics

Of the 54 participants recruited for the study, 34 were elite athletes, while 20 were recreational athletes. Participants were exclusively male (100%), with a mean age of 25 years (range, 18-39 years). The majority of the athletes (78%) were football players. Overall training times for both elite and recreational groups were 8.0 h/wk. Mean individual training hours were longer for the elite athletes, at 10.9 h/wk compared with 1.8 h/wk for the recreational group. The characteristics of the participants are summarized in Table 2.

Table 2

Participant Characteristics (N = 54)^a

Characteristic	Value
Age, y	25 ± 6.6
Sport played
Football	42
Futsal	2
Badminton	9
Sepak takraw	1
Overall training, h/wk	8.0 ± 5.43
Elite	10.9 ± 3.8
Recreational	1.8 ± 0.9

Data are presented as mean ± SD or No. of participants.

All 54 athletes completed the test-retest component, for a 100% response rate. The participants completed the questionnaire within 5 minutes, and the interviews regarding the questionnaire took between 10 and 15 minutes, on average.

Translation Process

Only minor discrepancies in the choices of synonyms and use of prepositions were found during the forward-translation process. Examples included minggu lepas instead of minggu lalu when referring to “past week,” as it is more precise, thus making the questionnaire more relevant if it is to be implemented for weekly monitoring. In certain local cultures, the term uzur for describing illness can be misunderstood as “menstruation.” Thus, the more socially acceptable term penyakit was opted for the questionnaire. We opted to use only “symptoms” instead of “symptoms/health complaints” in question 4, as it reflects the same thing in detecting any underlying health problem. One unresolved term was clarified with the original researcher; The category “other health problems” was explained as conditions that include overtraining syndrome and certain mental conditions. There was no other significant difficulty or comprehensibility issues in translating the questionnaire. The final version of the OSTRC-H.M is shown in Appendix Table A1.

Validity

Face Validity

Almost all athletes who underwent cognitive interviews (n = 30) rated the questionnaire in general as “understandable,” and they also agreed that the questions were relevant in detecting health problems in sports. There were a few minor suggestions for linguistic improvement. Some of the athletes mentioned that the term masalah kesihatan (“health problem”) to their understanding referred only to illness and not injury. Thus, the term was revised to kecederaan, penyakit, atau masalah kesihatan lain (“injury, illness, or other health problems”) to provide better comprehensibility.

Content Validity

The content validity was found to be good (Table 3). The item content validity ratio ranged from 0.71 to 1.00, while the CVI for the questionnaire was excellent at 0.93.

Table 3

Content Validity of the OSTRC-H.M^a

Item	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	CVR	CVI
Q1	X	X	X	X	X	X	X	1.0	0.929
Q2	X	X	X	X	X	X	X	1.0
Q3	X	X	X	X	X	X	X	1.0
Q4	X	−	X	X	X	X	X	0.714

“X” denotes the item was considered essential with respect to representativeness, consistency, and relevance; “–” denotes the item was considered nonessential with respect to representativeness, consistency, and relevance. CVI, content validity index; CVR, content validity ratio; OSTRC-H.M, Malay translation of the Oslo Sports Trauma Research Center Questionnaire on Health Problems; Q, question.

Construct Validity

The CFI showed a good model, with a value of 0.999. This was supported with an RMSEA value of 0.038. Subgroup analysis between the elite and recreational groups also displayed good and comparable CFIs. Results are summarized in Table 4. The factor-loading estimate as well as both the individual and overall residual values for the CFA were noted to be significant (P < .05 for all), as shown in Figure 1.

Table 4

Results of Confirmatory Factor Analysis for the OSTRC-H.M^a

Parameter	Overall	Elite Athletes	Recreational Athletes
CFI^b	0.999	0.998	0.998
RMSEA^c	0.038	0.024	0.068

CFI, comparative fit index; OSTRC-H.M, Malay translation of the Oslo Sports Trauma Research Center Questionnaire on Health Problems; RMSEA, root mean square error of approximation.

Values >0.95 are considered a good model.

Values <0.05 are considered a good model.

Figure 1.

Overall confirmatory factor analysis model for the OSTRC-H.M. Larger factor-loading estimate values indicate stronger contributions to the questionnaire domain. Values within circular arrows indicate the variability of the answers of the respondents (variance); the smaller the value, the more precise the items are. Statistically significant: *P < .05, **P < .01, ***P < .001. OSTRC-H.M, Malay translation of the Oslo Sports Trauma Research Center Questionnaire on Health Problems; Q, question.

Test-Retest Reliability and Internal Consistency

The overall mean score for the initial OSTRC-H.M was 4.12 ± 0.915, and the score for the retest 1 week later was 4.09 ± 0.65. The overall test-retest reliability of the questionnaire was good (ICC, 0.8; 95% CI, 0.63-0.88; P < .001). Per-item ICC values for test-retest reliability ranged from 0.55 to 0.80, with the highest being for Q3 (ICC, 0.8; 95% CI, 0.65-0.88; P < .001) (Table 5). The OSTRC-H.M showed good internal consistency, with a Cronbach α value of .84 (Table 5).

Table 5

Test-Retest Reliability and Internal Consistency of the OSTRC-H.M^a

Item	Test-Retest Reliability		Internal Consistency
Item	Mean ICC	P ^b	Corrected Item-Total Correlation^c
Q1	0.55	<.002	0.53
Q2	0.67	<.001	0.71
Q3	0.80	<.001	0.80
Q4	0.67	<.001	0.81
	Overall: 0.80	<.001	Cronbach α = .84

ICC, intraclass correlation coefficient; OSTRC-H.M, Malay translation of the Oslo Sports Trauma Research Center Questionnaire on Health Problems; Q, question.

Statistically significant at P < .05.

A corrected item-total correlation value of >0.3 indicates the item is suitable to be retained.

Discussion

The results of this study supported the feasibility, validity, and reliability of the OSTRC-H.M. The translated version reflected satisfactory measurement properties outcomes, which is in line with the established recommendations.

The OSTRC-H provides a new approach of recording all types of health problems in sport; it is sensitive and valid in documenting the pattern of acute injuries, overuse injuries, and illnesses in athletes.⁵ The questionnaire ticks off all the checklist items of a good patient-reported outcome measure: (1) it can provide information relevant to a health condition and goals, (2) it can be collected easily (eg, using a smartphone or tablet computer), (3) items are logical and easy to interpret, (4) the completion time is short, and (5) it is open-access.⁴ Hence, it can serve as a useful tool to monitor health problems among local athletes.

Many suggested cross-cultural adaptation guidelines, and recommendations have been found in the literature. In this study, the standard 6-stage translation was mainly adapted from the guidelines provided by Beaton et al.³ We managed to meet all 12 design requirements as provided by the COSMIN checklist for a good translation process.¹¹

Cognitive interviews were conducted on 30 athletes to provide face validation of the OSTRC-H.M and minimize nonresponse and response errors.¹⁷ The CVI was found to be 0.929 by the panel of 7 experts, exceeding the acceptable value of 0.83 as recommended.^9,20 CFA was conducted to measure construct validity, with a CFI of 0.999 and an RMSEA of 0.038 indicating a good model. The original English version of the OSTRC-H was initially designed to monitor health problems in elite athlete populations. However, upon confirmation with the original authors, the questionnaire can also be used in the recreational athlete, making it even more practical to be applied throughout a more heterogeneous sports population. Subgroup analysis between the elite and recreational groups displayed good and comparable CFIs (0.998 for both groups). This finding underscores that the OSTRC-H.M is applicable to both elite and recreational athletes.

Test-retest analysis was conducted in the study, with the second questionnaire distributed via email after 1 week after the first questionnaire. Response rate was 100% in which all 54 athletes completed the test-retest component. The overall test-retest reliability of the questionnaire was considered good (ICC = 0.8). According to Streiner et al,¹⁴ a test-retest period within 2 days to 2 weeks is recommended. However, a shorter period is susceptible to recall bias while a longer period allows too much time for natural changes to occur in the severity of injuries and illness, affecting the reliability of the questionnaire.¹⁴ This was seen during the study, as some of the elite athletes sustained injuries within the 1-week period of test-retest. They were training a mean of 10 hours per week and thus were susceptible to sustaining an injury, which prompted them to respond differently during the second questionnaire. This was further supported with findings by Jorgensen et al,⁷ who reported considerably lower ICC values after a test-retest period lasting 2 weeks (ICC = 0.62) compared with that of 1 week (ICC = 0.72). Otherwise, the OSTRC-H.M showed good internal consistency, with a Cronbach α value of .84, exceeding the target value of ≥0.70.

Limitations

There are several limitations to this study. We were unable to analyze criterion-related validity because of the absence of other known methods to capture health problems. This problem was also issued in other translations of the OSTRC-H, such as the Danish⁷ and Japanese¹⁰ versions. Another limitation noted in this study was that all the participants were male. We approached several sport clubs consisting of both male and female athletes; however, only male athletes showed interest in participating in the study. Thus, we were not able to assess whether the female counterparts would respond differently toward the translated version. However, during the development of the translation, the translators and committee members included both male and female participants. Given that the questionnaire consists of only 4 questions with wordings that are easy to understand, we expect the difference would not be significant.

Conclusion

The OSTRC-H.M was performed using established methodology and adapted culturally and validated. Furthermore, it was found to be valid and reliable and therefore acceptable for application in the Malay-speaking population. Further research should be conducted to establish the psychometric properties of the OSTRC-H.M on a more extensive population with a longer period of monitoring.

Footnotes

Appendix

Table A1

Malay Version of the Oslo Sports Trauma Research Center Questionnaire on Health Problems

Sila jawab kesemua soalan di bawah tanpa mengira sama ada anda mengalami sebarang masalah kesihatan atau tidak sepanjang minggu lepas. Tentukan pilihan yang paling sesuai untuk anda, dan sekiranya tidak pasti, sila cuba untuk berikan jawapan yang sebaik mungkin.

1. Pernahkah anda mengalami sebarang kesukaran untuk melibatkan diri dalam latihan dan pertandingan disebabkan oleh kecederaan, penyakit, atau masalah kesihatan yang lain sepanjang minggu lepas?

○ Penglibatan penuh. Tiada kecederaan, penyakit atau masalah kesihatan yang lain

○ Penglibatan penuh, tetapi dengan kecederaan, penyakit atau masalah kesihatan yang lain

○ Pengurangan penglibatan disebabkan oleh kecederaan, penyakit atau masalah kesihatan yang lain

○ Tidak dapat terlibat sama sekali disebabkan kecederaan, penyakit atau masalah kesihatan yang lain

2. Sehingga tahap mana anda mengurangkan jumlah latihan disebabkan kecederaan, penyakit atau masalah kesihatan lain sepanjang minggu lepas?

○ Tiada pengurangan

○ Tahap pengurangan yang sedikit

○ Tahap pengurangan yang sederhana

○ Tahap pengurangan yang banyak

○ Tidak dapat turut serta sama sekali

3. Sehingga tahap mana kecederaan, penyakit, atau masalah kesihatan yang lain menjejaskan prestasi anda sepanjang minggu lepas?

○ Tidak terjejas.

○ Terjejas dengan sedikit

○ Terjejas dengan sederhana

○ Terjejas dengan banyak

○ Tidak dapat turut serta sama sekali

4. Sehingga tahap mana anda mengalami gejala kecederaan, penyakit, atau masalah kesihatan yang lain sepanjang minggu lepas?

○ Tiada gejala

○ Tahap gejala yang sedikit

○ Tahap gejala yang sederhana

○ Tahap gejala yang teruk

Acknowledgements

The authors thank Dr. Md Edre bin Mohammad Aidid for his advice and input on statistical analysis. They also appreciate the technical assistance provided by Dr. Azlan bin Abdul Rahman and Muhammad Ashraff bin Muhammad Yazid during the athletes’ recruitment and data-collection process.

Final revision submitted October 23, 2023; accepted January 10, 2024.

The authors declared that there are no conflicts of interest in the authorship and publication of this contribution. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.

Ethical approval for this study was obtained from University of Malaya (ref No. 2022812-11465).

References

Bailon-Cerezo

Clarsen

Sanchez-Sanchez

Torres-Lacomba

Cross-cultural adaptation and validation of the Oslo Sports Trauma Research Center Questionnaires on overuse injury and health problems (2nd version) in Spanish youth sports. Orthop J Sports Med. 2020;8(12):2325967120968552.

Barbara

MB.

Factor analysis models: analytic models: viewing the structure of an assessment instrument from three perspectives. J Pers Assess. 2005;85(1): 17-32.

Beaton

Bombardier

Guillemin

Ferraz

MB.

Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186-3191.

Brukner

Clarsen

Cook

, et al. Brukner & Khan's Clinical Sports Medicine: Volume 1 Injuries. 5th ed. McGraw-Hill Education; 2017.

Clarsen

Ronsen

Myklebust

Florenes

Bahr

The Oslo Sports Trauma Research Center questionnaire on health problems: a new approach to prospective monitoring of illness and injury in elite athletes. Br J Sports Med. 2014;48(9):754-760.

Hirschmüller

Steffen

Fassbender

, et al. German translation and content validation of the OSTRC Questionnaire on overuse injuries and health problems. Br J Sports Med. 2017;51(4):260-263.

Jorgensen

Rathleff

Andreasen

Danish translation and validation of the Oslo Sports Trauma Research Centre questionnaires on overuse injuries and health problems. Scand J Med Sci Sports. 2016;26(12):1391-1397.

Kittinad

Chaijenkij

Tongsai

Validity and reliability of the Oslo Sports Trauma Research Center (OSTRC) questionnaire on overuse injury and health problem in Thai version. J Med Assoc Thailand. 2021;104(1):105-113.

Lynn

MR.

Determination and quantification of content validity. Nurs Res. 1986;35(6):381-385.

10.

Mashimo

Yoshida

Hogan

, et al. Japanese translation and validation of web-based questionnaires on overuse injuries and health problems. PLoS One. 2020;15(12):e0242993.

11.

Mokkink

Prinsen

Bouter

Vet

Terwee

CB.

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther. 2016;20(2):105-113.

12.

Mokkink

Terwee

Patrick

, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737-745.

13.

Pimenta

Hespanhol

Lopes

AD.

Brazilian version of the OSTRC Questionnaire on health problems (OSTRC-BR): translation, cross-cultural adaptation and measurement properties. Braz J Phys Ther. 2021;25(6):785-793.

14.

Streiner

Norman

Cairney

Health Measurement Scales: A Practical Guide to Their Development and Use. 5th ed. Oxford University Press; 2015.

15.

Timpka

Alonso

Jacobsson

, et al. Injury and illness definitions and data collection procedures for use in epidemiological studies in athletics (track and field): consensus statement. Br J Sports Med. 2014;48(7):483-490.

16.

Tsang

Royse

Terkawi

AS.

Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth. 2017;11(suppl 1):S80-S89.

17.

Wild

Grove

Martin

, et al; ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaption process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaption. Value Health. 2005;8(2):94-104.

18.

Willems

Verbestel

Calders

Lapauw

De Craemer

Test-retest reliability and internal consistency of a newly developed questionnaire to assess explanatory variables of 24-h movement behaviors in adults. Int J Environ Res Public Health. 2023;20(5):4407.

19.

Willis

GB.

Cognitive Interviewing: A Guide for Improving Questionnaire Design. SAGE Publications; 2005.

20.

Yusoff

MSB

. ABC of content validation and content validity index calculation. Educ Med J. 2019;11(2):49-54.