Allied health professionals cognitive workload scale in Malacca,Malaysia: A psychometric properties evaluation

Abstract

Background

Cognitive workload has emerged as one of the most important topics that must be understood and addressed because of its impact on errors and work performance that will compromise patient safety. With increasing patient health demands, Allied Health Professionals (AHPs) tend to have heavier cognitive workloads and impact on personal well-being.

Objective

Our study aims to determine the validity and reliability of a cognitive workload scale for AHP in Malacca, Malaysia and examine its effect on personal well-being.

Method

The instrument comprises items related to human reliability, human-computer interaction, decision-making, skilled performance, training, work stress and cognitive ergonomic effect. The content validation was conducted using the Content Validity Questionnaire with eight expert panels’ responses. 120 AHP working in Malacca were recruited using purposive sampling and took part in the study, completing the online questionnaires. To evaluate the validity based on relationships with other measures, the SMEQ scale was also administered. AMOS Version 24 and IBM SPSS Version 26 were used in data analysis.

Results

The instrument showed good content validity (CVI > 0.74). After the deletion of six items, the instrument has good convergent validity (AVE > 0.5), discriminant validity (HTMT Ratio < 0.85), composite reliability (CR > 0.7), internal consistency (alpha > 0.7), construct validity the ratio (1.865), CFI (0.900), PCFI (0.821), PNFI (0.738), RMSEA (0.085), IFI (0.90) and test-retest reliability (ICC > 0.6). The study found a positive correlation with the Malay version of SMEQ. This study also found that the AHP in the laboratory group have higher work stress and experiences more cognitive load effects, leading to lower personal well-being.

Conclusion

Overall, the final version of a scale measuring the cognitive workload is valid and reliable in assessing cognitive ergonomics among AHP. Testing cognitive workload on different work populations, including manual and non-manual workers, is recommended for future studies.

Keywords

psychological load validity reliability well-being health care provider

Introduction

Understanding cognitive workload in the healthcare sector has drawn much attention from researchers.^1–3 Knowing the cognitive workload status helps to control the unnecessary cognitive load while working.⁴ The healthcare sector is a very complex industry and is a combination of healthcare workers, human resource management, patients and family members. The need to know the status of cognitive workload, especially in the field of health is very important because excessive cognitive workload impacts errors and work performance that will affect patient safety.⁵ As a large multidisciplinary organization, ensuring good cognitive workload practice is a big challenge.

Cognitive workload describes the cognitive resource capacity required to perform the task requirements. Based on cognitive load theory, cognitive resources are limited for each individual. The demand for cognitive resources that exceed an individual's ability can have an impact on task performance.⁶ The decision-making process, skill performance, training, mental workload, stress and human-computer interaction will be disrupted if this cognitive load exceeds the ability of cognitive resources in a certain time.⁶ By knowing individual competencies and limitations, human interactions with other system components can be improved by optimizing cognitive processes. An excessive cognitive workload can impact well-being.⁷ Work exhausted is often caused by loose work control.^8–10 Multi-tasking, distractions at work and inadequate instruction can cause cognitive and psychological stress.¹¹

Cognitive load theory is not only used as a learning theory but also started to be used in the field of patient health care.¹² Cognitive load is divided into internal cognitive load and external cognitive load. Internal cognitive load is related to the natural nature of the task, for example, the allied health science profession itself has a high internal load due to the nature of its work that must deal with patients that involve mental load, decision making and interaction with technology. The external cognitive load is information other than the task itself such as individual competence, performing multiple tasks at one time and stress.¹² Germane Load, on the other hand, is the cognitive resource needed to facilitate the learning of new information so that skill performance can be improved. The germane load will not always exist if the individual cannot balance the internal and external load because the cognitive load has limited capacity resources to meet the needs of the task. Understanding the factors that contribute to cognitive load can form a strategy so that excessive cognitive load can be reduced, errors can be avoided and skill training and performance can be improved.¹³ In Malaysia, studies on cognitive workload are still far behind.¹⁴ There is a study trying to investigate the mental workload of pathology staff at a hospital in Malaysia during the COVID-19 pandemic.¹⁵ However, studies on other allied health professions still do not receive attention in this country.

AHPs are autonomous practitioners who work together with other health professionals to ensure that holistic health care is provided. According to the Second Schedule Amendment to the Allied Health Professions Order act as many as 16 professions are included under this act where each profession plays its own role and nature of work. The variety of professions is divided into 3 groups, namely the clinical, the laboratory and the public health.¹⁶ For example, the clinical group is directly involved in providing interventional treatment for more severe conditions while the public health group is more about promotion towards the formation of a healthy community. The laboratory group is more about providing support in preparing diagnostic tests. They also do not have direct contact with patients. The characteristics of each group are described as having different work, physical and work-pressure demands. In previous studies, physiotherapists have moderate job demands involving performing continuous tasks over a long period, using various techniques of handling and non-ergonomic postural which are at risk of musculoskeletal disorders.^17,18 Meanwhile, laboratory workers report work stress and fatigue due to workload.¹⁹ All three groups are exposed to excessive cognitive load and making inefficient decisions can make patients vulnerable to harm and reduce patients’ trust in health institutions.²⁰ However, training can improve the knowledge and performance of allied health professions skills, especially in decision-making.²¹ Furthermore, the focus of training is to meet job demands and competency requirements in the job performed.

Measuring cognitive load using subjective measures has proved to be the most widely used instrument. However, assessment tools to measure cognitive workload status using subjective measure that are available, are not based on studies on the Malaysian population.²² Furthermore, in Malaysia, ergonomic assessment only focuses on physical ergonomics in the workplace. There is still no standard assessment of cognitive workload in Malaysia.²³ Developing validated questionnaires in the target language is crucial in public health and medical research. It ensures that the results obtained from different groups and populations can be generalized and accurate conclusions.²⁴ Additionally, various languages represent distinct concepts that often require significant adjustments to convey them effectively.²⁵ Using imprecise language may lead to inadequate interpretability of the survey, failing to meet its stated objectives and invalidating any conclusions drawn from the data.²⁶

The majority of AHPs in Malaysia are Malays and their native language is Malay.²⁰ In addition, in Malaysia, the Malay language is the official national language used in government agencies including the Ministry of Health Malaysia. Studies have found that subjective questionnaires are highly dependent on the language and culture of their questionnaires where differences in meaning can lead to interpretation problems.²⁷ Language and culture can have a negative impact on the success of the interaction. For example, in a study conducted by Weng et al.,²⁸ although members work in the health sector, the use of languages other than their mother tongue is an obstacle for them to implement EBP in their place. Therefore, this questionnaire was developed in Malay language to minimize accuracy discrepancies when used by non-English speakers. Malay language is also used so that the delivery of ideas to be conveyed is effective.²⁹

The use of a self-developed scale is effective in investigating the mental workload of doctors in China.³⁰ The new instrument, through good validity and reliability, has helped to identify elements related to mental workload. The results findings can also implement related strategies to improve competence in health care delivery. Meanwhile, allied health professions are the third largest in the Malaysian Ministry of Health (MOH) sector after nurses and medical officers.³¹ As doctors and nurses are among the largest workforces in the health sector, the contribution of other healthcare workers is often overlooked.³² No matter how skilled an allied health worker is, they are still subject to common cognitive limitations. Cognitive limitations could constrain how much information is used by allied health workers when making decisions and judgments because complex tasks require more significant decisions to be made causing excessive cognitive load.³³ If an individual is put under a cognitively straining condition, their task performance can be impaired, affecting their overall performance and productivity when dealing with patients.

The status of cognitive workload in Malaysia especially in allied health professions is unknown. There is a need to examine the cognitive workload status among allied health professions that can influence personal well-being. Thus, the objective of this study is to determine the psychometric test of a scale that measures the level of cognitive workload of allied health professionals and examine its impact on well-being. Further investigation about the factors and effects of cognitive workload was carried out to figure out how to create a more efficient and safe system for allied health professionals.

Materials and method

Study design

This study is a quantitative study conducted in Malacca, Malaysia. The data was obtained and collected from December 2022 to March 2023. The self-developed questionnaire used in this study focused on the cognitive workload factors. To measure the well-being of life, the personal well-being index (PWI) was used in this study.

Participants

Eight experts with doctoral degrees in ergonomic rehabilitation or who have at least 20 years of related work experience were invited to assess the content validity of the Cognitive Workload Scale.

The target population is allied health professionals. The study population is comprised of allied health professionals in Malacca, Malaysia. The participants were sampled using the purposive sampling method. According to a report written Rotem and Roberts,²⁰ there are three allied health professional groups: clinical, laboratory, and public health. Thus, AHPs from all three groups were recruited.

Eligible participants are Malaysian AHPs who work in the Ministry of Health, Malaysia (MOH) hospitals. The participants must have at least one year of working experience. Allied health professionals who are on study leave, maternity leave, having brain injury and psychiatric illness will be excluded from the study. 120 allied health professionals participated in this study. For test-retest reliability, 30 participants were recruited. Demographics of expert panels involved in content validity tests and participants recruited for validity are as in Table 1.

Table 1.

Demographics of expert panels involved in the content validity test and participants recruited for validity.

Demographics	Expert Panels (N (%))	Participants involved in validity test (N (%))	Participants involved in test-retest reliability test (N (%))
Gender
Male	3 (37.50)	51 (42.50)	13 (44.80)
Female	5 (62.50)	69 (57.50)	16 (55.20)
Age Group (years)
20–29	0(0)	22 (18.33)	5 (17.24)
30–39	0(0)	64 (53.33)	16 (55.17)
40–49	3 (37.50)	28 (23.33)	6 (20.69)
50–59	5 (62.50)	6 (5)	2 (6.90)
Academic Qualification
Diploma	0 (0)	86 (71.67)	20 (69.00)
Bachelor's degree	5 (62.50)	33 (27.50)	9 (31.00)
Master's degree	2 (25.00)	1 (0.83)	0 (0)
Philosophical Doctrine	1 (12.50)	0 (0)
Working Experience (years)
1–9	0 (0)	42 (35.00)	9 (31.03)
10–19	0 (0)	64 (53.33)	17 (58.62)
20–29	4 (50.00)	11 (9.17)	2 (6.90)
30–39	4 (50.00)	3 (2.50)	1 (3.45)
Profession Group
Clinical Group	—	62 (51.67)	19 (65.52)
-Physiotherapy		17 (27.4)	4 (21.1)
-Occupational Therapy		28 (45.2)	8 (42.1)
-Dietician		5 (8.1)	3 (15.7)
-Speech Therapist		1 (1.6)	—
-Diagnostic Radiographer		11 (17.7)	4 (21.1)
Laboratory Group	—	39 (32.50)	2 (6.90)
-Medical Laboratory		13 (33.3)	2 (6.90)
Technologist
-Dental Technologist		16 (41.1)	—
-Medical laboratory		10 (25.6)	—
Scientist
Public Health Group	—	19 (15.83)	8 (27.59)
-Environmental Health		17 (89.5)	7 (87.5)
Officer
-Nutritionist		2 (10.5)	1 (12.5)
Race
Malay	-	108 (90.00)	27 (93.10)
Chinese	-	4 (3.30)	1 (3.45)
Indian	-	3 (2.50)	1 (3.45)
Others		5 (4.20)	0 (0)

Instrumentation

A self-developed questionnaire was used as the main instrument to measure the factors and effects of the cognitive workload scale among allied health professionals in Malacca, Malaysia. The language used in this questionnaire is Malay. The questionnaire consists of eight constructs for cognitive workload factors: mental workload (MW), work stress (WS), decision-making (DM), training (TR), human reliability (HR), human-computer interaction (HCI) and skilled performance (SP) (Table 2). The questionnaire also contains a few items related to the possible effects of cognitive load (CE). There are at least five items questioned under each factor. The item response format is Likert frequency with a scale of 11 points. Respondents are required to choose the most appropriate answer option, where the highest score is strongly agreed, and the lowest score is strongly disagree based on the answer statement.

Table 2.

Constructs and items of cognitive workload scale (In Malay).

Table 3.

Shows the HTMT value that has been achieved.

	MW	HR	HCI	DM	SP	TR	WS
HR	0.84
HCI	0.69	0.71
DM	0.70	0.82	0.62
SP	0.68	0.78	0.61	0.83
TR	0.52	0.51	0.50	0.50	0.57
WS	−0.10	−0.14	−0.07	−0.07	−0.13	−0.17
EF	0.09	−0.00	0.00	0.09	−0.01	−0.07	0.74

We also used the Subjective Mental Effort Questionnaire (SMEQ) in its Malay adaptation to measure the validity of cognitive workload with relationship to other instruments.29 The SMEQ is a subjective measure to assess cognitive workload. The SMEQ consists of a single scale with nine labels indicating the mental effort used to complete the task. The SMEQ has been employed in various laboratory and field studies with high validity and reliability.³⁴ Its use is easy, quick, inexpensive, and no special tool are needed.²²

Another instrument used was the Malay version of the Personal Wellbeing Index.³⁵ The Personal Wellbeing Index is used to determine the interaction effect of allied health professions groups and the level of cognitive workload on subjective well-being. The Personal Wellbeing Index measures the subjective well-being of an individual in terms of standard of living, health, achieving in life, relationships, safety, community connectedness, and future security. The Personal Wellbeing Index used an 11-point likert scale to indicate the respondents’ satisfaction with the items on the scale. “0” signifies no satisfaction at all, while “10” signifies being completely satisfied. All the items can be analyzed using their respective scores. The scores can also be summed up to determine the participants’ subjective Wellbeing.

Procedure and data collection

At the beginning of the research proposal paper is prepared. Then, the application for research ethics approval from from the Research Ethics Committee of The National University of Malaysia (JEP) and the National Medical Research Register (NMRR) was carried out for the conduct of research in the hospitals of the Ministry of Health Malaysia (MOH).

After the development of the Cognitive Workload Scale, the inclusion criteria for all the tests, including the construct validity test, correlation test with SMEQ, test-retest reliability, and two-way independent ANOVA test, were decided. Next, content validity testing of the questionnaire was conducted using the Content Validity Questionnaire with consent from expert panels, and the content validity index was calculated. Then, feasibility testing was carried out. All subjects were briefed about the study information and consent was obtained using an information sheet and an online consent form.

The Cognitive Workload Scale and Personal Wellbeing Index-Malay Version and SMEQ were distributed to the subjects who fulfilled the inclusion and exclusion criteria through an online platform. After two weeks, data were collected from 30 respondents for the second time for the test-retest reliability testing.³⁶ All the data was documented in Microsoft Excel. Then, data analysis was carried out using IBM SPSS and IBM SPSS AMOS. The discussion was made with the supervisor and research collaborator. Lastly, a conclusion was reached after data analysis.

Data analysis

The Item Content Validity Index (I-CVI) to determine the content validity of each item in the questionnaire was calculated. The Kappa Coefficient is another measure to assess content validity beyond the possibility of chance agreement. The I-CVI statistic to test content validity and Kappa were used in this study to ensure that consensus among experts was beyond chance.^37,38

The validity of the cognitive workload scale construct is done through confirmatory factor (CFA) to analyse the extent to which the items loaded on the questionnaire construct are valid.³⁹ In order to check unidimensional item, factor loading of each item is checked. Factor loading values >0.50 indicate acceptable unidimensionality.⁴⁰ Items that do not meet the specified factor loading values should be discarded. Next, the fit index was checked such as (i) Ratio, (ii) Comparative of Fit Index (CFI), (iii) Incremental of fit index (IFI), (iv) Parsimonious Normed fit index (PNFI), (v) The Root Mean Square Error of Approximation (RMSEA) and (vi) Parsimonious Comparative Fix Index (PCFI) according to the value suggested by Hu and Bentler⁴¹ and Meyers, Gamst.⁴² Modification Indices (MI) will be checked if the matching index is not reached where a high MI value indicates there are redundant items. Redundant items that are selected are discarded until the appropriateness index is reached.⁴⁰

To establish the overall construct validity of the questionnaire, convergent validity and discriminant validity were also measured in this study. The average variance extracted (AVE) computation verified the convergent validity. An AVE value of >0.50 confirms the acceptable convergent validity of the questionnaire.⁴³ The discriminant validity is measured by calculating the heterotrait-monotrait (HTMT) ratio of the correlation. An HTMT ratio value of >0.90 denotes a lack of discriminant validity.³⁹

The internal consistency was evaluated by calculating Cronbach's alpha and composite reliability. An alpha value of >0.70 indicates acceptable reliability of the construct. The composite reliability, which is >0.70, is required to achieve the composite reliability of the construct.⁴⁴

Evidence of validity based on relationships with other measures was analyzed by Pearson correlation between cognitive workload scale and SMEQ. Pearson's correlation r is used to see the direction of the relationship and the strength of the relationship between two variables in the form of continuous data. The value of the correlation coefficient (r) Positive, Zero or Negative Correlation is either negative or positive to show the relationship between the two variables. The value of the correlation coefficient (r) is referred to know the strength of the relationship between cognitive workload scale and SMEQ as follows: (r = 0.00); very weak (r < 0.20), weak (r < 0.35), medium (r < 0.50), high (r < 0.7), very high (r < 1.00).⁴⁵

The test-retest reliability of the questionnaire is evaluated using the intraclass correlation coefficient (ICC). Cicchetti⁴⁶ classified the strength of test-retest reliability into three groups: fair (0.40 to 0.59), good (0.60 to 0.74), and excellent (above 0.75). The standard error of measurement (SEM) was used to evaluate the absolute reliability of the items in the questionnaire. The Bland-Altman method determines the agreement between two measurements by assessing the limits of agreement (LoA). The plot will be plotted for each construct. All data values should fall within ±2 standard deviations of the mean difference.⁴⁷

The two-way ANOVA independent test was used to study the impact of two categorical independent variables on a continuous dependent variable. Assumptions like normality, independence, and homogeneity of variances ensure the result's validity. The main effects of each factor were examined to detect significant differences in dependent variable means across the factor levels. The interaction effect was assessed to determine if one factor's effect varies based on the level of the other. The overall F-test identifies if there are significant differences in dependent variable means across factor combinations. Post-hoc tests were used to explore significant group differences when the overall ANOVA test was significant, enabling detailed pairwise comparisons.

Ethical approval

This study received ethics approval from the X. Furthermore, this study has also been registered under the X. This study obtained ethics approval from the X for conducting research in Ministry of Health Malaysia (MOH) hospitals. In addition, the participants’ consent and agreement to participate in this study were obtained before proceeding with the data collection. All information obtained from the respondent, including their identity, would be kept confidential and only used for research purposes.

Results

Content validity

The items cognitive workload scale was initially generated based on previous literature^48–57 and assessed by eight experts for content validation. Based on the I-CVI scores, all items have values ranging from 0.88 to 1.00 in all variables of the content validity test. Since all items have a CVI of > 0.74 for all variables of the content validity test, all items were judged valid for relevance, clarity, simplicity, and ambiguity. This shows that all of the items are appropriate to be included in the Cognitive Workload Scale. Furthermore, modified Kappa Coefficient (K*) scores were calculated and showed values ranging from 0.88 to 1.00. This shows that all items are excellent.

Confirmatory factor analysis (CFA)

The eight-factor model was tested by CFA using the maximum likelihood estimation method. All items reached the set factor loading value of above 0.05, indicating that unidimensionality was achieved. At the initial review, it was found that the construct had failed to meet the value of the fit index that had been set. Therefore, the modification indices are evaluated to delete redundant items until the matching index value is reached. As a result, six items have been deleted. The goodness-of-fit model was as follows: the ratio (1.865), CFI (0.900), PCFI (0.821), PNFI (0.738), RMSEA (0.085) and IFI (0.90) (Table 4). Based on these criteria, the model was a good fit for the data.

Table 4.

Values of factor loadings, convergent validity, composite reliability and cronbach's alpha.

Construct	Item	Factor Loading	AVE (>0.50)	CR (>0.60)	Cronbach's Alpha (>0.70)
Mental Workload (MW)	MW1	0.91	0.68	0.89	0.87
	MW2	0.89
	MW3	0.84
	MW4	0.62
	MW5	Item was deleted
Human Reliability (HR)	HR1	0.86	0.73	0.93	0.93
	HR2	0.85
	HR3	0.90
	HR4	Item was deleted
	HR5	0.82
	HR6	0.84
Human-Computer Interaction (HCI)	HCI1	0.82	0.79	0.95	0.94
	HCI2	Item was deleted
	HCI3	0.89
	HCI4	0.81
	HCI5	0.96
	HCI6	0.94
Decision Making (DM)	DM1	0.97	0.83	0.97	0.97
	DM2	0.98
	DM3	0.91
	DM4	0.91
	DM5	0.89
	DM6	0.80
Skilled Performance (SP)	SP1	0.85	0.81	0.96	0.95
	SP2	0.93
	SP3	0.95
	SP4	0.93
	SP5	0.84
	SP6	Item was deleted
Training (TR)	TR1	0.97	0.93	0.98	0.98
	TR2	0.97
	TR3	0.96
	TR4	0.96
	TR5	Item was deleted
Work Stress (WS)	WS1	0.79	0.84	0.91	0.90
	WS2	Item was deleted
	WS3	0.73
	WS4	0.85
	WS5	0.81
	WS6	0.90
Cognitive Ergonomics Effect (EF)	EF1	0.80	0.60	0.86	0.85
	EF2	0.80
	EF3	0.65
	EF4	0.84

i) Fit Index: Ratio (<5*) = 1.865, RMSEAª (<0.10*) = 0.085, CFI^b (>0.90*) = 0.90, IFI^c (>0.90*) = 0.90, PNFI^d (0.50*) = 0.738, PCFI^e (>0.50*) = 0.821, * Threshold value of fit indices refers to previous studies by.^58,59

ii) a. Root Mean Square Error of Approximation, b. Comparative fit index, Incremental Fit Index, d. Parsimonious Normed Fit Index, e. Parsimonious Comparative Fix Index

Discriminant validity

Next, the HTMT criteria were used to assess discriminant validity. The value of the HTMT ratio between the constructs must be below 0.85 to confirm the establishment of discriminant validity. The calculated HTMT ratio in the range of −0.008 to 0.84 has met the set criteria (Table 3).

Convergent validity

The average variance extracted (AVE) was calculated to assess the convergent validity of each construct in the questionnaire. All constructs have passed the threshold for AVE (>0.50), indicating that the data is positively related to their construct (Table 4).

Internal consistency

Internal consistency of items was measured based on Cronbach's Alpha value. The value of 0.7 indicates that the items are highly interrelated in the questionnaire. The “TR” construct has the highest internal consistency, alpha = 0.98, followed by the “DM” construct, alpha = 0.97, and the “SP” construct, alpha = 0.95. The other constructs also have alpha values > 0.70, which denotes good internal consistency among all the items (Table 4).

Composite reliability

Besides, the constructs’ composite reliability (CR) was also evaluated to confirm the adequacy of the internal consistency of the items. A study by Hair, Hult⁴⁴ found that CR values of >0.7 indicate good reliability. The results of the CR of all constructs ranged from 0.86 to 0.98. This denotes that the questionnaire has high composite reliability (Table 4).

Test-retest reliability

A two-way mixed model was used to determine the intraclass correlation coefficient (ICC). A sample of 29 was required to acquire a minimum ICC of 0.70 and a maximum ICC of 0.90 with a power of at least 80%. The expected dropout rate was 20% for the test-retest. One of the 30 respondents was excluded from the study due to a large discrepancy with the other data. The ICC values calculated for all constructs ranged from 0.68 to 0.97 (Table 5). Only the “HR” construct showed test-retest reliability with an ICC value lower than 0.75, ICC = 0.68 (95% CI, 0.43–0.84), while all other constructs have ICC values above 0.75. The Standard Error of measurement (SEM) was calculated and is presented in Table 5. The “Skilled Performance” construct obtains the lowest SEM (1.11), while the “Human Reliability” construct obtains the highest SEM (3.32).

Table 5.

Standard deviation (SD), intraclass correlation coefficient (ICC), 95% confidence interval and standard error of measurement (SEM).

	SD	ICC	95% CI		SEM
	SD	ICC	Lower	Upper	SEM
Mental Workload (MW)	6.57	0.96	0.91	0.98	1.35
Human Reliability (HR)	5.86	0.68	0.43	0.84	3.32
Human-Computer Interaction (HCI)	8.26	0.97	0.94	0.99	1.45
Decision Making (DM)	6.44	0.94	0.87	0.97	1.64
Skilled Performance (SP)	5.42	0.96	0.91	0.98	1.11
Training (TR)	6.26	0.93	0.87	0.97	1.61
Work Stress (WS)	10.68	0.92	0.84	0.96	3.02
Cognitive Ergonomics Effect (EF)	10.36	0.96	0.92	0.98	2.02

Correlation between cognitive workload scale and SMEQ

The Pearson Correlation test results show that there is a significant relationship between the cognitive workload scale and the Malay version of SMEQ with a significant value, p < 0.01. With a significant value, p = 0.002 and a coefficient value, r = 0.277 shows a weak and positive correlation between the cognitive workload scale and the SMEQ Malay version.

Effects of the AHP profession group and work stress on PWI scores

A two-way independent ANOVA test showed a significant interaction between the AHP group and work stress on PWI scores (F (2, 114) = 4.135, p = 0.018). The significant interaction results are interpreted as in Figure 1. The figure shows a significant decrease in the personal well-being index (PWI) score when the work stress score is high for the laboratory group, compared to the clinical and public health groups.

Figure 1.

Effect of the AHP profession group and work stress on PWI scores.

Effects of the AHP profession group and cognitive load on PWI scores

A two-way independent ANOVA test showed a significant interaction between the AHP group and the effect of cognitive load on PWI scores (F (2, 114) = 4.604, p = 0.012). The significant interaction results are interpreted in the form of Figure 2. The figure shows a significant decrease in the personal well-being index (PWI) score when the cognitive load effect score is high for the laboratory group, compared to the clinical and public health groups.

Figure 2.

Effect of the AHP profession group and cognitive load effect on PWI scores.

Discussion

Although cognitive workload was studied about 60 years ago, only a few valid and reliable questionnaires have been created to effectively assess and analyze the factors and effects of cognitive workload.²² Therefore, this paper is aimed at determining the validity and reliability of the self-developed Cognitive Workload Scale. The final version of the cognitive workload scale was validated to evaluate the level of cognitive workload among allied health professionals. There are a total of eight constructs in the cognitive workload scale: Mental Workload (MW), Human Reliability (HR), Human-computer Interaction (HCI), Decision Making (DM), Skilled Performance (SP), Work Stress (WS), Training (TR), and Effects of Cognitive Load (CE). The content validity of the self-developed scale was shown to be good after being evaluated by experts.

However, redundancy was found in the CFA stage. Thus, six items were eliminated based on the appropriateness of the fit indices of the model. After item deletion, the final version of the questionnaire consists of 38 items. Each construct consists of at least four items. The CFA findings show a good fit for the questionnaire; at least one of all three types of fit indices (absolute, relative and parsimonious) is within an acceptable range. This proves that the construct validity of the questionnaire is good. Furthermore, the values of Cronbach's alpha and composite reliability show that all constructs possess high internal consistency and reliability. This indicates that all the items accurately and consistently measure the intended construct.

The cognitive workload scale shows good to excellent test-retest reliability. Seven constructs (Mental Workload, Human-Computer Interaction, Decision Making, Skilled Performance, Training, Work Stress and Cognitive Load Effects) show excellent test-retest reliability (ICC > 0.75), and one construct (Human Reliability) shows good test-retest reliability (0.60 < ICC < 0.74). A smaller SEM indicates better reliability and precision. Due to measurement error, four scores are the minimum change needed for “Human Reliability” and “Work Stress” to detect a significant difference. Three score changes are required by “Cognitive Load Effects”, and the six other constructs require two score changes to indicate an actual difference in the two measurements.⁶⁰ Although the sample size might be too small to conclude the test-retest reliability of the questionnaire, the results of the two measurements show that this questionnaire is reliable in evaluating the cognitive workload of allied health professionals based on all constructs in the questionnaire.

The results of the study found a positive but weak correlation between the cognitive workload scale and the Malay version of SMEQ. These findings provide evidence that these tools differ in content even though they are designed to measure cognitive workload constructs. SMEQ describes the mental effort as a whole for cognitive function, while the cognitive workload scale focuses by including the constructs of Human Reliability (HR), Human-computer Interaction (HCI), Decision Making (DM), Skill Performance (SP), Work Stress (WS), Training (TR) and Cognitive Load Effects (CE). Although the SMEQ is a simple and easy-to-use instrument, it may be useful to evaluate by looking more deeply at other cognitive workload factors.

In this study found laboratory groups were significantly associated with a decrease in personal well-being when the work stress scores were high. This has been noted in other studies by Garcia et al.⁶¹ and Kelly et al.¹⁹ The US Bureau of Labor Statistics has identified certain workplace factors that contribute to laboratory technicians’ stress. These include job conditions like performing monotonous and repetitive tasks, dealing with excessive workloads, and working overtime without choice. Laboratories often involve a challenging work environment with high work demands, strict work policies, and sometimes long working hours.⁶² This may be influenced by the nature of the work, which plays an important role in determining the diagnosis and planning treatment for the patient. To produce accurate results, precise concentration should be given while the process takes place, and the results should be completed within the specified time. They always need to work very cautiously to ensure the accuracy of the results. A small mistake they commit in the investigation result would turn the patients’ treatment entirely wrong.⁶³ Also, according to Kelly et al.,¹⁹ the shift schedule and job control are among the things that distinguish the medical laboratory profession from other professions. As can be seen, two groups, which are the clinical group and the public health group, are working normal shift hours and are involved directly with patient care. Even these groups also reported stress at work^64,65 but working at normal hours can provide a work-life balance.⁶⁶ The study notes that those who work the night shift are more likely to experience role overload in the night shift, the same as laboratory workers.⁶³ All these factors not only act as demotivating factors for employees but also highly affect their personal well-being.

This study also found that the AHP in the laboratory group experiences more cognitive load effects, leading to lower personal well-being. It is reported that laboratory workers are at high risk of getting work-related musculoskeletal disorder.⁶⁷ This is linked to tasks that have a high cognitive load, causing certain muscles to become more tense than normal, especially tasks that involve decision-making and human technology interaction.⁶⁸ Furthermore, multitasking activity causes changes in stress response and can subsequently cause service quality to decrease.⁶⁹ A study conducted by Leso et al.⁷⁰ found that those who work in shifts/long working hours, such as laboratory workers, can experience negative effects on cognition, slow response and fatigue. The effect on cognitive increases errors to occur. This study also revealed that clinical groups, such as radiography professionals, recorded a high level of well-being despite the impact of cognitive load effects. The results of this study differ from those recorded by Jacquet et al.⁷¹ who found that the well-being of radiographers was also poor. However, this study found that self-compassion helps in increasing the level of personal well-being.⁷¹ Self-compassion allows individuals to better cope with stress at work and develop greater well-being. While nutritionists in public health groups show that most nutritionists are satisfied with their jobs.⁷² Job satisfaction makes employees more enthusiastic, experience happiness at work and do not experience easy fatigue.⁷³ Furthermore, it can improve the quality of work.

More stress prevention strategies and work policies can be implemented for laboratory AHPs to improve their well-being. Time management tools, cognitive training programs and stress reduction techniques such as meditation and mindfulness can contribute to optimising workload. However, organizationally focused interventions are seen to be more successful in promoting healthier workplaces compared to individually focused interventions.⁷⁴ Studies find opportunities to speak up and more control and comprehensive employee involvement can improve the well-being of employees regardless of the type of intervention.⁷⁵ Based on cognitive load theory, working memory information is very limited. Allied health professionals need to know how to reduce distractions and avoid multitasking, especially in complex care processes to avoid errors.⁷⁶ Among the examples of interventions carried out by other health workers is wearing a “Do Not Disturb” jacket to reduce interference during specific procedures carried out on patients.⁷⁷ In addition, No Disturbance Zone signs and diversion strategies are used during medication administration procedures.^78,79 Examples of diversion strategies involve other staff doing other tasks example phone calls or patient requests so the process of patient procedure is not interrupted. Next, the “3 GOOD THINGS” intervention is seen to help overcome a decline in work, depression, work-life balance problems and happiness among health workers.⁸⁰

Limitation

The limitation of the study is that the online distribution of the questionnaire may cause some problems. The participant does not have the opportunity to request clarification on the items, which they feel are less understandable. Also, the instructions to fill out the questionnaire were not given verbally; participants had to understand the instructions themselves. This may cause misunderstandings and incorrect responses to the questionnaire, leading to inaccurate data obtained. For example, in the test-retest reliability test, one participant responded with data that had a large discrepancy with other data. Therefore, the participant was excluded from the study.

As the second smallest state in Malaysia, Malacca has only 3 districts and an estimated 1.3 per cent of the total land in Malaysia. The small geographical factor of the state resulted in a smaller sample size representing the entire allied health profession in Malaysia. A small sample size will be prone to sampling error. It means that statistical tests carried out on samples cannot be generalized to the values of the study population, which are called parameters. Next, the cognitive load scale was developed in Malay and sampled the Malaysian population. Therefore, further validation and adaptation for different languages is needed because Malaysia comprises various races.

Conclusion

In conclusion, the Cognitive Workload Scale demonstrated good validity and reliability across the eight constructs. It provides a robust tool for assessing cognitive workload among allied health professionals. Furthermore, the effect of cognitive workload factors has been found to impact the well-being of AHPs. Thus, healthcare organizations should consider implementing interventions addressing work stress and effects of cognitive load . Investment in user-friendly technology interfaces, providing ergonomic equipment and appropriate training, workload redistribution, task prioritization, and stress management programs are the interventions suggested to be applied by these organizations. By implementing these suggestions, healthcare organizations can create a conducive cognitive work environment for allied health professionals, improving their wellbeing, performance, and overall quality of care.

Footnotes

Acknowledgements

The authors would like to thank the participants for taking part in the study.

ORCID iDs

Shahirah Md Rasid

Hanif Farhan Mohd Rasdi

Ethical approval

This study was approved by the Research Ethics Committee, The National University of Malaysia (UKM PPl/111/8/JEP-2022-525) and the National Medical Research Register (NMRR ID-22-01822-HFY and NMRR ID-22-02035-3CM).

Informed consent

All respondents were informed of relevant information before being invited to participate in our study and online consent was obtained.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Kazemi

Smith

. Overcoming COVID-19 pandemic: emerging challenges of human factors and the role of cognitive ergonomics. Theor Issues in Ergon Sci 2023; 24: 401–412.

. Frontline health care workers’ mental workload during the COVID-19 pandemic: a cross-sectional study. Asia-Pac J Public Health 2021; 33: 303–305.

Carayon

Wust

Hose

, et al. Human factors and ergonomics in health care. In: Handbook of human factors and ergonomics. New Jersey: John Wiley & Sons Inc., 2021, pp. 1417–1437.

Privitera

. Promoting clinician well-being and patient safety using human factors science: reducing unnecessary occupational stress. Health 2022; 14: 1334–1356.

Stevens

Hulme

Salmon

. The impact of power on health care team performance and patient safety: a review of the literature. Ergonomics 2021; 64: 1072–1090.

Howie

Dharanikota

Gunn

, et al. Cognitive load management: an invaluable tool for safe and effective surgical training. J Surg Educ 2023; 80: 311–322.

Heponiemi

Gluschkoff

Vehko

, et al. Electronic health record implementations and insufficient training endanger nurses’ well-being: cross-sectional survey study. J Med Internet Res 2021; 23: e27096.

Nestor

O’Tuathaigh

O’Brien

. Assessing the impact of COVID-19 on healthcare staff at a combined elderly care and specialist palliative care facility: a cross-sectional study. Palliat Med 2021; 35: 1492–1501.

Mäkikangas

Leiter

Kinnunen

, et al. Profiling development of burnout over eight years: relation with job demands and resources. Eur J Work Org Psychol 2021; 30: 720–731.

10.

Khatab

Hanna

Rofaeil

, et al. Pathologist workload, burnout, and wellness: connecting the dots. Crit Rev Clin Lab Sci 2023; 61: 1–21.

11.

Christy

Duraisamy

. Ergonomics and employee psychological well being. Int J Manag 2020; 11: 435–438.

12.

Szulewski

Howes

van Merriënboer

JJG

, et al. From theory to practice: the application of cognitive load theory to the practice of medicine. Acad Med 2021; 96: 24–30.

13.

Vella

Hall

van Merrienboer

JJG

, et al. An exploratory investigation of the measurement of cognitive load on shift: application of cognitive load theory in emergency medicine. AEM educ Train 2021; 5: e10634.

14.

Mohd Pauzi

. Investigation on the effect of cognitive ergonomic of final year student workload. 2018.

15.

Mohd'Aizat

. Mental workload assessment among healthcare workers during pandemic COVID-19 at Hospital Tunku Azizah/Mohd'Aizat Mohd Basari . Doctoral dissertation, Universiti Malaya, 2020.

16.

Profile of Allied Health Professions in Ministry of Health Malaysia A Consultation Report by World Health Organization (WHO). In: Allied Health Sciences Division MP, Ministry of Health, Malaysia., editor. Policy & Strategic Planning Section, Allied Health Sciences Division, Medical Programme, Ministry of Health Malaysia 2020.

17.

Muaidi

Shanb

. Effects of work demands on physical therapists in the KSA. J Taibah Univ Med Sci 2016; 11: 56–62.

18.

Gorce

Jacquier-Bret

. Global prevalence of musculoskeletal disorders among physiotherapists: a systematic review and meta-analysis. BMC Musculoskelet Disord 2023; 24: 265.

19.

Kelly

Soles

Garcia

, et al. Job stress, burnout, work-life balance, well-being, and job satisfaction among pathology residents and fellows. Am J Clin Pathol 2020; 153: 449–469.

20.

Rotem

EPDA

Roberts

. Profile of Allied Health Professions in Ministry of Health Malaysia A Consultation Report by World Health Organization (WHO). In: Allied Health Sciences Division MP, Ministry of Health, Malaysia., editor. 2020.

21.

Coates

Clerke

. Training interventions to equip health care professionals with shared decision-making skills: a systematic scoping review. J Contin Educ Health Prof 2020; 40: 100–119.

22.

Fista

Azis

Aprilya

, et al. Review of cognitive ergonomic measurement tools. IOP Conf Ser: Mater Sci Eng 2019; 598: 012131.

23.

Guidelines On Ergonomics Risk Assessment At Workplace. In: Department Of Occupational Safety And Health MOHRM, editor. Department Of Occupational Safety And Health, Ministry Of Human Resources Malaysia: Department Of Occupational Safety And Health, Ministry Of Human Resources Malaysia; 2017.

24.

Valdez

Montenegro

Crawford

, et al. Translation frameworks and questionnaire design approaches as a component of health research and practice: a discussion and taxonomy of popular translation frameworks and questionnaire design approaches. Soc SciMed 2021; 278: 113931.

25.

Liu

Wang

Liu

, et al. What is the meaning of health literacy? A systematic review and qualitative synthesis. Fam Med Community Health 2020; 8: e000351.

26.

Valdez

Montenegro

Crawford

27.

Sha

Gabel

. The essential role of language in survey research. USA: Rti Press, 2020.

28.

Weng

Y-H

Kuo

Yang

C-Y

, et al. Implementation of evidence-based practice across medical, nursing, pharmacological and allied healthcare professionals: a questionnaire survey in nationwide hospital settings. Implement Sci 2013; 8: 1–10.

29.

Shahirah

Hanif Farhan

Dayana Hazwani

MSN

. Translation, validation and reliability testing of the subjective mental effort questionnaire (SMEQ) from English into the malay language. Med Health 2023; 18: 515–529.

30.

, et al. Physician mental workload scale in China: development and psychometric evaluation. BMJ Open 2019; 9: e030137.

31.

WHO. WHO Guidelines on Translation and Adaptation of Instruments. 2017.

32.

Walker

Stanton

Halvorsen

, et al. Allied health professionals: hidden but essential. In: Research handbook on contemporary human resource management for health care. USA: Edward Elgar Publishing, 2024, pp. 149–163.

33.

Kalakoski

Henelius

Oikarinen

, et al. Cognitive ergonomics for data analysis. Experimental study of cognitive limitations in a data-based judgement task. BehavInf Technol 2019; 38: 1–10.

34.

Bevan

Macleod

. Usability measurement in context. Behav Inf Technol 1994; 13: 132–145.

35.

Tinagaran

Mohd Rasdi

Harun

, et al. Psychometric properties of the malay version of the personal wellbeing index: a preliminary study among caregivers of children with autism spectrum disorder. Jurnal Sains Kesihatan Malaysia (Malaysian Journal of Health Sciences) 2023; 21: 157–165.

36.

Dutil

Bottari

Auger

. Test-retest reliability of a measure of independence in everyday activities: the ADL profile. Occup Ther Int 2017; 2017: 3014579.

37.

Polit

Beck

. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health 2006; 29: 489–497.

38.

Zamanzadeh

Rassouli

Abbaszadeh

, et al. Details of content validity and objectifying it in instrument development. Nurs Pract Today 2014; 1: 163–171.

39.

Ab Hamid

Sami

Mohmad Sidek

. Discriminant validity assessment: use of fornell & larcker criterion versus HTMT criterion. J Phys: Conf Ser 2017; 890: 012163.

40.

Awang

. A handbook on structural equation modeling. Bandar Baru Bangi, Selangor, Malaysia: MPWS Rich Resources, 2014.

41.

Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 1999; 6: 1–55.

42.

Meyers

Gamst

Guarino

. Applied multivariate research: Design and interpretation. London: Sage publications, 2016.

43.

Hair Jr

Sarstedt

Hopkins

, et al. Partial least squares structural equation modeling (PLS-SEM). Eur Bus Rev 2014; 26: 106–121.

44.

Hair

Hult

GTM

Ringle

, et al. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM). 2022.

45.

Senthilnathan SJAaS. Usefulness of correlation analysis. 2019.

46.

Cicchetti

. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 1994; 6: 284.

47.

Bland

Altman

. Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8: 135–160.

48.

Cain B. A review of the mental workload literature. 2007. Available from: https://apps.dtic.mil/sti/pdfs/ADA474193.pdf [cited 12 Feb 2023].

49.

Hancock G, Longo L, Young M, et al. Mental workload. In: Handbook of human factors and ergonomics. New Jersey: John Wiley & Sons Inc., 2021, pp. 203–226.

50.

Goyal

Tilley

Latifi

. Healthcare provider-centered: ergonomics of movement and functionality. In: The modern hospital. Switzerland: Springer, 2019, pp. 195–201.

51.

Spiegel

. Cognitive process elements of people decision-making. In: Encyclopedia of information science and technology. 4th ed. USA: IGI Global, 2018, pp. 2076–2084.

52.

Featherston

Downie

Vogel

, et al. Decision making biases in the allied health professions: a systematic scoping review. PLoS One 2020; 15: e0240716.

53.

Drews

Bakdash

. Simulation training in health care. Rev Hum Factors Ergon 2013; 8: 191–234.

54.

Shinners

Aggar

Grace

, et al. Exploring healthcare professionals’ understanding and experiences of artificial intelligence technology use in the delivery of healthcare: an integrative review. Health Informat J 2020; 26: 1225–1236.

55.

Liu

Zhang

Yin

, et al. Human errors and human reliability. In: Handbook of human factors and ergonomics. New Jersey: John Wiley & Sons Inc., 2021, pp. 514–572.

56.

Ahmed

Saada

Jones

, et al. Medical errors: healthcare professionals’ perspective at a tertiary hospital in Kuwait. PLoS One 2019; 14: e0217023.

57.

Chen

Fang

Guo

, et al. The moderation effects of task attributes and mental fatigue on post-interruption task performance in a concurrent multitasking environment. Appl Ergon 2022; 102: 103764.

58.

Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model: Multidiscip J 1999; 6: 1–55.

59.

Meyers

Gamst

Guarino

. Applied multivariate research : design and interpretation. Thousand Oaks: Sage Publications, 2006.

60.

Roaldsen

Måøy

ÅB

Jørgensen

, et al. Test-retest reliability at the item level and total score level of the Norwegian version of the spinal cord injury falls concern scale (SCI-FCS). J Spinal Cord Med 2016; 39: 317–326.

61.

Garcia

Kundu

Kelly

, et al. The American society for clinical pathology’s job satisfaction, well-being, and burnout survey of laboratory professionals. Am J Clin Pathol 2020; 153: 470–486.

62.

Alrawahi

Sellgren

Altouby

, et al. The application of herzberg's two-factor theory of motivation to job satisfaction in clinical laboratories in omani hospitals. Heliyon 2020; 6: e04829.

63.

Rajan

. Shift work, workload, and professionalism related motivators affecting job satisfaction: an empirical study among medical laboratory technicians. Health Econ Manag Rev 2023; 4: 1–21.

64.

Ewen

Jenkins

Jackson

, et al. Well-being, job satisfaction, stress and burnout in speech-language pathologists: a review. Int J Speech-Language Pathol 2021; 23: 180–190.

65.

Popova

Hahn

Morris

, et al. Exploring well-being: resilience, stress, and self-care in occupational therapy practitioners and students. OTJR (Thorofare N J) 2023; 43: 159–169.

66.

Priyadharshini

Divya

. Work-life balance in the healthcare industry. Univ J Econ 2023; XXVII: 91–98.

67.

AlNekhilan

AlTamimi

AlAqeel

, et al. Work-related musculoskeletal disorders among clinical laboratory workers. Avicenna J Med 2020; 10: 29–34.

68.

Deeney

O’Sullivan

. Effects of cognitive loading and force on upper trapezius fatigue. Occup Med 2017; 67: 678–683.

69.

Becker

Kaltenegger

Nowak

, et al. Biological stress responses to multitasking and work interruptions: a randomized controlled trial. Psychoneuroendocrinology 2023; 156: 106358.

70.

Leso

Fontana

Caturano

, et al. Impact of shift work and long working hours on worker cognitive functions: current evidence and future research needs. Int J Environ Res Public Health 2021; 18: 6540.

71.

Jacquet

Paucsik

Guy

J-B

, et al. Self-compassion and psychological well-being of radiographers at work. Int J Qual Studies Health Well-Being 2024; 19: 2287621.

72.

Costa

Stangarlin-Fiori

Bertin

, et al. Satisfaction of nutritionists who work in food service. Revista de Nutrição 2019; 32: e180168.

73.

Kuo

T-S

Chu

L-C

Kao

P-L

, et al. The effect of job satisfaction on psychological well-being for Taiwanese home-care workers, mediated by emotional labor. Healthcare (Basel) 2023; 11: 2514.

74.

Cohen

Pignata

Bezak

, et al. Workplace interventions to improve well-being and reduce burnout for nurses, physicians and allied healthcare professionals: a systematic review. BMJ Open 2023; 13: e071203.

75.

Fox

Johnson

Berkman

, et al. Organisational-and group-level workplace interventions and their effect on multiple domains of worker well-being: a systematic review. Work Stress 2022; 36: 30–59.

76.

Wiradhany

van Vugt

Nieuwenstein

. Media multitasking, mind-wandering, and distractibility: a large-scale study. Atten Percept Psychophys 2020; 82: 1112–1124.

77.

Berdot

Vilfaillot

Bezie

, et al. Effectiveness of a ‘do not interrupt’ vest intervention to reduce medication errors during medication administration: a multicenter cluster randomized controlled trial. BMC Nurs 2021; 20: 153.

78.

Raban

Westbrook

. Are interventions to reduce interruptions and errors during medication administration effective?: a systematic review. BMJ Qual Saf 2014; 23: 414–421.

79.

Kellogg

Puthumana

Fong

, et al. Understanding the types and effects of clinical interruptions and distractions recorded in a multihospital patient safety reporting system. J Patient Saf 2021; 17: e1394–ee400.

80.

Sexton

Adair

. Forty-five good things: a prospective pilot study of the three good things well-being intervention in the USA for healthcare worker emotional exhaustion, depression, work–life balance and happiness. BMJ Open 2019; 9: e022695.