Comparing workload and performance during demanding push force and mental tasks

Abstract

BACKGROUND:

The effects of performing occupational tasks that demand physical and mental efforts in combined and simultaneous approaches are unknown, especially when pushing force is analyzed as a physical effort and solving arithmetic problems as a mental effort.

OBJECTIVE:

In this study, physical and mental demands were simulated in a lab environment to assess the workload.

METHODS:

Using a push force dynamometer, the maximum push force strength of each participant was recorded, and the physical demand was simulated exerting the push force in low, medium, and high levels. Mental demands were simulated solving arithmetic tasks in low, medium, and high levels. Two experimental conditions were defined: (1) task with combined physical and mental workload (performing physical demands first and mental demands after) and (2) task in a simultaneous way (performing both demands at the same time. NASA-TLX Traditional and RAW were applied to assess the workload.

RESULTS:

The time to complete the tasks was significantly longer in the combined than the simultaneous approach, and performance was significantly higher in the combined than the simultaneous tasks.

CONCLUSION:

The combined approach obtained better results than simultaneous and Traditional NASA-TLX presented a significantly higher level of global workload index than RAW.

Keywords

Simultaneous combined NASA-TLX traditional RAW

1 Introduction

1.1 Relevance

In today’s work environments, cognitive demands are increasingly increasing on workers, while physical demands are decreasing, emerging two types of forms for evaluation: combined and simultaneous approach. These types of workload evaluation are characterized by using a physical load task and a mental load task when executing a task and without pausing to continue the next task is called the combined approach while executing both tasks at the same time is known as the simultaneous approach. Examples of these jobs are those performed by: computer numerical control machinery operators [1], assemblers [2, 3], drivers [4, 5], forklift operators [6], mechanics, machinery assembly [7, 8], industrial fitters [9], electromechanical maintenance technicians [10], specialized operators in numerical control centers [11], vulcanizers [12, 13], among others. With the introduction of new technologies in different work environments, high cognitive demands have been created that have caused secondary effects such as tension, fatigue, a feeling of monotony, stress, cardiovascular problems, musculoskeletal disorders (MSD), depression, substance abuse, psychiatric disorders mild, exhaustion, low self-perception of health, among others [14 –16].

There is a lack of studies that involve physical and mental workload, also the evaluations are not developed in a combined and simultaneous approach, nor comparisons between them. The use of pushing force is very common in industrial tasks, they are characterized where the orientation of the force is in a horizontal direction [17]. Gómez-Londoño and González-Correa [18] and Hoozemans et al. [19] defined the concept of pushing force as the deployment of a force exerted by a person in horizontal trajectory. The use of the pushing force at the international level is given through an evaluation of the tasks of this force focusing on the criteria determined in ISO 11228-2 [20], which is established in the investigations of Snook and Ciriello [21].

In Mexico, two official norms have been approved to identify and evaluate the mental and physical work components, the NOM-035-STPS-2018 and NOM-036-STPS-2019, respectively [22, 23]. The diseases with the highest incidence number in Mexico are hearing loss, pneumoconiosis, and respiratory conditions, but it is noteworthy that there are a significant number of occupational - physical injuries: musculoskeletal disorders [24]. These diseases group together: dorsopathy, enthesopathy, carpal tunnel syndrome, shoulder injuries, radial styloid tenosynovitis of Quervain syndrome, synovitis, tenosynovitis, bursitis, epicondylitis, and osteoarthritis. Therefore, musculoskeletal disorders currently represent the first type of occupational disease and heavy physical injuries [25]. According to data from the Mexican Social Security Institute [26], by 2020 there were 5948 cases of musculoskeletal disorders, out of a total of 15136 work-related illnesses, which represents 39% of all illnesses.

1.2 Evaluation methods

Ergonomic assessment methods usually focus on cognitive ergonomics (such as mental workload or usability) or physical ergonomics (such as physical workload or body posture assessment). However, the human task demands both cognitive and physical resources in a combined or simultaneous approach and there is a lack of methods able to assess these kinds of tasks. Thus, the purpose of combined-simultaneous assessment is to achieve a more comprehensive assessment approach and make assessment more effective than when using separate assessment methods for cognitive and physical ergonomic aspects [27]. For workload assessment, some methods allow obtaining the level of mental workload and physical workload perceptively. One of the most used methods is the NASA method (TLX). This method was developed by Hart and Staveland [28], is a subjective tool for obtaining workload, which has been used in two ways to obtain load levels: Traditional and RAW. The most widely used approach that makes use of the full algorithm is known as NASA-TLX Traditional [28 –32]. However, there are investigations where the pairwise comparisons (assessment phase) of the Traditional approach were eliminated arguing that there are no significant differences between the two approaches or some have eliminated variables as the physical effort of the analysis, saying that better results were obtained [33 –36]. This approach is known as NASA-TLX RAW. It distinguishes six dimensions of mental load (mental demand, physical demand, time demand, performance, effort, and level of frustration), from which it calculates a global index of mental load. This method can be used in two ways to obtain the mental workload level (Traditional and RAW) [37]. For example, DiDomenico and Nussbaum [38, 39], DiDomenico [40], Arellano et al. [15] and Young et al. [14] developed the evaluation of physical workload postures and mental workload during different evaluated activities. Many jobs are performed in a combined approach, evaluating them will allow finding work design deficiencies to prevent health problems and detect cognitive errors that can generate work problems [41]. Psychosocial risks have arisen at work affecting health and quality of work-life since they increase levels of fatigue and stress, causing various problems in workers, mentioned in the research by Gil-Monte [29].

1.3 Objectives

The objectives of the investigation were two. To assess the workload score during a task performed in combined and simultaneous approaches, as well as the effects on the perception of workload and performance during the development of a task that involves performing push force. And to compare the global workload index (GWI) of NASA-TLX Traditional (weighted) against NASA-TLX RAW (unweighted).

1.4 Limitations

In this research, only laboratory studies were carried out due to the global health contingency situation. The research was carried out with student volunteers between the ages of 18 and 35, performing simulations of industrial tasks in the laboratories of ergonomic product design and ergonomics.

2 Materials and methods

2.1 Study design

The study design was quasi-experimental, descriptive, cross-sectional, correlational, and comparative. The study was carried out in the design and testing laboratory of the company ErgoTechMx. The order of the tests at their different levels was random and participants provided their consent to the study before performing the tasks.

2.2 Sample

A total of 32 volunteers completed the study according to the following inclusion criteria: free of heart or respiratory problems, hypertension, or injuries in the last year before the experiment. Not having performed strenuous exercises in the two days before the session. Not have drunk alcohol two days before the experiment sessions. Non-smoking people. During the experiment, participants could be excluded if they manifest any musculoskeletal pain in the upper or lower extremities, headache, or mental exhaustion. Subjects wear comfortable clothing such as tennis, jeans or pants, cotton or polyester shirt, and cotton socks.

2.3 Variables

2.3.1 Independent

The items of the mental and physical workload are commonly evaluated in industrial tasks such as production lines and manual assembly lines. The levels of physical and mental workload were considered as independent variables based on the proposal of DiDomenico and Nussbaum [39] and Taylor et al. [43], who established values/scores for the variables. Accordingly, two independent variables were considered: physical workload and mental workload at three different levels (low, medium, and high) shown in Table 1.

Table 1
Independent variables

Independent

Physical workload Mental workload

Level Activity Level Activity

Low Exert push force (25%) Low Answer 60 arithmetic operations (sums)

Medium Exert push force (50%) Medium Answer 60 arithmetic operations (subtractions)

High Exert push force (75%) High Answer 60 arithmetic operations (multiplications)

Independent
Low	Exert push force (25%)	Low	Answer 60 arithmetic operations (sums)
Medium	Exert push force (50%)	Medium	Answer 60 arithmetic operations (subtractions)
High	Exert push force (75%)	High	Answer 60 arithmetic operations (multiplications)

2.3.2 Dependent

The dependent variables were determined considering the research of Cornejo Chávez [44], González [45], and Veltman and Gaillard [46] who gave it different levels (see Table 2).

Table 2
Dependent variables

Dependent

Name Unit of measurement Levels

Workload perception NASA-TLX with Likert scale 1 to 5 [25] Low: 1 to 2 Medium: 3 to 4 High: 5

Mental performance # of correct answers [18] Low: 0-33% Medium: 33-66% High: 67-100%

Perceived effort Borg scale 0 to 10 [26] Low: 0 to 3 Medium: 4 to 7 High: 8 to 10 [27]

Dependent
Workload perception	NASA-TLX with Likert scale 1 to 5 [25]	Low: 1 to 2 Medium: 3 to 4 High: 5
Mental performance	# of correct answers [18]	Low: 0-33% Medium: 33-66% High: 67-100%
Perceived effort	Borg scale 0 to 10 [26]	Low: 0 to 3 Medium: 4 to 7 High: 8 to 10 [27]

A questionnaire of the NASA-TLX method was given to the participants, where they answered their perception of the mental workload of the activities, as well, a section asking about general demographic information.

The variables for mental and physical workload were considered for both evaluations (combined and simultaneous), as well as the three levels (low, medium, and high) to test the different approaches-levels.

2.4 Materials

The following materials were used in the development of the laboratory experiment:

Force meter dynamometer: Threshold setting: 0.8 N. to 200 N., accuracy within 1% of the reading (designed and built for this study).

Transmitter box and cable to connect force meter to a computer (designed and built for this study)

Software to capture the force exerted (designed and built for this study)

Computer

Chronometer

Worktable

Anti-skid mat

Pen and paper (questionnaires)

2.5 Procedure and groups of activities

2.5.1 Reception of participants

Once participants arrived at the lab, they read and signed an informed consent form. The participants were informed about the layout of the experiment, and the procedure to determine their maximum strength was explained to the participants.

2.5.2 Maximum force determination and mental activities

Determination of the levels of physical activity for each participant consisted of two stages. In the first, the maximum strength of each participant was recorded three times (the average of the three values was used) using the Caldwell protocol [47] and granting a rest of ten minutes between each cycle to minimize the effects of fatigue. In the second stage, the corresponding percentages were determined for low (25%), medium (50%), and high (75%) of the maximum force (See Fig. 1).

For mental activity, a list of ten arithmetic operations (sums, subtraction, and multiplications), according to the level of workload demands, were answered. Participants were instructed to perform arithmetic tasks trying to minimize errors in the answers.

Fig. 1

Determination of maximum force.

2.5.3 Procedure for the combined approach

The force percentage is placed in the designed software, which emits an alarm that is activated when the participant reached the force value. This prevents the participants from exceeding the percentage of the desired force (See Fig. 1).

The participants get ready to start the experiment and push the force meter for five seconds.

Once the participant reaches the previously captured force level (depending on the level to be evaluated), the software emits a sound indicating the participant must stop.

Upon completion of the physical activity, the participant is asked to answer the questionnaire of arithmetic operations (ten items per sheet).

The person will repeat the push activity and respond to another series of arithmetic operations, until completing the six sheets.

In the end, time to rest is offered to the participant, and the final questionnaire is given where the perception of the activities carried out is answered.

Figure 2 offers a diagram of the combined approach.

Fig. 2

Pushing force activity in the combined approach.

2.5.4 Procedure for the simultaneous approach

The participants get ready to start the experiment and push the force meter for five seconds.

While the person performs the physical activity, different arithmetic operations to answer (ten per sheet) are asked orally by the researcher.

The same routine is developed (physical and mental activities) until complete the six sheets.

In the end, time to rest is offered to the participant, and the final questionnaire is given where the perception of the activities carried out is answered.

Figure 3 offers a diagram of the simultaneous approach.

Fig. 3

Pushing force activity in simultaneous approach.

2.6 Statistical data analysis

To analyze the behavior of the data of the method used, the data from the study were collected in Excel 365 for Windows and migrated to IBM Statistical Package for Social Science (SPSS v23). Descriptive statistics (mean, minimum, maximum, standard deviation) were calculated. Analysis of Variance (ANOVA) was applied using 95% confidence intervals to explore the differences between the six different tests. The Post Hoc Tukey test was used to identify homogeneous subsets with the different combinations. The normality of the data was checked by applying the Kolmogorov-Smirnov and Shapiro-Wilk tests. The Wilcoxon test was applied to explore the differences between the dimensions/variables/items of the NASA-TLX method in the combined and simultaneous approaches. The T-Student test was used to make comparisons between the Traditional NASA-TLX and NASA-TLX RAW. A confidence level of 95% was used for all data analyses.

3 Results

The sample consisted of 32 participants (13 women and 19 men). The average age of the study participants was 22 years (SD: 1.17). During the combined approach of the experiment, the average time to finish the tests (physical and mental) was 7.37 minutes on average while in the simultaneous approach, the average time was 3.09 minutes on an average per test (physical and mental).

3.1 Descriptive results

Table 3 shows the mean and SD of the workload scores of the six dimensions of the NASA-TLX for both approaches (simultaneous and combined), considering the three levels analyzed (low, medium, and high). Table 4 shows the percentage of performance achieved from the participants as well as the time to finish the tasks in both approaches (simultaneous and combined). Table 5 presents the descriptive analysis of the test results of the NASA-TLX Traditional and NASA-TLX RAW.

Table 3
Descriptive values of the 6 NASA-TLX items in the 6 situations analyzed

Approach level Mental demand Physical demand Temporary requirements Effort Performance Frustration

CL 2.69 (1.18) 1.97 (0.73) 2.25 (0.94) 3.09 (1.04) 4.00 (0.90) 2.88 (0.93)

CM 3.41 (0.93) 2.44 (0.86) 2.63 (0.93) 3.16 (0.75) 3.50 (0.90) 3.28 (0.91)

CH 4.53 (0.56) 3.69 (.95) 3.44 (0.86) 4.00 (0.71) 2.97 (0.98) 3.53 (0.93)

SL 2.69 (0.88) 1.97 (0.73) 3.41 (0.93) 3.09 (0.98) 3.78 (0.86) 2.94 (0.90)

SM 3.41 (0.82) 2.78 (0.78) 3.94 (0.83) 4.03 (0.74) 3.13 (0.70) 4.00 (0.79)

SH 4.03 (0.68) 3.69 (1.07) 3.78 (0.78) 4.16 (0.71) 2.47 (0.90) 4.47 (0.71)

Approach level	Mental demand	Physical demand	Temporary requirements	Effort	Performance	Frustration
CL	2.69 (1.18)	1.97 (0.73)	2.25 (0.94)	3.09 (1.04)	4.00 (0.90)	2.88 (0.93)
CM	3.41 (0.93)	2.44 (0.86)	2.63 (0.93)	3.16 (0.75)	3.50 (0.90)	3.28 (0.91)
CH	4.53 (0.56)	3.69 (.95)	3.44 (0.86)	4.00 (0.71)	2.97 (0.98)	3.53 (0.93)
SL	2.69 (0.88)	1.97 (0.73)	3.41 (0.93)	3.09 (0.98)	3.78 (0.86)	2.94 (0.90)
SM	3.41 (0.82)	2.78 (0.78)	3.94 (0.83)	4.03 (0.74)	3.13 (0.70)	4.00 (0.79)
SH	4.03 (0.68)	3.69 (1.07)	3.78 (0.78)	4.16 (0.71)	2.47 (0.90)	4.47 (0.71)

Combined low = CL; Combined medium = CM; Combined high = CH; Simultaneous low = SL; Simultaneous medium = SM; Simultaneous high = SH.

Table 4

Performance and evaluation test time

Participant	Performance percentage						Test duration (min)
	CL	CM	CH	SL	SM	SH	CL	CM	CH	SL	SM	SH
1	89	83	81	88	76	77	6.2	7.1	11.4	2.4	2.4	3
2	87	80	79	76	81	76	8.1	4.6	8.8	2.9	3	3.6
3	87	77	85	88	67	78	5.3	6.5	8.4	2.8	2.2	3.6
4	94	86	86	86	76	82	5.5	6.2	11.3	3	2	3.3
5	97	79	74	81	71	70	5.8	6.9	9.2	2.8	3.3	3.7
6	92	87	73	80	77	82	5.3	5.6	8.9	2.7	3.1	4
7	87	95	75	82	68	74	6.9	6.1	9.1	3	3.4	4.2
8	95	91	72	81	66	78	5.1	5.8	10.3	2.9	3	4.3
9	93	81	77	84	74	81	6.8	6.3	10.1	2.7	2.6	3.9
10	97	86	79	78	70	77	5.5	8.4	8.2	2.9	2.9	3.2
11	85	87	84	91	83	74	5.2	6.8	11.4	2.6	2.6	3.5
12	84	85	79	77	74	79	4.3	5.2	10.2	3	3.1	4.1
13	83	84	74	81	67	78	6.5	5.9	8.4	2.6	3.4	3.5
14	96	83	80	83	82	72	6.6	8.6	11.3	2.8	3.7	3.7
15	94	75	85	87	83	75	5.2	7.2	8.1	3	3.4	3.2
16	96	87	83	80	80	76	5.2	6.9	7.8	2.7	3.9	4
17	97	91	70	77	77	77	6.6	7.9	11.6	2.9	2.5	4.4
18	98	94	84	79	66	82	5.8	5.9	10.3	2.2	3.5	4
19	89	92	75	78	68	78	4.4	6.4	9.6	2.2	3.4	3.4
20	82	88	82	79	78	77	5.1	8.2	10.4	2.6	3.1	3.3
21	85	97	83	84	72	79	6.7	7.4	8.3	2.4	3	3.3
22	97	88	75	82	76	71	6.6	4.9	9.7	2.1	2.6	3.3
23	82	81	86	80	77	77	4.9	7.1	12.1	2.3	3.8	3
24	96	89	82	83	66	74	4.4	5.1	9.2	2.3	3.1	3.5
25	94	79	76	81	70	75	6.1	7.6	8.7	2.5	3.2	3.1
26	82	84	78	85	71	70	5.3	8.3	9.5	2.3	2.7	3.6
27	97	85	77	78	82	75	4.5	6.3	10.1	2.6	2.9	3.7
28	89	77	80	84	70	79	6.3	6.5	10.8	2.4	2.8	3.2
29	88	88	83	90	77	82	5.8	6.2	10.3	2.6	2.2	3.4
30	92	84	85	78	81	75	4.9	6.3	9.4	2.5	3.6	3.6
31	86	85	82	89	75	76	4.8	7.9	10.2	2.8	3.7	3
32	91	77	86	84	72	79	5.3	7.8	10.3	2.6	3.3	3.6
Average	90.65%	85.15%	79.68%	82.31%	74.16%	76.72%	5.66 min	6.68 min	9.79 min	2.63 min	3.04 min	3.57 min
	Combined: 85.16%			Simultaneous: 77.76%			Combined: 7.37 min			Simultaneous: 3.09 min

Table 5

Descriptive analysis of the GWI of the Traditional and RAW forms of NASA-TLX for the different combinations made

Descriptive values	NASA-TLX Traditional
	CL	CM	CH	SL	SM	SH
Mean	56.85	58.30	74.79	57.82	67.15	74.91
Standard deviation	13.31	9.33	8.80	8.38	7.35	10.37
Maximum	81.33	73.33	88.00	77.33	80.00	90.67
Minimal	34.67	44.00	57.33	49.33	56.00	48.00
Rank	46.66	29.33	30.67	28.00	24.00	42.67
Descriptive values	NASA-TLX RAW
	CL	CM	CH	SL	SM	SH
Mean	53.64	58.18	71.21	55.76	69.70	73.94
Standard deviation	11.93	9.68	7.82	7.92	5.94	8.51
Maximum	76.67	70.00	80.00	73.33	80.00	83.33
Minimal	36.67	40.00	56.67	43.33	60.00	53.33
Rank	40.00	30.00	23.33	30.00	20.00	30.00

Combined low = CL; Combined medium = CM; Combined high = CH; Simultaneous low = SL; Simultaneous medium = SM; Simultaneous high = SH.

3.2 Combined vs. simultaneous workload approach comparison

Comparisons of NASA-TLX data between simultaneous and combined assessments are presented in Table 6.

Table 6
Comparisons between the combined and simultaneous forms of NASA-TLX

Level Variable Mean Dif. of P-

Combined Simultaneous means value

Low MD 2.69 2.69 0.00 1.000

Medium MD 3.41 3.41 0.00 1.000

High MD 4.53 4.03 0.50 0.132

Low PD 1.97 1.97 0.00 1.000

Medium PD 2.44 2.78 –0.34 0.518

High PD 3.69 3.69 0.00 0.493

Low TD 2.25 3.41 –1.16 0.082

Medium TD 2.63 3.94 –1.31 0.138

High TD 3.44 3.78 –0.34 0.005

Low EF 3.09 3.09 0.00 1.000

Medium EF 3.16 4.03 –0.87 0.796

High EF 4.00 4.16 –0.16 0.539

Low PE 4.00 3.78 0.22 0.192

Medium PE 3.50 3.13 0.37 1.000

High PE 2.97 2.47 0.50 0.371

Low FR 2.88 2.94 –0.06 0.506

Medium FR 3.28 4.00 –0.72 0.432

High FR 3.53 4.47 –0.94 0.068

Level	Variable	Mean	Dif. of	P-
Low	MD	2.69	2.69	0.00	1.000
Medium	MD	3.41	3.41	0.00	1.000
High	MD	4.53	4.03	0.50	0.132
Low	PD	1.97	1.97	0.00	1.000
Medium	PD	2.44	2.78	–0.34	0.518
High	PD	3.69	3.69	0.00	0.493
Low	TD	2.25	3.41	–1.16	0.082
Medium	TD	2.63	3.94	–1.31	0.138
High	TD	3.44	3.78	–0.34	0.005
Low	EF	3.09	3.09	0.00	1.000
Medium	EF	3.16	4.03	–0.87	0.796
High	EF	4.00	4.16	–0.16	0.539
Low	PE	4.00	3.78	0.22	0.192
Medium	PE	3.50	3.13	0.37	1.000
High	PE	2.97	2.47	0.50	0.371
Low	FR	2.88	2.94	–0.06	0.506
Medium	FR	3.28	4.00	–0.72	0.432
High	FR	3.53	4.47	–0.94	0.068

Mental demand = MD; Physical demand = PD; Temporary demand = TD; Effort = EF; Performance = PE; Frustration = FR.

Only the variable temporary demand at a high level obtained significant differences between combined and simultaneous approaches.

3.3 Comparison of NASA-TLX Traditional vs. NASA-TLX RAW

3.3.1 NASA-TLX Traditional

The results of the ANOVA test show that there are significant workload differences in at least one of the six different combinations analyzed (F: 7.564, p: 0.000). Table 7 presents the result of the Post Hoc Tukey test showing that two groups were generated with the six different combinations.

Table 7
Result of the Post Hoc Tukey NASA-TLX Traditional test

Factor Subset for alpha = 0.05

1 2

COMB_LOW 57.86

SIM_LOW 58.83

COMB_MED 59.33

SIM_MED 68.17 68.17

COMB_HIGH 75.79

SIM_HIGH 75.91

Sig. .199 .498

Factor	Subset for alpha = 0.05
COMB_LOW	57.86
SIM_LOW	58.83
COMB_MED	59.33
SIM_MED	68.17	68.17
COMB_HIGH		75.79
SIM_HIGH		75.91
Sig.	.199	.498

3.3.2 NASA-TLX RAW

The results of the ANOVA test show that differences among the mean score of mental workloads for the six different combinations are present and that at least one combination is different (F: 10.061, p:0.000). Table 8 presents the Post Hoc Tukey test with its result of the three groups generated and their six different combinations.

Table 8
Result of the Post Hoc Tukey NASA-TLX RAW test

Factor Subset for alpha = 0.05

1 2 3

COMB_LOW 54.85

SIM_LOW 56.86

COMB_MED 59.69 59. 69

SIM_MED 70.79 70. 79

COMB_HIGH 72.53

SIM_HIGH 74.92

Sig. .868 .064 .897

Factor	Subset for alpha = 0.05
COMB_LOW	54.85
SIM_LOW	56.86
COMB_MED	59.69	59. 69
SIM_MED		70.79	70. 79
COMB_HIGH			72.53
SIM_HIGH			74.92
Sig.	.868	.064	.897

3.4 Comparisons between NASA-TLX Traditional vs. NASA-TLX RAW forms

Table 9 shows the T-Student tests for the six paired tests compared to each other of the NASA-TLX Traditional and NASA-TLX RAW. NASA-TLX Traditional obtained higher values in the combined-high level.

Table 9
T-Student paired comparisons between NASA-TLX and NASA-TLX RAW

Combination/level NASA-TLX form Half Dif of means Percentage difference Sig. (bilateral)

Comb low Traditional 56.85 3.21 5.65% 0.051

RAW 53.64

Comb medium Traditional 58.30 0.12 0.21% 0.912

RAW 58.18

Comb high Traditional 74.79 3.58 4.79% 0.036

RAW 71.21

Sim low Traditional 57.82 2.06 3.56% 0.123

RAW 55.76

Sim medium Traditional 67.15 2.55 3.80% 0.151

RAW 69.70

Sim high Traditional 74.91 0.97 1.29% 0.602

RAW 73.94

Combination/level	NASA-TLX form	Half	Dif of means	Percentage difference	Sig. (bilateral)
Comb low	Traditional	56.85	3.21	5.65%	0.051
	RAW	53.64
Comb medium	Traditional	58.30	0.12	0.21%	0.912
	RAW	58.18
Comb high	Traditional	74.79	3.58	4.79%	0.036
	RAW	71.21
Sim low	Traditional	57.82	2.06	3.56%	0.123
	RAW	55.76
Sim medium	Traditional	67.15	2.55	3.80%	0.151
	RAW	69.70
Sim high	Traditional	74.91	0.97	1.29%	0.602
	RAW	73.94

3.5 Performance and time analysis

Performance was analyzed considering two situations. 1) Compare the quotient between the number of correct answers and the number of questions, and 2) time to finish the test. Table 10 shows the performance of the evaluations in combined and simultaneous approaches. Table 11 shows the time obtained from the evaluations in combined and simultaneous approaches.

Table 10
Performance comparison between combined and simultaneous approaches

Performance Mean Dif. of means P-value Interpretation

Combined 51.09 4.43 0.075 No significant differences were found between the performance in the combined and simultaneous approaches.

Simultaneous 46.66

Performance	Mean	Dif. of means	P-value	Interpretation
Combined	51.09	4.43	0.075	No significant differences were found between the performance in the combined and simultaneous approaches.
Simultaneous	46.66

Table 11

Time comparison between combined and simultaneous approaches

Time	Average (min)	Dif. of means	P-value	Interpretation
Combined	7.37	4.28	.000	Time to perform the task is greater in the combined than the simultaneous approach.
Simultaneous	3.09

4 Discussion

This laboratory experiment was planned considering the studies of Ayaz et al. [48], Bligárd and Osvalder [27], DiDomenico [40], DiDomenico and Nussbaum [38, 39], Jung and Jung [49], and Young et al. [14], where the comparison of physical demands and mental demands were investigated and analyzed in a combined and/or simultaneous approach. In this study, the NASA-TLX method was used to assess the workload level, considering the Traditional version of the method and the RAW form [50, 51].

The objective of the comparative study was to find the differences in physical and mental workload during a pushing task, considering the workload in their combined and simultaneous approach. The NASA-TLX Traditional method and NASA-TLX RAW were applied and a sample of 32 participants (18 men and 14 women) completed the study. It was carried out 15 paired T-Student comparisons of the variables used in the NASA-TLX method between the combined and simultaneous approaches, in which there were significant differences in 1 of the 15 comparisons.

Significant differences were found during NASA-TLX evaluation in the combined approach in the variables mental demand, physical demand at the high level, and performance at a low level. While in the simultaneous approach, significant differences were found in the variables of physical demand, temporal demand, effort, and frustration at the high level, as well as in temporal demand at the medium level. Tubbs-Cooley et al. [52] found that mental demand, physical demand, temporary demand, and effort were higher than the overall workload. Likewise, Helton et al. [53] established that physical demand does not appear to be a substantial component of the workload and that the correlations between workload elements and performance were also different depending on the assessed level.

The ANOVA and Post Hoc Tukey tests revealed that 2 groups were formed with the variables: 1. Low combined, low simultaneous, medium combined, medium simultaneous; 2. High combined and high simultaneous in the Traditional NASA-TLX. Finding no significant differences between the workloads in low combined and low simultaneous, high combined and high simultaneous, while the evident differences were shown in the rest of the combinations. For the evaluation using the NASA-TLX RAW, 3 groups were created: 1. Low combined, low simultaneous, medium combined; 2. Medium simultaneous, high combined; 3. High simultaneous. Where no significant differences were found between the workloads in combined low, low simultaneous, and combined medium, while there were differences in the rest of the combinations. For example, Hernández Arellano et al. [15] found that high levels of workload have an important effect on performance. On the contrary, Wanyan et al. [54] concluded that with a high level of load, a significant reduction in task performance was observed.

Even though DiDomenico [39] concluded that there were no differences between the two NASA-TLX forms, in this study we found significant differences between the Traditional and RAW NASA-TLX, being higher in the Traditional NASA-TLX mean scores in all evaluations and levels than RAW scores, its percentage difference from 3% to 5% on average in 4 comparisons of 6. The difference obtained between both forms of NASA-TLX deduces that the GWI values of NASA-TLX RAW were reduced, therefore, the RAW form offers a devalued of GWI. Our results coincided with Byers et al. [55], Grier [34], and Schmidtke [56] that suggested as more appropriate the NASA-TLX Traditional for workload calculation.

Performance analysis showed that overall correct answers and test duration were higher in the combined assessment, obtaining 85.16%, with an average time of 7.37 minutes to finish one of the six tests and a greater number of correct answers than the simultaneous approach. Compared to the simultaneous assessment in which their results showed a 77.76% overall performance with an average time of 3.09 minutes of test response and a smaller number of correct answers. Lacome et al. [57] and Wanyan et al. [54] concluded that there is a better level of workload and performance with the combined approach of elements such as mental and physical workload, avoiding fatigue and muscle damage.

5 Conclusion

During the tasks of physical and mental workload in combined and simultaneous approaches, the participants manifested high scores of physical and mental demands during the high level of the tasks, as well as in medium level during the simultaneous approach. By performing the simultaneous tasks, participants manifested a greater workload of activities at all levels, especially at the high and medium levels. This is because being executing the physical task, participants had to solve the arithmetic operations simultaneously, which caused a greater and faster mental fatigue than the combined approach, therefore, lower levels of performance were obtained.

NASA-TLX method in its Traditional form obtained a higher level of GWI compared to the RAW form. The combined approach showed a significantly better performance than the simultaneous approach. The time spent on the combined tasks was higher than many simultaneous tasks since the physical and mental tasks are executed separately in the combined approach, while both are performed simultaneously at the same time in the simultaneous approach. It is important to keep in mind that the tasks executed simultaneously generate greater stress, fatigue/physical discomfort like headache, feeling uncomfortable, arm pain, back pain, among others [58], and therefore a low performance due to the situation of developing two tasks at the same time. Further comparative research is needed to contrast and extend the understanding of using the NASA-TLX method to assess workload in combined and simultaneous approaches.

Author contributions

All authors have been involved in the study design and research plan. A.F.C.M. and J.L.H.A. conducted the lab study. A.F.C.M. and C.A.O.O. were responsible for the analysis. JLHA, A.A.M.M., and Y.R.R. produced the primary manuscript. All authors have seen and contributed to the drafting and revision of the manuscript and have approved the final version for publication.

Ethical approval

The study was conducted according to the guidelines of the Declaration of Institutional Ethics and Bioethics Committee of the Autonomous University of Ciudad Juárez and approved by the Ethics and Bioethics Committee from the Autonomous University of Ciudad Juárez (approval number CIEB-2018-1-35).

Informed consent

Informed consent was obtained from all subjects involved in the study.

Footnotes

Acknowledgments

The authors thank the ErgoTechMx company for providing the design and testing laboratory facilities to conduct the study.

Conflict of interest

The authors declare no conflict of interest.

Funding

This research received no specific grant from any funding agency in the commercial or not-for-profit sectors.

References

Hernandez Arellano

, Serratos Perez

, Garcia Alcaraz

, Maldonado Macias

. Assessment of Workload, Fatigue, and Musculoskeletal Discomfort Among Computerized Numerical Control Lathe Operators in Mexico. IISE Trans Occup Ergon Hum Factors. 2017;5(2):65–81.

Azizoğlu

, İmat

. Workload smoothing in simple assembly line balancing. Comput Oper Res. 2018;89:51–7.

Ghenghea

. Workload analyse of assembling process. IOP Conf Ser Mater Sci Eng. 2015;95(1).

Faure

, Lobjois

, Benguigui

. The effects of driving environment complexity and dual tasking on drivers’ mental workload and eye blink behavior. Transp Res Part F Traffic Psychol Behav. 2016;40:78–90.

Marquart

, Cabrall

, de Winter

. Review of Eye-related Measures of Drivers’ Mental Workload. Procedia Manuf [Internet]. 2015;3(Ahfe):2854–61. Available from: https://dx-doi-org.web.bisu.edu.cn/10.1016/j.promfg.2015.07.783

Nneji

, Cummings

, Stimpson

. Predicting locomotive crew performance in rail operations with human and automation assistance. IEEE Trans Human-Machine Syst. 2019;49(3):250–8.

Abdous

, Finco

, Visentin

. Workload evaluation of industrial work: Existing methods and practical applications. Proc Summer Sch Fr Turco. 2018;2018-Septe(2011):395–401.

Spada

, Ghibaudo

, Gilotta

, Gastaldi

, Cavatorta

. Analysis of exoskeleton introduction in industrial reality: Main issues and EAWS risk assessment. Adv Intell Syst Comput. 2018;602(i):236–44.

Liu

, Brooks

, Subrahmaniyan

, Miller

, Oman

. Measuring The Time Course of Engineer Workload During Automation Mode Transitions. Proc Hum Factors Ergon Soc Annu Meet. 2019;63(1):1555–9.

10.

Uʇurlu

, Karahan

, Ünlü

, Abbasoʇlu

, Özhan Elbaş

, Avcl Işlk

, et al. The Effects of Workload and Working Conditions on Operating Room Nurses and Technicians. Work Heal Saf. 2015;63(9):399–407.

11.

Fallahi

, Motamedzade

, Heidarimoghadam

, Soltanian

, Miyake

. Assessment of operators’ mental workload using physiological andsubjective measures in cement, city traffic and power plant controlcenters. Heal Promot Perspect [Internet]. 2016;6(2):96–103. Available from: https://dx-doi-org.web.bisu.edu.cn/10.15171/hpp.2016.17

12.

Fritzsche

. Ergonomics risk assessment with digital human models in car assembly: Simulation versus real life. Hum Factors Ergon Manuf. 2010;20(4):287–99.

13.

Mazloumi

, Ghorbani

, Saraji

, Kazemi

, Hosseini

. Workload assessment of workers in the assembly line of a car manufacturing company. Iran Occup Heal. 2014;11(4):44–55.

14.

Young

, Brookhuis

, Wickens

, Hancock

. State of science: mental workload in ergonomics [Internet]. Vol. 58, Ergonomics. Taylor & Francis; 2015. p. 1-17. Available from: https://dx-doi-org.web.bisu.edu.cn/10.1080/00140139.2014.956151

15.

Hernandez Arellano

, Serratos Perez

, Alcaraz

JLG

, Maldonado Macias

. Assessment of Workload, Fatigue, and Musculoskeletal Discomfort Among Computerized Numerical Control Lathe Operators in Mexico. IISE Trans Occup Ergon Hum Factors. 2018 Apr;5(2):65–81.

16.

Chander

, Turner

, Swain

, Sutton

, McWhirter

, Morris

, et al. Impact of occupational footwear and workload on postural stability in work safety. Work. 2019;64(4):817–24.

17.

Pinto

, Córdova

, Quiceno

, Llambías

. Ecuaciones de empuje y arrastre ACHS: Relación entre el peso de una carga y la fuerza humana necesaria para moverla. Cienc Trab. 2012;45(1):228–32.

18.

Gómez-Londoño

, González-Correa

. Manual Pressure Force and Correlation With Anthropometric and Physical Condition in College Students. Biosalud [Internet]. 2012;11(2):11-9. Available from: http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S1657-95502012000200002&lng=en&nrm=iso&tlng=es

19.

Hoozemans

MJM

, Van Der Beek

, Frings-Dresen

MHW

, Van Dijk

FJH

, Van Der Woude

LHV

. Pushing and pulling in relation to musculoskeletal disorders: A review of risk factors. Ergonomics. 1998;41(6):757–81.

20.

Beker

. Las Normas ISO 11228 en el Manejo Manual de Cargas. XV Congr Int Ergon SEMAC [Internet]. 2009;1:17. Available from: http://www.semac.org.mx/archivos/congreso11/Pres09.pdf.

21.

Snook

, Ciriello

. The design of manual handling tasks: Revised tables of maximum acceptable weights and forces. Ergonomics. 1991;34(9):1197–213.

22.

Secretaría del trabajo y previsión social mexicana. Guía Informativa NOM-035-STPS-2018 Factores de riesgo psicosocial Identificación, análisis y prevención. D Of la Fed México [Internet]. 2018;26.. Available from: https://www.gob.mx/cms/uploads/attachment/file/503381/NOM035_guia.pdf.

23.

Secretaría del trabajo y previsión social mexicana. NORMA Oficial Mexicana OM-036-1-STPS-2018 Factores de riesgo ergonómico en el Trabajo-Identificación, análisis, prevención y control. D Of la Fed México. 2019;(200).

24.

Mehta

, Agnew

. Effects of concurrent physical and mental demands for a short duration static task. Int J Ind Ergon [Internet]. 2011;41(5):488–93. Available from: https://dx-doi-org.web.bisu.edu.cn/10.1016/j.ergon.2011.04.005

25.

Fernandes

, Gonçalves

DAG

, Conti

. Musculoskeletal Disorders. Dent Clin North Am [Internet]. 2018;62(4):553–64. Available from: https://doi.org/10.1016/j.cden.2018.05.004

26.

Instituto Mexicano del Seguro Social. Memoria Estadística IMSS 2020 [Internet]. Memoria estadística del Instituto Mexicano del Seguro Social. 2020 [cited 2021 Oct 14]. Available from: http://www.imss.gob.mx/conoce-al-imss/informes-estadisticas.

27.

Bligárd

, Osvalder

. CCPE: Methodology for a combined evaluation of cognitive and physical ergonomics in the interaction between human and machine. Hum Factors Ergon Manuf. 2014;24(6):685–711.

28.

Hart

, Staveland

. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Adv Psychol. 1988;52(C):139–83.

29.

Ramkumar

, Stappers

, Niessen

, Adebahr

, Schimek-Jasch

, Nestle

, et al. Using GOMS and NASA-TLX to Evaluate Human–Computer Interaction Process in Interactive Segmentation. Int J Hum Comput Interact. 2017.

30.

Liu

, Wanyan

, Xiao

, Zhao

, Duan

. Pilots ’ mental workload prediction based on timeline analysis. IOS Press. 2020;28(37):207–16.

31.

Rizzo

, Delany

, Longo

. ARROW @ TU Dublin Modeling Mental Workload via Rule-based Expert System: a Comparison with NASA-TLX & Workload Profile system: a comparison with NASA-TLX &. 2016;0–14.

32.

Zoaktafi

, Kazemi

, Choobineh

, Sabooryaraghi

, Nematolahi

, Zakerian

. Relationship between mental workload and salivary cortisol levels: A field study. Work. 2020;67(2):381–6.

33.

Schwartz

, Gerberich

, Albin

, Kim

, Ryan

, Church

, et al. The association between janitor physical workload, mental workload, and stress: The SWEEP study. Work. 2020;65(4):837–46.

34.

Faulhaber

. From Crewed to Single-Pilot Operations: Pilot Performance and Workload Management. Int Symp Aviat Psychol. 2019;283–8.

35.

Grier

. How High Is High ? a Meta-Analysis of Nasa-Tlx Global Workload Scores. Proc Hum Factors Ergon Soc 59th Annu Meet. 2015;1727–31.

36.

Nikulin

, Lopez

, Piñonez

, Gonzalez

, Zapata

. NASA-TLX for predictability and measurability of instructional design models: case study in design methods. Educ Technol Res Dev [Internet]. 2019;67(2):467–93. Available from: https://doi.org/10.1007/s11423-019-09657-4

37.

Said

, Gozdzik

, Roche

, Braun

, Julian

, Kaserer

. Validation of the Raw National Aeronautics and Space Administration Task Load Index (NASA-TLX) Questionnaire to Assess Perceived Workload in Patient Monitoring Tasks: Pooled Analysis Study Using Mixed Models Corresponding Author: 2020;22:1–13.

38.

DiDomenico

, Nussbaum

. Effects of different physical workload parameters on mental workload and performance. Int J Ind Ergon. 2011;41(3):255–60.

39.

DiDomenico

, Nussbaum

. Interactive effects of physical and mental workload on subjective workload assessment. Int J Ind Ergon. 2008;38(11-12):977–83.

40.

DiDomenico . An investigation on subjective assessments of workload and postural stability under conditions of joint mental and physical demands. Virginia Tech. 2003.

41.

Neumann

, Och

, B2

. Inventory of Tools for Ergonomic Evaluation. Arbetslivsrapport Nr [Internet]. 2006;21(January 2006):1401-2928. Available from: http://ludivine.mas.free.fr/ressources/neumann_tools_intervention.pdf%0Apapers2://publication/uuid/28EA2461-2DE6-42F7-8CA6-35DA6087C196

42.

Gil-Monte

. Riesgos psicosociales en el trabajo y saud ocupacional. Rev Peru Med Exp Salud Publica [Internet]. 2012;29(2):237–41. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22858772.

43.

Taylor

, Hoonakker

, Carayon

, Gurses

, Brown

, Mcguire

, et al. IIE Transactions on Healthcare Systems Engineering Measuring workload of ICU nurses with a questionnaire survey: the NASA Task Load Index (TLX) Measuring workload of ICU nurses with a questionnaire survey: the NASA Task Load Index (TLX). 2011;(December 2014):37–41.

44.

Cornejo Chávez

. Condiciones de trabajo y bienestar/malestar docente en profesores de enseñanza media de Santiago de Chile. Educ e Soc. 2009;30(107):409–26.

45.

Gonzalez

. Task workload and cognitive abilities in dynamic decision making. Hum Factors. 2005;47(1):92–101.

46.

Veltman

, Gaillard

AWK

. Physiological workload reactions to increasing levels of task difficulty. Ergonomics. 1998;41(5):656–69.

47.

Caldwell

, Chaffin

, Dukes-Dobos

, Kroemer

KHE

, Laubach

, Snook

, et al. A Proposed Standard Procedure for Static Muscle Strength Testing. Am Ind Hyg Assoc J. 1974;35(4):201–6.

48.

Ayaz

, Shewokis

, Bunce

, Izzetoglu

, Willems

, Onaral

. Optical brain monitoring for operator training and mental workload assessment. Vol. 59, Neuroimage. 2012. p. 36–47.

49.

Jung

, Jung

. Establishment of overall workload assessment technique for various tasks and workplaces. Int J Ind Ergon. 2001;28(6):341–53.

50.

Hart

. NASA-TLX: 20 Years Later. Proc Hum Factors Ergon Soc Annu Meet. 2006;904–8.

51.

Noyes

, Bruneau

DPJ

. A self-analysis of the NASA-TLX workload measure. Ergonomics. 2007;50(4):514–9.

52.

Tubbs-Cooley

, Mara

, Carle

, Gurses

. The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses. Intensive Crit Care Nurs [Internet]. 2018;46:64–9. Available from: https://doi.org/10.1016/j.iccn.2018.01.004.

53.

Helton

, Funke

, Knott

. Measuring workload in collaborative contexts: Trait versus state perspectives. Hum Factors. 2014;56(2):322–32.

54.

Wanyan

, Zhuang

, Zhang

. Improving pilot mental workload evaluation with combined measures. Biomed Mater Eng. 2014;24(6):2283–90.

55.

Byers

, Bittner

, Hill

. Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? In: Advances in industrial ergonomics & safety. 1989.

56.

Schmidtke

. A comparison between the responsiveness of selected physiological and subjective mental workload indicators during real-world driving scenarios. Dep Hum Kinet Ergon Rhodes Univ 2019 Makhanda, South Africa. 2020.

57.

Lacome

, Carling

, Hager

, Dine

, Piscione

. Workload, fatigue, and muscle damage in an under-20 rugby union team over an intensified international tournament. Int J Sports Physiol Perform. 2018;13(8):1059–66.

58.

Davila

, Meltzer

, Hallbeck

, Stone

, Money

. Physical discomfort, professional satisfaction, and burnout in vascular surgeons. J Vasc Surg [Internet]. 2019;70(3):913-920.e2. Available from: https://doi.org/10.1016/j.jvs.2018.11.026

Independent
Physical workload		Mental workload
Level	Activity	Level	Activity
Low	Exert push force (25%)	Low	Answer 60 arithmetic operations (sums)
Medium	Exert push force (50%)	Medium	Answer 60 arithmetic operations (subtractions)
High	Exert push force (75%)	High	Answer 60 arithmetic operations (multiplications)

Dependent
Name	Unit of measurement	Levels
Workload perception	NASA-TLX with Likert scale 1 to 5 [25]	Low: 1 to 2 Medium: 3 to 4 High: 5
Mental performance	# of correct answers [18]	Low: 0-33% Medium: 33-66% High: 67-100%
Perceived effort	Borg scale 0 to 10 [26]	Low: 0 to 3 Medium: 4 to 7 High: 8 to 10 [27]