Development of a sitting posture monitoring system for children using pressure sensors: An application of convolutional neural network

Abstract

BACKGROUND:

Today, sedentary lifestyles are very common for children. Therefore, maintaining a good posture while sitting is very important to prevent musculoskeletal disorders. To maintain a good posture, the formation of good postural habit must be encouraged through posture correction. However, long-term observation is required for effective posture correction. Additionally, posture correction is more effective when it is performed in real time.

OBJECTIVE:

The goal of this study is to classify nine representative sitting postures of children by applying a machine learning technique using pressure distribution data according to the sitting postures.

METHODS:

In this study, a customized film-type pressure sensor was developed and pressure distribution data from nine sitting postures was collected from seven to twelve year-old children. A convolutional neural network (CNN) was applied to classify the sitting postures and three experiments were conducted to evaluate the performance of the model in three applicable usage scenarios: usage by familiar identifiable users, usage by familiar, but unidentifiable users, and usage by unfamiliar users.

RESULTS:

The results of our experiments revealed model accuracies of 99.66%, 99.40%, and 77.35%, respectively. When comparing the recall values for each posture, leaning left and leaning right postures had high recall values, but good posture, leaning forward, and crossed-legs postures had low recall values.

CONCLUSION:

The results of experiments indicated that CNN is an excellent classification method to classify the posture when the pressure distribution data is used as input data. This study is expected to contribute a development of system to aid in observing the natural sitting behavior of children and correcting poor posture in real time.

Keywords

Classification pressure distribution machine learning sensing cushion

1 Introduction

Sedentary lifestyles are very common for modern people and it is considered important to maintain a good posture while sitting because inappropriate sitting postures may lead to musculoskeletal disorders [1 –5]. Proper sitting posture is especially important for children [6, 7]. The postural habits formed during childhood are likely to carry over to adulthood because these habits do not change easily once they are formed [8, 9]. Additionally, musculoskeletal disorders that develop during childhood can be more dangerous than those that develop during adulthood. Back pain during childhood caused by improper posture can be a major risk factor for the development of lumbar diseases as growth occurs [10, 11]. These diseases are difficult to treat once they develop and are likely to cause similar diseases in the future [12]. Furthermore, they can be a deterioration factor for the physical development of children [13]. Therefore, it is important to form good postural habits during childhood.

To effectively encourage children to form good postural habits, it is desirable for posture observation and correction to be performed in real time. Traditionally, posture corrections have been made in such a way that human observers (ergonomic specialists, occupational therapists, teachers, parents, etc.) monitor a child’s sitting behavior, identify the causes of postural problems, and encourage the child to sit properly [14 –16]. However, this process has two major drawbacks. The first is a drawback of the observation method. This method requires significant time and resource investment, meaning it is difficult perform long-term observation. Additionally, reliability problems may arise because of variance between observers [17]. The second drawback is a problem with the correction method. The effectiveness of this method may decrease over time because it is difficult for users to remember guidelines and maintain good posture. Zheng and Morrell postulated that real-time posture correction may be more effective for learning and maintaining good posture based on the results of interviews with ergonomic specialists and occupational therapists [18]. Because of these problems, a system that automatically monitors and corrects posture in real time is needed.

In order to develop such a system, an automatic posture classification system is required to collect objective measurements from a subject and determine their posture. Such systems can be roughly divided into two types according to the conditions for sensing. The first is a type of system that observes posture by attaching various devices to the subject’s body. For instance, sensors such as accelerometers, gyroscopes, or goniometers can be attached to the subject’s body and directly measure their postures based on inertial measurements of body parts and joints [19 –21]. Systems that use markers for motion capture also belong to this type [22 –24]. In this type of system, it is possible to observe the posture of a subject accurately and without any complex data processing because of the directness of measurement. However, such systems are not usable in the real world because of their hardware complexity and natural motion inhibition. According to Fradet and his collegues, such systems require careful selection of attachment points for sensors (or markers) to acquire accurate data, meaning users would require expert knowledge [25]. Additionally, the attached sensors (or markers) could inhibit the natural movement of subjects [26]. The second type is systems that do not require any devices to be attached to a subject’s body. Marker-less motion capture systems belong to this type. In such systems, a subject is filmed by a depth camera and RGB-camera, and the posture of the subject is calculated using a skeleton-tracking algorithm [26 –30]. Based on recent developments in image processing technology, a system using only an RGB-camera has been developed [31]. These systems do not physically limit the movement of subjects, meaning they allow more natural movement. However, these systems can still limit the natural behavior of subjects because they can give subjects a negative impression of being monitored, which also raises privacy concerns [17]. Because of the drawbacks mentioned above, several attempts have been made to develop novel posture monitoring systems using pressure sensors [17 , 32–35]. These systems predict a subject’s posture based on center of pressure, pressure distribution, etc. Such systems do not require any devices other than a chair with built-in sensors and do not physically or psychologically interfere with the movement of subjects, meaning they are suitable for use in the real world and enable observation of the natural behavior of subjects. However, because these systems do not directly measure body parts or joint positions, researches have struggled to identify optimal hardware configurations and analysis techniques with the goal of predicting more posture more accurately.

In previous studies, Tan et al. recruited adult subjects aged 18 to 60 years and obtained pressure data for a total of ten sitting postures [36]. Commercial mat-type pressure sensors (consisting of a grid of 42×48 force-sensitive resistors (FSRs)) were attached to the seat pan and backrest of a chair. The obtained pressure distribution data were analyzed by PCA-based classification algorithms to predict sitting postures. Zemp et al. performed a sitting posture prediction study on adults aged 24 to 64 years [35]. A total of 16 FSRs were attached to the seat pan, backrest, and armrest of a chair. A total of seven sitting postures were predicted. In addition to pressure distribution data, they utilized backrest angle data captured by motion-modules equipped with accelerometers, gyroscopes, and magnetometers. Five different classification algorithms were applied to predict sitting postures: support vector machine, multinomial regression, neural network, random forest, and boosting. Chenu et al. predicted nine sitting postures with varying waist and hip positions [37]. A total of twelve adult subjects participated and commercial mat-type pressure sensors (32×32 FSRs) were attached to the seat pan of a chair. Instead of using a machine learning technique, posture classification was performed based on the pre-mapped pressure distributions of each posture. Meyer et al. predicted a total of sixteen sitting postures using two different mat-type pressure sensors [33]. One was a commercial sensor (32×32 FSRs) and the other was a custom-made sensor consisting of 240 FSRs. Compared to other related studies, they predicted a more diverse array of postures. A Naïve Bayes classifier was applied to classify the postures. Xu et al. developed a textile pressure sensor mat called the “eCushion,” consisting of 256 sensors (16×16 FSRs), and attached it to the seat pan of a chair [17]. They predicted seven postures using a dynamic time warping-based classification method. Zemp et al. used a custom-made mat-type pressure sensor (8×8 FSRs) called the “SIT-CAT” to predict seven sitting postures [38]. Adult subjects aged 25 to 57 years participated and the random forest method was applied as a prediction algorithm. Bao et al. attempted to predict sitting postures using fewer sensors by attaching only five FSRs to the seat pan of a chair [39]. Five static postures and two activities (swaying and shaking) were selected for testing and a density-based clustering method was applied to classify the postures. Liang and his colleague tried to find optimal number of pressure sensors and their placement on seat [40]. To achieve this goal, they acquired the data of 15 postures from 15 participants and conduct classification experiment using AdaBoost algorithm. Zhou et al developed wireless smart blanket which has 1024 (32×32) sensing points for sitting posture monitoring [41]. The data of 12 states (11 postures and not seated condition) were collected from 16 participants. Especially they used simple algorithm based on correlation score vector considering time-efficiency of calculation. Huang and his colleague classified 8 postures using 64 sensing units in 52 by 44 piezo-resistive sensor array [42]. They put the system on the seat and collect the data from 16 participants and classified the postures using ANN algorithm. The system developed by Roh et al. had 4 load cells which are located under seat cushion [43]. They acquired the data of 6 postures from 9 participants in the main test. They compared the multiple algorithms (SVM, Linear and quadratic discriminant analysis, Naïve Bayes algorithm, random forest and Decision tree). The results showed that SVM was the most effective algorithm. Jeong and Park classified 8 postures using customized sensing system [44]. The system consisted of 6 (2×3) FSRs on seat and 6 (2×3) distance sensors on back rest. The data was acquired from 36 participants and KNN algorithm was utilized as a classifier.

Previous studies predicted sitting postures using pressure data obtained by custom-made or commercial pressure sensors. The pressure sensors were typically attached to the seat pan of a chair and it can be reasonably assumed that the pressure distribution obtained from the seat pan is indispensable for predicting posture. For classification algorithms, researchers adopted various algorithms based on their hardware configurations and extracted input features. Although the number of predicted postures was different, most studies included an upright posture, leaning of the upper body in four directions (left, right, forward, backward), and postures for each foot crossing over the other. However, these studies had certain limitations. First, they acquired pressure data for selected postures, which are defined qualitatively. Considering the facts that each posture has intra-variability and the boundary of each posture is unclear, prediction results may be less reliable, even if the accuracy of classification is high. Second, these studies have mainly collected the data from adults. Even for the same posture, the pressure distributions of children may be different than those of adults. This is because children have distinctively different physical characteristics from adults in terms of body proportions, as well as the length, weight, and geometry of cervical vertebrae and joints [45]. According to Moes, various parameters that determine pressure distributions are influenced by demographic and anthropometric characteristics, such as body mass, gender, stature, somatotype, and body fat percentage [46]. It means that posture prediction systems developed for adults have limitations in reproducing the same performance when applied to children. Therefore, to develop a reliable system for predicting the postures of children, it is necessary to acquire data of qualitatively defined postures from children. In this study, a mat-type pressure sensor was developed and inserted into the seat pan of a chair for non-invasive observation and pressure distribution data for pre-defined postures of children were collected. A convolutional neural network (CNN) model, which is a famous deep learning algorithm for image classification [47, 48], was applied to classify sitting postures based on collected data. CNN has been successfully applied to image classification in various fields such as object recognition in an autonomous driving environment, medical diagnosis, face recognition [49 –51], and the acquired pressure distribution data has a two-dimensional image structure. Thus, it can be seen that it is reasonable to apply CNN when classifying pre-defined sitting postures. Three independent experiments were performed considering three applicable usage scenarios: usage by familiar identifiable users, usage by familiar, but unidentifiable users, and usage by unfamiliar users.

2 Methodology

2.1 Hardware configuration and measurement characteristics

In order to acquire the pressure distribution data for sitting postures, a conventional chair for children (RA-070SDSF, Duoback Korea, Seoul, Korea) was equipped with a self-developed sensing cushion containing a film-type pressure sensor (Fig. 1). The chair could be adjusted for location of backrest, height of seat pan, and position of footrest to fit each participant’s body size. The size of the sensor was 318×318 mm and it included 64 (8×8) FSRs (TechStorm, Seoul, Korea). The pressure distribution data was recorded at a 10 Hz frequency and 12-bit resolution. The data was transmitted to a smartphone (Galaxy Note 3, Samsung, Seoul, Korea) through a Bluetooth network using appropriate applications.

Fig. 1

Hardware configuration. (a) Chair with a sensing cushion, (b) structure of sensing cushion, (c) customized film-type pressure sensor inserted into sensing cushion, (d) structure of sensor mat.

2.2 Environment for data acquisition

For efficient data acquisition, one chair with a sensor, one desk, two simple protractors, and two cameras were prepared for each of two participants. The pressure distribution data for each posture was measured while participants sat at the desk, which is used in Korean elementary schools. The protractors were installed to the right and rear of the participants to verify that they correctly assumed the prescribed posture. The cameras were installed to the left and front of the participants so that lateral and front views of the participants could be filmed. The dimensions of the environment are presented in Fig. 2.

Fig. 2

Data acquisition environment.

2.3 Data acquisition

2.3.1 Participants

A total of 24 healthy 7 to 12 year-old children (11 boys, 13 girls) were recruited for data acquisition. Their average age was 10.13 years (SD = 1.62). A body discomfort chart and visual analogue scale were used to screen unhealthy participants [52] and all participants passed. Prior to data collection, the participants and caregivers were informed regarding the purpose and procedure of the study, and they gave consent to participate. The anthropometric information of the participants is listed in Table 1.

Table 1
Anthropometric information of participants

Measuring posture Anthropometric dimensions Mean SD Min. Max.

Standing Height (mm) 1422.04 101.84 1182.00 1630.00

Weight (kg) 40.45 10.06 21.30 56.00

Sitting Height (mm) 729.88 44.08 641.00 817.00

Shoulder height (mm) 477.46 32.33 417.00 550.00

Elbow-seat height (mm) 196.96 20.09 153.00 226.00

Knee height (mm) 431.50 37.23 347.00 502.00

Popliteal height (mm) 366.42 31.01 292.00 422.00

Buttock-Knee length (mm) 481.79 45.54 360.00 544.00

Buttock-Popliteal length (mm) 406.83 42.38 290.00 455.00

Buttock-Abdominal thickness (mm) 216.13 32.07 162.00 270.00

Hip width (mm) 306.33 36.76 218.00 366.00

Measuring posture	Anthropometric dimensions	Mean	SD	Min.	Max.
Standing	Height (mm)	1422.04	101.84	1182.00	1630.00
	Weight (kg)	40.45	10.06	21.30	56.00
Sitting	Height (mm)	729.88	44.08	641.00	817.00
	Shoulder height (mm)	477.46	32.33	417.00	550.00
	Elbow-seat height (mm)	196.96	20.09	153.00	226.00
	Knee height (mm)	431.50	37.23	347.00	502.00
	Popliteal height (mm)	366.42	31.01	292.00	422.00
	Buttock-Knee length (mm)	481.79	45.54	360.00	544.00
	Buttock-Popliteal length (mm)	406.83	42.38	290.00	455.00
	Buttock-Abdominal thickness (mm)	216.13	32.07	162.00	270.00
	Hip width (mm)	306.33	36.76	218.00	366.00

Note. SD: Standard deviation.

2.3.2 Postures

In this study, nine postures were selected considering the commonly predicted postures in previous studies [10 , 28–32] and commonly observed postures in Korean children [53, 54] (Fig. 3): (a) good posture, (b) leaning forward, (c) leaning left, (d) right foot over left, (e) leaning right, (f) left foot over right, (g) sitting at the front edge, (h) slouching, (i) crossed-legs. To cover a wider range of postures, sub-postures were included in some of the selected postures. The good posture includes a slightly leaning backward posture and upright posture (Fig. 3 (a)). The leaning forward posture includes a normally leaning forward posture and head down on the desk posture (Fig. 3 (b)). The leaning left posture includes a normally leaning left posture, leaning left and forward posture, and leaning left and slouching posture (Fig. 3 (c)). The right foot over left posture includes a right foot fully over left posture and right foot slightly over left posture (Fig. 3 (d)). The leaning right posture includes a normally leaning right posture, leaning right and forward posture, and leaning right and slouching posture (Fig. 3 (e)). The left foot over right posture includes a left foot fully over right posture and left foot slightly over right posture (Fig. 3 (f)). The definition of each posture was specifically defined based on the position of body parts, such as the hips, upper body, and legs (Table 2). In particular, the definitions for the upper body and hip positions for each posture were established according to previous studies related to workload assessment of sitting work and anthropometric recommendations for the usage of chairs [55 –58].

Fig. 3

Front and side views of selected sitting postures. (a) Good posture (left: slightly leaning backward posture, right: upright posture), (b) leaning forward posture (left: normally leaning forward posture, right: head down on the desk posture), (c) leaning left posture (left: normally leaning left, center: leaning left and forward posture, right: leaning left and slouching posture), (d) right foot over left posture (left: right foot fully over left posture, right: right foot slightly over left posture), (e) leaning right posture (left: normally leaning right posture, center: leaning right and forward posture, right: leaning right and slouching posture), (f) left foot over right posture (left: left foot fully over right posture, right: left foot slightly over right posture), (g) sitting at the front edge posture, (h) slouching posture, (i) crossed-legs posture.

Table 2

Definitions of selected sitting postures

Selected posture	Included posture	Description	Posture discrimination criteria
			Hip position	Upper body position	Leaning on the backrest?	Leg position
Good posture	Slightly leaning backward	A person leans slightly leaning backward comfortably. The back is well-supported by the backrest	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	Extension: ∼10°	Yes	Both thighs remain parallel to the ground
				Left Lateral Bend: ∼10°
				Right Lateral Bend: ∼10°
	Upright	A person puts both feet flat on the floor and is sitting upright comfortably. The back is not supported by the backrest	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	Flexion: ∼10°	No	Both thighs remain parallel to the ground
				Left Lateral Bend: ∼10°
				Right Lateral Bend: ∼10°
Leaning forward	Normally leaning forward	A person puts their arms on the desk and tilts their upper body forward to support it with the desk	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	Flexion: 10° ∼	No
	Head down on the desk	A person puts their arms on the desk while placing their head on the desk	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	The upper body or head touches the desk or lower arm.	No
Leaning left	Normally leaning left	A person leans their upper body to the left only	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	Flexion: ∼10°	Yes
				Extension: ∼10°
				Left lateral bend: 10° ∼
	Leaning left and forward	A person leans their upper body to the left and forward	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	Flexion: 10° ∼	No
				Left lateral bend: 10° ∼
	Leaning left and slouching	A person sits on the front part of the seat leaning their upper body to the left and backward against the backrest	Less than 50% of the buttock-popliteal is in contact with the seat	Extension: 10° ∼ (*Extension until fully leaning against the back plate in the defined hip position)	Yes
				Left lateral bend: 10° ∼
Right foot over left	Right foot fully over left	A person puts their right leg fully over their left leg	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat			Knees are touching
	Right foot slightly over left	A person puts their right leg slightly over their left leg	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat			Right ankle resting on left knee
Leaning right	Normally leaning right	A person leans their upper body to the right only	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	Flexion: ∼10°	Yes
				Extension: ∼10°
				Right lateral bend: 10° ∼
	Leaning right and forward	A person leans their upper body to the right and forward	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat	Flexion: 10° ∼ Right lateral bend: 10° ∼	No
	Leaning right and slouching	A person sits on the front part of the seat leaning their upper body to the right and backward against the backrest	Less than 50% of the buttock-popliteal is in contact with the seat	Extension: 10° ∼ (*Extension until fully leaning against the back plate with the defined hip position) Right lateral bend: 10° ∼	Yes
Left foot over right	Left foot fully over right	A person puts their left leg fully over their right leg.	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat			Knees are touching
	Left foot slightly over right	A person puts their left leg slightly over their right leg.	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat			Left ankle resting on right knee
Sitting at the front edge		A person sits on the front of the seat without resting on the back of the chair.	Less than 50% of the buttock-popliteal is in contact with the seat		No
Slouching		A person sits on the front of the seat and leans their upper body backward against the backrest.	Less than 50% of the buttock-popliteal is in contact with the seat	Extension: 10°∼ (*Extension until fully leaning against the back plate with the defined hip position)	Yes
Crossed-legs		A person bends both knees inward, placing each foot on the knee of the opposite leg.	More than 80% and less than 95% of the buttock-popliteal is in contact with the seat			Both knees are bent and crossed inward positioning both feet on the opposite legs

2.3.3 Procedure

Each participant sat on the chair in front of the desk. The chair and desk were adjusted to fit their body based on anthropometric recommendations [58]. Each participant was given a verbal description of postures with supplementary figures until they could understand and perform each posture correctly. Next, pressure data for each posture were collected for thirty to forty seconds. During this time, participants were allowed to move within the acceptable ranges in the definition of each posture (Table 2). Movement was facilitated to prevent repetitive measurements of the same data during the measurement period and acquire data for various sub-postures that can occur within the definition of each posture. The participants were carefully monitored in real time to verify that they moved within the definition of each posture. This was then checked again using the recorded video. The selected postures were given in counterbalanced order. The total time for data acquisition was approximately an hour for each participant, including time for setup, instruction, and breaks. All participants received monetary compensation for their efforts.

2.4 Classification algorithm

A CNN was used to predict sitting postures. CNNs which was introduced by LeCun and Bengio [59] have become widely used in various fields, such as speech and image recognition [60 –62], and have been proven to be reliable and powerful by many researches. A CNN is a type of multi-layer perceptron, but it is specialized for two-dimensional inputs by introducing convolution and sub-sampling techniques. Conventional neural networks cannot reflect characteristics of dimensionality because they consider inputs linearly. CNNs can also be optimized to handle computer vision tasks by introducing filters. A CNN consists of two types of layers called convolutional and pooling layers, which effectively reduce the number of weight parameters. Therefore, the complexity of a CNN is reduced by introducing these types of layers.

Because the acquired data (8×8) were too small to be optimal inputs for a CNN, it was necessary to adjust the inputs to a more appropriate size (16×16) through linear interpolation. This interpolation can represent inputs more densely and can smoothly represent the differences between pressure values (see Fig. 4 (a) and Fig. 4 (b)). After interpolation, the inputs were min-max normalized to make pressure the distributions for the same posture from different subjects more similar (see Fig. 4 (b) and Fig. 4 (c)).

Fig. 4

Transformation process for raw pressure distribution data. (a) Pressure distribution map from raw data, (b) refined pressure distribution map from interpolation, (c) refined pressure distribution map from normalization.

In this sturdy, the applied CNN consisted of four convolution and sub-sampling layers. Table 3 presents hyper-parameters and tuned values in our CNN. Hyper-parameters were set to the optimal value by validating 30% after training 70% of all sample data. In each layer, 50 feature maps were generated and 2×2 filters were used between the convolution and sub-sampling layers. Following the two pairs of convolution and sub-sampling layers, a fully connected layer with 100 neurons was connected to the output layer. The full architecture is illustrated in Fig. 5.

Table 3

Hyper-parameters and their tuned values in CNN

Hyper-parameters	Values
1st convolutional features	50
2nd convolutional features	50
3rd convolutional features	50
4th convolutional features	50
Max-pooling size	2*2
# of neurons of fully connected neural networks	100
Iterations	20,000
Activation functions	ReLU
Batch size	300

Note. Rectified Linear Units (ReLU) function (f) is defined as f(x) = max (0,x).

Fig. 5

Structure of the CNN. C, S, and F denote convolution, sub-sampling, and fully connected layers, respectively.

3 Results

The collected data was used to train the CNN model to classify the nine postures. The performance of the model was evaluated considering three applicable usage scenarios. The first is usage by a familiar identifiable user. In this situation, the user’s pressure distribution data for each posture is in the known dataset and the system classifies postures with the user’s own data set. The second scenario is usage by a familiar unidentifiable user. In this situation, the user’s pressure distribution data for each posture is in the known dataset and the system must classify the posture using the entire dataset because it does

not recognize the user. The final scenario is usage by an unfamiliar user. In this situation, the system must classify the posture of the user using data from other users. The methodology and results of the experiments for each usage scenario are described in order in sections 3.1, 3.2, and 3.3.

3.1 Results of experiment 1

Experiment 1 was conducted using individual data. For each participant, 70% of their own data were randomly selected as a training set and the remaining 30% were used as a test set. Table 4 shows the results of predicting postures for each participant. The average training rate and accuracy were 99.88% (97.00 –100.00) and 99.66% (95.00 –100.00), respectively.

Table 4
Results of experiment 1

Participant ID Training rate [% ] Accuracy [% ]

1 100.00 100.00

2 100.00 100.00

3 100.00 100.00

4 100.00 100.00

5 100.00 100.00

6 100.00 100.00

7 100.00 100.00

8 100.00 100.00

9 100.00 100.00

10 100.00 100.00

11 100.00 100.00

12 100.00 99.80

13 97.00 95.00

14 100.00 100.00

15 100.00 99.80

16 100.00 99.00

17 100.00 100.00

18 100.00 99.40

19 100.00 100.00

20 100.00 99.00

21 100.00 99.80

22 100.00 100.00

23 100.00 100.00

24 100.00 100.00

Avg. 99.88 99.66

Participant ID	Training rate [% ]	Accuracy [% ]
1	100.00	100.00
2	100.00	100.00
3	100.00	100.00
4	100.00	100.00
5	100.00	100.00
6	100.00	100.00
7	100.00	100.00
8	100.00	100.00
9	100.00	100.00
10	100.00	100.00
11	100.00	100.00
12	100.00	99.80
13	97.00	95.00
14	100.00	100.00
15	100.00	99.80
16	100.00	99.00
17	100.00	100.00
18	100.00	99.40
19	100.00	100.00
20	100.00	99.00
21	100.00	99.80
22	100.00	100.00
23	100.00	100.00
24	100.00	100.00
Avg.	99.88	99.66

3.2 Results of experiment 2

Experiment 2 was conducted using the entire dataset. For each user, 70% of the entire posture dataset was randomly chosen as a training set and the remaining 30% was used as a test set. The training rate and accuracy were 100.00% and 99.40%, respectively (Table 5).

Table 5
Results of experiment 2

Training rate [% ] Accuracy [% ]

Total 100.00 99.40

	Training rate [% ]	Accuracy [% ]
Total	100.00	99.40

3.3 Results of experiment 3

The leave-one-out method was used in experiment 3. This experiment was performed to predict postures for each participant using the data of the other 23 participants as the training set. Table 6 summarizes the training rates and accuracies when data from each participant were used as test data. The average training rate and accuracy were 99.79% (99.33 –100.00) and 77.35% (61.00 –93.20), respectively.

Table 6
Results of experiment 3

Participant ID Training rate [% ] Accuracy [% ]

1 99.67 76.40

2 99.67 73.00

3 100.00 80.20

4 99.67 84.20

5 100.00 77.00

6 99.67 81.60

7 99.67 71.00

8 100.00 92.20

9 99.67 73.00

10 100.00 70.80

11 99.67 69.00

12 99.67 91.80

13 99.67 72.20

14 100.00 93.20

15 100.00 61.00

16 100.00 65.40

17 99.67 80.40

18 100.00 77.40

19 100.00 64.40

20 100.00 65.40

21 99.67 88.60

22 99.67 82.60

23 99.33 80.00

24 99.67 85.60

Avg. 99.79 77.35

Participant ID	Training rate [% ]	Accuracy [% ]
1	99.67	76.40
2	99.67	73.00
3	100.00	80.20
4	99.67	84.20
5	100.00	77.00
6	99.67	81.60
7	99.67	71.00
8	100.00	92.20
9	99.67	73.00
10	100.00	70.80
11	99.67	69.00
12	99.67	91.80
13	99.67	72.20
14	100.00	93.20
15	100.00	61.00
16	100.00	65.40
17	99.67	80.40
18	100.00	77.40
19	100.00	64.40
20	100.00	65.40
21	99.67	88.60
22	99.67	82.60
23	99.33	80.00
24	99.67	85.60
Avg.	99.79	77.35

4 Discussion

4.1 Overall results

The accuracies of classification in experiments 1 and 2 were very good, but the accuracy in experiment 3 was relatively low (Fig. 6). The same result was obtained in a previous study [36]. This result suggests that if a user’s posture data is included in the training data, it has a significant effect on classification performance. This is because pressure distributions differ according to the physical characteristics of users, even for the same posture [34].

Fig. 6

Summary of experimental results.

The average accuracy of classification in experiment 1 was 99.66%. This result is superior when compared to the results of a previous study (95% in [36]). Although Tan et al. predicted more postures, they also equipped expensive commercial pressure sensor mats on both the backrest and seat pan of a chair [36]. Therefore, it can be concluded that the hardware and classification method used in this study are very effective and efficient at predicting sitting postures.

The average accuracy of classification in experiment 2 was 99.4%. This result is superior or equivalent to the results of similar experiments in previous studies (96% in [36]; 85.9% in [17]; 94.2% in [39]; 99.6% in [40]; 80% in [41]; 82.2% in [42]; 97.2% in [43]; 92% in [44]). Considering the fact that previous studies classified fewer postures (five postures in [39]; seven postures in [17]; 8 postures in [42, 44]; 6 postures in [43]) or used a larger number of sensors (8,064 sensors in [36]; 256 sensors in [17]) or small amount of data (15 participants in [40]; 16 participants in [41]), the performance of the sensor and CNN could be interpreted as being superior to previously applied.

The average accuracy of classification in experiment 3 was 77.35%. This is slightly lower than the experimental results from leave-one-out method tests in previous studies (79% in [36]; 81% in [33]; 90.9% in [35]; 82.7% in [38]). This result can be attributed to the fact that relatively few sensors were used and that sensors were attached to only the seat pan of a chair in this study. This argument is supported by the results of a previous study [33]. Meyer et al. compared the performance of two static sitting posture classification systems [33]. One used only pressure data from the seat pan and the other used both pressure data from seat pan and backrest. The results demonstrated that if pressure data from the backrest was not used, the accuracy of classification fell dramatically from 81% to 55%. Zemp et al. used a smaller number of pressure sensors to predict posture, but attached pressure sensors to the backrest and armrest in addition to the seat pan [35]. Additionally, a motion module was attached to their chair to utilize the backrest angle as an input variable. Zemp et al. only used 64 pressure sensors attached to the seat pan, but also classified fewer postures (seven postures) [38].

4.2 Recognition rate for each posture

The accuracies in experiments 1 and 2 were close to 100%. However, the accuracy in experiment 3 was significantly lower, so it is necessary to discuss the recognition rate (same as recall value) for each posture in this experiment. The recall value for each posture is presented in Fig. 7. While the recall values for the leaning left and right postures were high, the recall values for the upright, leaning forward, and crossed-legs postures were low. As shown in Fig. 8, the postures with high recall values had distinct pressure distributions and the postures with low recall values did not. There were many cases where the upright posture was misclassified as the leaning forward posture, leaning forward posture was misclassified as the upright posture, and crossed-legs posture was misclassified as the leaning forward posture (Table 7). As shown in Fig. 8, the pressure distributions of the upright posture and leaning forward posture are similar. The fact that that these two postures share many common features has already been demonstrated in a previous study [17]. However, the similarity in pressure distributions between the crossed-legs posture and leaning forward posture was an unexpected result because these two postures are clearly very different. This result is noteworthy because the crossed-legs posture is a very common posture in Asia. Further study is needed to distinguish between these two independent postures accurately.

Fig. 7

Recall value for each posture.

Fig. 8

Example pressure distribution maps of nine postures from different participants (Top: Participant 23, Bottom: Participant 18). (a) Good posture, (b) leaning forward posture, (c) leaning left posture, (d) right foot over left posture, (e) leaning right posture, (f) left foot over right posture, (g) sitting at the front edge posture, (h) slouching posture, (i) crossed-legs posture.

Table 7

The confusion matrix for experiment 3

	Assigned sitting postures										Recall value
		Good	Leaning forward	Leaning left	Right foot over left	Leaning right	Left foot over right	Sitting at the front edge	Slouching	Cross-legs
True sitting postures	Good	1006	284	41	17	18	67	4	0	58	0.673
	Leaning forward	442	679	56	50	32	69	36	12	91	0.463
	Leaning left	119	21	2288	12	9	3	14	35	34	0.903
	Right foot over left	31	48	62	614	5	8	1	23	24	0.752
	Leaning right	46	25	3	9	2309	25	32	23	21	0.926
	Left foot over right	23	60	11	2	34	629	0	14	51	0.763
	Sitting at the front edge	0	1	19	0	52	0	657	76	2	0.814
	Slouching	6	29	25	7	16	54	66	560	0	0.734
	Cross-legs	27	166	16	28	19	3	0	1	540	0.675

Table 8

Combined confusion matrix

		Assigned sitting postures		Recall value
		Good	Poor
True sitting	Good	1006	489	0.673
postures	Bad	694	9811	0.934

4.3 Further application as a discriminator of good posture

It is important to predict exactly what type of posture users assumed, but it is also important to discriminate between whether or not users assumed good posture for posture correction. In this section, the performance of the proposed system when it is used as a good posture discriminator is evaluated. Evaluation was performed with a focus on cases when the system is applied to unfamiliar users for reasons described in Section 4.2. Table 8 is the combined confusion matrix of Table 7 for two postures: good posture and poor posture (all postures other than good posture). The recall value for good posture was 0.67 and the recall value for poor posture was 0.93. It should be noted that positive predictive value (PPV) and negative predictive value (NPV) are more important than the other performance metrics when considering application of the system in the real world. In this situation, PPV is the probability that a user’s actual posture was good when the system predicted that the posture was good and NPV is the probability that the user’s actual posture was bad when system predicted that the posture was bad. The reason why PPV and NPV are particularly important in a posture discrimination system is that the actual posture of a user is unknown in a real usage situation and only the prediction result from the system can be used as a cue for the true posture of the user. The PPV and NPV of the suggested good posture discriminator system were 0.59 and 0.95, respectively. However, the training data used in this study does not represent the actual sitting posture behavior of the entire child population because the number of collected data for each posture was controlled. Because PPV and NPV depend on the prevalence of good posture, these values may vary between users because sitting posture behavior is different for each user. In the absence of additional training data, it can be assumed that calculated recall values of good and bad posture are applied to all users equally. Under this assumption, the change patterns in PPV and NPV with the prevalence of good posture are presented in Fig. 9. As shown in the figure, the discriminant performance of the system changes according to the prevalence of good posture. Both PPV and NPV are greater than 0.7 when the prevalence of good posture is between 0.21 and 0.55. False discovery rate (FDR), with is the complement of PPV, is critical for the purposes of the system. If bad postures are misjudged as good postures, the opportunity for posture correction may disappear. If the system was applied to users with very poor postural habits, then FDR would be high. This means that the effectiveness of the system may be low initially because it views bad posture as good posture. However, it can be expected that FDR will decrease rapidly as the prevalence of good posture increases as users correct their improper sitting behavior through use of the system. In contrast, when a user with very good postural habits uses the system, the system may underestimate how often the user assumes good posture because false omission rate, which is the complement of NPV, would be high. If we improve the recall value for each posture through acquisition of additional training data and improvement of the classification algorithm, the performance of the good posture discriminator system would be better and more stable, irrespective of users.

Fig. 9

Prevalence of good posture vs. system prediction performance (PPV: Positive predictive value; NPV: Negative predictive value).

5 Conclusion

Sedentary behavior is common in daily life and the formation of proper sitting postural habits during childhood is important for preventing musculoskeletal disorders that may lead to back pain and hinder physical development. Improper postural habits during childhood are likely to remain in adulthood and cannot be easily corrected by oneself. For these reasons, we collected the pressure distribution data of nine postures from children using self-developed film-type pressure sensor, and classified sitting postures using CNN. The sitting postures selected for prediction were defined qualitatively and quantitatively to provide more information to users during practical use of the system. Furthermore, considering economic efficiency and practical application suitability, sitting posture data obtained noninvasively in real time by using a custom-made film-type pressure sensor. When measuring the data for each posture, more data from each posture was collected by providing degrees of freedom for the upper body. This precaution guarantees greater versatility in actual use. A CNN algorithm was applied to predict sitting postures and its performance was excellent in three different experiments compared to previous studies, considering the number of pressure sensors and wide range of each posture. However, there are still some postures which are not included in our study. For the results of this study to be more useful in real-world, more postures need to be predicted. In the future, we will collect the data of additional postures and improve the classification algorithm for the development of robust and accurate posture monitoring system. Furthermore, we will study effective methods, which will help children to be aware of their sitting posture and correct improper posture in real-time. In spite of some limitations, it is expected that the applied techniques and the results of this study could contribute the development of posture monitoring and correcting systems which can be used in personal sitting environments or public institutions, such as schools.

Footnotes

Acknowledgments

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF 2019R1A6A3A01095083). The authors would like to thank DBK Co., Ltd. and Algorigo Inc. for their cooperation in this study.

Conflict of interest

None to report.

References

Cecco

, De Cunzo

, Napolitano

, Rega

, Fiorillo

, Califano

. School combo-desk comfort assessment: a method for weighting postural factors that affect the overall perceived comfort while performing different activities. In: The Second International Conference on Comfort (ICC2019) Delft. 2019.

Fiorillo

, Anzisi

, Carbone

, Califano

, Naddeo

. A comfort evaluation tool for sitting postures: the case of Library chairs. Work. 2019.

Naddeo

, Califano

, Fiorillo

. Identifying factors that influenced wellbeing and learning effectiveness during the sudden transition into eLearning due to the COVID-19 lockdown. Work. 2021;68(1):45–67.

Naddeo

, Califano

, Vallone

, Cicalese

, Coccaro

, Marcone

, et al. The effect of spine discomfort on the overall postural (dis) comfort. Appl Ergon. 2019;74:194–205.

Naddeo

, Califano

, Vink

. The effect of posture, pressure and load distribution on (dis) comfort perceived by students seated on school chairs. Int J Interact Des Manuf. 2018;12(4):1179–88.

C-W

, Lu

J-M

. Evaluation of the Indonesian National Standard for elementary school furniture based on children’s anthropometry. Appl Ergon. 2017;62:168–81.

Syazwan

, Azhar

MNM

, Anita

, Azizan

, Shaharuddin

, Hanafiah

, et al. Poor sitting posture and a heavy schoolbag as contributors to musculoskeletal pain in children: an ergonomic school education intervention program. J Pain Res. 2011;4:287.

Floyd

, Ward

. Anthropometric and physiological considerations in school, office and factory seating. Ergonomics. 1969;12(2):132–9.

Yeats

. Factors that may influence the postural health of schoolchildren (K-12). Work. 1997;9:45–55.

10.

Grimmer

, Nyland

, Milanese

. Longitudinal investigation of low back pain in Australian adolescents: a five-year study. Physiother Res Int. 2006;11:161–72.

11.

Harreby

, Nygaard

, Jessen

, Larsen

, Storr-Paulsen

, Lindahl

, et al. Risk factors for low back pain in a cohort of Danish school children: an epidemiologic study. Eur Spine J. 1999;8(6):444–50.

12.

Troup

, Forema

, Baxter

, Brown

. Volvo Award in Clinical Sciences: The Perception of Back Pain and the Role of Psychophysical Tests of Lifting Capacity. Spine. 1987;12:645–57.

13.

Tanner

, Whitehouse

, Marubini

, Resele

. The adolescent growth spurt of boys and girls of the Harpenden growth study. Ann Hum Biol. 1976;3:109–26.

14.

Dockrell

, Earle

, Galvin

. Computer-related posture and discomfort in primary school children: The effects of a school-based ergonomic intervention. Comput Educ. 2010;55(1):276–84.

15.

Dugan

. Teaching the body: A systematic review of posture interventions in primary schools. Educ Rev. 2018;70(5):643–61.

16.

Williams

, Jacobs

. The effectiveness of a home-based ergonomics intervention on the proper use of computers by middle school children. Work. 2002;18(3):261–8.

17.

, Huang

M-C

, Amini

, He

, Sarrafzadeh

. ecushion: A textile pressure sensor array design and calibration for sitting posture analysis. IEEE Sens J. 2013;13(10):3926–34.

18.

Zheng

, Morrell

. Comparison of visual and vibrotactile feedback methods for seated posture guidance. IEEE T Haptics. 2013;6:13–23.

19.

, Chae

, Moon

, Seo

, Pan

. Four-joint motion data based posture classification for immersive postural correction system. Multimed Tools Appl. 2017;76:11235–49.

20.

Wong

, Wong

. Detecting spinal posture change in sitting positions with tri-axial accelerometers. Gait Posture. 2008;27:168–71.

21.

Worsley

, Rebolledo

, Webb

, Caggiari

, Bader

. Monitoring the biomechanical and physiological effects of postural changes during leisure chair sitting. J Tissue Viability. 2017;27(1):16–22.

22.

Korakakis

, Sideris

, Giakas

. Sitting bodily configuration: A study investigating the intra-tester reliability of positioning subjects into a predetermined sitting posture. Manual Ther. 2014;19:197–202.

23.

Kuo

, Tully

, Galea

. Video analysis of sagittal spinal posture in healthy young and older adults. J Manip Physiol Ther. 2009;32:210–5.

24.

O’Sullivan

, Clifford

, Hughes

. The reliability of the CODA motion analysis system for lumbar spine analysis: a pilot study. Physiother Pract Res. 2010;31(1):16–22.

25.

Fradet

, Tiernan

, Mcgrath

, Murray

, Braatz

, Wolf

. The use of pressure mapping for seating posture characterisation in children with cerebral palsy. Disabil Rehabil Assist Technol. 2011;6(1):47–56.

26.

Choi

, Park

. Estimation of sitting posture by using the combination of ground reaction force. J Mech Sci Technol. 2015;29:1657.

27.

Harutyunyan

, Moldoveanu

, Balan

, Morar

, Dascalu

M-I

. A SYSTEM FOR IMPROVING IT OFFICE EMPLOYEES HEALTH USING AN UNCONVENTIONAL USER INTERFACE. Univ Politeh BUCHAREST Sci Bull Ser C-ELECTRICAL Eng Comput Sci. 2017;79(1):3–16.

28.

Hassani

, Kubicki

, Mourey

, Yang

. Advanced 3D movement analysis algorithms for robust functional capacity assessment. Appl Clin Inform. 2017;8:454–69.

29.

Liu

, Li

, Zhang

, Ye

. Healthy human sitting posture estimation in RGB-D scenes using object context. Multimed Tools Appl. 2017;76:10721–39.

30.

Wiedemann

, Planinc

, Kampel

. Ergonomic Monitoring of Office Workplaces Using Kinect. In: International Workshop on Ambient Assisted Living. Springer; 2014. pp. 275–8.

31.

Sánchez

, Loram

, Darby

, Holmes

, Butler

. A video based method to quantify posture of the head and trunk in sitting. Gait Posture. 2017;51:181–7.

32.

Lee

, Shin

. Feasibility study of sitting posture monitoring based on piezoresistive conductive film-based flexible force sensor. IEEE Sens J. 2015;16(1):15–6.

33.

Meyer

, Arnrich

, Schumm

, Troster

. Design and modeling of a textile pressure sensor for sitting posture classification. IEEE Sens J. 2010;10:1391–8.

34.

Schrempf

, Schossleitner

, Minarik

, Haller

, Gross

. PostureCare–Towards a novel system for posture monitoring and guidance. IFAC Proc Vol. 2011;44(1):593–8.

35.

Zemp

, Fliesser

, Wippert

, Taylor

, Lorenzetti

. Occupational sitting behaviour and its relationship with back pain–A pilot study. Appl Ergon. 2016;56:84–91.

36.

Tan

, Slivovsky

, Pentland

. A sensing chair using pressure distribution sensors. IEEE/ASME Trans Mechatronics. 2001;6(3):261–8.

37.

Chenu

, Vuillerme

, Demongeot

, Payan

. A wireless lingual feedback device to reduce overpressures in seated posture: A feasibility study. PLoS One. 2009;4(10):e7550.

38.

Zemp

, Tanadini

, Plüss

, Schnüriger

, Singh

, Taylor

, et al. Application of machine learning approaches for classifying sitting posture based on force and acceleration sensors. Biomed Res Int. 2016;2016.

39.

Bao

, Li

, Ge

, Bao

. Sitting Posture Recognition based on data fusion on pressure cushion. TELKOMNIKA Indones J Electr Eng. 2013;11(4):1769–75.

40.

Liang

, Cao

, Liu

. Smart cushion: A practical system for fine-grained sitting posture recognition. In: 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE; 2017. pp. 419-24.

41.

Zhou

, Singh

, Yildirim

, Prifti

, Zurian

, Yuncosa

, et al. Smart blanket: a real-time user posture sensing approach for ergonomic designs. In: International Conference on Applied Human Factors and Ergonomics. Springer; 2017. pp. 193–204.

42.

Huang

, Gibson

, Yang

. Smart chair for monitoring of sitting behavior. KnE Eng. 2016;274-80.

43.

Roh

, Park

, Lee

, Hyeong

, Kim

, Lee

. Sitting posture monitoring system based on a low-cost load cell using machine learning. Sensors. 2018;18(1):208.

44.

Jeong

, Park

. Developing and Evaluating a Mixed Sensor Smart Chair System for Real-time Posture Classification: Combining Pressure and Distance sensors. IEEE J Biomed Heal Informatics. 2020.

45.

Straker

, Burgess-Limerick

, Pollock

, Coleman

, Skoss

, Maslen

. Children’s posture and muscle activity at different computer display heights and during paper information technology use. Hum Factors. 2008;50(1):49–61.

46.

Moes

. Variation in sitting pressure distribution and location of the points of maximum pressure with rotation of the pelvis, gender and body characteristics. Ergonomics. 2007;50:536–61.

47.

Rawat

, Wang

. Deep convolutional neural networks for image classification: A comprehensive review. Neural Comput. 2017;29(9):2352–449.

48.

Ciresan

, Meier

, Masci

, Gambardella

, Schmidhuber

. Flexible, high performance convolutional neural networks for image classification. In: Twenty-second international joint conference on artificial intelligence. 2011.

49.

Gao

, Cheng

, Wang

, Li

, Zhao

, Li

. Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment. IEEE Trans Ind Informatics. 2018;14(9):4224–31.

50.

Iqbal

, Sameem

MSI

, Naqvi

, Kanwal

, Ye

. A deep learning approach for face recognition based on angularly discriminative features. Pattern Recognit Lett. 2019;128:414–9.

51.

Rasti

, Teshnehlab

, Phung

. Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks. Pattern Recognit. 2017;72:381–90.

52.

Kelly

, Dockrell

, Galvin

. Computer use in school: its effect on posture and discomfort in schoolchildren. Work. 2009;32(3):321–8.

53.

Min

C-K

. A review on the KSG(Korean industrial standard for student’s desk and chair) design guideline. J Korean Inst Educ Facil. 2007;14(3):17–26.

54.

Song

S-R

, Pan

Y-H

, Jeong

J-H

. A study on context of children’s library and user behavior modeling. J HCI Soc Korea. 2008;3(1):33–8.

55.

Hignett

, McAtamney

. Rapid entire body assessment (REBA). Appl Ergon. 2000;31:201–5.

56.

Keyserling

, Brouwer

, Silverstein

. A checklist for evaluating ergonomic risk factors resulting from awkward postures of the legs, trunk and neck. Int J Ind Ergon. 1992;9(4):283–301.

57.

Openshaw

, Taylor

. Ergonomics and design a reference guide. Allsteel Inc., Muscatine, Iowa; 2006.

58.

Panagiotopoulou

, Christoulas

, Papanckolaou

, Mandroukas

. Classroom furniture dimensions and anthropometric measures in primary school. Appl Ergon. 2004;35:121–8.

59.

LeCun

, Bengio

. Convolutional networks for images, speech, and time series. Handb brain theory neural networks. 1995;3361(10):1995.

60.

Abdel-Hamid

, Mohamed

, Jiang

, Deng

, Penn

, Yu

. Convolutional neural networks for speech recognition. IEEE/ACM Trans audio, speech, Lang Process. 2014;22(10):1533–45.

61.

Krizhevsky

, Sutskever

, Hinton

. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012;25:1097–105.

62.

Sainath

, Mohamed

, Kingsbury

, Ramabhadran

. Deep convolutional neural networks for LVCSR. In: 2013 IEEE international conference on acoustics, speech and signal processing. IEEE; 2013. pp. 8614-8.

Development of a sitting posture monitoring system for children using pressure sensors: An application of convolutional neural network

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSION:

Keywords

1 Introduction

2 Methodology

2.1 Hardware configuration and measurement characteristics

2.3.1 Participants

2.4 Classification algorithm

3.1 Results of experiment 1

Table 5 Results of experiment 2 Training rate [% ] Accuracy [% ] Total 100.00 99.40

4.1 Overall results

Footnotes

Acknowledgments

Conflict of interest

References

Table 5
Results of experiment 2

Training rate [% ] Accuracy [% ]

Total 100.00 99.40