A crowdsourcing approach for personalization in human activities recognition

Abstract

The technology trend of context-aware computer systems carries the promise of more flexible automated systems, with a high degree of adaptation to the user’s situation, but it implies as a precondition that the context information (such as the place, time, activity, preferences, etc.) is indeed available. One very important aspect of the user context is the activity in which the human is currently involved. Human Activity Recognition (HAR) has become a trending topic in the last years because of its potential applications in pervasive health care, assisted living, exercise monitoring, etc. Most of the works on HAR either require from the user to label the activities as they are performed so the system can learn them, or rely on a trained device that expects a “typical” ideal user. The first approach is impractical, as the training process easily becomes time consuming, expensive, etc., while the second one drops the HAR precision for many non-typical users. In this work we propose a “crowdsourcing” method for building personalized models for HAR by combining the advantages of both user-dependent and general models, finding class similarities between the target user and the community users. We evaluated our approach on 4 different public datasets and showed that the personalized models outperformed the user-dependent and user-independent models when labeled data is scarce.

Keywords

Activity recognition wearable sensors accelerometer model personalization

1. Introduction

The technology trend of context-aware computer systems [64, 10] carries the promise of more flexible automated systems, with a high degree of adaptation to the user’s situation, but it implies as a precondition that the context information (such as the place, time, activity, preferences, etc.) is indeed available. One very important aspect of the user context is the activity in which the human is currently involved, because it highly influences her/his needs and expectations. Inferring the current activity being performed by an individual or group of people can provide valuable information in the process of understanding the context and situation in a given environment and as a consequence, personalized services can be delivered.

Of course one way of getting which activity the user is involved in is to ask her/him directly, by requiring from users to declare it in some computer interface, but this would be a burden that is of course better to avoid, and in practice almost no user would be constantly declaring his/her activity. It is way better to automatically recognize the human activity from clues such as the location, position, movements, etc. This is why in the last years Human Activity Recognition (HAR) [14] has gained a lot of attention. In the HAR wide range of applications we can find health and elder care [40, 51, 28, 62], sports [52, 36], indoor location systems [32], etc. There are several approaches to recognize human activities, e.g., from video cameras [19, 61], and even the Microsoft Kinect sensor [68, 55]. Some works have used sensors installed in the environment like binary switches, RFID tags, proximity sensors, motion sensors, etc. [25, 20, 51]. Recently, the use of wearable sensors has become the most common approach to recognize physical activities because of its unobtrusiveness and ubiquity – specifically the use of accelerometers [44, 11, 52, 50] because they are already embedded in several wearable devices and they raise less privacy concerns than other types of sensors, like video cameras.

One of the challenges in HAR systems lies in the training process. There are basically two options from which to choose: one is to apply a general model for a “generic” human user, lowering very much the precision for each individual user, and making it unusable except for the most basic activities, for which a general model is enough, as is the case for the pedometer function or for alerting a user who sits down too long; basic functions like these are now available in some fitting devices and smart watches. The other option is to train the device for each individual user, but then the problem is that training the device is a very tedious, time consuming activity, and also prone to errors; and as a consequence, most users abandon the training process altogether.

Another related problem with personalization in HAR is that at the initial state of a system there is no information about a specific user (in our case, sensor data and labels). In the field of recommender systems (e.g., movie, music, book recommenders) this is known as the cold-start problem [63, 6] and it includes the situation when there is a new user but nothing or little is known about him/her, in which case it becomes difficult to recommend an item, service, etc. It also encompasses the situation when a new item is added to the system but since no one has yet rated, purchased or used that item, then it is difficult to recommend it to the users.

The key idea of our approach is to assume that there is a community of users of the considered device, and that some of the users contribute with training data for themselves, but these data can be reused as well for some other users, especially if they “behave” in a similar way to the contributing user. Of course, the proportion of training data would be small compared to the quantity of users, but even with few available training data it is possible to take advantage of it. We do not expect, however, to reach the performance of a user-dependent device with full training data, but to have a comparable performance with much less effort from each individual user.

So, we will focus on the situation when there is a new user in the system and we want to infer his/her physical activities from sensor data with high accuracy even when there is little information about that particular user, assuming that the system already has data from many other users and also assuming that their associated data is already labeled. We are thus attempting to use a “crowdsourcing” approach which consists in using collective data to fit personal data.

The key insight in our approach is that we will use the scarce labeled data from the target user to select a subset of the other users’ data based on class similarities to build a personalized model. The rational behind this idea is that the way people move varies between individuals so we want to exclude instances from the training set that are very different from those of the target user in order to remove noise.

This work is based on our previous conference paper [22], with the additions of evaluating the use of an oversampling technique to avoid class imbalance problems, as well as the impact of choosing different clustering quality indices for the selection of the optimum number of groups in the clustering step of the proposed algorithm and additional statistical methods to validate our experimental results; also additional explanations, figures and tables are included given the ampler extension of a journal paper as this one.

This paper is organized as follows: Section 2 presents some related work. Section 3 details the process of building a Personalized Model. Section 4 presents the datasets used in this work. The experiments are described in Section 5. Finally in Section 6 we draw our conclusions.

2. Related work

From the reviewed literature, broadly three different types of models in HAR can be identified–namely: General, User-Dependent, and Mixed models.

General Models (GM): Sometimes also called User-Independent Models, Impersonal Models, etc. A model is constructed using a fixed set of users, with the hope that they will be similar to all other users, so the model will be also applicable to these ones.

User-Dependent Models (UDM): They are also called User-Specific Models. In this case, individual models are trained and evaluated for the given user using just her/his own fully labeled data.

Mixed Models (MM): Called also Hybrid models [48]. This type of model tries to combine GMs and UDMs in the hope of adding their respective strengths, and usually is trained using all the aggregated data without distinguishing between users.

Building UDMs and GMs is the standard way of evaluation in hand gesture recognition systems [73, 7, 38, 23] and also in pervasive health monitoring systems such as automatic stress detection [53, 49]. There are also some works in HAR that have used the UDM and/or GM approach [71, 27, 34, 74]. The disadvantages of GM are mostly related to their lack of precision, because the data from many dissimilar users is just aggregated. This limits the GM HAR systems to very simple applications such as pedometers and detection of long periods of sitting down. The disadvantages of UDM HAR systems are related to the difficulties of labeling the specific users’ data, as the training process easily becomes time consuming and expensive, so in practice users avoid it.

For UDMs, several techniques have been used to help users label the data, as it is the weakest link in the process. For example, Lara develops a mobile application [56] in which the user can select several activities from a predefined list. Anguita [8] video-records the data collection session and then manually labels the data. Some other works have used a Bluetooth headset combined with speech recognition software to perform the annotations [33] whereas others [24] take annotations manually. There are also some works that have used a crowdsourcing approach to label activities from video [29, 42]. Furthermore, once the data is labeled, the boundaries between labels may be shifted so further pre-processing to correct boundaries may be needed [35]. In any case, labeling personal activities remains being very time-consuming and undesirable indeed.

From the previous comments, apparently MMs look like a very promising approach, because they could cope with the disadvantages of both GM and UDM, but in practice combining the strengths of both has proved to be an elusive goal; as noted by Lockhart and Weiss [48], no such system has made it to actual deployment.

There have been several works that have studied the problem of scarce labeled data in HAR systems [26, 67, 44]and used Semi-supervised learning methods [17] to deal with the problem; however they follow a Mixed model approach, i.e., they do not distinguish between users.

Model personalization/adaptation refers to training and adapting classifiers for a specific user according to his/her own needs. Building a model with data from many users and using it to classify activities for a target user will introduce noise due to the diversity between users. Lane et al. [41] showed that there is a significant difference for the walking activity between two different groups of people (20–40 and 65 $+$ years old). Parviainen [58] also argued that a single general model for activity classification will not perform well due to individual differences and proposed an algorithm to adapt the classification for each individual by only requesting binary feedback from the user.

In Lu’s work [49]a model adaptation algorithm (Maximum A Posteriori) is used for stress detection using audio data. Zheng et al. [75] used a collaborative filtering approach to provide targeted recommendations about places and activities of interest based on GPS traces and annotations. They manually extracted the activities from text annotations whereas in this work the aim is to detect physical activities from accelerometer data.

Abdallah et al. [5] proposed an incremental and active learning approach for activity recognition to adapt a classification model as new sensory data arrives. Vo [72] proposed a personalization algorithm that uses clustering and a Support Vector Machine that first trains a model using data from user A and then personalizes it for another person B; however they did not specify how should user A be chosen. This can be seen as a $1\rightarrow n$ relationship in the sense that the base model is built using data from a specific user A and the personalization of all other users is based solely on A. The drawback of this approach is that user A may be very different from all other users which could lead to poor final models. Our work differs from this one in that we follow a $n\rightarrow 1$ approach which is more desirable in real world scenarios, i.e., use data already labeled by the community users to personalize a model for a specific user.

Lane [41] personalizes models for each user by first building Community Similarity Networks (CSN) for different dimensions such as: anatomical similarity, lifestyle similarity and sensor-data similarity. Our study differs from this one in two key aspects: First, instead of looking for inter-user similarities we find similarities between classes of activities. This is because two users may be similar overall but still, there may be activities that are performed very differently between them. Second, we just use accelerometer data to find similarities since other types of data (age, locations, height, etc.) are usually not available or raise privacy concerns. Furthermore, we evaluated the proposed method on 4 different public datasets collected by independent researchers.

In this work we will use an approach that is between GMs and UDMs, so it could be seen as a variation of Mixed Models, but here we use the small amount of labeled data from the considered user to select a subset of the other users’ activities instances, instead of just aggregating the user data with any other user data. This selection is made based on class similarities and the details will be presented in Section 3.

Our approach builds on the assumption that a community of users have already labeled data, in the style of the “crowdsourcing” systems [12]. We actually assume a very big community of users of a given device (like the nearly 10 millions of active users of the Fitbit devices); with that community size, even a very small percentage of labeling (say an average of only 1 label reported per user) is enough for having a very big data bank (10 million data points). Of course, there are some difficult aspects related to crowdsourced data, mentioned in the given reference [12], related to low-quality work by some users, unbalance of populations, even spammers, but with very large numbers it is possible to deal with those problems with specialized algorithms.

In our approach we also assume that the considered user will supply some little labeled data. Of course, it would be better to avoid asking for labeled data altogether, but the analysis we will present in this paper shows that the value of a little labeled data from the considered user for improving the accuracy is of a paramount importance.

Other authors like Nguyen [54] make use of crowdsourcing-style algorithms for activity recognition, also requiring from the user very small labeling, but they use completely different methods (semi-supervised learning and active learning), and their source of information is ambient sound, instead of accelerometers.

3. Personalized models

In this section we describe how a Personalized Model (PM) is trained for a given target user $u_{t}$ . A General Model (GM) includes all instances from users $U_{\textit{other}}$ , where $U_{\textit{other}}$ is the set of all users excluding the target user $u_{t}$ . In this case there may be differences between users on how they perform each activity (e.g., some people tend to walk faster than others) so this approach will introduce noisy instances to the train set, and thus the resulting model will not be very accurate when recognizing activities for $u_{t}$ .

The idea of building a PM is to use the scarce labeled data of $u_{t}$ to select instances from a set of users $U_{\textit{similar}}$ , where $U_{\textit{similar}}$ is the set of users similar to $u_{t}$ according to some similarity criteria. In our approach, we look for similarities per class (that is, activity type) instead of a per user basis, i.e., the final model will be built using only the instances that are similar to those of $u_{t}$ for each activity type. Procedure 1 presents the proposed algorithm to build a PM based on class (activity type) similarities.

Procedure 1
Build PM
1:
${\rm T}\leftarrow\left\{{}\right\}$ $\triangleright$ Start with an empty train set
2:
for $c$ in $C$ do $\triangleright$ For each class
3:
$\textit{data}_{t}\leftarrow\textit{subset}(\tau_{t},c)$ $\triangleright$ $\tau_{t}$ is the target user’s train set
4:
$\textit{data}_{\textit{other}}\leftarrow\textit{subset}(\textit{instances}(U_{% \textit{other}}),c)$
5:
$\textit{data}_{\textit{all}}\leftarrow\textit{data}_{t}\cup\textit{data}_{% \textit{other}}$
6:
Cluster $\textit{data}_{\textit{all}}$ using k-means for $k=2..\textit{UpperBound}$ and select the optimal $k$ according to some clustering quality index.
7:
$S\leftarrow$ $\operatorname{arg\,max}_{g\in G}\left|{\textit{data}_{t}\cap g}\right|$ $\triangleright$ $G$ is the set of the resulting $k$ groups
8:
${\rm T}\leftarrow{\rm T}\cup S\cup\textit{data}_{t}$
9:
end for
10:
$\textit{weight}({\rm T})$ $\triangleright$ Assign a weight to each instance such that the importance of $\tau_{t}$ increases as more training data of the target user is available.
11:
Build model using training set ${\rm T}$ .

The procedure starts by iterating through each possible class (activity type) $c$ . Within each iteration, instances of class $c$ from the $u_{t}$ train set $\tau_{t}$ and all the instances of class $c$ that belong to all other users are stored in $\textit{data}_{\textit{all}}$ . The function $\textit{subset}(set,c)$ returns all the instances in $s e t$ of class $c$ which are then saved in $\textit{data}_{t}$ . Function $\textit{instances}(U)$ returns all the instances that belong to the set of users $U$ . Next, all instances in $\textit{data}_{\textit{all}}$ are clustered using the k-means algorithm for $k=2\ldots\textit{UpperBound}$ . For each $k$ , the Silhouette* clustering quality index [60] of the resulting groups is computed and the $k$ that produces the optimal quality index is chosen. A clustering quality index [9] is a measure of the quality of the resulting clustering based on compactness and separation. The Silhouette index was chosen because it has been shown to produce good results with different datasets [9]. Next, instances from the cluster in which the majority of instances from $\textit{data}_{t}$ ended up are added to the final training set ${\rm T}$ . Also all instances from $\textit{data}_{t}$ that ended up in other clusters are added to ${\rm T}$ to make sure all the data from $u_{t}$ are used. After the for loop, all instances in ${\rm T}$ are assigned an importance weight as a function of the size of $\tau_{t}$ such that instances from the $u_{t}$ train set have more impact as more training data is available for that specific user. The exponential decay function $y=(1-r)^{x}$ was used to assign the weights where $r$ is a decay rate parameter and $x=\left|{\tau_{t}}\right|$ . The weight of all instances in ${\rm T}$ that are not in $\tau_{t}$ is set to $y$ and the weight of all instances in $\tau_{t}$ is set to $1-y$ . Finally, the model is built using ${\rm T}$ with the new instances’ weights. Note that the classification model needs to have support for instance weighting. For the model itself we used a decision tree implementation called rpart [69], which supports instance weighting. This implementation produces binary decision trees by recursively finding the variable that best splits the data into two subgroups [70].
4. Datasets

We conducted our experiments with 4 publicly available datasets from the UCI Machine Learning repository [46]. The criteria for selecting the datasets were:

1.
The dataset must include simple activities.
2.
It must contain data collected from several users.
3.
The information of which user generated each instance must be included.
4.
Each class (activity type) should have several instances per user.

Now we describe the details about each of the datasets that met the criteria to be considered in our experiments. We also include information about the processing steps we made for each of the datasets.

D1: Chest Sensor Dataset. This dataset has data from a wearable accelerometer mounted on the chest [15, 1]. The data were collected from 15 participants performing 7 different activities. The sampling rate was set at 52 Hz. The sensor returns values for the $x$ , $y$ and $z$ axes. The included activities are: 1) working at computer, 2) standing up, walking and going up/down stairs, 3) standing, 4) walking, 5) going up/down stairs, 6) walking and talking with someone, 7) talking. The labeling was carried out by asking the participants to annotate sequentially the activities they performed. To reduce signal noise, a moving average filter with a window length of 10 was applied to the raw accelerometer data for each axis. We extracted 16 common statistical features on fixed length windows of size 208 which corresponds to 4 seconds. The features were: mean for each axis, standard deviation for each axis, maximum value of each axis, correlation between each pair of axes, mean of the magnitude, standard deviation of the magnitude, mean difference of the magnitude and area under the curve of the magnitude. The features were ranked with a filter method based on information gain [16, 59] and the top 10 were kept. The resulting total number of instances were 9 188.

D2: Wrist Sensor Dataset. This dataset is composed of the recordings of 14 simple activities performed by a total of 16 volunteers with a tri-axial accelerometer mounted on the right wrist [13, 4]. The set of activities includes: 1) brush teeth, 2) climb stairs, 3) comb hair, 4) descend stairs, 5) drink glass, 6) eat meat, 7) eat soup, 8) getup bed, 9) liedown bed, 10) pour water, 11) sitdown chair, 12) standup chair, 13) use telephone and 14) walk. The data was collected in the volunteers’ homes and a supervisor labeled the data while the volunteer performed one of the activities of interest. The sampling rate was set at 32 Hz. The same pre-processing steps and the same set of features as dataset 1 were extracted from a window of size 128 that corresponds to 4 seconds. This resulted in a total of 3 098 instances.

D3: WISDM Dataset. This dataset was collected by 36 subjects while performing 6 different activities [39, 3]. The data was recorder using a smartphone with a sampling rate of 20 Hz. The data was collected with an application running on a phone which allows to label the activity being performed through a graphical user interface. The dataset already contained 46 features which were extracted from fixed length windows of 10 seconds each. The activities include: 1) downstairs, 2) jogging, 3) sitting, 4) standing, 5) upstairs and 6) walking. The total number of instances are 5 418.

D4: Smartphone Dataset. This database was built from the recordings of 30 subjects performing activities of daily living while carrying a waist-mounted smartphone with embedded inertial sensors [8, 2]. The activities in this database include: 1) walking, 2) walking upstairs, 3) walking downstairs, 4) sitting, 5) standing and 6) laying. The experiments were video recorded to perform the labeling. The sampling rate was set at 50 Hz. For our experiments we used a subset of this dataset that was distributed in the ‘Data analysis’ course [45] which consists of 21 users. The dataset already includes 561 extracted features from the accelerometer and gyroscope sensors. The total number of instances for the 21 users are 7 352.

For all the datasets the features were normalized between 0–1. Table 2 shows a summary of the datasets and its characteristics.

Table 1
Datasets summary

Abbr. Name # subjects # classes # instances

D1 Chest sensor 15 7 9 188

D2 Wrist sensor 16 14 3 098

D3 WISDM 36 6 5 418

D4 Smartphone 21 6 7 352

Table 2
Distribution of selected number of groups according to the Silhouette index for each dataset

${k=2}$ ${k=3}$ ${k=4}$

D1 25.8% 38.5% 35.6%

D2 64.3% 32.5% 3%

D3 54% 27% 19%

D4 85% 7.7% 7.5%

5. Experiments and results

Abbr.	Name	# subjects	# classes	# instances
D1	Chest sensor	15	7	9 188
D2	Wrist sensor	16	14	3 098
D3	WISDM	36	6	5 418
D4	Smartphone	21	6	7 352

	${k=2}$	${k=3}$	${k=4}$
D1	25.8%	38.5%	35.6%
D2	64.3%	32.5%	3%
D3	54%	27%	19%
D4	85%	7.7%	7.5%

Several works in HAR perform the experiments by first collecting data from one or several users and then evaluating their methods using k-fold cross validation (being 10 the most typical value for $k$ ) on the aggregated data [39, 43, 56, 52, 30, 66, 21, 65]. For a $k=10$ this means that all the data is randomly divided into 10 subsets of approximately equal size. Then, 10 iterations are performed. In each iteration a subset is chosen as the test set and the remaining $k-1$ subsets are used as the train set. This means that 90% of the data is completely labeled and the remaining 10% is unknown, however, we are considering situations it is more likely that just a fraction of the data will be labeled. In our experiments we want to consider the situation when the target user has just a small amount of labeled data. Our models’ evaluation procedure consists of sampling a small percent $p$ of instances from $u_{t}$ to be used as the train set $\tau_{t}$ and use the remaining data to test the performance of the General Model, User-Dependent Model and our proposed Personalized Model.

To preserve the class distributions, stratified random sampling was used, i.e., perform simple random sampling within each class. This will ensure that at least one instance per class is chosen. We chose $p$ to range between 1% to 30% with increments of 1. To reduce sampling variability, for each $p$ percent we performed 5 stratified sampling iterations for each user and reported the averaged results.

Figures 2–4 show the results of averaging the accuracy of all users for each $p$ percent of data used as train set. The plots also show the results of a general model 2 (magenta curve) which is the same as the general model but also including the target user training instances; we show this curve for establishing that the improvement is not due to adding the user’s data, but to the application of our method. For D1 (Fig. 2) the PM clearly outperforms the other models when the labeled data is between 1% and 10% (the curve for PM-2 will be explained soon). The general model shows a stable accuracy since it is independent of the user. The general model 2 presents an overall performance similar to the general model but with a small increase as more training data from the target user is added. For the rest of the datasets the PM shows an overall higher accuracy except for D2 (later we will analyze why this happened). The recall plots (which can be found in Appendix A) have a similar shape.

Figure 1.

D1 – Chest sensor dataset. PM-2 is explained below.

Figure 2.

D2: Wrist sensor dataset.

Figure 3.

D3: WISDM dataset.

Figure 4.

D4: Smartphone dataset.

Table 2 shows the distribution of the resulting number of groups selected by the Silhouette index. For datasets D2, D3 and D4 the predominant number of groups was 2. For D1 $k=3$ was the predominant value. Table 4 shows the average number of labeled instances per class (activity type) and its corresponding approximate time in seconds for each $p$ percent of training data. For example for D3 we can see how just using 3 labeled instances per class the PM achieves a good classification accuracy ( $\approx$ 0.8).

Tables 4 and 6 show the difference of average overall accuracy and recall (from 1% to 30% of labeled data) between the PM and the other two models. Here we can see how the PM significantly outperforms the other two models in all datasets except for the accuracy in D2 when comparing PM – UDM, case in which the difference is negative. This may be due to the user-class sparsity of the dataset, i.e., some users just performed a small subset of the activities. This situation will introduce noise to the PM. In the extreme case when a user has just 1 type of activity, obviously it would be enough to always predict that activity. However, the PM is trained with the entire set of possible labels from all other users in which case the model will predict labels that are not part of that user. To confirm this, we visualized and quantified the user-class sparsity of the datasets and performed further experiments. First we computed the user-class sparsity matrices for each dataset. These matrices are generated by plotting what activities were performed by each user. A cell in the matrix is set to 1 if a user performed an activity and 0 otherwise. The sparsity index is computed as 1 minus the proportion of 1 s in the matrix. In datasets D1 and D4 all users performed all activities giving a sparsity index of 0. Figures 6 and 6 show the user-class sparsity matrices of datasets D2 and D3 respectively. D2 has a sparsity index of 0.6 whereas for D3 it is 0.18. For D2 this index is very high (more than half of the entries in the matrix are 0); further, the number of classes for this dataset is also high (12). From the matrix we can see that several users performed just a small number of activities (in some cases just 1 or 2 activities). One way to deal with this situation is to train the model excluding activities from other users that were not performed by the target user. Figures 2–4 (gray dotted line PM-2) show the results of excluding types of activities that are not in $u_{t}$ . As expected, for datasets with low or no sparsity the results are almost the same (with small variations due to random initial k-means centroids). For D2 which has a high sparsity the accuracy significantly increased. This shows evidence that the user-class distribution of the dataset has an impact on the PM and that this can be alleviated by excluding the classes that are not relevant for a particular user.

Table 3

Average number of labeled instances per class for each dataset and corresponding approximate time in seconds per class

avg. #instances/secs.	1%	5%	10%	15%	20%
D1	1/4	5/20	9/36	14/56	18/72
D2	1/4	1/4	2/8	3/12	3/12
D3	1/10	2/20	3/30	4/40	5/50
D4	1/2.5	3/7.6	6/15.3	9/23.0	12/30.7

Table 4

Difference of average overall accuracy (from 1% to 30% of labeled data) between the PM and the other two models

	PM – GM	PM – UDM
D1	19.1%	2.7%
D2	12.5%	$-$ 4.5%
D3	6.4%	22.4%
D4	4%	25.5%

Table 5

Difference of average overall recall (from 1% to 30% of labeled data) between the PM and the other two models

	PM-GM	PM-UDM
D1	15.5%	8.6%
D2	18.1%	13.9%
D3	7.4%	34.4%
D4	4%	28%

Table 6

Mean overall accuracy between 1% and 30% of training data for different clustering quality indices (PM-2)

	Silhouette index	PBM index
D1	74.5%	74.8%
D2	67.1%	67.2%
D3	78.5%	78.4%
D4	90.7%	90.2%

Figure 5.

D2: Wrist sensor dataset user-class sparsity matrix.

Figure 6.

D3: WISDM dataset user-class sparsity matrix.

Since the method selects just the similar classes, it may introduce some class imbalance [37]. To account for this, we used the oversampling method called SMOTE [18] to make sure the classes are balanced. Figure 7 shows a comparison between PM-2 and PM-SMOTE, which adds the oversampling step before building the final classifier. In this figure we can see that performing the oversampling did not produce any significant improvement on the results (we did the comparison for all 4 datasets with similar results).

Table 7

Results of the statistical tests with 95% confidence intervals.

	PM/GM		PM/UDM
	t-test	Mann-Whitney	t-test	Mann-Whitney
D1: Chest sensor	$p\ll 0.01$ CI (0.16, 0.21)	$p\ll 0.01$	$p\ll 0.05$ CI (0.006, 0.048)	$p\ll 0.01$
D2: Wrist sensor	$p\ll 0.01$ CI (0.10, 0.14)	$p\ll 0.01$	$p\ll 0.01$ CI ( $-$ 0.059, $-$ 0.031)	$p\ll 0.01$
D3: Wisdm	$p\ll 0.01$ CI (0.05, 0.07)	$p\ll 0.01$	$p\ll 0.01$ CI (0.17, 0.27)	$p\ll 0.01$
D4: Activities	$p\ll 0.01$ CI (0.02, 0.06)	$p\ll 0.01$	$p\ll 0.01$ CI (0.16, 0.34)	$p\ll 0.01$

Figure 7.

D3: WISDM dataset. Comparison between PM-2 and PM-2 with SMOTE oversampling.

We also evaluated the impact of choosing another clustering quality index to identify the optimal number of groups for Algorithm 1. In this case we also tested with the PBM index which was chosen because it is more recent and in the original paper [57] it was claimed that the results were superior than other previously proposed indices. Table 6 shows the comparison between using Silhouette and PBM indices for the mean overall accuracy between 1% and 30% of training data (PM-2). The results show that there is no significant difference in the selection between the two clustering quality indices.

Figures 8–11 show the resulting confusion matrices. The anti-diagonal represents the recall of each individual activity. For datasets D1, D2 and D3 the recall of the general model seems to be skewed towards the walking activity which is also the most common one. For the personalized and user-dependent model, the recall is more uniformly distributed (the anti-diagonal has a more solid color). For all the datasets the major source of error was between the walking and stairs activities.

Figure 8.

D1: Chest sensor dataset confusion matrix.

Figure 9.

D2: Wrist sensor dataset confusion matrix.

Figure 10.

D3: WISDM sensor dataset confusion matrix.

Figure 11.

D4: Activities dataset confusion matrix.

To validate our results we used a two-tail paired t-test with a significance level $\alpha=$ 0.05 to see whether or not there is a significant difference in the performance between the proposed Personalized Model and the General Model and User Dependent Model. We also performed a Mann-Whitney U test which does not assume normality in the data. Table 7 shows the results of the statistical tests. From this table we can see that all tests resulted in a statistically significant performance increase except in the case when comparing PM against UDM for dataset D2, which resulted in a statistically significant reduction in performance, which is the case when the sparsity is high. After excluding the classes that are not part of the target user the PM clearly outperformed the UDM (Fig. 2 gray line). Figures 12–15 show the box plots between the PM and the other two.

Figure 12.

D1: Chest sensor dataset box plots.

Figure 13.

D2: Wrist sensor dataset box plots.

Figure 14.

D3: WISDM sensor dataset box plots.

Figure 15.

D4: Activities dataset box plots.

6. Conclusions

We have presented a “crowdsourcing” method for taking advantage of training data from a community of users in order to get data useful for personalizing a device for a given user, improving the accuracy in recognizing human physical activities. With this approach, we can get the benefits of personalized training reducing drastically the effort of a painful individual training. One of the main insights in our approach is that we select, from the universe of training data generated by the community of users, the data of users which behave in a most similar way to the user under consideration.

We evaluated the proposed method on 4 independent human activity datasets. The experimental data shows that with just 5% of users’ training data we can achieve accuracy close to 70% (as for instance in Fig. 2), in situations where the (user-independent) general model gives an accuracy below 60% and a user-dependent model (with data just from the considered user) would get an accuracy $\approx$ 60%. So we have evidence to claim that our approach is a good compromise that gets nearly the advantages of the general model (almost no need for individual training) as well as those of personalized models (high accuracy).

The main contribution of our approach is to refine the community data identifying clusters of classes that behave in a similar way with respect to an activity, and then using training data only from that cluster. In the case of datasets with high sparsity, the performance problems were attenuated to a great extent by excluding types of activities from other users that were not performed by the target user.

One of the limitations of our approach is that it assumes that at least there is a small amount of labeled data; however it does not consider the case when there is no labeled data at all for the considered user. This scenario will be considered in future work. In the present paper we also assumed that the users collected the data using the same type of device, accelerometers in particular. An interesting future direction would be to also take into account the heterogeneity of devices. Another future direction is to intend to carry this type of crowdsourcing-based selection of training data on long-term/complex activities [31, 47], like commuting, shopping, cooking, dining, etc.

Footnotes

Acknowledgments

Enrique Garcia-Ceja would like to thank Consejo Nacional de Ciencia y Tecnología (CONACYT) and the Intelligent Systems research group at Tecnologico de Monterrey for the financial support in his PhD. studies.

A. Appendix

Recall plots for the Personalized Models experiments of the 4 datasets.

Figure 16.

D1 – Chest sensor dataset.

Figure 17.

D2: Wrist sensor dataset.

Figure 18.

D3: WISDM dataset.

Figure 19.

D4: Smartphone dataset.

References

Activity recognition from single chest-mounted accelerometer data set, https://archive.ics.uci.edu/ml/datasets/Activity+Recognition+from+Single+Chest-Mounted+Accelerometer 2012. Accessed: February 18 2015.

Human activity recognition using smartphones data set, http://archive.ics.uci.edu/ml/datasets/Human+Activity+Recognition+Using+Smartphones, 2012. Accessed: February 18 2015.

Activity prediction dataset. http://www.cis.fordham.edu/wisdm/dataset.php, 2012. Accessed: February 18 2015.

Dataset for adl recognition with wrist-worn accelerometer data set, https://archive.ics.uci.edu/ml/datasets/Dataset+for+ADL+Recognition+with+Wrist-worn+Accelerometer, 2014. Accessed: February 18 2015.

Abdallah

Gaber

Srinivasan

and Krishnaswamy

, StreamAR: Incremental and active learning with evolving sensory data for activity recognition, in: Tools with Artificial Intelligence (ICTAI), 2012 IEEE 24th International Conference on 1 (Nov 2012), 1163–1170. doi: 10.1109/ICTAI.2012.169.

Ahn

H.J.

, A new similarity measure for collaborative filtering to alleviate the new user cold-starting problem, Information Sciences 178(1) (2008), 37–51. ISSN 0020-0255. doi: https://dx-doi-org.web.bisu.edu.cn/10.1016/j.ins.2007.07.024. URL https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0020025507003751.

Akl

and Valaee

, Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing, in: Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, IEEE (2010), 2270–2273.

Anguita

Ghio

Oneto

Parra

and Reyes-Ortiz

J.L.

, Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine, in: Ambient Assisted Living and Home Care Bravo

Hervás

and Rodríguez

, eds, volume 7657 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2012, pp. 216–223. ISBN 978-3-642-35394-9. doi: 10.1007/978-3-642-35395-630. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/978-3-642-35395-6_30.

Arbelaitz

Gurrutxaga

Muguerza

Pérez

J.M.

and Perona

, An extensive comparative study of cluster validity indices, Pattern Recognition 46(1) (2013), 243–256. ISSN 0031-3203. doi: https://dx-doi-org.web.bisu.edu.cn/10.1016/j.patcog.2012.07.021. URL https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S003132031200338X.

10.

Baldauf

Dustdar

and Rosenberg

, A survey on context-aware systems, International Journal of Ad Hoc and Ubiquitous Computing 2(4) (2007), 263–277.

11.

Banos

Galvez

J.-M.

Damas

Pomares

and Rojas

, Window size impact in human activity recognition, Sensors 14(4) (2014), 6474–6499. ISSN 1424-8220. doi: 10.3390/s140406474. URL https://www-mdpi-com.web.bisu.edu.cn/1424-8220/14/4/6474.

12.

Barbier

Zafarani

Gao

Fung

and Liu

, Maximizing benefits from crowdsourced data, Computational and Mathematical Organization Theory 18(3) (2012), 257–279.

13.

Bruno

Mastrogiovanni

and Sgorbissa

, A public domain dataset for adl recognition using wrist-placed accelerometers, in: Robot and Human Interactive Communication, 2014 RO-MAN: The 23rd IEEE International Symposium on, (Aug 2014), 738–743. doi: 10.1109/ROMAN.2014.6926341.

14.

Brush

Krumm

and Scott

, Activity recognition research: The good, the bad, and the future, in: Proceedings of the Pervasive 2010 Workshop on How to do Good Research in Activity Recognition, Helsinki, Finland, (2010), 17–20.

15.

Casale

Pujol

and Radeva

, Personalization and user verification in wearable systems using biometric walking patterns, Personal and Ubiquitous Computing 16(5) (2012), 563–580. ISSN 1617-4909. doi: 10.1007/s00779-011-0415-z. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/s00779-011-0415-z.

16.

Chandrashekar

and Sahin

, A survey on feature selection methods, Computers & Electrical Engineering 40(1) (2014), 16–28.

17.

Chapelle

Schölkopf

Zien

et al., Semi-Supervised Learning, MIT press Cambridge, 2006.

18.

Chawla

N.V.

Bowyer

K.W.

Hall

L.O.

and Kegelmeyer

W.P.

, SMOTE: Synthetic minority over-sampling technique, Journal of Artificial Intelligence Research (2002), 321–357.

19.

Choi

and Savarese

, A unified framework for multi-target tracking and collective activity recognition, in: Computer Vision ECCV 2012 Fitzgibbon

Lazebnik

Perona

Sato

and Schmid

, eds, volume 7575 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2012, pp. 215–230. ISBN 978-3-642-33764-2. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/978-3-642-33765-9_16.

20.

Cook

Krishnan

and Rashidi

, Activity discovery and activity recognition: A new partnership, Cybernetics, IEEE Transactions on 43(3) (June 2013), 820–828. ISSN 2168-2267. doi: 10.1109/TSMCB.2012.2216873.

21.

Fang

Srinivasan

and Cook

D.J.

, Feature selections for human activity recognition in smart home environments, Int J Innov Comput Inf Control 8 (2012), 3525–3535.

22.

Garcia-Ceja

and Brena

, Building personalized activity recognition models with scarce labeled data based on class similarities, in: Ubiquitous Computing and Ambient Intelligence. Sensing, Processing, and Using Environmental Information (UCAmI 2015), volume 0 of UCAMI, Springer (December 2015).

23.

Garcia-Ceja

Brena

and Galván-Tejada

C.E.

, Contextualized hand gesture recognition with smartphones, in: Pattern Recognition Martínez-Trinidad

J.F.

Carrasco-Ochoa

J.A.

Olvera-Lopez

J.A.

Salas-Rodríguez

and Suen

, eds, volume 8495 of Lecture Notes in Computer Science, Springer International Publishing, 2014, pp. 122–131. ISBN 978-3-319-07490-0. doi: 10.1007/978-3-319-07491-713. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/978-3-319-07491-7_13.

24.

Garcia-Ceja

Brena

R.F.

Carrasco-Jimenez

J.C.

and Garrido

, Long-term activity recognition from wristwatch accelerometer data, Sensors 14(12) (2014), 22500–22524. ISSN 1424-8220. doi: 10.3390/s141222500. URL https://www-mdpi-com.web.bisu.edu.cn/1424-8220/14/12/22500.

25.

Tao

Pung

H.K.

and Lu

, epsicar: An emerging patterns based approach to sequential, interleaved and concurrent activity recognition, in: Pervasive Computing and Communications, 2009. PerCom 2009. IEEE International Conference on, (March 2009), 1–9. doi: 10.1109/PERCOM.2009.4912776.

26.

Guan

Yuan

Lee

Y.-K.

Gavrilov

and Lee

, Activity recognition based on semi-supervised learning, in: Embedded and Real-Time Computing Systems and Applications, 2007. RTCSA 2007. 13th IEEE International Conference on, (Aug 2007), 469–475. doi: 10.1109/RTCSA.2007.17.

27.

Guenterberg

Ghasemzadeh

and Jafari

, Automatic segmentation and recognition in body sensor networks using a hidden markov model, ACM Trans Embed Comput Syst 11(S2) (Aug 2012), 46:1–46:19. ISSN 1539-9087. doi: 10.1145/2331147.2331156. URL http://doi.acm.org/10.1145/2331147.2331156.

28.

Han

Lee

Sarkar

A.M.J.

and Lee

Y.-K.

, A framework for supervising lifestyle diseases using long-term activity monitoring, Sensors 12(5) (2012), 5363–5379. ISSN 1424-8220. doi: 10.3390/s120505363. URL https://www-mdpi-com.web.bisu.edu.cn/1424-8220/12/5/5363.

29.

Heilbron

F.C.

and Niebles

J.C.

, Collecting and annotating human activities in web videos, in: Proceedings of International Conference on Multimedia Retrieval, ICMR ’14, ACM, New York, NY, USA (2014), 377–384. ISBN 978-1-4503-2782-4. doi: 10.1145/2578726.2578775. URL http://doi.acm.org/10.1145/2578726.2578775.

30.

Hung

Englebienne

and Kools

, Classifying social actions with a single accelerometer, in: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’13, ACM, New York, NY, USA (2013), 207–210. ISBN 978-1-4503-1770-2. doi: 10.1145/2493432.2493513. URL http://doi.acm.org/10.1145/2493432.2493513.

31.

Huynh

Fritz

and Schiele

, Discovery of activity patterns using topic models, in: Proceedings of the 10th International Conference on Ubiquitous Computing, UbiComp ’08, ACM, New York, NY, USA (2008), 10–19. ISBN 978-1-60558-136-1. doi: 10.1145/1409635.1409638. URL http://doi.acm.org/10.1145/1409635.1409638.

32.

Jin

Toh

H.-S.

Soh

W.-S.

and Wong

W.-C.

, A robust dead-reckoning pedestrian tracking system with low cost sensors, in: Pervasive Computing and Communications (PerCom), 2011 IEEE International Conference on, (March 2011), 222–230. doi: 10.1109/PERCOM.2011.5767590.

33.

Khan

Lee

Y.-K.

Lee

and Kim

T.-S.

, Accelerometers position independent physical activity recognition system for long-term activity monitoring in the elderly, Medical & Biological Engineering & Computing 48(12) (2010), 1271–1279. ISSN 0140-0118. doi: 10.1007/s11517-010-0701-3. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/s11517-010-0701-3.

34.

Khan

A.M.

Lee

Y.-K.

Lee

S.Y.

and Kim

T.-S.

, A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer, Information Technology in Biomedicine, IEEE Transactions on 14(5) (2010), 1166–1172.

35.

Kirkham

Khan

Bhattacharya

Hammerla

Mellor

Roggen

and Ploetz

, Automatic correction of annotation boundaries in activity datasets by class separation maximization, in: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, UbiComp ’13 Adjunct, ACM, New York, NY, USA (2013), 673–678. ISBN 978-1-4503-2215-7. doi: 10.1145/2494091.2495988. URL http://doi.acm.org/10.1145/2494091.2495988.

36.

Koskimaki

and Siirtola

, Recognizing gym exercises using acceleration data from wearable sensors, in: Computational Intelligence and Data Mining (CIDM), 2014 IEEE Symposium on, IEEE (2014), 321–328.

37.

Kotsiantis

Kanellopoulos

Pintelas

et al., Handling imbalanced datasets: A review, GESTS International Transactions on Computer Science and Engineering 30(1) (2006), 25–36.

38.

Kühnel

Westermann

Hemmert

Kratz

Müller

and Möller

, I’m home: Defining and evaluating a gesture set for smart-home control, International Journal of Human-Computer Studies 69(11) (2011), 693–704. ISSN 1071-5819. doi: https://dx-doi-org.web.bisu.edu.cn/10.1016/j.ijhcs.2011.04.005. URL https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S1071581911000668.

39.

Kwapisz

J.R.

Weiss

G.M.

and Moore

S.A.

, Activity recognition using cell phone accelerometers, SIGKDD Explor Newsl 12(2) (Mar 2011), 74–82. ISSN 1931-0145. doi: 10.1145/1964897.1964918. URL http://doi.acm.org/10.1145/1964897.1964918.

40.

Lane

N.D.

Mohammod

Lin

Yang

Ali

Doryab

Berke

Choudhury

and Campbell

, Bewell: A smartphone application to monitor, model and promote wellbeing, in: 5th International ICST Conference on Pervasive Computing Technologies for Healthcare, (2011), 23–26.

41.

Lane

N.D.

Choudhury

Campbell

A.T.

and Zhao

, Enabling large-scale human activity inference on smartphones using community similarity networks (Csn), in: Proceedings of the 13th International Conference on Ubiquitous Computing, UbiComp ’11, ACM, New York, NY, USA (2011), 355–364. ISBN 978-1-4503-0630-0. doi: 10.1145/2030112.2030160. URL http://doi.acm.org/10.1145/2030112.2030160.

42.

Lasecki

W.S.

Weingard

Ferguson

and Bigham

J.P.

, Finding dependencies between actions using the crowd, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, ACM, New York, NY, USA (2014), 3095–3098. ISBN 978-1-4503-2473-1. doi: 10.1145/2556288.2557176. URL http://doi.acm.org/10.1145/2556288.2557176.

43.

Lee

Y.-S.

and Cho

S.-B.

, Activity recognition using hierarchical hidden markov models on a smartphone with 3d accelerometer, in: Hybrid Artificial Intelligent Systems Corchado

Kurzyński

and Woźniak

, eds, volume 6678 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2011, pp. 460–467. ISBN 978-3-642-21218-5. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/978-3-642-21219-2_58.

44.

Lee

Y.-S.

and Cho

S.-B.

, Activity recognition with android phone using mixture-of-experts co-trained with labeled and unlabeled data, Neurocomputing 126 (2014), 106–115. ISSN 0925-2312. doi: https://dx-doi-org.web.bisu.edu.cn/10.1016/j.neucom.2013.05.044. URL https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0925231213006966.

45.

Leek

, Data analysis online course, https://www.coursera.org/specializations/jhu-data-science, 2014.

46.

Lichman

, UCI machine learning repository, 2013. URL http://archive.ics.uci.edu/ml. Accessed: February 18 2015.

47.

Liu

Peng

Liu

and Huang

, Sensor-based human activity recognition system with a multilayered model using time series shapelets, Knowledge-Based Systems (2015).

48.

Lockhart

J.W.

and Weiss

G.M.

, Limitations with activity recognition methodology & data sets, in: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, UbiComp ’14 Adjunct, ACM, New York, NY, USA (2014), 747–756. ISBN 978-1-4503-3047-3. doi: 10.1145/2638728.2641306. URL http://doi.acm.org/10.1145/2638728.2641306.

49.

Frauendorfer

Rabbi

Mast

M.S.

Chittaranjan

G.T.

Campbell

A.T.

Gatica-Perez

and Choudhury

, StressSense: Detecting stress in unconstrained acoustic environments using smartphones, in: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, UbiComp ’12, ACM, New York, NY, USA (2012), 351–360. ISBN 978-1-4503-1224-0. doi: 10.1145/2370216.2370270. URL http://doi.acm.org/10.1145/2370216.2370270.

50.

Mannini

and Sabatini

A.M.

, Machine learning methods for classifying human physical activity from on-body accelerometers, Sensors 10(2) (2010), 1154–1175. ISSN 1424-8220. doi: 10.3390/s100201154. URL https://www-mdpi-com.web.bisu.edu.cn/1424-8220/10/2/1154.

51.

Martínez-Pérez

F.E.

González-Fraga

J.A.

Cuevas-Tello

J.C.

and Rodríguez

M.D.

, Activity inference for ambient intelligence through handling artifacts in a healthcare environment, Sensors 12(1) (2012), 1072–1099. ISSN 1424-8220. doi: 10.3390/s120101072. URL https://www-mdpi-com.web.bisu.edu.cn/1424-8220/12/1/1072.

52.

Mitchell

Monaghan

and O’Connor

N.E.

, Classification of sporting activities using smartphone accelerometers, Sensors 13(4) (2013), 5317–5337.

53.

Muaremi

Arnrich

and Tröster

, Towards measuring stress with smartphones and wearable devices during workday and sleep, BioNanoScience 3(2) (2013), 172–183. ISSN 2191-1630. doi: 10.1007/s12668-013-0089-2. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/s12668-013-0089-2.

54.

Nguyen-Dinh

L.-V.

Blanke

and Tröster

, Towards scalable activity recognition: Adapting zero-effort crowdsourced acoustic models, in: Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia, ACM (2013), 18.

55.

Wang

and Moulin

, RGBD-HuDaAct: A Color-Depth Video Database for Human Daily Activity Recognition, in: Consumer Depth Cameras for Computer Vision Fossati

Gall

Grabner

Ren

and Konolige

, eds, Advances in Computer Vision and Pattern Recognition, Springer London, 2013, pp. 193–208. ISBN 978-1-4471-4639-1. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/978-1-4471-4640-7_10.

56.

Lara

Ó.D.

Pérez

A.J.

Labrador

M.A.

and Posada

J.D.

, Centinela: A human activity recognition system based on acceleration and vital sign data, Pervasive and Mobile Computing 8(5) (2012), 717–729. ISSN 1574-1192. doi: https://dx-doi-org.web.bisu.edu.cn/10.1016/j.pmcj.2011.06.004. URL https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S1574119211000794.

57.

Pakhira

M.K.

Bandyopadhyay

and Maulik

, Validity index for crisp and fuzzy clusters, Pattern Recognition 37(3) (2004), 487–501.

58.

Parviainen

Bojja

Collin

Leppänen

and Eronen

, Adaptive activity and environment recognition for mobile phones, Sensors 14(11) (2014), 20753–20778. ISSN 1424-8220. doi: 10.3390/s141120753. URL https://www-mdpi-com.web.bisu.edu.cn/1424-8220/14/11/20753.

59.

Romanski

and Kotthoff

, Package fselector, 2014.

60.

Rousseeuw

P.J.

, Silhouettes: A graphical aid to the interpretation and validation of cluster analysis, Journal of Computational and Applied Mathematics 20 (1987), 53–65.

61.

Sadek

Al-Hamadi

Michaelis

and Sayed

, A fast statistical approach for human activity recognition, International Journal of Intelligence Science 2 (2012), 9.

62.

Sanchez

Martinez

Campos

Estrada

and Pelechano

, Inferring loneliness levels in older adults from smartphones, J Ambient Intell Smart Environ 7(1) (Jan 2015), 85–98. ISSN 1876-1364. URL http://dl.acm.org/citation.cfm?id=2724694.2724701.

63.

Schein

A.I.

Popescul

Ungar

L.H.

and Pennock

D.M.

, Methods and metrics for cold-start recommendations, in: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM (2002), 253–260.

64.

Schilit

Adams

and Want

, Context-aware computing applications, in: Mobile Computing Systems and Applications, 1994. WMCSA 1994. First Workshop on, IEEE (1994), 85–90.

65.

Shoaib

Bosch

Incel

O.D.

Scholten

and Havinga

P.J.M.

, Fusion of smartphone motion sensors for physical activity recognition, Sensors 14(6) (2014), 10146–10176. ISSN 1424-8220. doi: 10.3390/s140610146. URL https://www-mdpi-com.web.bisu.edu.cn/1424-8220/14/6/10146.

66.

Siirtola

Laurinen

Haapalainen

Roning

and Kinnunen

, Clustering-based activity classification with a wrist-worn accelerometer using basic features, in: Computational Intelligence and Data Mining, 2009. CIDM ’09. IEEE Symposium on, (Mar 2009), 95–100. doi: 10.1109/CIDM.2009.4938635.

67.

Stikic

Van Laerhoven

and Schiele

, Exploring semi-supervised and active learning for activity recognition, in: Wearable Computers, 2008. ISWC 2008. 12th IEEE International Symposium on, IEEE (2008), 81–88.

68.

Sung

Ponce

Selman

and Saxena

, Human activity detection from RGBD images, CoRR, abs/11070169, (2011).

69.

Therneau

Atkinson

Ripley

and Ripley

M.B.

, Package rpart, https://cran.r-project.org/web/packages/FSelector/index.html, 2015.

70.

Therneau

T.M.

and Atkinson

E.J.

, An introduction to recursive partitioning using the rpart routines, Technical report, Technical Report 61, 1997.

71.

Varkey

Pompili

and Walls

, Human motion recognition using a wireless sensor-based wearable system, Personal and Ubiquitous Computing 16(7) (2012), 897–910. ISSN 1617-4909. doi: 10.1007/s00779-011-0455-4. URL https://dx-doi-org.web.bisu.edu.cn/10.1007/s00779-011-0455-4.

72.

Q.V.

Hoang

M.T.

and Choi

, Personalization in mobile activity recognition system using-medoids clustering algorithm, International Journal of Distributed Sensor Networks (2013).

73.

Pan

Zhang

and Li

, Gesture recognition with a 3-d accelerometer, in: Ubiquitous Intelligence and Computing, Springer (2009), 25–38.

74.

Zhang

and Sawchuk

A.A.

, A feature selection-based framework for human activity recognition using wearable multimodal sensors, in: Proceedings of the 6th International Conference on Body Area Networks, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering) (2011), 92–98.

75.

Zheng

V.W.

Cao

Zheng

Xie

and Yang

, Collaborative filtering meets mobile recommendation: A user-centered approach, in: AAAI 10 (2010), 236–241.