The user attribution problem and the challenge of persistent surveillance of user activity in complex networks

2.1. Research questions

These are the research questions that have provided the original motivation for this study:

Is it possible to recognize specific users among many in the network by observing and classifying their historical communication behavior and be at least as accurate in the classification process measured using recall as when leveraging comparable classification approaches?

2.2. Assumptions

Two assumptions were made for this study:

HTTP (port 80) traffic was selected as the most representative user communication type traffic since it is used by web browsers which require direct user intervention to navigate. An implication of this study is that it will not be possible to separate any traffic initiated by applications which do not require user intervention/direction but still utilize port 80.

2.3. Related work

The popularity of wireless devices and the rise in supported bandwidth by WiFi and cellular 4G networks has brought to the forefront the user attribution problem in the context of mobility scenarios. Between 2009 and 2013 several researchers have tackled this problem. Two generic frequency based approaches have emerged from this work. One leverages source IP address based identification to track users and uses frequency of access to visited web destinations to perform inference. Results from this approach are good in terms of accuracy and scalability but accuracy decreases dramatically when the source IP changes. The other approach leverages behavior-based identification which forgoes tracking via the use of a source IP address and only uses frequency of access to visited web destinations. Results are promising in terms of recall accuracy but accuracy does not scale well since frequency of visited web sites does not provide enough unique differentiation among different users especially when few popular web sites dominate test data sets.

In 2009 Kumpošt and Matyáš [32] took on the user attribution problem by leveraging vectors of destination IP addresses bound to specific source IP addresses and classified users based on similarity between train and test data measured using TF-IDF. Experiments results are mixed showing 21% false alarms for SSH, and false alarm rates of 70% and 60% for HTTP and HTTPS. The authors blame the poor results on students moving across campus and getting assigned different IP addresses. In 2010 Herrmann, Gerber, Banse and Federrath [25] use behavior-based identification to tackle the user attribution problem so that users are identified based on access frequencies of web destinations within a fixed user time window which satisfies classification based on a Multinomial Naïve Bayes (MNB) classifier. Experiment results show correct user identification for 50% of the 28 users 80% of the time. Assumption of conditional independence among sites visited limits the scalability of this solution. In 2010, Yang [46] also proposed behavior-based identification. During inference, support and lift are computed to record the strength of patterns of visited web sites followed by calculating the Euclidean difference between learned user patterns and newly inferred ones. Experiment results show 87% accuracy for 100 users using the support based inference. However, the author acknowledges the difficulty of scaling up the number of users due to the inability of the approach to link up consecutive user sessions belonging to the same user. In order to address the scalability problem Yang suggests, as future research, combining behavior-based identification with the use of a tracking identifier like a source IP address. In 2012, Banse, Herrmann, and Federrath [4] use the triplet ⟨ epoch, source IP, destination IP ⟩ to identify user sessions by aggregating all events that share that same epoch (time frame) and source IP based on a Multinomial Naïve Bayes classifier. Experiment results show correct user identification for 88% of about 2100 user sessions. The authors also acknowledge that changing the source IP address frequently decreases accuracy (60% every 3 h, 49% every hour). In 2013, Hermann, Banse and Federrath [24] use again the triplet ⟨ epoch, source IP, destination IP ⟩ to identify user sessions by aggregating all events that share that same epoch (time frame) with a source IP based on a comparison of three classification approaches: (1) 1-Nearest Neighbor Classifier using Jaccard coefficient and Cosine Similarity), (2) Multinomial Naïve Bayes and (3) using lift and support as proposed in Yang [46]. The best accuracy results record up to 85% recall, using MNB, for over 3000 users with the IP address changing every 24 h. Recall accuracy degrades when the source IP address changes frequently (65% every 3 h, 54% every hour). In Yang’s study, behavioral profiling was meant to be used as an additional authentication mechanism (like a behavioral biometric). Therefore, Yang could assume that the learning algorithms will have access to a quite large set of labeled sessions for each user. In fact, the cited result of 87% recall for 100 concurrent users was achieved with 200 training sessions per user (to derive the support-based patterns for the profiles) and 100 test sessions, which were processed as a whole to obtain the support-based profiles, which were to be linked to the training sessions. In contrast, the work by Herrmann et al. [24] links singular sessions.

Previous research on web mining [21,37,41] shows that utilizing the source IP is a poor choice for identifying users when the source IP address changes as is the case in mobility scenarios. Previous research also shows that utilizing exclusively behavior based identification does not scale well. To understand why consider using the approach proposed in the literature to identify users that visit 3 popular web sites, say A, B, C. In this case, there exists a single identifiable pattern ⟨ A , B , C ⟩ distinguishable only based on the user frequency of access of each web site. Now consider recording the order of visits to web sites as an additional way to classify unique patterns. This approach would increase six fold the number of unique patterns: ⟨ ABC ⟩ , ⟨ ACB ⟩ , ⟨ CAB ⟩ , ⟨ CBA ⟩ , ⟨ BAC ⟩ , ⟨ BCA ⟩ . Finally, consider taking into account the time when web sites are visited so that sites visited at approximately the same time represent a single user timed sequence. Now the number of unique patterns increases even more: ⟨ ABC ⟩ , ⟨ ACB ⟩ , ⟨ CAB ⟩ , ⟨ CBA ⟩ , ⟨ BAC ⟩ , ⟨ BCA ⟩ , ⟨ A ⟩ , ⟨ B ⟩ , ⟨ C ⟩ , ⟨ AB ⟩ , ⟨ BC ⟩ , ⟨ AC ⟩ , ⟨ CB ⟩ , ⟨ CA ⟩ , ⟨ BA ⟩ . Increasing the number of unique identifiable patterns helps address the presence of popular web sites in the data set creating conditions for unique differentiation among user patterns that enables to adequately address the user attribution problem.

The next few sections will introduce many terms, computations and symbols which are defined in Appendix I. Figure 7 shows a typical HTM with its inputs. Input sequence ( IS ) is the input sequence being matched by a given HTM layer. For HTM layer 1, IS is a sequence of web destinations (in a real cellular network these are extracted from HTTP requests) and for HTM layers 2 and 3 it is a sequence of temporal groups (variable order Markov chains) matched in the layer below. At layer 1, input is of the form: Timestamp ⟨ TS , Dest ⟩ where the format of this input is fully described in Section 3.1.1.

At HTM layers 2 and 3 input is of the form: λ L x ⟨ FFB g i , FFB g i + 1 , FFB g i + 2 , … ⟩ where 2 ⩽ x ⩽ 3 represent layers above the first HTM layer and 1 ⩽ i ⩽ the number of Markov chains at HTM layer L x − 1 . λ L x is a vector of feed forward beliefs, which measures how well inputs match learned sequences and is a vector where only one index is filled based on which temporal group g i matched the input at the lower layer L x − 1 of the HTM. The λ L x vector represents the input for HTM layers 2 and 3 where the index of the entry in the vector that is filled represents the feed forward belief of the specific temporal group g i received from lower HTM layers. For instance, vectors λ L 2 ⟨ 0 , 0.4 , 0 ⟩ and λ L 2 ⟨ 0.8 , 0 , 0 ⟩ , show layer 2 of the HTM receiving from layer 1 a sequence of temporal groups S g 21 ( g 2 , g 1 ) with feed forward belief values of 40% and 80% for temporal groups 2 and 1 respectively.

3.1.1. Session identification

In this study, the TCP timestamp TS value [29] is used to identify and track user sessions only during the training phase of experiments. The use of TCP timestamps was inspired by the work of Kohno, Broido and Claffy [30]. These authors proposed device recognition by fingerprinting devices via detection of changes in clock skews among different devices using the TCP Timestamp option. Kohno et al., believe that their approach can be used to identify the same physical device among a large number of devices since there exist variability in the clock skew of different physical devices, and it holds that the clock skew for a given device is constant and independent of network access technology. This approach differs from the way in which TCP timestamps are used in this study, where they are leveraged to track directly user sessions and consider clock skew not as a unique fingerprint for devices but instead noise that will decrease the tracking capabilities of this session identification approach. In general, any fingerprinting approach which identifies a logical source (e.g. source IP address) or a physical device as done in Kohno et al. suffers in deployments which obfuscate the source. This becomes particularly problematic in session identification when trying to link up TCP packets belonging to the same user. Consider fingerprinting a device as proposed in Kohno et al. which requires the TCP timestamp value to be passed unchanged through a middle box. Many middle boxes deployed in cellular networks act as “proxies” by splitting the TCP connection between the device and the origin server into two separate TCP connections which cause the origin server to negotiate TCP timestamp options with the middle box instead of the device [28]. This causes the middle box to be mistaken for the real source. This is not a problem when TCP timestamps are used for session identification as proposed in this study. The ability to link up TCP packets belonging to the same user during training will not be impacted whether tapping of communication traffic occurs between device and middle box or middle box and origin server. This is because this approach does not track the source but connections and in this case both connections are surrogates for the single end to end connection between device and origin server. We are not aware of other work that leveraged TCP timestamps to track communication sessions as was done for this study.

Training of HTMs is completely unsupervised and leverages the tracking strength of the TS value to identify consecutive web visits as belonging to the same user session (observation). This is different from the supervised training approach used by Yang in her experiments where a label (user-id) was used to train her inference model. During training, in this study, session identification and user identification are one and the same. During the inference stage the assumption that a specific session belongs to a given user no longer holds and instead the TS value is only used to identify an anonymous session (a set of consecutive web visits belonging to an unknown user that make up an observation). The task of assigning an anonymous session to a specific user is carried out by the Markov chains performing inference within the different layers of each HTM based on past learned patterns of users’ sessions (web visits). All HTMs attempt to recognize each anonymous session and only one HTM will be able to recognize it better than the other HTMs based on its past training.

Beacken et al. [6] have discovered that the TCP Timestamp field used for iPhones always starts at the same date/value when the device is restarted but for android devices, the TCP timestamp value on device power up is random. They state that this allows one to be able to distinguish iPhones from android type devices. In this study, the TCP time stamp value, a 32 bit value, which implements a virtual clock on each device, is used to uniquely identify unique sessions associated with a given user. The prototype built for this study identifies multiple communication sessions during the training phase of learning that belong to different users by tracking the unique TS value (TS) of each device. During training, all communication input associated with a given ⟨ TS ⟩ value within a given time window is fed to a hierarchical temporal memory (HTM) to identify the communication patterns associated with sessions belonging to different users. These communication patterns are defined in terms of the destinations (Dest, a number mapping to the IP address of the web site) visited by this user. The timestamp, calculated from the input at HTM layer 1, has a resolution of 1 millisecond and represents the passage of time with respect to the arrival of input to the HTM. The time stamp is specifically needed to distinguish multiple ⟨ TS , Dest ⟩ input pairs immediately following each other with potentially the same TCP time stamp values, as either all arriving at the same time or at different times.

The algorithm in Fig. 5 was used to implement communication session identification during the training phase of classification and selection of appropriate HTMs to perform communication pattern identification.

Each HTM Ux once created runs a virtual clock with a 1 ms resolution used to track the TS value of sessions associated with this HTM. The allowed TS clock window was computed as follows: Allowed-TS-Clock-Window = [ TS v + Clock ( ) ] ± Clock-Skew-Factor . The computation TS v + Clock ( ) needs to account for wrap around at 2 32 . The Clock-Skew-Factor is a fixed maximum allowed clock skew.

Unfortunately, using a fixed window offset from the currently received TCP TS counter to measure clock skew, can potentially either underestimate (lose a single tracked user session) the clock skew with a window that is too small or overestimate (identify a single user session as belonging to multiple user sessions) the clock skew with a window that is too large. A possible way to address this problem is to allow for dynamic resynchronization of the HTM TS counter with a tracked source based on how much of an offset (within a window) a given new received TCP timestamp is from the existing HTM TS counter. This approach would use the new TCP time stamp received as the new TS counter value each time the new TS value is within the window but does not match exactly the current HTM TS counter. This could address the potential increase in clock skew that occurs over time overcoming the limitations of a fixed HTM TS counter. With this newly proposed approach, it will be possible to use a small window size since the algorithm is able to adjust to clock skew over time. The benefit of this approach, as well as determining the best size for the clock skew window, is an area of further research that should be based on the empirical results of studying the characteristics of clock skew of mobile devices in real mobile networks.

OPEN IN VIEWER

Fig. 5.

TCP timestamp session identification algorithm.

Each HTM learns and then performs inference. Learning occurs in an unsupervised manner, starting from the bottom layer of the HTM, one layer at a time. Layer 1 learns first. After that, layer 2 learns and once layer 2 is done learning layer 3 completes learning. During training a new HTM is created for each user each time a not seen before user session (based on TCP timestamp tracking) is encountered. Learning entails both spatial and temporal learning. Spatial learning at layer 1 covers identification of individual sequences of web destinations, while at layer 2 and 3 it covers identification of individual sequences of coincidences (temporal groups representing Markov chains matched from the layer below).

Initial learning is completed at each HTM layer with creation of a single Markov graph representing all learned sequences within that HTM layer. At the end of training this single graph is split into many variable order Markov chains by merging all nodes that are most highly connected into one of several Markov chains based on a depth first traversal of the Markov graph (see Fig. 6) thus ensuring that sequences are maintained and not broken up and that sequences held in Markov chains do not overlap.

OPEN IN VIEWER

Fig. 6.

Algorithm to create Markov chains from a single Markov graph.

These Markov chains represent destinations or coincidences (temporal groups) that are highly temporally correlated based the specific temporal order in which they follow each other.

After learning is completed at a given HTM layer, playback occurs. During playback, each HTM at layer L n which completed learning is used to bootstrap learning for the layer above L n + 1 using the already learned sequences at layer L n . Playback (an approach introduced in this study) improves the time it takes to train the HTM and allows higher layers to learn higher level concepts that are consistent with the lower level concepts learned by the layers below. In the playback stage, learned sequences at layer n are generated in increasing order of time, so that layer n + 1 can correctly learn higher level concepts from the layer below. This in effect simulates the HTM been retrained on the same input used to train the layer below. In order to generate sequences in increasing time order (from oldest to most recent), each node in the Markov graph holds a FIFO queue of timestamps. Each time stamp represents the time when a node was created or modified by updating or adding incoming or outgoing links to/from this node. Time ordered sequence generation is achieved by traversing the Markov graph at each layer of the HTM, starting from the “start” state, while removing from the front of the FIFO queues timestamps associated with nodes with the least recent (oldest) timestamp for each transition up to the “final” state.

During inference the HTM layer collects input until a sequence is formed. The spatial and temporal poolers at each layer of the HTM ensure that sequences are created so that nodes that follow each other in space (sequential order of inputs) and time (timely order of inputs) are grouped together creating a sequence matched against learned sequences of coincidences (stored within variable order Markov chains g i ). The HTM spatial and temporal poolers terminate a sequence and start another under one of the following terminating conditions: (TC₁) A fixed maximum input size has been processed. (TC₂) A maximum learned inter destinations arrival rate is exceeded. (TC₃) The same destination is already present in the sequence (HTM version 1). Note that only HTM layer 1 uses terminating condition TC₂.

Each HTM layer matches the input sequence collected against learned sequences held in Markov chains and finds the best (longest) matching learned sequence (LLS). Each HTM layer computes the feed forward belief (λ) of the best matching learned sequence (LLS). This feed forward belief combines the degree of membership (how well input matches learned sequences) and persistence (how often a matched learned sequence is visited). Belief propagation occurs when HTM layer n passes as input the feed forward belief (λ) to HTM layer n + 1 . Belief propagation for casual (Bayesian) networks was first proposed by Judea Pearl [36] and then adapted to HTMs by Deleep George [19].

The output that is sent to the Max Output Layer (see Fig. 8) from each HTM includes identification of the specific HTM and provides the feed forward belief of the matched observation input across all layers of that HTM. The Max Output Layer aggregates the feed forward beliefs ( λ Output ) of up to one observation worth of data (50 web sites) from each HTM using one of seven HTM algorithms and then selects the HTM (user) with the maximum aggregated feed forward belief value among all HTMs as the one that best matches the HTM layer 1 input.

HTMs use one of seven algorithms proposed in this study (detailed in Section 3.5) to aggregate feed forward beliefs and to determine how well an observation matches HTMs’ learned input. Each one of the seven different HTM algorithms (based on the HTM layer where they are applied 1 or 3) combines feed forward beliefs associated with a given input observation based on one of three generic algorithms: average, weighted sum and path probability. The average based algorithm simply computes the averages of feed forward beliefs associated with a given observation. Weighted sum algorithms (BottomUp and TopTop) use weights proportional to the size of an observation matched by a given feed forward belief. The BottomUp algorithm aggregates the feed forward belief with weights proportional to an observation at layer 1 so that the degree of membership calculation at higher layers will be impacted by this weight as the feed forward belief travels up the HTM layers (see Table 1). The TopTop algorithm aggregates the feed forward belief with weights proportional to an observation matched at layer 3. The Path Probability algorithm leverages the idea of independence among feed forward beliefs and simply multiplies together the feed forward beliefs belonging to a given observation.

OPEN IN VIEWER

Fig. 7.

Hierarchical temporal memory layers.

As shown in Fig. 7 the key computation performed by HTMs creates groups of feed forward beliefs ( λ ⟨ FFBs ⟩ ) that match the input presented to each HTM layer. The idea is to find the longest learned sequence (LLS) out of all the learned sequences in all of the Markov chains in a given HTM layer which best matches (has the highest degree of membership ( DM ) as shown in Eq. (8)) the input sequence ( IS ). The calculation of DM and persistence ( SP Eq. (7)) of the matched LLS represents how well inferred input ( IS ) is matched against learned input. In terms of notation: (Abbreviation) Function name( Arg 1 , … , Arg n ) represents a function “Function name” which takes n arguments Arg 1 , … , Arg n and is referenced using the abbreviated name “Abbreviation”. The rest of this section presents the key calculations necessary to compute feed forward beliefs (Eqs (9) and (10)) that are calculated within each HTM layer before being propagated up to next layer. Refer to Appendix I for specific explanations of abbreviations used in this section. ( ALLLS_Cond ) Adjusted Length LLS Condition ( IS , cLLS ) = if IS is a substring of cLLS and IS 1 = cLLS 1 and | cLLS | > | IS | , ( ALLLS ) Adjusted Length LLS ( IS , cLLS ) = | IS | , ALLLS _ Cond , max ( | IS | , | cLLS | ) , otherwise , (1) ( LCS m ) Longest Common Subsequence Measure ( IS , cLLS ) = | LCS ( IS , cLLS ) | ALLLS ( IS , cLLS ) , (2) ( LCSU m ) Longest Common Substring Measure ( IS , cLLS ) = | LCS u ( IS , cLLS ) | ALLLS ( IS , cLLS ) .

See Appendix C for an example of how to use Eqs (1) and (2). ( Fq ) Frequency of visits ( cLLS i ) = number of times cLLS i occurs in Markov chain g i (3) within this HTM layer across all Markov chains in that HTM layer , NLS = number of all learned sequences within a given HTM layer , (4) ( Ps ) Persistence ( cLLS ) = Fq ( cLLS ) NLS , Similarity Weights = Ws + Wu + Wp = 1.0 , (5) where in this study Ws = 0.495 , Wu = 0.495 , Wp = 0.01 , ( SS ) Sequence Similarity ( IS , cLLS ) (6) = ( LCS m ( IS , cLLS ) × Ws ) + ( LCSU m ( IS , cLLS ) × Wu ) , (7) ( SP ) Sequence Persistence ( cLLS ) = ( Ps ( cLLS ) × Wp ) , (8) ( DM ) Degree of Membership ( IS , cLLS ) (8a) = max LLS ( SS ( IS , cLLS ) ) when | ∀ s , SS ( IS , cLLS ) > SS ( IS , s ) | = = 1 , (8b) = max LLS ( SS ( IS , cLLS ) , SP ( cLLS ) ) when | ∀ s , SS ( IS , cLLS ) > SS ( IS , s ) | > 1 , where s = all learned sequences, including cLLS , in a given HTM layer .

The best matching LLS is that candidate LLS (cLLS), in a given HTM layer, that has the maximum degree of membership value when measured against the input sequence ( IS ). For instance, assume IS = ⟨ S 1 , S 3 ⟩ which matches cLLS 1 ⟨ S 1 , S 2 , S 3 , S 4 ⟩ and cLLS 2 = ⟨ S 1 , S 3 , S 6 ⟩ in Fig. 14, then LLS which satisfies DM ( IS , cLLS ) is cLLS 2 since: LCS m ( IS , cLLS ) = LCS m ( IS , cLLS 2 ) = 2 / 2 , LCSU m = ( IS , cLLS ) = LCS u ( IS , cLLS 2 ) = 2 / 2 , LCS m ( IS , cLLS ) = LCS m ( IS , cLLS 1 ) = 2 / 4 , LCSU m = ( IS , cLLS ) = LCS u ( IS , cLLS 1 ) = 1 / 4 , SS ( IS , cLLS 1 ) = ( 0.5 × 0.495 ) + ( 0.25 × 0.495 ) , SS ( IS , cLLS 2 ) = ( 1 × 0.495 ) + ( 1 × 0.495 ) , SS ( IS , cLLS 2 ) > SS ( IS , cLLS 1 ) then based on (8a) max LLS ( SS ( IS , cLLS ) ) is cLLS 2 .

When more than one cLLS sequence exists with the same maximum sequence similarity values then the more persistent of these cLLS sequences is chosen. Consider a different example where IS = ⟨ S 1 , S 6 ⟩ and from Fig. 14 cLSS 1 = ⟨ S 1 , S 8 , S 6 ⟩ and cLSS 2 = ⟨ S 1 , S 3 , S 6 ⟩ , in this case Eq. (8b) is used since two sequences have the same sequence similarity values, namely | SS ( IS , cLLS 1 ) = SS ( IS , cLLS 2 ) | > 1 . In this case, LLS = ⟨ S 1 , S 3 , S 6 ⟩ since SP ( cLLS 2 ) = 2 / 16 > SP ( cLLS 1 ) = 1 / 16 . ( FFB ) Feed Forward Belief ( IS , LLS ) (9) = min ( 1.0 , DM ( IS , LLS ) + SP ( LLS ) ) where min is the minimum function. FFB ( IS , LLS ) is the feed forward belief used with all HTM algorithms except for path probability. The feed forward belief applied with the path probability HTM algorithm (see Section 3.5 for details) is shown below: ( FFB_PP ) Path Probability of LLS ( LLS ) = P ( LLS ) = P ( LLS 1 ) × P ( LLS 2 | LLS 1 ) × P ( LLS 3 | LLS 1 LLS 2 ) × ⋯ (10) × P ( LLS n | LLS 1 ⋯ LLS n − 1 ) , where: (11) P ( LLS 1 ) = Fq ( LLS 1 ) NLS , if LLS 1 = IS 1 , penalty = 0.0001 , otherwise , where penalty is an error probability for mismatches against IS . (12) Fq ( LSS j → k ) = frequency of visits across node transition j → k , P ( LLS k | LLS j ) = Fq ( LLS j → k ) Fq ( LLS j ) where k , j > 1 .

3.4. Feed forward belief propagation calculations

Feed forward beliefs propagate through HTM layers, as shown in Fig. 7, following the formulas shown in Table 1. Feed forward beliefs exiting the output (last) HTM layer will be directed to the Max HTM Output Layer, shared across all HTMs, which aggregates feed forward beliefs for an each observation worth of inputs (see Fig. 8).

The input activation level equation shown below measures the strength of a match between input and learned sequences and is further defined in Appendix I. ( IAL S g i … i + N ) Input Activation Level ( S g i … i + N , FFB g i ) (16) = min ( 1.0 , ∑ i = 1 | S g i … i + N | FFB g i | S g i … i + N | ) .

The seven HTM algorithms described in previous sections are implemented based on the three generic algorithms defined by equations 18, 19 and 20. The algorithms are deployed mainly in the Max Output Layer and also across the HTM layers for the Path Probability and BottomUp algorithms. Figure 8 shows λ Output which represents the feed forward belief output from layer 3 of an HTM to the Max Output Layer and S λ Output k … k + N which is a sequence of such feed forward beliefs. These parameters are used by the Max Output Layer to compute the highest valued feed forward belief for a given observation using the formula shown below. ( MAX_FFB_HTMs ) Max Feed Forward Belief for HTMs ( S λ Output k … k + N ) (17) = max HTM 1 … M ( HTM_Algorithm a ( S λ Output k … k + N ) ) , where a is one of 3 generic HTM algorithms (Average, Weighted Sum, Path Probability), 1 ⩽ M ⩽ number of HTMs/users learned during training, HTM 1 … M is the name of all HTMs learned during training.

OPEN IN VIEWER

Fig. 8.

HTM Max Output Layer inputs and output.

The three generic HTM algorithms which process feed forward beliefs within the Max HTM Output Layer are shown below: NB = | S λ Output k … k + N | is the number of beliefs per observation , (18) Average ( S λ Output k … k + N ) = ( ∑ k = 1 NB λ Output k ) / NB , (19) Weighted Sum ( S λ Output k … k + N , IS k , OS ) = ∑ k = 1 NB ( λ Output k × | IS k | OS ) , where IS k is HTM layer 1 input sequence associated with λ Output k and OS  = observation size (50 web destinations at HTM layer 1) (20) Path Probability ( S λ Output k … k + N ) = ∏ k = 1 NB λ Output k .

See Appendix E for an example on how the Max HTM Ouput Layer utilizes HTM algorithms. Table 2 provides more details about which HTM layer provides inputs to the Max output layer for specific HTM algorithms.

How to read Table 2: The Average and BottomUp HTM algorithms use the average generic HTM algorithm with inputs to Max HTM Output Layer from HTM layers 1 and 3 (see Fig. 8). The TopTop HTM algorithm uses the weighted sum algorithm with inputs exclusively from layer 3. Allowing HTM algorithms to generate inputs from either HTM layer 1 or 3 was done to enable verification of HTMs with and without HTM hierarchies. The BottomUp algorithm, takes the average of feed forward beliefs at the Max HTM Output Layer computed using the weighted sum at layer 1 (see Table 1, Eq. (15)). This was done to normalize differences in calculations due to large number of beliefs produced from layer 1 versus the few number of beliefs produced from layer 3 of HTMs.

Three hundred and thirty experiments (E₁) were conducted using synthetic data without simulating context drift which used 1000, 5000, and 10,000 visited web destinations. These experiments simulated 5, 20, 50, 100, 500 users accessing the network using 5 days’ worth of train data and either one or two days’ worth of test data or just 3 observations (one observation “Obs” equals 50 visited web destinations) worth of test data per user. For instance, Table 21 in Appendix H shows that for 5 users and 1000 visited web destinations, 11 experiments are executed using 5 train days and 1 test day (5/1), 11 experiments using 2 days’ worth of test data (5/2), 11 experiments using 3 observations (3 Obs) worth of test data for a total of 33 experiments. On the other hand for 500 users using 5000 destinations Table 21 shows 11 experiments being executed using 3 observations in the test data set.

All 132 concept drift experiments (E₂) shown in Table 22 in Appendix H involved visits to 1000 web destinations with 5 users, with 5, 10, 15, 20 training days and 2, 3, 4, 5 test days’ worth of test data respectively (see Appendix F for a description of how concept drift was simulated using a random walk algorithm). Concept drift is introduced in the form of 20% new destinations and 10% new transitions between destinations not in the original train data set. The first line of Table 22 represents the baseline (no concept drift).

In this study, users are identified based on their normal behavior and anomalous traffic is introduced as a form of noise to understand how much user identification accuracy is lost when noise (in the form of an attack) is introduced in the normal communication traffic patterns of users. The idea is not to identify the attack traffic but to identify normal traffic (tied to a specific user) in spite of the presence of embedded attack traffic (noise). The assumption is that when noise is introduced in the form of a phish or DOS attack from the device, it is due to the device having been compromised, possibly based on a download of an infected application. As the subscriber uses his mobile device to browse the Internet (normal behavior) the malicious app is at work in the background, launching its phish or DOS attacks. Table 23 in Appendix H shows twenty-one experiments (E₃) which were conducted with HTM algorithms by simulating denial of service attacks, embedded within synthetic network data, where the attack is initiated from individual devices during the test phase to a number of destinations (5, 10, 20) learned at train time. The destinations are attacked repeatedly over time (within a time interval of 5, 10, 20 ms and spaced by a fixed time interval of 5 ms). The idea is to determine how well the HTM can continue to identify users before and after the attack. In these experiments 10 users are used and 4 of them are assumed to be infected and to start DOS attacks during the test phase. The motivation for attacking destinations learned at train time is based on the assumption that perpetrators of DOS attacks typically target sites or services hosted on high-profile web servers such as on-line retailers, banks, credit card payment gateways which are likely to have been visited by the user, thus making the attack less likely to be detected (less conspicuous).

Table 24 in Appendix H shows twenty-one experiments which simulate phishing attacks (E₄), embedded within synthetic network data, which were run using HTM algorithms. For these experiments, attacks are initiated from individual devices during the test phase where unique destinations (1, 3, 5) are randomly selected from outside the user training data set (to simulate access to never visited before web phish sites) and attacked within a time interval (1 ms, 3 ms, 5 ms) spaced by random time intervals (1 min–1 h).

Ten scalability experiments (E₅) were also run using the two best performing HTM algorithms to measure the ability of the HTM to accurately identify users as the number of users increased from 150 to 500 users

4.2. Experiments using real network data

The next set of user identification experiments used real network data collected from a CDMA/LTE cellular data network in North America over a period of approximately a month. Experiments were conducted against HTMs using the following parameters: 5 and 10 users, 5 train days and 1 test day, 5 train days and 2 test days, 10 train days and 3 test days. The actual number of different web destinations visited by all users over the month was: 4903 destinations for 5 train days/1 test day, 5221 destinations for 5 train days/2 test days, 6672 destinations for 10 train days/3 test days.

The data originally collected from the network was for 50 users for a period of one month, unfortunately only 10 users used enough communication data to support the train and test timelines proposed for this study.

One hundred twenty six user identification experiments (E₇, E₈, E₉) were conducted using real network data running seven HTM algorithms. Table 25 in Appendix H shows the configuration for these experiments. For instance, 10 users leveraging 5 days of train data and 1 day worth of test data (5/1).

Twenty four user identification experiments were run using alternate Markov based algorithms (E₆). Forty two experiments (E₉) uncovered shortcomings in the HTM state machines when handling repetitive consecutive web destinations embedded in the real network input data set. The HTM was modified to address these shortcomings and the same 42 user identification experiments (E₈), using the same data set, were run to determine if the HTM accuracy could be improved. Finally, repetitive web destinations occurring at the exact same time (same timestamp) were removed from the input data set and the 42 same user identification experiments were run again (E₇).

Twenty one experiments which identify users in spite of simulated DOS attacks (E₁₀) and 21 experiments which identify users in spite of simulated Phish attacks (E₁₁) were also run using real network data (using parameters shown in Tables 23, 24).

The inability to collect TCP timestamps from the real cellular network limited session identification experiments to utilizing synthetically created TCP timestamps. It thus became necessary to conduct a set of experiments (E₁₂) to determine how noise introduced by different session identification algorithms impacts train data and the ensuing inference accuracy of HTMs. Session Identification experiments were performed by creating training data sets for the HTM that use one of three session identification algorithms: (1) Source IP, (2) Sliding Window, (3) TCP Timestamp. The train data set to be modified by the session identification algorithms uses real network data. The experiments include a preliminary step which runs the session identification algorithms against real network data to produce a new altered train data set that is modified based on the bias introduced by each session identification algorithm run under conditions that introduce noise. The experiment would then train the HTM with this altered train data set and use the original real network data as the test data set. Using the “Source IP” algorithm, all input with the same source IP address belongs to the same user. The “Sliding Time Window” algorithm selects the first (oldest) HTTP request in a time window based on the source IP address and assign it to user-x, then all subsequent HTTP requests within the time window for that source IP address, belong to the same user-x. As long as data is available for user-x within the window over time, then that session belongs to user-x otherwise that session is assigned to a new user (source IP address) selected at random based on users who have data falling within the sliding time window. Note that the sliding window approach presented in other related literature [4,46] only specified that requests occurring together in time belong to the same session. No other detail was given as to how a specific session was identified among others occurring at similar times. Thus, the use of the oldest source IP in a given time window as the seed for identifying a given user is proposed in this paper in support of this approach. The TCP Timestamp algorithm uses the TCP Timestamp values within a clock skew window to track different users.

A total of 18 session identification experiments were run where the source IP and TCP Timestamps leveraged real life scenarios to alter the original train data set. This was done by random simulation of recycling of the same source IP address among users as done by NATs and web proxy middle-boxes and by random simulation of re-attachment of a device with a new source IP as done when users move across networks. For TCP Timestamp, data loss was randomly simulated by creating holes in the data stream as well as random simulation of device power off/on. The number of users in these experiments is 5 and 10 for 5 train days, with the following additional experiment parameters: (Source IP): 10% recycle source IP address and 10% access network re-attaches; (Sliding Window): Sliding window size in seconds (1, 3, 5, 60); (TCP Timestamp): 10% data loss and 10% device power on/off.

In order to determine if it was possible to further improve HTM user attribution accuracy with real network data in the presence of real concept drift, continuous learning logic was added to the HTM. Continuous learning was implemented by allowing the output of the Max HTM Output Layer, which identifies which HTM x a given observation belongs to, to be sent back to layer 1 of that HTM x so that it can learn that observation. HTM x then during inference uses a mechanism similar to “playback” to learn the just received observation across all HTM layers. Two types of continuous learning were implemented: (1) Continuous Baseline which lets the Max HTM Output Layer send feedback to the correct HTM that matched the given observation. (2) Continuous Inference which lets the Max HTM Output Layer send feedback to the inferred HTM (which could be right or wrong) that matched the given observation. Twelve user identification experiments (E₁₃) with continuous learning were conducted for 5 and 10 users using the BottomUp approach run at HTM layer 3.

In order to determine if continuous learning could improve the HTM accuracy in spite of DOS or Phish attacks, 6 experiments (E₁₄) were run. These experiments were conducted using real network data for 10 users using 5 days’ worth of train data and 2 days’ worth of test data, using the BottomUp Layer 3 HTM algorithm and applying simulated DOS and Phish attacks configured with the same parameters previously described for these experiments in Tables 23, 24.

5.1. Results using synthetic data sets

Based on experiment set (E₁), alternate MC based algorithms with one day worth of test data never produced recall accuracy statistics above 42% (see Table 4 below) and when the test data set consisted of 3 observations, these algorithms never produced recall statistics over 66% (see Table 5). In contrast, HTM algorithms produced accuracy statistics (recall statistics) as high as 99% for a sample of 100 users with one day worth of test data as shown in Table 4 and 99% recall accuracy for a sample of 500 users with 3 observations worth of test data as shown in Table 5.

Why do Markov chains based algorithms perform so poorly in these experiments? Fig. 9 shows, the PPM statistics for the experiments run with 5 users with results shown in Table 5. The percentage of hits and misses were computed for all K orders across all users. The PPM algorithm starts at the highest K order ( K = 3 ) and each time the context (input) of size K of the input is not matched the algorithm scales down to a lower K order (matches a shorter portion of the input). Figure 9 shows that the PPM algorithm operates at K order = 0 about 80% of the time. This means that 80% of the time the PPM algorithm fails to match its input, applies a penalty to the path probability for the input and moves down to a lower K order Markov graph until it reaches K order = 0. This explains the poor performance of PPM and other higher K order algorithms (3rd Order MC, All-K) and also explains why higher order Markov chain algorithms have accuracy performance recall output values similar to lower order Markov chain algorithms.

OPEN IN VIEWER

Fig. 9.

PPM matches and miss matches per K order = 3 (E₁).

Accuracy reported by HTM algorithms scales better with increasing number of users and web destinations than the accuracy reported by alternate Markov chains based algorithms as shown in Table 6. Table 6 shows the difference in recall accuracy between experiments (E₁, Table 21, Appendix H) run for 5 and 100 users, with different number of visited web destination (1000, 5000, 10,000), over 5 train days and 1 test day using synthetic data. The high and low recall values of all HTM algorithms and alternate algorithms were recorded and the difference between accuracy values for 5 and 100 users was tabulated. Table 6 shows that HTM algorithms scale better (smaller differences) than alternate algorithms with a maximum of 13% loss in accuracy when tracking 1000 web destinations moving from 5 to 100 users compared to 41% loss in accuracy for alternate MC based algorithms running equivalent experiments. For 5000 and 10,000 web destinations, the scale factor for the HTM algorithm improves even more and is as low as 1% for high recall values.

To further understand how accuracy is specifically impacted by the number of destinations in the data set, it was observed that synthetic data accuracy performance increases substantially (E₁, Table 21, Appendix H) as number of destinations increases from 1000 to 5000 (from 54% recall accuracy to 99% for 500 users with 5 days of train data and 3 observations of test data), minimally from 5000 to 10,000. Synthetic data accuracy scalability measured for increasing number of users with 5000 web destinations visited (E₅) is high for the top 2 best performing HTM algorithms (BottomUp at layer 1 and TopTop) consistently at 99% recall accuracy for 150, 250 350, 450, 500 users (see Fig. 10).

OPEN IN VIEWER

Fig. 10.

Accuracy scalability synthetic data, 5 train and 1 test days (E₅).

The weighted sum based HTM algorithms (BottomUp at layers 1 and 3 and TopTop) outperformed all other HTM algorithms. To understand why, let us compare the average based algorithm to the weighted sum based algorithm. Consider an observation of size 5 (5 web sites visited) where the input received by the HTM is ⟨ 1 , 2 , 3 , 4 , 5 ⟩ which matches 100% of the learned input as 5 distinct nodes (5 sequences each of size 1). That is, for the average HTM algorithm FFB = ( 1 + 1 + 1 + 1 + 1 ) / 5 = 1.0 (100%). Now consider the calculation (as done for the BottomUp algorithm) for the average weighted sum proportional to the matched input FFB = [ ( 1 / 5 ) + ( 1 / 5 ) + ( 1 / 5 ) + ( 1 / 5 ) + ( 1 / 5 ) ] / 5 = 1 / 5 (20%). Now assume a new input ⟨ 1 , 2 , 3 , 4 , 5 ⟩ which matches 100% of the learned input as a single sequence of 5 nodes (1 sequence of size 5). In this case, using the average algorithm FFB = 1 / 1 = 1 (100%), while using the average weighted sum algorithm FFB = [ 5 / 5 ] / 1 = 1 (100%). The average algorithm produces the same recall accuracy whether an entire sequence is matched or only individual elements of the sequence are matched, while weighted sum based algorithms give more weight to longer sequences over shorter ones increasing the discriminating power of the solution.

In order to verify the ability of the HTM to handle noise, several experiments were conducted. Experiments (E₂) using synthetic data with simulated concept drift via “Random Walk” show a reduction in accuracy of up to 25%, indicating that the HTM is susceptible to random splitting of learned sequences. Concept drift which adds new connections at the end of learned sequences reduces accuracy only by up to 11%. Experiments which simulated DOS attacks (E₃) run across all HTM algorithms with synthetic data reduced accuracy by up to 11%, while experiments which simulated Phish attacks (E₄) reduced accuracy only by up to 5%.

Experiments were also conducted with real network data collected over a period of a month from a cellular data network. Train data sets ranged from 5 to 10 days and 1, 2, 3 test days. Results, at first were modest (E₉). For instance, for 5 users with 5 days’ worth of train data and 2 days’ worth of test data produced results with recall accuracy as high as 81% for HTM algorithms and 10% for Markov chains based algorithms. Visual observation of this data set showed a high recurrence of repeating continuous patterns of a single destination (e.g. 48, 48, 48, 48) within observations compared to similar measurements for synthetic data. This was confirmed by intra-observation repetitiveness (IOR) measurements which for some users were as high as 94%, indicating that within a user observation on average there were 47 repeating destinations out of 50. Calibration of real network data run against the HTM also showed poor performance with the best HTM algorithms (BottomUp and TopTop) scoring recall accuracy values (for experiments E₉ with 5 users, 5 train and 2 test days’ worth of data) ranging from 71% to 100%. Unexpectedly, alternate Markov chain algorithms in calibration tests with the same data set performed very well with the lowest recall accuracy value of 99%. It appears that alternate Markov chain based algorithms perform well when test and train data are very similar but when the data set differ as in the user identification experiments (E₆) then recall accuracy scores do not go above 10%.

The HTM was modified (version 2 HTM2++) in the implementation of the sequence termination condition (TC₃) of the HTM spatial and temporal poolers to account for continuous repeated destinations. The same set of experiments was repeated (E₇) but this time any repeated destinations that occurred at the exact same time were removed from the real network dataset. The reduction was applied to all train and test data files and accounted for a total reduction in repeated destinations of about 35%. The IOR values decreased and recall accuracy increased. For instance, with real network data for 10 users with 5 train days and 1 test day, IOR decreased by 7% and 9% for train and test data sets respectively while recall accuracy increased by 8% with the recall accuracy values shown in Table 7.

To determine if results using 10 users with real network data had statistical significance we took the experiment from Table 7 using real network data for 10 users for 5 train days and 1 test day. This experiment produced a “high” recall accuracy value of 87% (86.7). We repeated the same experiment 30 times, skipping the first 5, 10, 15, 20, …up to 150 destinations for each experiment. This mimics starting an experiment at a different place in the real data stream. The value of α chosen was 0.01, the research hypothesis was that the recall accuracy over these experiments was greater than 85.8%. The calculated sample standard deviation was 0.0084, the standard error mean was 0.0015, and the mean was 0.8625, while the z-score was 2.92. These results are significant at the 0.0018 level.

OPEN IN VIEWER

Fig. 11.

Accuracy comparisons of all HTM versions (E₇, E₈, E₉) including removal of same time destinations.

Why was HTM1 (the original version of the HTM) unable to recognize repetitive continuous patterns? HTM1 broke up repetitive patterns instead of treating them as sequences. So pattern, 1, 2, 3, 1, 2, 3, 1, 2, 3 was seen as pattern 1, 2, 3 occurring 3 times (which is good), but sequence 2, 2, 2, 2, 2, 2, 2, 2 was seen as a single destination 2 visited 8 times. This means that a user who seldom visits destination 2 and another who visits it in a sequence will produce analogous similarity statistics since for a single repeating continuous destination, HTM1 does not see a sequence of destinations but only a single element. HTM version 2 (HTM2) modifies terminating condition TC₃ to continue to process repeated destinations already in the sequence until a new destination not already in the sequence is encountered or terminating conditions TC₁ or TC₂ are met. In order to understand the recall accuracy improvements between experiments E₉, E₈, and E₇, Fig. 11 shows the results for user identification tests run with 5 users with 5 train and 2 test days’ worth of real data. HTM1 represents version 1 of the HTMs without the fix to address continuous repetitive patterns (E₉), HTM2 is the second version of the HTM which addresses repetitive continuous patterns but is run on the same real data set as HTM1 (E₈). HTM2++ is HTM2 run on the real data set where web destinations repeated at the exact same time are removed from the input (E₇). The baseline in Fig. 11 is based on running the experiments on the equivalent synthetic data set. The association of higher levels of IOR measurements with the inferior accuracy results was further investigated and experiments were run (beyond experiments reported in Table 3) with the same data set for 10 users (5 train days/1 test day), this time completely eliminating repeating continuous patterns of a single destination within observations in the input to determine if this would further positively impact accuracy results. While IOR continued to decrease (additional 8% for both train and test data sets), unexpectedly, accuracy never improved beyond 87%.

While investigating these results further, the real network data set was run through an algorithm that computed and thus measured concept drift by converting the concept drift generator for synthetic data to a concept drift detector for real network data. This concept drift detector identified changed user behavior in the test data set due to visits to new connections among existing nodes ( CD e ) and new connections to new nodes ( CD n ) as shown in Table 8.

The average CD e concept drift is 20% and CD n is 7% which is very similar to the simulated concept drift levels used for synthetic data. In order to determine the impact of concept drift on recall accuracy for real network data experiments, the HTM prototype was further modified to continuously learn during inference (E₁₃). Preliminary results, using only the BottomUp approach run at HTM layer 3, show that recall accuracy improves with the real network data for 5, 10 users for 5 train days and 1, 2, 3 test days. Specifically, recall accuracy for 5 users improved on average 4% and 2% for continuous baseline and continuous inference respectively and 6% and 1% respectively for 10 users. Table 9 shows details of these results.

Experiments with real network data which simulated DOS attacks (E₁₀) run across all HTM algorithms reduced accuracy by up to 8% while experiments which simulated Phish attacks (E₁₁) reduced accuracy only by up to 4%. Figure 12 shows the results of (E₁₄) using continuous learning to mitigate the user attribution effects under a DOS attack. The baseline, “normal” in Fig. 12, represents experiments run without continuous learning. The results show that when a subset of users (four out of ten) is under attack during the inference phase, it is possible to get improved attribution recognition accuracy (even over scenarios where no attacks are present) when the HTM algorithm can learn perfectly (continuous baseline). On the other hand, using inference to select which sequences to learn during continuous learning (continuous inference) produces mixed results. DOS attacks conducted during the inference phase produce decreased recognition accuracy performance when continuous inference learning is enabled possibly due to the fact that attacked sites were already learned by this user before the attack took place and do not create new distinctive patterns. Phish attacks instead produced better recognition accuracy performance possibly due to the fact that attacked sites were new and not learned until after the attack making it easier to infer them correctly.

OPEN IN VIEWER

Fig. 12.

Recall accuracy using continuous learning during DOS and Phish attacks (E₁₄).

The motivation for conducting session identification experiments is to determine how much different session identification approaches applied during HTM training impact the ability of the HTM to infer accurately (when used under conditions that emulate real life scenarios specific to each session identification algorithm).

Session identification experiment results showed that the sliding window session identification algorithm (which operates normally with some level of randomness) when measured against “perfect” tracking algorithms such as source IP address and TCP timestamp (that are exposed to simulated real life conditions which introduce noise in the data set) can outperform both of these algorithms. The average recall accuracy for all experiments across 5 and 10 users is shown in Fig. 13. To put things in perspective, a window size of 1 min, produced a change in the original train data set (loss of web destinations) of 57%, yet recall accuracy during inference is reported at 92% (Sliding Window worst result), compared to the best performing TCP timestamp (with 10% data loss and 10% device resets) which reports average recall accuracy value of 83%. These results warrant further study into the impact of noise on the session identification algorithms presented in this study.

OPEN IN VIEWER

Fig. 13.

Aggregate session identification recall accuracy results.

7.1. Limitations in implementation

The HTM prototype was completely written in Java. The performance scalability of HTMs measured in terms of run-time and space (memory needed at run-time) was a challenge in this study which limited the experiments to a maximum of 500 users. All experiments were executed on a Quad i7-3820QM 2.7–3.7 GHz with 16 GB RAM laptop. Threading (one thread per HTM) and caching (of already computed results derived by performing inference traversal of the Markov chains within each layer of the HTM) were two techniques that considerably improved the inference performance of the HTM allowing completing the experiments for 500 users in reasonable times. When first implemented threads improved run-time performance by almost 100% so that running the HTM algorithms for 2 users would take about 30 min to complete in single threaded mode but using multiple threads the experiment would complete in 16 min. By adding caching of results of Markov chain searches, performance dropped from 16 min to 6 min for the same set of experiments. Further optimizations in how threads were used (limiting the number of concurrent threads to 8) and other enhancements in the cache algorithms to maximize cache hits and minimize collisions resulted in run times of 16 min to 1 h and 15 min for 5 to 100 users using 5 days’ worth of synthetic train data and 1 day worth of test data and 7 min to 5 h for 5 to 500 users for 5000 destinations using 5 days’ worth of synthetic train data and 3 observations worth of test data. In contrast, the highest run time for alternate Markov chain based algorithms was 3 min for equivalent tests which involved 500 users. Alternate Markov chains algorithms have much better run times since discovery of the start of the input sequence is determined in constant time and matching of the sequence against the “context” occurs in time proportional to the size of the input since all alternate Markov chain algorithms are based on extensions of a 1st order Markov graph which matches the input completely based on the on the very first web destination in the sequence. HTM algorithms on the other hand have search run times that are proportional to the size of the entire input (all sequences) learned at training time as well as the size of the input sequence since HTMs perform an exhaustive search of all Markov chains at each HTM layer.

The amount of runtime memory needed by HTMs also proved to be a limiting factor in being able to extend experiments beyond 500 users. The HTM was run with a JVM setting of 14 GB of RAM but a limiting factor of the HTM design is the need for the Max HTM Output Layer (as shown in Fig. 8) to receive and hold one observation’s worth of feed forward beliefs from each HTM before being able to decide which HTM has the “best” feed forward belief. Increasing the number of users increases the number of HTMs which also increases the amount of RAM main memory needed to run the experiment. When the Java JVM starts to run out of the allocated RAM memory and starts to use hard drive virtual memory, run-time performance deteriorates dramatically eventually preventing forward progress.

7.2. Limitations in approach

There are a few limitations tied to the use of TCP timestamps as a communication session identification algorithm. In this study, the TCP timestamp session identification is used in two situations. The first occurs during training of an HTM when session identification is used to identify up to 50 observations as belonging to a given source associated with a specific HTM representing a given user. The second occurs during inference when session identification is used to create anonymous sessions for each observation which are distributed to all HTMs so that each HTM can perform inference on the observation to determine how well the observation matches learned input for each HTM. During the training phase the following scenarios are not allowed as they will corrupt the training data set:

A single user using multiple devices at training time would appear as many different sources and thus one HTM would be created for each source. This is a problem because each HTM created at train time identifies a different user when actually training is occurring for only one user. When many users share the same device at train time, the sessions produced are corrupted. These sessions will include data from multiple users and would thus not be representative of any given user. When a user recycles his/her device, the TCP timestamp is reset and this user session will appear as a new source and thus would incorrectly cause a new HTM to be created, thus identifying a new, non-existing user. During the inference phase when anonymous sessions are created only the second scenario is disallowed as the multiple users sharing the same device will create individual sessions with corrupted data belonging to multiple users. During the inference phase a user can use one device to train the HTM and a different device to perform inference. In addition, during inference, when anonymous sessions are created, the device under test can be recycled, as long as partial sessions (containing less than 50 observations) produced because recycling interrupts creation of the previous observation, are discarded. In general, the effect of a recycling device is the restart of a TCP IP connection which resets the TCP timestamp’s TS value. TCP connections are also reset when TCP connections go through middle boxes (Web proxies, NATs, Firewalls, Load Balancers) which split a single TCP connection from device to origin server into two TCP connections; one connection from the device to the middle box and the other from the middle box to the origin server.

If the TCP approach has the reported limitations why was this approach chosen for this study? The TCP timestamp approach provides a reliable way to “track” user communication sessions in a way that is independent of the location or network a device attaches to, thus enabling support for mobility. The authors believe that in real life scenarios, users recycle their mobile devices infrequently, and do not share their personal mobile devices with others. Another practical challenge with the use of TCP timestamp is due to the possibility of loss of tracking accuracy due to clock skew, approaches to address these challenges were presented in Section 3.1.1. A final limitation brought to bear in this study is found in experiments (E₁₄) where DOS attacks during user identification experiments are countered by using “continuous learning”. The results show consistent worse recall accuracy performance when using continuous inference learning. This leads to the observation that continuous learning can improve user identification recall accuracy in the presence of “concept drift” but when presented with repeated continuous patters found in typical DOS attacks, it can perform poorly.

7.3. Further research directions

An assumption made in this study is that users can be identified by learning web sites they visit. An interesting extension of this idea would be to study the user attribution problem in the context of peer-to-peer communication as is the case for messaging and voice calls. A key question would be: Can users be identified based on peer-to-peer patterns of communication? A related question would be: Is the power law at work in peer-to-peer communication scenarios as well? That is, do people tend to communicate (message or call) with a few set of users very often and with many other users infrequently? The security implications of extending the user attribution problem to cover peer-to-peer communication scenarios are especially relevant in the area of detection of spoofing of sources in peer-to-peer communication (e.g. recently RFC 7375 “Secure Telephone Identity Threat Model” has defined voice attacks where the calling party can be impersonated by an attacker).

As mobility will continue to dominate our future and as the Internet of people becomes the Internet of things, the user attribution problem will eventually morph into a device/user attribution problem. It would be interesting to run experiments that utilize all communication traffic originating from a device, not just HTTP traffic, to extend the attribution problem from a user to a source (device/user).

Due to the promising results achieved in this study it is important to extend experiments that seek to improve recall accuracy utilizing a larger real-network data set studying further the impact on user identification accuracy of techniques like continuous learning. The HTM framework will be made available to researchers who wish to extend this work by contacting the authors.