Contactless recognition technology of pedestrian lateral excitation based on dual perspective of mediapipe

Abstract

The remarkable lateral oscillatory occurrences observed in renowned footbridges, such as the Solferino Bridge in France, the Millennium Bridge in the UK, and the Oda Bridge in Japan, have garnered widespread attention. The intricate nature of pedestrian-induced lateral vibrations necessitates a comprehensive investigation into their underlying mechanisms, with the crux lying in the identification of the lateral excitation model. Presently, most prevailing detection approaches predominantly rely on contact-based equipment, such as force sensors and displacement sensors. However, these approaches suffer from two main drawbacks. Firstly, contact measurement necessitate a substantial number of sensors and entail higher experimental costs. Secondly, when extending contact-based measurements to accommodate multiple individuals, complications arise, including cumbersome installation, heightened technical complexity, and escalated experimental costs. Therefore, we present a novel approach, based on visual technology, to expeditiously recognize the lateral excitation induced by pedestrians. Termed the Dual Perspective of Mediapipe (DPM), this method harnesses the power of Mediapipe, coupled with dual camera models, to precisely unravel pedestrian gait particulars and lateral forces. The efficacy of the proposed approach is validated through a comparative analysis with existing pedestrian gait data. It is noteworthy that this approach offers significant advantages: non-contact with the test pedestrians, thereby ensuring the accuracy of gait information; convenient and straightforward arrangement of measurement equipment; and low experimental costs.

Keywords

footbridge pedestrian-induced vibration vision technology pedestrian lateral gait quick recognition

Introduction

Over the last thirty years, several footbridges worldwide have encountered large lateral vibration events, such as the Millennium Bridge in London, Toda Park Bridge in Japan and the Solferino Bridge in Paris. When the number of pedestrians on the footbridge reaches a critical threshold, even a slight increase in number of pedestrians triggers an abrupt oscillation of the bridge(Dallard et al., 2001; Fujino et al., 1993; Newland, 2003). Since then, extensive research has been conducted to investigate the pedestrian-induced lateral vibration of footbridge. Several intricate mechanisms underlying these events have been unveiled, encompassing synchronization, human-structure interaction, and crowd dynamics.

Over two decades of extensive research, although significant advancements have been made in this field, the underlying mechanism of pedestrian-induced lateral vibration remains unclear. Initially, it was believed that the synchronous resonance between pedestrians and footbridges was the main cause of pedestrian-induced vibration. Fujino and Siringoringo (2016) proposed a model for the lateral force of pedestrian, representing it as a simple harmonic force with a random phase. This model was applied to the Toda Bridge and successfully predicted the amplitude of footbridge vibrations, aligning well with actual observations. The experiments conducted by Arup on the Millennium Bridge revealed a direct proportionality between the lateral force exerted by pedestrians and the local velocity of the footbridge. Building upon this finding, Dallard et al. (2001) introduced the concept of equivalent damping coeffecient $C_{p}$ , which serves as an estimation for determining the critical number of pedestrians that cause lateral vibration instability. The above models are mainly based on the synchronization mechanism between pedestrians and structures, and are commonly referred to as synchronous models. As research continued, scholars began to focus on the interaction between pedestrians and footbridges. It was believed that the self-excited force induced by pedestrians can also cause vibration instability. Ingólfsson et al. (2011) carried out pedestrian experiments on a lateral vibrating treadmill and proposed a human-induced lateral force model that is correlated with the vibration frequency and amplitude of the structure. Based on the Barker’s biomechanical theory, Macdonald (2009) proposed the Inverted Pendulum Model (IPM). This model simplifies the human body as a mass supported by two massless rigid legs and emphasizes that the body balance is controlled by adjusting the foot position. Picozzi et al. (2022) analyzed the acceleration of a footbridge caused by a single pedestrian and proposed the design response spectra of footbridge, taking into account the variability in gait parameters between pedestrians and the uncertainty in the dynamic parameters of the footbridge. Yang et al. (2022) proposed a 3D spring-loaded IPM to analyze the interaction between pedestrians and vibrating surfaces, and also used PD control to adjust the balance of pedestrian, which showed strong robustness. Although several models have been proposed for the interaction between pedestrians and footbridges, these models still lack sufficiently accurate experimental test data. Therefore, further experiments are necessary to observe pedestrian gait behavior. At present, the primary techniques for acquiring pedestrian gait information involve either directly measurement through the placement of force sensors within the structure, or alternatively, affixing displacement sensors to pedestrians and subsequently performing inverse calculations of Newton’s second law. Besides, scholars utilized treadmills in walking experiments to simulate the walking process for extended durations, effectively reproducing the vibration environment of a footbridge. Carroll et al. (2014) employed a visual target system to measure the motion data of different body parts, subsequently analyzing and comparing the motion patterns between the IPM and the center of mass (CoM) of the subjects. The results showed that the IPM accurately simulated the displacement and velocity variations of subjects’ CoM. The research (Bocian et al., 2015) employed both VR technology and motion capture technology to examine the gait of pedestrians on a vibrating structure. And they find that pedestrians adapt their gait when exposed to both vibrating structures with or without VR simulation. They also observed a consistent evolution in the relative phase between individuals and the vibrating structure. Claff et al. (2017) carried out vibration simulation experiments on a footbridge using an infrared motion-tracking marker device and a pressure-sensing insole to evaluate the IPM and reveal the mechanism of bridge vibration instability.

The aforementioned experiment aims to acquire pedestrian gait through contact measurement. However, it is evident that certain limitations exist in this approach: Firstly, the direct sensor-to-body contact unavoidably influences the subjects and fails to fully replicate natural walking, thereby compromising data accuracy. Secondly, the extensive number of sensors result in high experimental costs and subsequent maintenance requirements. On the other hand, deep learning has made significant advancements currently, and convolutional neural network has a strong feature extraction ability, which made it widely used in human posture recognition through vision technology. Previously, Toshev and Szegedy (2014) integrated deep learning with human gesture recognition and proposed Deep Pose, which is a deep neural network (DNN)-based method for recognizing gestures that employs DNN regression to predict the coordinates of key points on the human body. Zhou and Zhang (2020) proposed a multi-model-guided regression algorithm, which integrates multivariate human body information including visual features, skeletal poses, audio signals, etc., and combines it with a multi-class SVM to recognize and predict human body action. Fu et al. (2020) proposed a two-step Kalman filter and a fuzzy logic-based adaptive tuning method, where the environmental information is used as an input for adaptive tuning by the sensors. This method ultimately captures the human posture and motion. Ma et al. (2020) proposed an improved GaitSet network for human gait recognition, which utilizes human silhouette and posture as key information to avoid the influence of clothing differences on recognition. Ding et al. (2020) proposed a human pose recognition algorithm based on multi-feature and rule learning. They defined a 219-dimensional feature vector containing angles and distances. They also used bagging and random subspace methods to classify human poses, achieving the pose recognition. Bourahmoune et al. (2022) proposed an intelligent sitting posture training scheme based on the LifeChair IoT cushion, which achieves real-time monitoring of sitting posture as well as highly accurate recognition. Xu et al. (2023) proposed a robust Abnormal Human-Posture recognition, and OpenPose is used for human skeletal data extraction. Van Hauwermeiren et al. (2020) used a static camera and a UAV camera to simultaneously observe the crowd trajectories on a footbridge and used color segmentation to detect them, in which the resulting detection accuracy reaches 2∼3 cm. While vision technology has demonstrated remarkable performance in human posture recognition, its application to pedestrian lateral force detection remains relatively underexplored. Therefore, we aim to bridge this gap by applying vision technology for pedestrian lateral force recognition, offering novel perspectives and methodologies. This paper proposes a novel visual recognition approach, namely Dual Perspective of Mediapipe (DPM), for precise identification of pedestrians' gait information and lateral forces. The approach is compared with experiments conducted by Carroll et al. (2013) to verify its effectiveness and carries out a preliminary application - validation of the IPM. DPM employed a vision-based, non-contact gait recognition technique that enhances accuracy in data collection and pedestrian walking restoration. This approach improves the quality of test data, thereby enabling us to further validate the IPM. Compared to traditional contact measurement, DPM also offers simplified equipment requirements, reduced experimental costs, and only utilizes two cameras as the key equipment. The cameras only need to meet basic recording specifications for successful identification. Besides, the setup of measurement equipment is simple and the utilization of neural networks ensures fast processing speed. Consequently, DPM exhibits remarkable attributes including high accuracy and cost-effectiveness and serves as a reliable research tool for recognizing pedestrian gait information and lateral forces under constrained conditions.

Dual-perspective visual recognition method

Convolutional neural networks (CNNs) are widely used in the field of visual recognition, and their powerful feature extraction ability has established them as a key tool in human pose recognition. For single person pose recognition, the most common approach involves inputting RGB images into a neural network to predict the positions of various joint points in the human body. Subsequently, the surrounding scene features are analyzed, and the corresponding joint points are connected. With the continuous development of single person pose recognition algorithms, their efficiency and accuracy have been consistently improving. A paradigm shift from traditional numerical coordinate-based methodologies to the innovative heat map estimation techniques has endowed single person pose recognition with invaluable utility across diverse scenarios and conditions.

In this study, we introduce a pioneering approach to visual recognition, employing a dual-perspective methodology built upon the lightweight convolutional neural network, Mediapipe(Lugaresi et al., 2019). Our method capitalizes on the swift and precise attributes inherent to Mediapipe in the realm of single-person pose recognition, while synergistically integrating the capabilities of the dual-perspective model for measuring detection points’ distances. It is noteworthy that the binocular camera exhibits superior accuracy and 3D reconstruction capabilities in comparison to its monocular counterpart. This facilitates a more precise capture of pedestrians' lateral gait characteristics, thereby enabling a more reliable and accurate pedestrian gait recognition process.

Visual recognition network of mediapipe

This study employed a lightweight convolutional neural network, namely Mediapipe, to accurately recognize posture of pedestrian. Mediapipe encompasses a suite of modules that integrate various models, including human pose recognition, gesture recognition, and facial recognition. For the purpose of human posture detection, Mediapipe leverages the BlazePose lightweight convolutional neural network. This network combines heat map and regression methods to accurately identify the coordinates of key points of the human posture. During the inference stage, BlazePose incorporates a detector-tracker architecture. The detector component identifies the presence of a complete human body in the current frame, while the tracker predicts the coordinates of key points. In cases where the tracker fails to predict the posture information for the next frame, it reruns the detector to recognize the human body and initialize the detector once again. Given that the human body exhibits a wide range of movements and certain movements may be obstructed by limbs, BlazePose enhances detection efficiency and accuracy by utilizing facial detection instead of limb detection. This choice is motivated by the fact that the face undergoes relatively smaller movement changes compared to the torso, enabling face detection to handle various complex environments and changes in human posture. However, it is important to note that this approach has a limitation: the prediction process relies on the visibility of the human face.

BlazePose’s framework is structured around the integration of heat map, offset, and regression methods, which are divided into two main components: the key point detection network and the key point regression network. During the training phase, a supervised lightweight embedding approach is employed, utilizing heat map and offset losses to refine the network’s output features. These refined features are then utilized by the regression networks. During the inference phase, the corresponding output layer is removed, resulting in a streamlined network structure. Through careful verification, it has been observed that this modified network architecture significantly accelerates the operation speed without compromising accuracy. The specific details of this network structure can be found in Figure 1b, which provides a visual representation of its components and their connections.

Figure 1.

(a)Human body’s 33 marker points identified by Mediapipe, (b)Framework of Blazepose.

In the estimation of human posture, Mediapipe demonstrates remarkable precision in identifying 33 key marker points distributed across the human body, as illustrated in Figure 1(a). These marker points encompass crucial information, including the coordinates and confidence levels associated with facial features, arms, trunk, legs, and other body parts. Thanks to its exceptional prediction speed and accuracy, Mediapipe is particularly well-suited for indoor single-person treadmill walking experiments. This visual recognition framework can be effectively utilized in the study of pedestrian-induced lateral vibrations on footbridges. By harnessing Mediapipe for efficient and high-precision recognition of human poses, in conjunction with the proposed algorithm for information extraction, the integration and extraction of pedestrian gait information can be achieved, offering a reliable solution within the constraints of limited research resources and conditions.

Camera model based on dual-perspectives

A dual-perspective camera setup is used to measure human posture, providing the following advantages: 1) The human posture information in the two 2D images is used to convert the actual recognition point displacement. This makes the displacement data sufficiently accurate. 2) The effect of lens distortion is reduced. During a person’s walking process, two cameras simultaneously record the motion in their respective viewpoints. Gait information, such as step frequency, step width, CoM displacement, velocity, and acceleration, is extracted based on the Center of Mass (CoM) and Center of Pressure (CoP) recognition point coordinates using Mediapipe. At the same time, the camera lens may have certain deviations in production accuracy and assembly process, resulting in stretching near the edge of the portrait. To reduce measurement errors caused by image stretching, two cameras should be aligned with the CoM and CoP so that they are directly imaged in the center.

The dual-perspective camera setup used in DPM is illustrated in Figure 2. The two cameras are positioned along the extension line of the treadmill’s central axis. The longitudinal plane, represented by the yellow plane in Figure 2, is perpendicular to the ground and passes through the central axis. Camera 1 is horizontally aligned with the position of the human’s CoM and provides the CoM perspective. Camera 2 is horizontally aligned with the position of the human’s foothold points, known as the CoP, and provides the CoP perspective. In this setup, the human’s CoM is captured at the image center of Camera 1, while the human’s CoP is captured at the image center of Camera 2.

Figure 2.

Schematic diagram of visual capture of human posture from dual perspective.

The horizontal distance $z$ from the identification point on the screen in each camera can be calculated by measuring the pixel difference. Through idealized geometric transformations, $z$ can be used to determine the actual position of the identification point (marker point) $a$ , resulting in the displacement value of the identification point.

As depicted in Figure 3, a specific dual-perspective camera model is presented in detail. In Figure 3(a), the main view of the dual-perspective camera model is shown along the x-axis direction. Two rectangular boxes represent the imaging frames of the upper and lower cameras. Figure 3(b) presents a cross-sectional view of the dual-perspective camera model along the longitudinal plane. In Figure 3(a), $a_{i}$ (i = 1, 2, 3) the specific marker point’s position, and $a_{0}$ , is positioned in the longitudinal plane. Points $a_{1}$ and $a_{2}$ are the locations where the marker point swings the most from side to side at other times. As the pedestrian walks, it can be approximated that this point oscillates cyclically along the y-axis between points $a_{1}$ and $a_{2}$ in the same vertical plane. As illustrated in Figure 3(b), $p_{u 0}$ and $p_{d 0}$ represent the imaging points of $a_{0}$ on the upper and lower cameras, respectively. Their coordinates expressed as ( $x_{u 0}$ , $y_{u 0}$ ) and ( $x_{d 0}$ , $y_{d 0}$ ). z₀ is the distance from $a_{0}$ to the line between cameras, while $f_{u}$ , $f_{d}$ , $c_{u}$ ,and $c_{u}$ denote the focal lengths and optical center positions of cameras. The optical center coordinates expressed as ( $c_{u x}$ , $c_{u y}$ ) and ( $c_{d x}$ , $c_{d y}$ ). Before the experiment, the camera focal lengths were adjusted to be consistent, i.e., $f_{u}$ = $f_{d}$ = $f$ . Based on the geometric relationship, we can derive the following equation:

\frac{L - (y_{d 0} - c_{d y}) - ({c_{u y} - y}_{u 0})}{z_{0} - f} = \frac{L}{z_{0}}

(1)

In many cases, the marker point’s position is not aligned with the longitudinal plane. For example, there may be an offset

x_{s}

between the lateral coordinate of the imaging point

p_{u}

and the optical center

c_{u}

of the camera, as illustrated in Figure 3(a). Assuming the coordinates of the imaging points

p_{u}

and

p_{d}

are represented by (

x_{u 1}

y_{u 1}

) and (

x_{d 1}

y_{d 1}

) respectively, thus:

{f_{u}}^{2} + {x_{s}}^{2} = {f_{u}}^{'}^{2}

(2)

\frac{L - (y_{d 1} - c_{d y}) - ({c_{u y} - y}_{u 1})}{z_{1} - {f_{u}}^{'}} = \frac{L}{z_{1}}

(3)

Figure 3.

Schematic diagram of dual camera model:(a) Main-view; (b) Cross-sectional view; (c) Offset of the imaging point; (d) Top-down view.

The lateral distance $z_{1}$ of the point $a_{1}$ , which is not on the longitudinal plane, can be calculated using equations (2) and (3). Additionally, $z_{2}$ can also be calculated in the same way. From the geometrical relationship, as shown in Figure 3(d), we can obtain further information.

{z_{1}}^{2} - {z_{0}}^{2} = {x_{1}}^{2}

(4)

Extraction of CoM displacement and gait information

The visual recognition network, implemented through the utilization of Mediapipe, demonstrates the capacity to discern and identify 33 pivotal landmarks within the human physique. However, it regrettably falls short in capturing the indispensable information pertaining to the CoM of pedestrian. Carroll’s seminal research illuminates the intrinsic mechanics of human locomotion, elucidating the preponderant role played by the Head, Torso, and Pelvis (HTP) in generating lateral forces(Carroll et al., 2013). Astonishingly, these HTP forces account for a substantial majority, encompassing 50.4% of the total body mass. This empirical evidence effectively corroborates the fundamental tenets of the simplified model theory, specifically the IPM, which posits the simulation of the human body as a mass entity, bolstered by a rigid limb devoid of mass. Carroll ascertained the location of CoM in the corporeal domain to reside superior to the pelvis through meticulous investigation. thereby substantiating the reliability of this theoretical framework. Drawing inspiration from Carroll’s pioneering research, this study similarly situates the center of mass position above the pelvis as the definitive location within the human physique. The process of ascertaining the CoM’s precise coordinates necessitates the utilization of specific identification points, namely 11, 12, 23, and 24, as illustrated in Figure 4. Notably, the midpoint “a,” derived from the intersection of identification points 11 and 12, as well as the midpoint “b,” formed by the points 23 and 24, serve as cardinal indicators of the torso’s central axis. The CoM’s spatial coordinates manifest themselves at a distance corresponding to 0.8 times the length of “ab” from the aforementioned central axis of the torso. Thus:

x_{C o M}^{i} = (\frac{x_{23}^{i} + x_{24}^{i}}{2} - \frac{x_{11}^{i} + x_{12}^{i}}{2}) \times (1 - 0.2) + \frac{x_{11}^{i} + x_{12}^{i}}{2}

(5)

y_{C o M}^{i} = (\frac{y_{11}^{i} + y_{12}^{i}}{2} - \frac{y_{23}^{i} + y_{24}^{i}}{2}) \times 0.2 + \frac{y_{11}^{i} + y_{12}^{i}}{2}

(6)

In the aforementioned equation,

x_{C o M}^{i}

、and

y_{C o M}^{i}

represent the horizontal and vertical coordinates of the CoM position points in the i-th frame of the image, while

x_{n}^{i}

and

y_{n}^{i}

(where n takes values 11, 12, 23, 24) denote the coordinates of the marker points in the ith frame image. By combining equations (5) and (6) with the measured actual distance using the dual-perspective model, it becomes possible to determine the actual displacement of the CoM. Subsequently, by differentiating this displacement, the velocity and acceleration can be obtained. Carroll’s research elucidates that the lateral force exerted by pedestrians primarily originates from the combination of the head, torso, and pelvis (HTP), which accounts for approximately 50.4% of the body weight. Therefore, based on the acceleration derived from the previous step, the lateral force exerted by pedestrians can be calculated by applying Newton’s second law, considering it to be 50.4% of the body weight.

Figure 4.

Schematic diagram of marker points of human’s Centre of Mass(CoM).

The position of the CoP is determined by identifying points 27 and 28 in the Figure 1. In the IPM, the CoP’s position is defined as the point where the massless rigid rod supporting the body makes contact with the ground. It is assumed that the CoP’s position remains fixed during each step’s support phase. Among the 33 marker points provided by Mediapipe, there are six points associated with the foot region, respectively points 27∼32. However, points 29∼32, which correspond to the heel and toes, tend to exhibit significant fluctuations in position due to the rotational movement of the ankle during walking. In other words, when visually recognizing adjacent frames, these points may deviate considerably from their actual positions. On the contrary, points 27 and 28, representing the ankle, are relatively more stable. Consequently, only points 27 and 28 are considered for extracting step frequency information and determining the CoP’s position.

When pedestrians walk, the coordinates of the CoP change over time. A step taken with the same foot can be divided into three periods: the pre-foot-lift period, step fall period, post-foot-lift period, and forward step period.

• Pre foot lift period: In this period, pedestrians lift one foot and move it forward until it touches the ground.

• Step fall period: This period represents the pedestrian’s contact with the ground, supporting the entire body. Meanwhile, the other foot enters the later stage of foot lift.

• Post foot lift period: After the step fall period, the foot that was supporting the body is lifted, preparing for the forward step.

• Forward step period: This period involves the forward movement of the foot that just landed, while the other foot enters the pre-foot-lift period.

This process alternates between the two feet, creating a steady forward gait for the human body. Throughout these four stages, the CoP’s coordinates during the descent period remain relatively stable. However, during the Pre foot lift Period, Step fall period, and Forward step period, the CoP is separated from the ground. In the Pre and Post foot lift period, the CoP reaches its highest position above the ground, resulting in a local peak in the vertical coordinate of the CoP ( $P_{y}$ ), known as the peak area of CoP.

Based on the changes in CoP’s coordinates described earlier, this study has developed a peak recognition algorithm. The purpose of this algorithm is to identify the CoP’s peak area for each step cycle during the entire walking duration of pedestrians. Additionally, it separates the forward step period, which exhibits certain fluctuations in CoP’s coordinates. The algorithm ultimately outputs the stable period of the CoP, which corresponds to the time point when the pedestrian takes each step. Furthermore, the study utilizes the actual distance exchange algorithm of the dual-perspective model to calculate the actual distance between the foothold point of each step and the vertical plane. This calculation provides the coordinates of the foothold’s points. The separation process involved in this algorithm is illustrated in Figure 5, which likely provides a visual representation of the steps and stages involved in the algorithm. The vertical coordinate timescale of CoP during the walk is denoted by $P_{y} (t)$ .

Figure 5.

Process of peak recognition algorithm.

Verification and testing

Experimental campaign

In this study, a single-person indoor walking experiment was conducted on a stationary treadmill. The primary objective of this experiment was to assess the reliability and accuracy of the DPM in capturing pedestrian gait information. To achieve this, we compared the gait data obtained through DPM with the experimental data(Carroll et al., 2013). By comparing the results obtained from DPM with Carroll’s experimental data, the scholars aimed to evaluate the performance and accuracy of DPM in capturing and analyzing pedestrian gait information.

In the experiment, a treadmill model SHUA-T5170P was used, which is depicted in Figure 6a. Like most treadmills, it allows for arbitrary adjustment of the treadmill track speed. To simulate pedestrian walking under real conditions as closely as possible, a rhythmic audio with a fixed frequency was recorded for each participant prior to the experiment. During the experiment, the participant wore a pair of headphones and gradually adjusted their stride frequency to match the rhythm audio being played. Simultaneously, the speed of the treadmill was adjusted until the participant’s stride frequency fully matched the rhythm audio. This process aimed to replicate the natural walking pattern of the participant. The experiment and simulation were carried out under two different lateral step frequency conditions: 0.85 Hz and 0.93 Hz. These specific frequencies were likely chosen to investigate the effects of different step frequencies on the reliability and accuracy of DPM in capturing pedestrian gait information.

Figure 6.

Indoor treadmill experiment: (a) The treadmill (SHUA-T5170P) used; (b) Schematic diagram of sensor placement for the treadmill; (c) Indoor single pedestrian experiment site.

The treadmill boasts a running bandwidth spanning 0.62 meters, complemented by a running belt measuring 1.45 meters in length, all while maintaining a slope setting of 0°. This generous expanse of running belt accommodates the ambulatory requirements of the test subjects, ensuring ample room for maneuver, even in scenarios necessitating adjustments to the step frequency during locomotion. Furthermore, a meticulously arranged array of 300 square film pressure sensors, each with a side length of 40 mm, was discreetly positioned beneath the treadmill track, as elegantly illustrated in Figure 6b. These sensors possess the remarkable ability to discern and quantify the dynamic fluctuations in applied pressure. Whenever a test subject graces one of these film sensors with their footfall, it instantaneously relays real-time feedback on the pressure signal to an off-site monitoring platform. In this way, we can monitor the foothold coordinates and the corresponding time of pedestrian locomotion. The precision of step width ascertained by DPM is validated through a comparative analysis with the data obtained from slender film sensors. The subjects enlisted for this experiment encompassed five individuals, distinguished by their varying height and masses. Comprehensive details are tabulated in Table 1.

Table 1.

Physical data of subject and experimental conditions.

Number of subjects	Height (m)	Weight (kg)	Speed of the treadmill (km/h)	Walking duration (s)
1	1.70	54	3.2	20
2	1.74	65	3.6	20
3	1.66	60	3.2	20
4	1.75	65	3.2	20
5	1.80	68	3.3	20

The experimental setup is illustrated in Figure 6c below. Following the dual-perspective camera model described in Chapter 1.2 and considering the features of Mediapipe, it is essential to position both cameras in a manner that captures the complete human body and face comprehensively. Camera 1 is mounted at a height of 1.2 m above the ground and horizontally aligned with the center of the human body. Camera 2 is positioned 0.2 m above the ground and horizontally aligned with the feet. The cameras operate at a frame rate of 30fps. The visual recognition process does not require significant CPU or GPU usage. In this study, we used a 12th Gen Intel(R) Core(TM) i5-12600 KF CPU and an NVIDIA GeForce RTX 2060 GPU, which can recognize up to 30 frames per second. After completing the recognition of the human gesture points, we imported the results into the gait extraction program, which was run under the Python 3.10 environment. The process of DPM recognition involves extracting gait information from a 2-minute pedestrian walking video in just 2 minutes. As shown in Table 2, the relevant equipment information and costs are listed. The identification data is derived after smoothing the CoM displacement trajectory data with 3 times mean filtering. This process averages small changes in each frame and is not affected by anomalous signals.

Table 2.

Information on experimental equipment.

Computer hardware	CPU: 12th Gen Intel(R) Core(TM) i5-12600 KF, GPU: NVIDIA GeForce RTX 2060
Recognition efficiency	Identified in 2 minutes (2-minute-long video)
Equipment for filming	720p resolution, 30fps frame rate
Required equipment	2 cameras with stabilizing brackets, mass storage devices
Main cost consumption of the experiment	Two pieces of qualifying filming equipment

We conducted a comprehensive validation of the recognition of human CoM motion and gait information by DPM. In Section 3.2, we compared the data obtained from sensors installed on the treadmill with the data derived from DPM. Specifically, we examined the temporal evolution of step points and step widths to assess the accuracy of DPM in identifying pedestrian step point and step width information. In Section 3.3, we compared the CoM lateral force determined by DPM with the human ground reaction force (GRF) measured by Carroll using visual target technology. This comparison was performed in both the time and frequency domains, providing further evidence to support the accuracy of the gait information obtained through the methodology employed in this study. Finally, in Section 4, we conducted numerical simulations of the IPM and compared the displacement, velocity, and acceleration time histories of the human centroid CoM between the two datasets. This analysis served to validate the rationality of the IPM.

Step width recognition and verification

One of the pedestrian gait parameters that can be recognized by DPM are the foothold points position and stride width of each step taken by the pedestrian. As discussed in Chapter 1.2, within the framework of the dual-perspective model, the distance between each step point and the longitudinal plane, represented by the centerline of the treadmill, can be determined. Prior to commencing the experiment, the participant adjusts their walking pace according to rhythmic audio cues, while the off-field personnel adapt the treadmill speed to match the participant’s stride frequency. Once the adjustments are finalized, the participant begins walking in a smooth and natural manner. Simultaneously, the two cameras commence capturing the position of the CoP of the subject’s body, while a thin film sensor located beneath the treadmill track records the position of each foothold made by the subject.

As depicted in Figure 7(a) presents the variations in step width and foothold points’ position, as measured by the DPM and the thin film sensor, respectively, for subject 1 at a step frequency of 0.85 Hz. Notably, the foothold points’ position obtained via the thin film sensor corresponds to the collective coordinates of multiple sensors, each encompassing the area where a step is taken. These coordinates represent the geometric center of the pressure-bearing sensors. Consequently, we can reasonably approximate the coordinate position of the pressure sensor as the actual location of each subject’s step. From Figure 7(a) and 7(b), it is evident that the foothold points’ coordinates derived from the DPM exhibit a remarkable concordance with the data directly measured by the thin film sensor. The DPM proves highly accurate in capturing the position of the CoP, as well as documenting the timing and location of each ground contact event. Figure 7(c) and 7(d) illustrates the comparative analysis of temporal variations in step width. Regarding step width, the trend observed in the DPM’s data aligns closely with the measurements obtained from the sensor. The relative error between the visual measurements of the two subjects and the sensor data falls within the range of 0 to 19%, signifying that the step width values calculated via the DPM closely approximate the actual values.

Figure 7.

(a) Comparison of foothold points of Subject 1 (step frequency of 0.85 Hz) (a) Comparison of foothold points of Subject 2 (step frequency of 0.93 Hz) (c) Comparison of step width of Subject 1 (step frequency of 0.85 Hz); (d) Comparison of step width of Subject 2 (step frequency of 0.93 Hz).

Identification and verification of lateral force

In this section, we aim to compare the subject’s lateral forces obtained through the DPM with experiment data (Carroll et al., 2013). The comparative analysis will be conducted in both the time and frequency domains. By employing the principle of inverse calculation based on Newton’s second law, DPM has the capability to discern the lateral forces exerted by pedestrians. During the act of walking, each segment of the human body exhibits a lateral motion pattern. By measuring the acceleration of each limb segment and multiplying it by the corresponding mass, the lateral force of each limb segment can be calculated. These individual forces can then be accumulated to determine the overall lateral force exerted by the subject. This study categorizes the limb combinations that primarily contribute to lateral excitation into the following five groups: 1) Head, Torso, Pelvic (HTP), 2) Left upper arm and right upper arm, 3) Left forearm and right forearm, 4) Left thigh and right thigh, and 5) Left calf and right calf. Referring to the adult male body parameters specified in GB/T 17245-2004, we adopt the relative mass of human limb segments illustrated in Table 3 below. The lateral forces were calculated by accumulating 5 combinations of limb segments from equation (7).

F_{L a t} = \sum_{i = 1}^{5} m_{i} {\ddot{x}}_{i}

(7)

Table 3.

Relative mass of human limb segments.

Limb segment	Relative mass (%)
Head, Torso, Pelvis(HTP)	50.0
Left upper arm and right upper arm	2.43
Left forearm and right forearm	1.25
Left thigh and right thigh	14.19
Left calf and right calf.	3.67

To capture the lateral forces exerted during pedestrian locomotion, Carroll introduced a visual target technology employing inverse dynamics analysis to comprehensively examine the intricate kinematic patterns exhibited by pedestrians. In his study, Carroll devised a modified treadmill capable of lateral vibrations, effectively emulating the walking conditions experienced on a vibrating bridge. Through this innovative approach, the proposed methodology successfully reproduced the lateral force, commonly referred to as the ground reaction force (GRF), during walking across various amplitudes and frequencies.

Carroll’s experiment encompasses the acquisition of subject lateral force from two distinct perspectives. Firstly, the load imposed is directly obtained by 4 accelerometers installed on the treadmill. The resulting ground reaction force (GRF) derived from this approach is referred to as the direct GRF and is denoted as $F_{m e a n s}$ . This method of measuring the lateral force demonstrates a relatively high level of objectivity and accuracy, thereby serving as a benchmark for comparison. The calculation for the direct GRF is provided by equation (8):

F_{m e a n s} = (\sum_{i = 1}^{4} F_{c, i}) - m_{d} {\ddot{x}}_{d}

(8)

F_{c, i}

is the load imposed on the ith of the 4 load cells.

m_{d}

is the mass of the treadmill deck and associated suspended structure and

{\ddot{x}}_{d}

is the lateral acceleration of the treadmill deck.

In addition to the aforementioned approach, Carroll employs visual targeting technology and sensors positioned on different anatomical regions of the human body to capture the acceleration of various limb segments. Subsequently, the corresponding forces are calculated utilizing Newton’s second law of motion. This reproduced ground reaction force is denoted as the reproduced GRF( $F_{r e p}$ ). The calculation formula for $F_{r e p}$ is as follows:

F_{r e p} = \sum_{i = 1}^{15} m_{s, i} {\ddot{x}}_{s, i}

(9)

Where

m_{s, i}

and

{\ddot{x}}_{s, i}

are the mass and CoM (frontal plane) acceleration of the ith of 15 body segments described previously.

Carroll conducted experiments on a non-vibrating treadmill and compared the forces obtained through two distinct measurement methods. The results revealed a substantial level of agreement between the two methods, both in the time and frequency domains. This finding underscores the reliability and accuracy of the visual targeting technology employed to acquire pedestrian lateral forces. However, due to the limited number of experiments conducted by Carroll on a stationary treadmill, only the time history information and corresponding power spectral density of the lateral force were obtained. Consequently, this study further conducted a comparative analysis based on these two aspects of data to effectively showcase the capability of visual technology in discerning individual lateral forces. This research endeavor establishes a technical foundation for future investigations to extend beyond individual experiments and encompass crowd experiments.

Figure 8(a) illustrates the lateral acceleration of each limb of subject 2 as identified through the DPM. Conversely, Figure 8(b) showcases the lateral forces exerted by various combinations of body segments of subject 2, also identified using DPM, within the 5∼10 second timeframe. It is evident that the predominant source of pedestrian lateral forces stems from the HTP combination, accounting for over 80% of the total lateral forces. This outcome aligns with the findings of Carroll’s experiment. Despite the legs and arms contributing to a portion of the overall body mass, employing the HTP combined motion as a representation of the entire human body during walking is deemed reasonable. This result supports the simplified theory of the IPM, while also indicating the feasibility of utilizing a single-point measurement method (such as measuring the CoM acceleration) for conducting crowd experiments.

Figure 8.

(a)Lateral acceleration of each limb segment of subject 2 obtained by DPM (b) Lateral force of each limb segment of subject 2 recognized by DPM.

Figure 9(a) depicts the variation curve of subject acceleration between the DPM and Carroll’s measurements within the 0∼10 second time interval. Upon comparing the two datasets, it was observed that the lateral force obtained by Carroll exhibited a distribution approximately around 5% of the body weight, with a peak value slightly below 10%. Conversely, the lateral force identified by DPM predominantly exhibited a distribution around 4.5%, with a peak value of approximately 8%. When considering other subjects within DPM and Carroll, the data exhibited a significant level of agreement within this magnitude range. Although slight variations in the shape and numerical values of the curve changes were observed, these deviations can be attributed to differences in the subjects' individual body data and walking habits.

Figure 9.

Comparison between the data of DPM and Carroll’s experiment: (a) Comparison between recognition data of DPM and Carroll’s experiment (b) Discrete harmonics of lateral forces from Carroll’s experiment; (c)Discrete harmonics of lateral forces obtained by DPM.

In terms of the frequency domain, as illustrated in Figure 9(b) and 9(c), the distribution of the two main frequencies in DPM exhibited a general similarity to Carroll’s direct Ground Reaction Force (GRF) data at lower frequencies. The amplitude at the primary frequency of 0.93 Hz was slightly lower compared to Carroll’s data, and the amplitude of harmonic waves within the frequency band around 3 Hz was slightly smaller than the data obtained in Carroll’s experiment. DPM did not detect the high frequency components (above 4 Hz) of lateral force, which is related to the frame rate of the camera equipment used in this experiment. For higher frequency components, it is possible to capture them using a camera device with a higher frame rate (e.g. 60 fps). As shown in Figure 10, the detection results of the DPM at 30fps and 60fps are compared. Under the same experimental conditions, DPM is able to detect the third frequency component of pedestrian’s lateral force at 60fps. The scholar can select filming equipment with an appropriate frame rate to capture the lateral force of pedestrian based on their research needs. Typically, high frequency components above 4 Hz have low amplitudes and are difficult to identify, while the first two orders of components at low frequencies are adequate for most pedestrian lateral force studies.

Figure 10.

Recognition results of DPM at different frame rates: (a) 30fps (b) 60fps.

Based on the aforementioned verification, DPM has demonstrated effectiveness in recognizing pedestrian gait on stationary structures. In contrast to conventional contact measurement approaches, DPM leverages visual technology to acquire precise pedestrian gait information which minimizes the disturbances to pedestrians, thereby it ensures that the gait data is accurate and reliable. Furthermore, this method can be employed for further investigations, such as recognizing the gait of a crowd and deriving crowd loading models. The subsequent section presents a preliminary application of DPM - validation of the IPM.

Application of DPM: Verification of IPM

The pedestrian-induced lateral vibration of footbridge constitutes a complex nonlinear problem involving biomechanics. Among various models proposed, the IPM has gained relatively high recognition. However, there still exists some uncertainties, as well as some doubts persisting in synchronous model rather than the IPM being more suitable for explaining pedestrian-induced lateral vibrations. The IPM is not fully accepted by the public. Therefore, more experimental data are needed to support the validity of IPM. Moreover, most current experiments validating the IPM rely on contact gait measurements which tend to cause interference during experimentation, thereby compromising the credibility of test data. In view of this, this section validates the IPM using the DPM, which can supplement the current validation basis. The experimental comparisons in Section 2 demonstrate the reliable and excellent performance of the DPM in pedestrian gait recognition. The notable advantage of the proposed method lies in its non-contact measurement capability, enabling pedestrians to maintain a natural walking state while accurately representing their walking conditions and ensuring data accuracy.

The IPM, initially proposed by Macdonald (2008), incorporates various biomechanical theories and serves as a simulation framework for representing pedestrians as inverted pendulums. In this model, CoM of the human body is sustained by two massless rigid legs, while the point of contact between the rigid legs and the ground is referred to as the CoP, as illustrated in Figure 11. The IPM primarily focuses on the frontal surface motion of pedestrians, which is perpendicular to their forward direction. During each step, the CoP remains fixed, and the motion of the CoM is derived through force balance considerations.

Figure 11.

Inverted pendulum model (IPM) detail.

As shown in Figure 11, when the step is supported by the right foot, there is the following force balance equation:

- m L^{2} \ddot{θ} = m g L \cos θ + m \ddot{x} L \sin θ

(10)

In the equation,

m

represents the mass of pedestrians; L denotes the distance from CoP to CoM;

g

represents the acceleration of gravity,

x

represents the absolute horizontal displacement of the structure,

θ

represents the angle between the legs and the horizontal plane. In the Figure 11,

F

represents the lateral force and

u

represents the distance from CoP to the midline of the structure. Considering that the angle between the pedestrian’s legs and the horizontal plane is typically close to 90° in most scenarios, equation (10) can be simplified as follows:

\ddot{y} + Ω_{p}^{2} (u - y) = - \ddot{x}

(11)

Where

Ω_{p} = \sqrt{g / L}

In the absence of horizontal structural vibrations, the displacement solution for the CoM as presented in equation (11) is as follows:

y = u + (y_{0} - u) \cosh [Ω_{p} (t - t_{0})] + \frac{v_{0}}{Ω_{p}} \sinh [Ω_{p} (t - t_{0})]

(12)

t

is the walking time,

t_{0}

、

y_{0}

、

v_{0}

represents the corresponding time, horizontal position, and CoM speed at the beginning of each step.

Assuming a fixed stride cycle with a smooth transition from one foot to the other, and considering a lateral stride frequency denoted as $f_{p}$ , when $t = t_{0} + 1 / 2 f_{p}$ , the corresponding velocity of the CoM is $- v_{0}$ . At the beginning of each step, the initial displacement $y_{0}$ of the CoM is zero. Consequently, the displacement y and the lateral force F of the CoM can be obtained using the following equation:

y = u {1 - \cosh [Ω_{p} (t - t_{0})] + \tanh (Ω_{p} / 4 f_{p}) \sinh [Ω_{p} (t - t_{0})]}

(13)

F = m u Ω_{p}^{2} {\cosh [Ω_{p} (t - t_{0})] - \tanh (Ω_{p} / 4 f_{p}) \sinh [Ω_{p} (t - t_{0})]}

(14)

Where

u = \pm δ / 2

, δ is the width of each step, when the first step starts from the right foot, t = 0,

t_{0} = n / 2 f_{p}

(

n

= 0, 1, 2 ... is an integer).

Verification of IPM based on DPM

The initial set of experimental data was obtained from subject 1, and the comparison was conducted based on the principles of the IPM. In the absence of structural vibrations, we select a set of parameters closest to those of subject 1 for the simulation of the IPM. The actual measurement indicated a step width of $δ$ = 0.22 m, while the horizontal step frequencies, $f_{p}$ , were set to 0.85 Hz and 0.93 Hz, respectively. Figure 12(a) illustrates the results of the IPM simulation and DPM recognition for a subject step frequency of 0.85 Hz. The CoM displacement exhibited a remarkable agreement between the two approaches, particularly from the 8th to the 18th seconds. In the initial 5 seconds of the test, deviations were observed due to the subject’s initial adjustment of stride frequency and deviation from the treadmill’s center line. However, as the subject gradually adapted to the walking rhythm guided by auditory cues, a stable walking posture was achieved, resulting in an improved alignment between the presented data and the ideal data simulated by the IPM. In the case of a real walker, it is conceivable that environmental factors could impede their ability to maintain a consistent walking frequency. Furthermore, external disturbances may introduce deviations in the positioning of subject on the treadmill, thereby causing the displacement trajectory to deviate from a complete alignment with the ideal IPM. Similarly, Figures 12(b) and 12(c) demonstrate a high degree of consistency between the velocities and accelerations obtained from the IPM simulation and the experimental data, following a short period of pace adjustment after the 8th second.

Figure 12.

Comparison between data of DPM and IPM ( $f_{p}$ = 0.85 Hz): (a) CoM displacement trajectory curve; (b) CoM velocity variation curve; (c) CoM acceleration change curve.

In the case of another subject, subject 2, the testing procedure involved a different lateral step frequency of $f_{p}$ = 0.93 Hz, accompanied by a measured step width of $δ$ = 0.18 m. Utilizing these parameters for the IPM simulation, the comparative curve is presented in Figure 13. Regarding the identification of CoM displacement, with the exception of the initial 5-second stage dedicated to step adjustment at the onset of the experiment, the subsequent time periods exhibit a high degree of consistency with the simulated CoM displacement derived from the IPM. Concerning speed and acceleration, during the stable period following the step adjustment, the information obtained from DPM slightly deviates from the theoretical values of the IPM, but remaining relatively consistent overall. To facilitate a more comprehensive comparison between DPM and IPM, and to quantitatively demonstrate the similarity between the two datasets, Table 4 presents the cross-correlation coefficients between DPM and IPM data. The values in Table 4, ranging between 0.8 and 0.95, indicate a high correlation among CoM displacement, velocity, and acceleration. This suggests that the IPM theory accurately and reliably simulates pedestrian motion.

Figure 13.

Comparison between data from DPM and IPM ( $f_{p}$ = 0.93 Hz): (a) CoM displacement trajectory curve; (b) CoM velocity variation curve; (c) CoM acceleration change curve.

Table 4.

Cross correlation coefficients between DPM and IPM.

Number	CoM’s displacement	CoM’s velocity	CoM’s acceleration
Subject 1	0.9078	0.9454	0.8692
Subject 2	0.9270	0.9214	0.8672

Figure 14 depicts the temporal profiles of lateral forces obtained through DPM identification and IPM 's simulation. The results of subject 1’s identification, as presented in Figure 14(a), illustrate that, excluding the initial step adjusting phase, the simulated values from the IPM exhibit a high degree of consistency with those identified by DPM, displaying a similar trend of change. Moreover, the peak forces observed in each step align closely with the measured values obtained through DPM.

Figure 14.

Comparison of identification data from DPM and lateral force of IPM’s imitation: (a) Subject 1 ( $f_{p}$ = 0.85 Hz) (b) Subject 2 ( $f_{p}$ = 0.93 Hz).

In conclusion, the IPM yields plausible simulation outcomes for pedestrian locomotion on a stationary structure, demonstrating satisfactory agreement in terms of both pedestrian motion and lateral forces.

Conclusion

In this study, we introduce a novel dual-perspective visual recognition technology based on Mediapipe, named DPM, to accurately recognize and capture gait information and lateral forces exerted by an individual pedestrian walking on a treadmill. The walking experiments demonstrate that the HTP combination generates over 80% of the lateral force exerted by pedestrians, confirming the plausibility of its simulation of pedestrian walking and validating the feasibility of single-point measurements in crowded experiments. The DPM identification is highly consistent with Carroll’s experimental results, with some differences primarily arising from individual variations. The frequency domain analysis reveals that the DPM successfully detects the low-frequency components of the lateral force, while failing to fully capture the high-frequency harmonics due to limitations imposed by the frame rate (fps = 30) of the recording device. This limitation can be overcome by employing a camera with a higher frame rate. However, for the pedestrian force, the low frequency components (i.e., first and third order) are dominant, so the low frequency data is sufficient to describe the pedestrian excitation.

Subsequently, after comparative analysis of the experimental data, we further validated the IPM. We select specific parameters of the IPM based on the subject’s body for simulation. It is found that the simulated values of the IPM are highly consistent with the measured data of the subjects, with the same trend of change. And the peak value of the lateral forces is also similar, except for a certain deviation at the beginning of the pace adjustment. The consistent outcomes across various experimental subjects provide compelling evidence that IPM effectively replicates the kinematic characteristics of pedestrians and has wide individual applicability. This implies that IPM can serve not only as a tool for analyzing the motion of individual pedestrians, but also for simulating and predicting the behavior of large-scale crowds, enabling us to anticipate the evolution of crowd dynamics and devise appropriate preventive and reactive measures against lateral vibrations on footbridges. The proposed method has exhibited commendable proficiency in recognizing pedestrian gait information. It is believed that it can also be applied to the recognition of crowd gaits in the future. Nonetheless, the present study has certain limitations: 1) The current implementation of DPM lacks the capability to recognize pedestrian vertical forces. 2) The diversity of experimental conditions should be expanded, for instance, by incorporating a wider range of prescribed step frequencies and walking speeds.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Research on the shear resistance mechanism of a new prefabricated UHPC composite tenon steel composite structure, Guangdong Natural Science Foundation project (general project) (2022A1515011023) and Deepening research on pedestrian induced lateral vibration of pedestrian bridges based on Kuramoto synchronous model, Guangdong Natural Science Foundation project (general project) (2022A1515011703).

ORCID iD

Yingfeng He

References

Bocian

Macdonald

Burn

, et al. (2015) Experimental identification of the behaviour of and lateral forces from freely-walking pedestrians on laterally oscillating structures in a virtual reality environment. Engineering Structures 105: 62–76.

Bourahmoune

Ishac

Amagasa

(2022) Intelligent posture training: machine-learning-powered human sitting posture recognition based on a pressure-sensing IoT cushion. Sensors 22: 5337.

Carroll

Owen

Hussein

(2013) Reproduction of lateral ground reaction forces from visual marker data and analysis of balance response while walking on a laterally oscillating deck. Engineering Structures 49: 1034–1047.

Carroll

Owen

Hussein

(2014) Experimental identification of the lateral human–structure interaction mechanism and assessment of the inverted-pendulum biomechanical model. Journal of Sound and Vibration 333: 5865–5884.

Claff

Williams

Blakeborough

(2017) The kinematics and kinetics of pedestrians on a laterally swaying footbridge. Journal of Sound and Vibration 407: 286–308.

Dallard

Fitzpatrick

Flint

, et al. (2001) The London millennium footbridge. Structural Engineer 79: 17–21.

Ding

Liu

, et al. (2020) Human posture recognition based on multiple features and rule learning. International Journal of Machine Learning and Cybernetics 11: 2529–2540.

Zhang

, et al. (2020) Capture of 3D human motion pose in virtual reality based on video recognition. Complexity 2020: 1–17.

Fujino

Siringoringo

(2016) A conceptual review of pedestrian-induced lateral vibration and crowd synchronization problem on footbridges. Journal of Bridge Engineering 21: C4015001.

10.

Fujino

Pacheco

Nakamura

, et al. (1993) Synchronization of human walking observed during lateral vibration of a congested pedestrian bridge. Earthquake engineering & structural dynamics 22: 741–758.

11.

Ingólfsson

Georgakis

Ricciardelli

, et al. (2011) Experimental identification of pedestrian-induced lateral forces on footbridges. Journal of Sound and Vibration 330: 1265–1284.

12.

Lugaresi

Tang

Nash

, et al. (2019) Mediapipe: a framework for building perception pipelines. arXiv preprint arXiv:1906.08172.

13.

Wei

Long

(2020) A gait recognition method based on the combination of human body posture and human body contour. In: Journal of Physics: Conference Series. Bristol, England: IOP Publishing, 012031.

14.

Macdonald

(2008) Pedestrian-induced vibrations of the Clifton Suspension Bridge, UK. In: Proceedings of the Institution of Civil Engineers-Bridge Engineering. London, UK: Thomas Telford Ltd, 69–77.

15.

Macdonald

(2009) Lateral excitation of bridges by balancing pedestrians. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 465: 1055–1073.

16.

Newland

(2003) Vibration of the London Millennium Footbridge. In: Department of Engineering, University of Cambridge. www2.eng.cam.ac.uk/∼den/ICSV9_06.htm .

17.

Picozzi

Avossa

Ricciardelli

(2022) Probabilistic assessment of footbridge response to single walkers. Archive of Applied Mechanics 92: 1913–1927.

18.

Toshev

Szegedy

(2014) Deeppose: human pose estimation via deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Silver Spring, MD: IEEE Computer Society Press, 1653–1660.

19.

Van Hauwermeiren

Van Nimmen

Van den Broeck

, et al. (2020) Vision-based methodology for characterizing the flow of a high-density crowd on footbridges: strategy and application. Infrastructures 5: 51.

20.

Guo

H-C

(2023) Robust abnormal human-posture recognition using OpenPose and multi-view cross-information. IEEE Sensors Journal 23: 12370–12379.

21.

Yang

, et al. (2022) A spring-loaded inverted pendulum model for analysis of human-structure interaction on vibrating surfaces. Journal of Sound and Vibration 522: 116727.

22.

Zhou

Zhang

(2020) Human action recognition toward massive-scale sport sceneries based on deep multi-model feature fusion. Signal Processing: Image Communication 84: 115802.