Measuring and modeling human bodies with a novel relocatable mechatronic sensor-net

Abstract

Anthropometric parameter based human modeling has been attractively explored and applied in human body related product design industries. However, how to conveniently obtain precise anthropometric parameters and how to make the human modeling results fit the anthropometric parameters well are still unsolved problems. In this paper, a smart garment attached with a mechatronic sensor-net has been designed to measure multiple body dimensions precisely and simultaneously. Geometrically, this sensor-net corresponds to a feature curve-net. A 3D human model is statistically reconstructed with body measurements initially and is further optimized with the shape constraints from the feature curve-net. Experimental results verify the effectiveness of our approach in both human body measurement and 3D human body modeling.

Keywords

human modeling human body measurement mechatronic sensor-net curve-net statistical modeling

The dimensions and 3D model of human bodies are the fundamental data for many human body related applications, especially in 3D computer-aided garment design,^1,2 garment sizing³ or customization,^4–6 and virtual try-on systems.^7,8 Technologies that can precisely measure and reconstruct 3D human bodies are eagerly needed.

Human body dimensions can be obtained either by contact or non-contact measurement tools. Non-contact measurement is mostly achieved by using laser-based or vision-based devices, such as 3D scanners or RGB-D cameras.^9–11 These devices can accurately obtain the 3D surfaces of human bodies, and the feature points and dimensions can be extracted from the 3D human models.^12,13 However, these devices have a limitation that they cannot “see” the surface occluded by another object, such as the human body part under clothing. Statistical methods for estimating the shape of a human body under clothing have been proposed,^14–16 but the estimation accuracy usually cannot be guaranteed. To reduce the influence of clothing, the consumers are usually asked to take off most of their clothes and leave close-fitting underwear only in scanning, which however, is emotionally unacceptable for some consumers because of individual privacy.

In the case of customers wearing clothes, a technology that can still measure their dimensions and create their 3D human model with acceptable accuracy is very useful. This will avoid the inconvenience and embarrassment of customers needing to undress or change clothes when measuring their body dimensions, particularly in applications located in open environments; for example, customers experiencing clothes in front of virtual try-on mirrors placed in shopping malls.

Currently, contact measurement is still the most widely used method in human body measurement. Human body dimensions are measured manually with a soft tape or other professional manual measurement tools. During measurement, clothes will be fastened to the surface of the human body by the tape. Therefore, this method is less sensitive to user's clothes compared to laser-based or vision-based measurement. Though the operation sounds straightforward, some inconveniences do exist for the consumers. Firstly, it is not convenient for the consumer to measure his/her body dimensions without assistance from someone else. Usually, the measurement is carried out by a practitioner. Secondly, some consumers may object to practitioners of the opposite gender, especially in measuring the dimensions of sensitive body parts. Thirdly, human body dimensions are measured asynchronously, and the whole measurement may take several to tens of minutes. During the measurement, human body measurements may change due to the breath or unconscious limb swinging, which influence the measurement results. Therefore methods or tools enabling a user to measure her/his body dimensions simultaneously without, or with less, assistance would be useful.

A contact measurement, such as soft tape based measurement, can only produce one-dimensional data; however, 3D models are necessary for applications such as 3D garment design and virtual try-on systems. To set up the mapping between body dimensions and 3D human models, statistical human modeling has been explored as an attractive option.^17–19 However, these methods are mostly based on global optimization; dimensions of the reconstructed 3D human model may deviate substantially from the actual measurements.

The goal of this paper is to provide a portable and easy-to-operate device with which multiple body dimensions can be measured simultaneously and 3D human models can be reconstructed effectively. We achieve our goal in two aspects, as shown in Figure 1. Firstly, we have designed a wearable measurement device which is formed by a mechatronic sensor-net attached on a coat. Multiple body dimensions can be measured simultaneously when the consumer wears this coat. Secondly, we have proposed a constrained statistical method to reconstruct 3D human models. Statistical human shape space is established based on a large number of 3D scanned human models. When the dimensions of a consumer are measured, her/his 3D model is initially reconstructed by mapping the dimensions to human shape space. From the initial human model, feature curves are extracted out and optimized to further constrain the shape deformation of the human model. In the method, the 3D scanned human model data are obtained from reference 20, which provides about 1500 example human models for each gender.

Figure 1.

Schematic diagram of human body measurement and reconstruction based on a mechatronic sensor-net.

In summary, our approach provides a novel device that can simultaneously measure multiple dimensions, which takes an active step toward improving the efficiency and convenience of contact measurement. Our approach also offers a new locally constrained statistical human modeling method, which makes the 3D human modeling results match the dimensions well. Our approach has the following academic contributions.

A novel wearable device for measuring multiple human body dimensions simultaneously has been designed. The device is formed by a garment attached with a mechatronic sensor-net. Each body dimension is measured as the length of a tensioned and inextensible thread which links each paired sensor.

A method for reconstructing a 3D feature curve-net that reflects the geometry of tensioned threads has been proposed. This feature curve-net describes a geometric wireframe of the measured human body.

A progressive 3D human body reconstruction method has been proposed. The 3D human model is initialized by statistical mapping from body dimensions to the human shape space, then it is optimized with the local shape constraints from the feature curve-net.

To our best knowledge, this work is the first attempt to measure human body shape in such a way. With our device and method, the geometric body shape of a consumer can be efficiently and effectively perceived by putting the garment onto him/her, which can potentially solve the body shape awareness problems in many human body related applications.

The remainder of this paper is structured as follows. After reviewing related works in the second section, the relation between anthropometric parameters and the human shape space is established in the third section. A mechatronic sensor-net for measuring human bodies is designed in the fourth section. In the fifth section, based on the measurement, feature curve-nets are generated to optimize the human model. The sixth section gives the experimental results. The whole paper is concluded in the seventh section.

Related works

Various methods and devices for improving the automation of measuring and reconstructing human bodies have been explored. We review the literature from two categories: non-contact and contact.

Non-contact approaches

The most representative devices are laser scanners⁹ or multi-stereo systems.²¹ The shape of a human body is initially captured as a point cloud, based on which 3D human model is reconstructed. Before getting a completed human model, manual interactions are usually required to enrich the human surface texture²¹ or tackle the holes and noises in the raw data.^17,22

Single RGB image²³ or multiple image²⁴ based approaches use the human body silhouettes extracted from the images to reconstruct 3D human models. RGB-D images provide an additional depth channel compared to RGB images, which can capture more geometric information. With the depth data, 3D human models can be reconstructed by using statistical modeling methods^14,15 or implicit surface modeling methods.^25,26

Contact approaches

The most common contact measurement tools are soft tapes and other manual measuring tools. Anthropometric data can be measured with the guidance of ISO standards.²⁷ Such tools have some merits and inconveniences, as mentioned in the earlier section. To overcome the inconveniences, wearable devices for body measurement have been studied. Xu et al.²⁸ used a vision based method to compute the spatial positions of color markers on flexible belts that are fixed on the human body. They used these spatial positions to reconstruct the 3D human body. Uhm et al.²⁹ designed a vest with RGB-IR markers for human body measurement. The straight distances between each paired markers are deduced from the known physical sizes of a background, and then the distances are converted to arc lengths by multiplying a ratio to get the circumferences of the human body.

The one-dimensional measurements have a natural relation to the 3D surface shape of the human body. Statistical human modeling methods, such as PCA (Principle Components Analysis) based methods, have been proposed to set the mapping between human body measurements and 3D human models.^18,19,30,31

Characteristics of existing approaches

Summarily, non-contact approaches capture the 3D shape of the human body first, and then the anthropometric dimensions are extracted from the 3D human model.³² They have the merit that they can get as many dimensions as needed. However, they also have a limitation that they cannot “see” the body part under a garment. For accuracy, consumers are usually asked to dress only in close-fitting underwear when they are measured, which makes some users feel uncomfortable because of individual privacy. Additionally, the high prices of the laser scanners hinder many users. The costs of RGB-D based devices are much lower, but their accuracy is not high enough; as reported in references 33 and 34, the average waist girth error in human modeling results based on RGB-D data is high at 6.5 cm and 6.2 cm.

Contact approaches are less sensitive to the user's clothing, because clothing will be attached to the human body by the contact tools. Therefore, in the process of measurement, the consumers can wear light clothing, either close fitting or loose. Moreover, the contact measurement tools are mostly portable and with a low price, which can benefit most users. Based on the human body dimensions, a 3D human model can be statistically reconstructed. The reconstruction methods are mostly global optimization based. As a result, the dimensions on the 3D reconstructed human model may differ substantially from the initial measurements. However, literature that mentions how to tackle such a problem is rare.

Our goal is to design an efficient and effective device for measuring human body dimensions and provide a dimensions-constrained statistical human modeling method. Compared to existing contact approaches,^28,29 our approach provides a mechatronic sensor-net to measure the arc lengths on a human body surface directly and does not need a special background. Furthermore, we propose a method for reconstructing a feature curve-net based on the measurement data. This feature curve-net is further used to optimize the 3D human model to match the measurements well. Furthermore, the effects of different anthropometric parameters on human modeling results have also been evaluated with experiments.

Relation between anthropometric parameters and human shape space

Statistical human modeling approaches based on anthropometric parameters reconstruct human bodies by projecting anthropometric parameters onto the human body shape space. We select the anthropometric parameters by following the ISO standard of measuring human body dimensions for clothes.²⁷

Human body shape space

The human body shape space is spanned by a number of high dimensional orthogonal axes which are computed from a large number of uniformed structured human models by using PCA. For each example human model with N vertices, its geometry is represented as a 3N dimensional column h_i. Let $H = [h_{1} - \bar{h}, - \bar{h}, \dots, h_{m} - \bar{h}]$ , and $\bar{h}$ be the average of {h_i}. Eigenvectors {y_i} and eigenvalues {λ_i} can be decomposed from HH^T. The first k eigenvectors with their eigen values meeting the following inequality are selected to form a matrix Y.

τ \leq \sum_{i = 0}^{k} λ_{i} \sum_{i} λ_{i}

(1)

where τ is a constant and 0 < τ≤1 (in our case τ = 0.98). For an individual human model h, it can be approximately expressed as

b = Y^{T} (h - \bar{h})

(2)

where b is a k dimensional vector of coefficients. In other words, given a particular b, the geometry of a corresponding human model h can be approximated as

\hat{h} = Y \cdot b + \bar{h}

(3)

In our approach, we adopt the human models from reference 20, two thirds of which are selected as the initial training set and the rest serve as the testing set.

Mapping from anthropometric parameters to the human body shape space

We learn the relation between b in equation (3) and human anthropometric parameters l = [l₁,… l_i,… l_m]^T with the expectation that b can be deduced from l in the form b = f(l). Consequently, equation (3) can be expressed in an alternative way as

\hat{h} = Y \cdot f (l) + \bar{h}

(4)

Mostly, $f (l)$ is set as a linear function,¹⁸ i.e. $b = f (l) = P [l 1]^{T}$ where P is a projection matrix and can be trained as follows

arg mi n_{P} (Σ_{i} | | P [l_{i} 1]^{T} - b_{i} | | 2)

(5)

With equations (5) and (4), human models can be reconstructed with anthropometric parameters.

The PCA model described in an earlier section and the mapping in equation (5) are all linear. However, the shape changes between different human bodies are generally nonlinear. To reduce the shape error in 3D human modeling, we optimize human body modeling results with the shape constraints of the feature-curve net, as will be detailed in the fifth section.

Mechatronic sensor-net for measurement

Anthropometric parameters in our approach

In the ISO standard of measuring human body dimensions for clothes,²⁷ tens of anthropometric parameters are defined. However, it is not necessary to involve all of them in the 3D human modeling. For instance, 8, 13, and 30 anthropometric parameters are adopted in references 35, 18, and 36, respectively. The influence of anthropometric parameters on human modeling results will be analyzed in the Experimental results section.

In our approach, human body shape is measured with inextensible soft threads under tensioned states. To measure both transverse dimensions and longitudinal dimensions, the distribution of threads is designed as shown in Figure 2. Each thread links two sensors, and these sensors can be repositioned on the coat. The design of these sensors will be detailed in the next subsection. Geometrically, these threads span a feature curve-net, as shown in Figure 2(c). The length of each curve corresponds to an anthropometric parameter. Two additional height parameters are adopted: one is the human body height, the other is the hip height, as shown by the lines with arrows in Figure 2(a). For the hip height, a sensor is draped straight from the side of the hip to touch the ground, as shown in Figure 2(b). For the human body height, it is manually set, since most people know their height exactly.

Figure 2.

Anthropometric parameters in our approach. (a) Curves for measuring a 3D human model. (b) Wearable mechatronic sensor-net. (c) Feature curve-net.

We partition a girth into several parts so that the human shape variation can be distinguished in a more discriminative way. For instance, the female chest girth is partitioned into four subparts, i.e. (1) length between two breast tips, which reflects the chest breadth; (2) curve length between two posterior armpit points, which reflects posterior interarmpit breadth; (3) two lengths, each of which is the curve length from one breast tip to one posterior armpit point on the same side, which reflects the chest depth.

Design of mechatronic sensor-net

A coat attached with curve length measurement sensors has been designed to synchronously obtain the anthropometric parameters, as shown in Figure 2(b). The length measurement sensor (as shown by the yellow boxes in Figure 2(b)) has been designed as an electronization of a rolled soft tape. The structure diagram of the sensor is shown in Figure 3(a). A soft thread with large tensile modulus is rolled on a drum. When the thread is pulled, the drum will be rotated. Rotation of the drum is resisted by a coiled spring inside the drum. Therefore, the thread is always tensioned under the force of the coiled spring; when the thread is released, it will be drawn back.

Figure 3.

Measurement sensor. (a) Inner structure diagram. (b) Measurement sensor adhered on a human body.

A grating encoder is coaxially fixed on the drum. When the drum rotates, the bars and openings on the grating encoder pass between a light emitter and receiver. The emitter and receiver are linked to a counter which counts how many openings have been passed. This counting number is synchronously sent to the human modeling system through WiFi. Consequently, the rotating angle of the drum can be calculated, and the length of thread that pulled out of the sensor can be computed.

To make it suitable for measuring various shapes of human bodies, the coat is made of thin and elastic material. The measurement sensors are adhered onto the coat using hook-and-loop fasteners, as shown in Figure 3(b), so that during measurement the positions of the sensors can be adjusted and fixed on the coat as needed. For each soft thread, it is rolled on one sensor's drum and is pulled out from the sensor through a guide hole. The active end of the thread is tied on a pin, and this pin can be inserted into a pinhole on another sensor. Therefore, the distance between these two relevant sensors can be measured.

The sensor can not only measure the curve length between two points on a human body but also can measure the height from one point to the ground as a plumb, as shown by the sensor in the red ellipse in Figure 3(b). We measure the hip height in such a way, as indicated in Figure 2(a).

Generating feature curve-net for human model optimization

With the measurements obtained in the previous section, a 3D individual human model can be generated using equations (4) and (5). However, a human modeling method based on equations (4) and (5) is a global linear optimization; the corresponding measurements extracted from the 3D human model may deviate from the actual measurements. To refine the 3D human modeling result, we optimize the feature curves on the 3D human model with the constraints of actual measurements from the sensor-net and use these feature curves as shape constraints to adjust the shape of the 3D human model.

Feature curve shape space

We optimize the feature curves in the statistical shape space so that the optimized results conform with statistical reality. The statistical models are created based on the 3D feature curves extracted from the 3D scanned human models provided by reference 20. To extract feature curves, feature points on human models are first extracted out,¹² and then geodesic curves between every two relative feature points are generated. Secondly, the convex curve of each geodesic curve is computed, as shown in Figure 2(a); these are regarded as the counterparts of the tensioned threads linking mechatronic sensors, as illustrated in Figure 2.

For each kind of curve, its statistical model is set up using a PCA method similar to that in the section on human body shape space. Thus, for a feature curve, c_j, extracted from a reconstructed human model, it can be filtered by the feature curve shape space as

c_{j} \bar{c_{j}} + Y^{c_{j}} \cdot {\hat{b}}^{c_{j}}, {\hat{b}}^{c_{j}} = (Y^{c_{j}}) T \cdot (c_{j} - \bar{c_{j}})

(6)

where

\bar{c_{j}}

is the average curve of the jth feature curves on the 3D scanned human models,

Y^{c_{j}}

is a matrix whose columns are eigenvectors decomposed from the covariance matrix of the feature curves.

Feature curve optimization with constraints of curve length

For each feature curve c_j extracted from a reconstructed human model, the length of its convex curve is denoted l(c_j). l(c_j) may deviate from the actual measurement l_j much. We refine c_j by reducing | l(c_j)- l_j| in the feature curve shape space.

One simple way is to scale c_j with a ratio λ = l_j/l(c_j), as shown in the process flagged by green arrows in Figure 4, and the corresponding result is shown as the green curve in Figure 4. Such a method is adopted in literature.³⁶ However, the relationship between the initial reconstruction result and the true result is usually not scaled in such a simple uniform way. To address this issue, we use curve segments to capture more shape information; for instance, the waist curve is comprised of a front part and a back part, as shown in the top left image in Figure 4. Correspondingly, the length scale for the front part and the back part can be computed as λ_k = l_j,k/l(c_j,k), k = 1,2, and usually λ₁≠λ₂.

Figure 4.

Adjustment of the feature curve by length.

For curve segments {c_j,k} with their scales {λ_k}, due to their different scales, if each c_j,k is scaled with λ_k directly, the adjacent curve segments will lose their continuity. To keep the continuity, they are adjusted progressively, as shown in the process flagged by the red arrows in Figure 4. The detailed method is as follows.

{c_j,k} are scaled with the smallest scale λ_s = min(λ_k), as shown step 1 in Figure 4;

{λ_k} is updated with the adjusted curve segments. If each λ_k is very close to 1, end the processing; otherwise, go to step (3).

For c_j,k with λ_k ≠ 1, the scale on each point on c_j,k is interpolated smoothly from one endpoint of c_j,k to its midpoint. The scale is computed as f(s, λ_k ± 1, λ_k) = (1 − s)λ_k ± 1 + sλ_k where s is the ratio between the curve length from the current point to the relative endpoint against the curve length from the midpoint to the endpoint; λ_k ± 1 is the scale on the curve segment before or after the current curve segment. Update c_j,k, as shown in step 2 in Figure 4. Go to step 2.

The above curve adjustments are treated geometrically; however, the result may deviate from a reasonable shape. We refine c_j by filtering it with equation (6), as shown in step 3 in Figure 4. After the above treatment, the curve length of c_j, l(c_j), is usually very close to the input length l_j; we scale c_j by l_j/l(c_j) to rectify the curve length.

As a comparison, as shown in the top right image in Figure 4, the shape of the adjusted curve generated by our approach can be obviously different from that generated by a mere uniform scale. For the mere uniform scale method, the shape of the adjusted result will not change, only its size changes; while the shape of curve adjusted by our approach varies according to the difference of curve length, which reflects the real situations more reasonably.

Adjusting the human model with constraints of the feature curve-net

The feature curve adjustment and human model optimization are implemented alternately; as shown in Figure 5, the feature curves are firstly extracted out from the initial reconstructed human model; secondly, the transverse curves are optimized and used to deform the human model; thirdly, on the deformed human model, the longitudinal curves are extracted, optimized, and further used to deform the human model. Therefore, both the feature curve-net and the human model are optimized progressively, which can help keep the connection between the feature curves.

Figure 5.

Feature curve-net constrained human model optimization.

We deform the human model h by minimizing the constrained Laplacian mesh deformation energy. For each feature curve, c_j on the human surface and its optimized shape ${\hat{c}}_{j}$ denote the sampled points on them being {p_j,k} and { ${\hat{p}}_{j, k}$ } correspondingly. As p_j,k is on the human surface, it can be parameterized by the relevant triangle △(v_l,0, v_l,1, v_l,2) containing p_j,k as

p_{j, k} = α v_{l, 0} + β v_{l, 1} + γ v_{l, 2}, α + β + γ = 1

(7)

where (

α, β, γ

) is the barycentric coordinate of p_j,k in the triangle. The human model is deformed by solving the following deformation energy minimizing function.

min_{v_{i}} (λ_{1} \sum_{k} ‖ α v_{l, 0} + β v_{l, 1} + γ v_{l, 2} - \hat{p} k j ‖ 2 + λ_{2} \sum_{i} ‖ L (v_{i}) - δ_{i} ‖ 2)

(8)

where the first term is to make the human surface meet the constraints of the feature curves; the second term is for making the deformation smooth throughout the human surface;

λ_{1}

and

λ_{2}

are weights; L(v_i) and δ _i are the Laplacian operator and Laplacian coordinate of vertex v_i, and are computed as

L (v_{i}) = δ_{i} = v_{i} - \sum_{j \in N (i)} w_{i, j} v_{j}, \sum_{j \in N (i)} w_{i, j} = 1

(9)

where N(i) = {j|(i,j)∈E} is the set of adjacent vertices of v_i; w_ij is a cotangent weight relative to edge v_iv_j.³⁷ As

v_{i}

is to be determined in equation (8),

δ_{i}

cannot be directly computed using equation (9). Instead, we compute

δ_{i}

δ_{i} = T_{i} δ_{i}^{0}

, where

δ_{i}^{0}

is computed from the initial human model;

T_{i}

is a transformation matrix which is iteratively optimized by using as-rigid-as-possible technology.³⁸

Experimental results

A number of experiments have been done to evaluate the performance of our approach. Firstly, the validity of the anthropometric parameters we selected for human modeling is evaluated in the next section. Secondly, the performance of our measurement based 3D human modeling method is tested based on the data set provided by reference 20. Thirdly, our mechatronic sensor-net is used to measure the dimensions of human bodies in different situations, and 3D human models are generated based on the measurements. Then, the potential applications of our approach are explored.

Influence of anthropometric parameters on human model reconstruction

Among tens of anthropometric parameters defined in the ISO standard of measuring human body dimensions for clothes,²⁷ usually only parts of them will be adopted in 3D human modeling. As a reference, 8, 13, and 30 anthropometric parameters are adopted in references 35, 18, and 36, respectively. We experimentally analyze the influence of anthropometric parameters on the human modeling results, based on which we choose the parameters for our human modeling approach.

The anthropometric parameters used in references 35, 18, and 36 include non-geometric parameters such as gender, weight, etc., and geometric parameters such as heights, width, lengths, and girths, as shown in the figures in the second row in Table 1. For human weight, it can be computed as a linear regression of human body volume.¹⁸ For evaluation, we use equations (4) and (5) to reconstruct 3D human models with the anthropometric parameters, and the shape errors between the modeling results and the ground truth are computed. Generally, the shape error can be computed as a point-to-surface distance, which however, is very sensitive to human poses; even swinging an arm a little can lead to a relatively big distance between one hand of the modeling results and the ground truth surface. Instead, we adopt girth error and longitudinal error. The convex curves {

C_{j}^{i}

} indicate the girths are extracted out, as shown in the white curves in the third row and fourth row in Table 1. The girth error on the jth curve is computed as follows

e_{j}^{G} = \frac{1}{n} \sum_{j} ‖ C_{j}^{i} - {\tilde{C}}_{j}^{i} ‖

(10)

where

C_{j}^{i}

and

{\tilde{C}}_{j}^{i}

are the lengths of the jth convex curve on the ith reconstructed human model and ith ground-truth human model, respectively; n is the number of reconstructed human models. In our experiments, one third of the human models provided by reference 20 are used for testing, i.e. n is about 500. The longitudinal error is computed as the average bone length difference between the reconstructed human models and the ground truth

e_{j}^{H} = \frac{1}{n} \sum_{j} ‖ L_{j}^{i} - {\tilde{L}}_{j}^{i} ‖

(11)

where

L_{j}^{i}

and

{\tilde{L}}_{j}^{i}

are the jth bone length on the ith reconstructed human model and its corresponding ground truth, respectively. The bones that are used for evaluation are similar to those in reference 36, as shown by the figure in the second row and fourth column in Table 1.

Table 1.

Evaluating anthropometric parameters

For intuitive illustration, the values of $e_{j}^{G}$ and $e_{j}^{H}$ are mapped to different colors, and the shape errors are displayed on a certain human model, as shown in Table 1. The color on regions in between girth curves is interpolated smoothly.

From the experimental results, it can be determined that shape errors can be reduced by using a greater number of proper anthropometric parameters in the regions where the shape error is big. As shown by the comparison between the fourth column and the second column in Table 1, by involving more parameters on the shoulder and limbs, the shape error on these areas can be reduced.

In the literature,^18,35,36 parts of parameters are measured “inside” the human body, such as bone lengths, as shown by the straight lines inside the 3D human models in the third and fourth columns in Table 1. To measure these parameters, a 3D human model is required beforehand, which is feasible for 3D scanner or RGB-D sensor based situations. However, it is impossible to measure them by physical threads. In our approach, we replace them with lengths of threads enveloping the human body, as shown in Figure 2(a) and the figure in the second row and last column in Table 1. The measurements are recorded as the tensioned length between paired sensors, and these sensors are positioned on the following points: cervical point, two breast tips, two posterior armpits, two anterior armpits, two acromions, left wrist point, three points on the waist section—left lateral point, right lateral point, and central back—and three points on the hip section—left lateral point, right lateral point, and central back point.

From Table 1, it can be determined that our approach performs well on human body reconstruction; particularly, the shape error on the upper part of the body is low. It should be pointed out that, in our approach, most anthropometric parameters are measured from the upper part of a human body, since at present, most of the sensors are distributed on the upper body. Shape errors on regions other than the upper body are comparatively larger. The shape error can be reduced by measuring more parameters on the limbs, such as the upper arm circumference and thigh circumference, etc.

Evaluating our measurement based 3D human modeling on 3D scanned data

The tensioned threads on our sensor-net correspond to 3D feature curves on human models; therefore, the performance of our measurement based 3D human modeling can be virtually evaluated on 3D scanned human models. We use 3D scanned human models provided by reference 20 as the ground truth data; from these data, feature curves are extracted out and their lengths are computed, as shown in Figure 6(a). Subsequently, we use these lengths to reconstruct the feature curves and human models, and the shape errors between the ground truth and the reconstructed results are computed.

Figure 6.

Human reconstruction. (a) Ground truth. (b) Reconstruction with equations (4) and (5). (c) Optimizing human model with uniform scaled feature curves. (d) Optimizing human model with our approach. (e) Waist curves from (a), (b), (c), (d) are colored in green, red, blue, and grey, respectively; (f), (g), (h) are the point-to-surface shape error between model (a) and models (b), (c), and (d), respectively.

With the length data, a human model can be reconstructed using equations (4) and (5), as shown in Figure 6(b), and the related input lengths are listed in Table 2. The reconstructed human model is further optimized in two ways, as proposed earlier: one is uniform scale based optimization, as shown in Figure 6(c), and the other is differentiated scale based optimization, as shown in Figure 6(d). As listed in Table 2, the girth errors on the human model can be reduced by either of these two methods. However, the human shape of the optimization is different, as shown in Figure 6(e) where the waist curves from Figure 6(a), Figure 6(b), Figure 6(c), and Figure 6(d) are colored in green, red, blue, and grey, respectively. Comparatively, our differentiated scale based optimization method can better reduce this shape difference. To reflect the shape difference between the ground truth

h

and the human modeling result

\hat{h}

well, point-to-surface (PTS) distances from

\hat{h}

h

are computed as follows

ε = d (M T (\hat{h}), T (h))

(12)

where

ε

is an n-dimensional vector with each element being the PTS distance on its one vertex; T(h) is a transformation which transforms the 3n-dimensional h into a 4 × n matrix V, as V = T(h) = [[

v_{1}^{T}

1]^T [

v_{2}^{T}

1]^T… [

v_{n}^{T}

1]^T];

d (\cdot, \cdot)

is the function for computing PTS distances between two human models. M = [R t] is a rigid transformation matrix with R and t being a 3 × 3 rotation matrix and a three-dimensional translation. M is computed by minimizing

ε

ε

is visualized as shown in Figures 6(f) to (h), respectively. The visualization results also prove the effectiveness of our feature curve-net based human model optimization approach.

Table 2.

Girths on human models

Model	Chest girth (mm)	Waist girth (mm)	Hip girth (mm)
Figure 8(a)	1094.1	949.7	1154.4
Figure 8(b)	1087.2	943.8	1136.7
Figure 8(c)	1093.3	949.7	1156.5
Figure 8(d)	1092.2	950.8	1154.1

Our feature curve-net based human model optimization method can also be applied as an extension of existing anthropometric parameter based human reconstruction methods. For instance, the human model reconstruction results obtained from references 35, 18, and 36 can be further optimized to reduce the shape error, as shown in the figures in the third row of Table 1. It should be pointed out that, as each girth is represented by one measurement in this literature,^18,35,36 the human models are optimized using the uniform scale based method.

Evaluating human body measurements and modeling with the mechatronic sensor-net

For the mechatronic sensor-net, its precision of length measurement is mainly determined by the grating encoder and the size of the drum. Denoting the diameter of the drum as d and the number of openings on the grating encoder as k, the length measurement precision can be approximately computed as (πd/k). In our approach, d = 37.4 mm, m = 100, therefore the measurement error can be controlled within 1.2 mm; such a measurement precision is high enough to meet the requirements of most human body related product industries. As a comparison, the scanning accuracy of available 3D whole-body scanners is up to 5 mm, e.g. VITUS XXL is up to 1 mm, TC² is up to 3 mm, and Size Stream is up to 5 mm.³⁹

For evaluating the effectiveness of our approach, a number of experiments have been done. In the experiments, we ask the volunteers to wear the same coat attached with the mechatronic sensor-net. The coat is made of cotton and spandex blended material, which is a kind of elastic material. Table 3 lists the measurements and 3D human modeling results of six volunteers. Their heights are from 164 cm to 183 cm, their chest girths are from 91 to 116 cm, their hip girths are from 90 to 108 cm, and their waist girths are from 71 to 97 cm.

Table 3.

Mechatronic sensor-net based human body measurement and modeling

Due to the body shape variety between different persons, the positions of the measurement sensors on the coat need to be relocated. As the sensors are mostly distributed on the front part and side part of the coat, they can be interactively adjusted by the users themselves; three sensors are on the central back, it is also not difficult for most of the users to adjust them. Generally, the sensors can be well repositioned within two minutes.

To test the performance of our approach in different situations, the volunteers are asked to dress in different garments, make different poses, take off, or put on, shoes during measuring. With the measurement data, 3D human models are generated based on the method provided in the fifth section. From the experimental results listed in Table 3, the following points can be drawn.

Wearing shoes, or not, will affect the measurement of hip height, as shown by the measurement data of E_1 and E_2 in Table 3. The influence on the 3D human modeling results is mainly reflected as a change of leg length, while the influence on the shape of upper body is much smaller, as shown in Figure 7(a). This is because, in our 3D human modeling approach, the human model is constrained by the feature curve-net and the curve lengths on the upper body barely change in the two situations.

The clothes on a human body can influence the measurement results, as listed in F_2, F_3, and F_4 in Table 3 where the dimensions of the same person wearing different clothes are measured. In measuring, the clothes are fastened to the surface of the human body by the tensioned threads. Therefore, the influence of cloth looseness on measurement results can be reduced. As a result, the transverse measurements vary with the thickness of the cloth accordingly, as listed in Table 3. With these data, 3D human models are reconstructed correspondingly; as illustrated in Figure 7(b), the shape difference changes smoothly in different situations, which also proves that the thickness of cloth, rather than the looseness, affects the measurements and 3D modeling of the human body in our approach.

Our sensor-net can also measure persons in different poses. When a person makes different poses, his/her dimensions change accordingly, as shown in F_1 and F_2.

Figure 7.

Comparison between 3D human modeling results. (a) Modeling results of tester E with shoe (E_1, the left image) or without shoe (E_2, the right image). (b) Modeling results of tester F with T-shirt (F_2), shirt (F_3), and sweater (F_4), respectively; the color on the second and third images reflects the point-to-surface distance between F_3 and F_2, F_4 and F_2, respectively.

Limitations of our approach

From the above points, for accuracy, consumers are encouraged to wear thin and slight clothes, such as a T-shirt or shirt, during measuring. It might also be practical to make a corresponding reduction in the measured dimensions according to the thickness of different clothes.

It should be pointed out that, as the research focus of this paper is on mechatronic sensor-net and 3D human modeling, at present, we only use a ready-made coat. For subjects who are slim, wrinkles on the coat will appear. For measurement accuracy, the design of the coat is to be improved in future work, so that the coat can be close fitting to most human bodies.

It also should be pointed out that the volunteers are all Asian. Ideally, the statistical human body shape space should be established with 3D Asian human data. However, due to the lack of such available data, we adopt 3D European and American human data to set the human body shape space. As a result, the reconstructed human models contain the characteristics of European and American human bodies, such as fullness and strength. More realistic 3D human modeling results can be obtained if a database of 3D Asian human models is available.

Exploration on potential applications of our approach

Our approach can potentially be applied in many human body related areas, such as clothing customization and clothing sales. In clothing customization, one problem is how to transfer the clothing design results that are initially finished on a standard human body to consumers. One solution to this problem is adjusting the 3D clothing design results according to the shape differences between the standard human model and the consumers.^5,6 Therefore, 3D human models of the consumers are necessary. Our approach provides a practical method for such an application. As shown in Figure 8, the dimensions of the consumer (A in Table 3) are first measured and, based on these, a 3D human model can be generated with our approach, as shown by the image in Figure 8(b). Then, by adopting the method provided in reference 6, a 3D garment designed on a reference human model, as shown in Figure 8(a), can be transferred to the 3D human model of the consumer, as shown in Figure 8(b).

Figure 8.

Application to 3D clothing customization. (a) 3D garment on a standard human model. (b) Transfer of the 3D garment onto an individual human model.

Our approach may also be applied in the area of clothing sales. Nowadays, online e-commerce has become a very important route for clothing sales. The virtual try-on system is regarded as an effective tool for consumers to perceive the wearing effects on themselves when buying clothes online. How to dress the consumer in the virtual garment is the key technical problem in such systems. With our approach, the 3D human model can first be generated, as shown in Figure 8(b). The pose difference between this 3D human model and the human figure in the image, as shown in Figure 9(a), can be reduced by adjusting the pose of the 3D human model based on the skeleton,⁴⁰ as shown in Figure 9(b). The virtual garment can finally fit onto the 3D human model as shown in Figure 9(c). By projecting the 3D garment onto the image, a try-on result of a consumer wearing a virtual garment can be obtained, as shown in Figure 9(d).

Figure 9.

Application to a virtual try-on system. (a) Initialize human model with measurements, where the 3D human model is rendered semi-transparent. (b) Adjust the 3D human model to align the human image. (c) Fit virtual garment onto the 3D human model. (d) Project 3D garment model onto the human image for virtual try-on.

Conclusions

This paper has proposed a method for measuring and modeling human bodies with a novel mechatronic sensor-net. Inextensible soft threads with controllable lengths are used to measure the surface distance between two relative sensors. These threads form a feature curve-net, and their geometric shape can be statistically reconstructed with their lengths. A 3D human model is initially reconstructed with the thread lengths by adopting the statistical mapping from the thread lengths to the human body shape space. It is further optimized with the shape constraints from the feature curve-net. Our approach provides a convenient and economic method for measuring and modeling of a human body with relatively high precision and can potentially be applied in human body related areas, such as clothing design and virtual try-on.

Currently, only the device for measuring the upper part of the human body is presented in this paper. However, our method can be adapted to measure the dimensions of limbs. Thus, more geometric information of a human body can be captured. Consequently, a more accurate human body model can be reconstructed. This work will be done in the near future.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the National Natural Science Foundation of China (No. 51575481, 61732015, 61472355). Zhejiang Provincial Natural Science Foundation of China (LY18F020004), Research Funding of Zhejiang University Robotics Institute, and 973 Project of China(#2015CB058103).

References

Bartle

Sheffer

Kim

, et al. Physics-driven pattern adjustment for direct 3D garment editing. ACM Trans Graphics 2016; 35(4): 11.

. Modeling 3D garments by examples. Comput Aided Des 2014; 49: 28–41.

Gill

. A review of research and innovation in garment sizing, prototyping and fitting. Text Progr 2015; 47(1): 1–85.

Wang

. Rule-free sewing pattern adjustment with precision and efficiency. ACM Trans Graphics 2018; 37(4): 53.

. Customizing 3D garments based on volumetric deformation. Comput Ind 2011; 62(7): 693–707.

Liu

, et al. Feature curve-net-based three-dimensional garment customization. Text Res J 2013; 83(5): 519–531.

Jiang

Sun

, et al. Transferring and fitting fixed-sized garments onto bodies of various dimensions and postures. Comput Aided Des 2019; 106: 30–42.

Wang

, et al. Fitting 3D garment models onto individual human models. Comput Graphics 2010; 34(6): 742–755.

Daanen

Harr

. 3D whole body scanners revisited. Display 2013; 34: 270–275.

10.

Markiewicz

Witkowski

Sitnik

, et al. 3D anthropometric algorithms for the estimation of measurements required for specialized garment design. Expert Syst Appl 2017; 85: 366–385.

11.

Cheng

Chen

Martin

, et al. Parametric modeling of 3D human body shape—A survey. Comput Graphics 2017; 71: 88–100.

12.

Wang

. Automatically constructing skeletons and parametric structures for polygonal human bodies. Comput Graphical Int 2007, pp. 151–156.

13.

Wang

. CyberTape: an interactive measurement tool on polyhedral surface. Comput Graphics 2004; 28(5): 731–745.

14.

Chen

Zeng

, et al. Optimizing human model reconstruction from RGB-D images based on skin detection. Virtual Reality 2016; 20: 159–172.

15.

Chen

Wang

Zeng

, et al. Reconstructing 3D human models with a Kinect. Comput Anim Virtual Worlds 2016; 27(1): 72–85.

16.

Hasler

Stoll

Rosenhahn

, et al. Estimating body shape of dressed humans. Comput Graphics 2009; 33: 211–216.

17.

Anguelov

Srinivasan

Koller

, et al. SCAPE: Shape Completion and Animation of People. ACM Trans Comput Graphics 2005; 24: 408–416.

18.

Koo

Park

Choi

, et al. Example-based statistical framework for parametric modeling of human body shapes. Comput Ind 2015; 73: 23–38.

19.

Chu

Tsai

Wang

, et al. Exemplar-based statistical model for anthropometric parametric design of human body. Comput Ind 2010; 61(6): 541–549.

20.

Pishchulin

Wuhrer

Helten

, et al. Building statistical shape spaces for 3D human modeling. Pattern Recognit 2017; 67: 276–286.

21.

Bogo

Romero

Loper

, et al. FAUST: Dataset and evaluation for 3D mesh registration. IEEE Int Conf Comput Vision Pattern Recognit 2014, pp. 3794–3801.

22.

Allen

Curless

Popovic

. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans Comput Graphics 2003; 22(3): 587–594.

23.

Guan P, Weiss A, Balan A, et al. Estimating human shape and pose from a single image. In: IEEE International Conference on Computer Vision 2009, Kyoto, Japan, 29 September-2 October 2009, pp.1381–1388.

24.

Balan A, Sigal L, Black M, et al. Detailed human shape and pose from images. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007, Minneapolis, MN, USA, 17-22 June 2007, pp.1–8.

25.

Newcombe RA, Izadi S, Hilliges O, et al. Kinectfusion: Real-time dense surface mapping and tracking. In: 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland, 26–29 October 2011, pp. 127–136.

26.

Dou M, Taylor J, Fuchs H, et al. 3D scanning deformable objects with a single RGBD sensor. In: IEEE Conference on Computer Vision and Pattern Recognition, 2015, Boston, MA, USA, 7–12 June 2015, pp.493–501.

27.

ISO 8559–1. Size Designation of Clothes – Part 1: Anthropometric definitions for body measurement, Geneva: ISO, 2017.

28.

Deng

, et al. Modeling 3D human body with a smart vest. Comput Graphics 2018; 75: 44–58.

29.

Uhm

Park

. Fully vision-based automatic human body measurement system for apparel application. Measurement 2015; 61: 169–179.

30.

Wuhrer

Shu

. Estimating 3D human shapes from measurements. Mach Vision Appl 2013; 24: 1133–1147.

31.

Baek

Lee

. Parametric human body shape modeling framework for human-centered product design. Comput Aided Des 2012; 44(1): 56–67.

32.

Stančić

Musić

Zanchi

. Improved structured light 3D scanner with application to anthropometric parameter estimation. Measurement 2013; 46(1): 716–726.

33.

Tong

Zhou

Liu

, et al. Scanning 3d full human bodies using kinects. IEEE Trans Visualization Comput Graphics 2012; 18: 643–650.

34.

Chen

Dang

Cheng

, et al. Fast capture of personalized avatar using two Kinects. J Manuf Syst 2014; 33: 233–240.

35.

Seo

Magnenat-Thalmann

. An example-based approach to human body manipulation. Graphical Models 2004; 66(1): 1–23.

36.

Zhang

Zheng

Magnenat-Thalmann

. Example-guided anthropometric human body modeling. Virtual Comput 2015; 31(12): 1615–1631.

37.

Meyer

Desbrun

Schröder

, et al.,Discrete differential-geometry operators for triangulated 2-manifolds. In: Hege

H-C

Polthier

(eds). Visualization and Mathematics III, Heidelberg: Springer-Verlag, 2003, pp. 35–57.

38.

Sorkine O and Alexa M. As-rigid-as-possible surface modeling. Symposium on Geometry Processing Barcelona, Spain, 4–6 July,2007, pp.109–116.

39.

Daanen

Psikuta

3D body scanning. In: Nayak

Padhye

(ed). Automation in garment manufacturing, Amsterdam, Netherlands: Elsevier, 2018, pp. 237–252, 2018. .

40.

. Skeleton driven animation based on implicit skinning. Comput Graphics 2011; 35(5): 945–954.