Computational design of shape-changing robotic mannequin based on 3D human models

Abstract

A shape-changing robotic mannequin is a mechatronic robot used to simulate different human body shapes. With such a robot, the dressing effects of a ready-made garment on different human bodies can be simulated and evaluated, which is very useful for garment design and garment online sales. Currently, the robotic mannequin surfaces are mostly designed into patches. However, the methodological lack of design result evaluation and shape deformation control hinders the performance of the robotic mannequins. In this paper, the robotic mannequin is computationally designed and optimized based on a large number of three-dimensional scanned human bodies, which endows the robot with a high simulation capability. The robot is designed into three layers comprising a skin layer composed of patches, a muscle layer consisting of shape-controllable elastic bars, and a skeleton layer made of linear actuators. The skeleton layer controls the overall shape change of the robot, and the muscle layer attached on the skeleton adjusts the bending of the skin layer, which makes the robot deformation predictable and controllable. A prototype of the robotic mannequin has been made on which the simulations of various human bodies have been experimented, and examples of predicting the dressing effects of ready-made garments on different human bodies via augmented virtual try-ons have been experimented as well.

Keywords

robotic mannequin computational design shape-changing human models

Objects with the ability to change their physical shape such as shape-changing interfaces have been receiving increasing attention.¹ In this paper, a novel shape-changing robotic mannequin that can simulate the shapes of human bodies is designed. It can be applied to garment design and retailers. By putting a garment onto the robotic mannequin, its dressing effects on various human bodies can be simulated by deforming the robotic mannequin. This allows a fashion designer to check whether a designing garment is suitable for the customer. It also allows people to perceive garment dressing effects on themselves when shopping online, which is very useful to reduce the high return rate caused by mismatched sizes.

There are two key problems in robotic mannequin research. The first one is about structure design: how to make the robotic mannequin have a strong possibility in simulating various human body shapes. The second is about shape simulation: how to deform the robotic mannequin to effectively simulate the objective shapes. To address these issues, the goal of this paper is to propose a new methodology for computationally designing the structure of a robotic mannequin, and to provide methods on how to predict and control the shape deformation of the robotic mannequin, so that the robotic mannequin is capable of simulating a large variety of human bodies shapes. Also, a method on how to apply the robotic mannequin to simulate the dressing effects of a garment on different human bodies is proposed to demonstrate the potential applications of the robotic mannequin.

Studies have been carried out for exploring robotic mannequin structure.^2–6 Most of them are patches-based, that is, the surface of a robotic mannequin is composed of several patches. In the literature,^3,5 the patches are rigid, and the robot is deformed by displacing the patches. Consequently, the silhouettes of the deformation results are usually not smooth. Abels et al.² used elastic rods to link the corners of adjacent flexible patches. By pushing or dragging the elastic rods, the rods slide and bend over the flexible patches, which force the patches to bend. Such a structure increases the smoothness of deformation; however, the loose coupling deformation between the rods and the patches makes it hard to predict the deformation results. Recently, Li et al.⁶ proposed a new robotic mannequin that imitates the feature curves on three-dimensional (3D) human models by flexible belts. Their approach simulates the wireframes of human bodies. This paper improves their work by adding a computationally designed skin layer and proposing a method to make the deformation of the robotic mannequin predictable and controllable.

In simulating slim human bodies, the patches on a robot tend to move inward, which might cause interference between adjacent patches; conversely, in simulating plump human bodies, the patches tend to move outward, and gaps between adjacent patches inevitably affect the simulation accuracy. Therefore, it is important to predict the simulation results; especially, interference between adjacent patches should be predicted and prevented. However, most of the existing approaches lack the feasible method of predicting simulation results of the robotic mannequin.

To enhance the simulation ability of the robotic mannequin, in this paper the robot is computationally designed into three layers based on a large number of 3D scanned human models. Firstly, as shown in Figure 1(a), the skin layer composed of patches is iteratively designed and optimized by evaluating its shape errors in virtually simulating 3D scanned human bodies. The ideal translations and bending deformations of patches are used in simulating various human bodies to design the skeleton layer and muscle layer, respectively. Secondly, the muscle layer is designed as elastic bars, with one side enveloped by flexible belts and attached to the patches and the other side joined on linear actuators. When the flexible belts are tensioned, they force the elastic bars to be bent, which further deforms the patches. Thirdly, the linear actuators are spatially distributed to form the skeleton layer and control the displacements of the patches. Thus, both the displacements and bending of patches can be controlled, which makes the robotic mannequin capable of simulating a large variety of human body shapes.

Figure 1.

Design of a robotic mannequin and its application in a garment dressing effect simulation: (a) designing the robotic mannequin into three layers based on 3D scanned human bodies, where ① is the device for controlling the flexible belt length, ② is the linear actuator, ③ is the elastic bar, and ④ is the flexible belt; (b) prototype; (c) target human body; (d) driving the robot and the garment on it by the target human body; and (e) transferring the garment image onto the target human body to show the garment dressing effect.

After the robot is made, as shown in Figure 1(b), it can simulate the shapes of human bodies, as shown in Figure 1(c) and (d). The garment on the robot deforms just like when dressing the target human body. Thus the dressing effect on the target human body can be simulated in an augmented reality way by transferring the image of the garment on the robot, as shown in the bottom image in Figure 1(d), onto the target human body image, as shown in Figure 1(e).

Our approach has the following features:

A three-layered robotic mannequin has been designed, comprising a skeleton layer, a muscle layer and a skin layer, which respond to the overall shape change, local bending and surface deformation, respectively. Each layer is computationally designed based on a large number of 3D scanned human models, which enables the robot to simulate human bodies in a large variety of shapes.

The robotic mannequin surface is controlled as a parametric surface by taking the endpoints of the linear actuators on the skeleton layer and the shape controllable flexible belts on the muscle layer as the control points and control curves, which make the robotic mannequin deformation predictable and controllable.

Based on the robotic mannequin, the deformations of a garment on various human bodies can be simulated, which provides a reliable solution to perceive the dressing effects of a garment on different human bodies. A method of displaying the dressing effects via augmented virtual try-ons has been proposed.

Related works

Our work relates to shape-changing devices,^1,7,8 3D human simulation⁹ and virtual try-on systems.^10,11 The related works are briefly reviewed in these areas.

Shape-changing interface

A growing body of research work on shape-changing devices has been carried out. The most relative works are on shape-changing displays and soft robotics.

Shape-changing displays represent virtual data through reconfigurable geometry, so that users can perceive the data more intuitively.^1,12 Most of the existing shape-changing displays are actuation pin-based. Densely arranged linear actuators are used to represent 2.5-dimensional (2.5D) shapes by adjusting their displacements.^13–15 To get a continuous surface, the actuation pins are usually covered by a piece of cloth.¹⁶ But cloth is too soft to build a smooth curved surface.

Comparatively, the shapes of human bodies are 3D rather than 2.5D, and rather than displacing, bending of the robotic mannequin surface is required to simulate various human body shapes. This requires a more complicated mechatronic structure than most of the shape-changing interfaces.

Soft robots are generally made of soft materials.⁸ They have infinite degrees of freedom and can be bent, twisted and expanded in a large range. Most of them are driven by variable-length tendons or fluidic actuation.¹⁷ The variable-length tendons are usually in the form of tension cables¹⁸ or shape-memory alloy actuators,^19,20 and they are embedded in soft robots to achieve body movements. Fluidic actuation is applied to inflate the inner channels to deform a soft robot.^21,22 Other actuators produced by intelligent materials have also been designed for soft robots, such as dielectric active polymer²³ and shape-memory polymer.²⁴ These materials are deformed under external physical fields. The soft materials provide infinite degrees of freedom and make the control of soft robots very challenging. This paper tries to find a practical solution to the inverse shape simulation of the robotic mannequin, that is, computing the parameters of the actuators to make the shape deformation of the robotic mannequin approximate the target human body.

3D human model simulation

A robotic mannequin can be regarded as the mechanization of a shape deformable human model. At present, methods of human model simulation can be classified into two categories. One category is interactive geometry deformation, which uses a parametric surface to control the shape of a human model.^25,26 The other category is built upon statistical analysis of 3D scanned human models.^9,27 By training the mapping between the feature parameters and the statistical human model,^28,29 the shape of a human model can be naturally edited with parameters, such as body dimensions.

Comparatively, a 3D human model can have a large number of degrees of freedom to describe the local shape details; however, for a robotic mannequin, the number of actuators that can be installed is very limited due to the small inner space of the robot. Thus in designing a proper mechanical structure it is very important to endow the robotic mannequin with a high simulation capability using just a few actuators.

Virtual try-on

In recent years, online sales have become a main channel for garment retailers. Technologies that can conveniently evaluate the dressing effects of a garment on individuals without physically wearing it are eagerly needed. Virtual try-on systems have been explored to meet this requirement. Technically, these systems can be classified into two categories. The first category is for garment customization, in which the sizes of a garment are adjusted according to the body shapes of individuals.^10,30–32 The second category is evaluating the dressing effects of ready-made garments on individuals with the sizes of the garments fixed,¹¹ which are very useful for garment retailers. Physics-based clothing simulation is usually applied to deform the garment.^11,33 However, it is still very difficult to get highly realistic simulation results, especially when the garments are made of multiple materials and with rich geometric details. Moreover, the retailers usually lack the design information of the ready-made garments; how to efficiently and economically digitalize theses garments is still an open question.

Image-based virtual try-on approaches have also been proposed.^34,35 Generally, a garment image is taken from a dressed reference human body, and then it is deformed to fit the customer’s figure. However, such an approach lacks the physical mechanism and cannot reflect the real garment deformation, especially when the body shape difference between the reference human body and the customer is large. Learning-based approaches have been proposed for virtual try-on,³⁶ but they are also not physical. The key to improve the reality of image-based try-on is to reduce the body shape difference between the reference human body and the customer. In this paper, such a problem is solved by adopting a shape-changing robotic mannequin to simulate the customers’ body shapes; the garment images from the robots are then taken for further transferring, as will be detailed in the “Experimental results.”

Computational design of the skin layer

Methodologically, the patches on the skin layer and their ideal deformations for mimicking various human models are firstly designed, and then the skeleton layer and the muscle layer are designed to drive the patches to approximate the ideal deformations.

Design principle

The skin of the robotic mannequin should have the physical characteristics of super elasticity and proper bending stiffness, so that it can be stretched and bent to simulate different human body shapes and the results are stiff enough to bear the weight of garments. Our approach meets this requirement by designing the skin layer into two sub-layers. The first sub-layer is composed of patches that are bendable but hard to stretch (see Figure 1(a) and (b)). The second sub-layer is an elastic tight garment covering the patches to make a continuous surface.

To simulate human bodies, the patches need to be displaced and bent. Interference between adjacent patches occurs when the sizes of any gaps become zero or negative in the simulation of small-sized human bodies. Enlarging the initial gaps’ sizes can reduce the number of interference occurrences. However, it is a double-edged sword, because the gaps tend to be too large when simulating large-sized human bodies. To find a reasonable balance, the patches are interactively optimized by evaluating the virtual simulation against 3D scanned human bodies. The flow chart in Figure 2 comprises three steps and each step is further detailed in the section “Computation design.” For description clarity of the three steps, several symbols are defined in Table 1.

Figure 2.

Flow chart of the skin layer design.

Table 1.

Definition of symbols

Symbol	Description
$\bar{H}$	Template human model whose shape is initialized as the average shape of human models and optimized later
H_i	An individual human model
$\bar{R}$	Surface of robotic mannequin, composition of 3D patches, designed on $\bar{H}$
R_i	Deformed $\bar{R}$ after the simulation of H_i
A_i	3D patches on H_i and have the same mesh structure with $\bar{R}$

3D: three-dimensional.

Step 1. Initial design. $\bar{R}$ is initially designed by drawing cutting curves on $\bar{H}$ , and then trimming $\bar{H}$ , as shown in Figure 2(a). The patches of $\bar{R}$ are parametrically encoded on $\bar{H}$ , and then its counterparts A_i on an individual human model H_i can be decoded, as shown in Figure 2(b) and (c), respectively.

Step 2. Design evaluation. Patches on $\bar{R}$ are displaced and bent to fit A_i, as shown in Figure 2(d). Each simulation result is evaluated by detecting the interferences between adjacent patches and by computing the shape error against the ground truth.

Step 3. Design refinement. Based on the evaluation results, the design is refined to enhance the simulation capability of the robotic mannequin, as shown in Figure 2(e) and (f).

Computational design

The processing steps of the design principle are detailed as follows.

Initial design

$\bar{H}$ is initially set as the average shape of thousands of 3D scanned human bodies provided by Pishchulin et al.³⁷ On $\bar{H}$ , curves are drawn to cut out patches to get $\bar{R}$ , as shown in Figure 3(b) and (c). These patches will be made of bendable but hard to stretch materials. The patches are translated and bent accordingly, when driving the robotic mannequin to simulate targets. The effective shape of the robotic mannequin in simulation can be regarded as the joining of the patches. To make the robotic mannequin able to simulate various shapes of human bodies, the cutting curves are designed to be approximately orthogonal to the gradient of shape difference between various human bodies. Physiologically, the shape difference between various human bodies is mainly reflected by two aspects:

The height or width difference caused by the length of bones, such as the body height, shoulder width, and so on;

The girth difference caused by muscles and fat, such as the waist girth, chest girth, and so on.

Figure 3.

Design and simulation of patches: (a) human body and its skeleton; (b) drawing boundary curves of three-dimensional patches on $\bar{H}$ ; (c) trimming $\bar{H}$ with curves to get $\bar{R}$ ; (d) human model H_i; (e) recovering patches on H_i to get A_i; (f) aligning each patch of $\bar{R}$ to A_i; (g) bending each patch of $\bar{R}$ to fit H_i; (h) detecting interferences between adjacent patches on $\bar{R}$ in simulation (the interfered vertices are rendered in red).

In the first aspect, the cutting curves are designed to be approximately orthogonal to the skeleton (as shown in Figure 3(a)), such as the horizontal curve segments on the waist–hip part and the V-shaped curve segments on the shoulder–neck part, as shown in Figure 3(b). In the second aspect, as the muscles are attached on the skeleton and the fat covers the muscles, the girth curves are almost orthogonal to the skeleton; therefore the relative cutting curves are designed approximately parallel to the skeleton, such as the longitudinal curves on the waist–hip part and the curve segments on the shoulder part that are almost parallel to the shoulder silhouette. Moreover, the shape features of the human body should be preserved, such as the contours of female breast. With the above considerations, the cutting curves are resulted in Figure 3(b).

The gaps between patches are initially designed to have an equal width. $\bar{R}$ is encoded on $\bar{H}$ by parameterizing each vertex v_i∈ $\bar{R}$ as (t_j, b _j ) where t_j is the jth triangle on the mesh of $\bar{H}$ that contains the projection of v_i, and b _j is the barycentric coordinates of the projection on t_j. For a human model as shown in Figure 3(c), by using {(t_j, b _j )}, its patches, A_i, can be decoded, as shown in Figure 3(d).

Design evaluation

To simulate a human body H_i with its patches A_i (Figure 3(d) and (e)), $\bar{R}$ (Figure 3(c)) is deformed as follows.

Firstly, each patch $P_{j}^{\bar{R}}$ on $\bar{R}$ is rigidly aligned to its counterpart on A_i, as shown in Figure 3(f). The alignment is achieved by finding a translation $(d_{i, j}^{-} t_{j}^{-} + d_{i, j}^{⊥} t_{j}^{⊥})$ where $t_{j}^{-}$ and $t_{j}^{⊥}$ are the normalized horizontal direction and normalized vertical direction, respectively, and $d_{i, j}^{-}$ and $d_{i, j}^{⊥}$ are the value of displacements under these two directions, respectively. This translation meets

\begin{matrix} {min}_{d_{i, j}^{-}, t_{j}^{-}, d_{i, j}^{⊥}, t_{j}^{⊥}} (\sum_{i} \sum_{k} ‖ p_{j, k} + (d_{i, j}^{-} t_{j}^{-} + d_{i, j}^{⊥} t_{j}^{⊥}) - q_{i, j, k} ‖^{2}), \\ d_{i, j}^{⊥} = d_{i, m}^{⊥}, \forall (P_{j}^{\bar{R}}, P_{m}^{\bar{R}}) \in L \end{matrix}

(1)

where

p_{j, k}

is the position of the kth vertex on

P_{j}^{\bar{R}}

, and

q_{i, j, k}

is its counterpart on A_i;

(P_{j}^{\bar{R}}, P_{m}^{\bar{R}}) \in L

means that patch

P_{j}^{\bar{R}}

and patch

P_{m}^{\bar{R}}

are on the same horizontal section, and they should have the same vertical displacement in simulation.

{ $t_{j}^{-}$ } and { $t_{j}^{⊥}$ } are used to guide the design of spatial distribution of linear actuators on the skeleton layer, as will be detailed in the section “Design of the skeleton layer and muscle layer.” Ideally, for each target human model, $t_{j}^{-}$ should be altered adaptively, which requires more than one actuator to work cooperatively to produce $t_{j}^{-}$ . Due to the limited installation space inside the robotic mannequin, $t_{j}^{-}$ is set as a constant direction, and is calculated from equation (1) in a least squares manner under a large number of target human model samples. Thus the horizontal displacement of each patch is controlled by only one actuator.

Secondly, $P_{j}^{\bar{R}}$ is bent to fit the surface of H_i, as shown in Figure 3(g). By taking the patches as bendable but non-stretchable cloth, they are simulated physically based on a mass-spring model. For each vertex on $P_{j}^{\bar{R}}$ , its closest point on H_i is extracted as the target point to guide the bending. Small step is set in simulation iteration to gradually fit H_i.

Thirdly, the simulation result, R_i, is evaluated. Interferences between adjacent patches on R_i are detected and the shape error between R_i and H_i is calculated.

As shown in Figure 3(h), for each vertex on R_i, a ray is cast along its normal direction to check whether it intersects the adjacent patch. If any intersection exists, the local region around the vertex tends to be interfered. Such vertices are colored in red in Figure 3(h). This interference detection is accelerated by embedding the simulation result into an octree, and only the interaction between the ray and the triangles on the patches that are inside or intersect with the same leaf node on the octree is computed.

When simulating large-sized human bodies, gaps on $\bar{R}$ tend to enlarge. These gaps are filled by the elastic tights in physical situations, as shown in Figure 1(d) and (e). In the design stage, this physical case is virtually simulated as the interpolating transitional surfaces between the boundaries of adjacent patches, and it is achieved by triangulating the boundaries, as shown in Figure 4(c) and (d). To compute the shape error between the ground truth and the simulation result, these two models are embedded into an octree; for each vertex on the simulation result, its nearest point on the ground truth is searched out by projecting it onto the triangles of the ground truth that are in the same leaf node on the octree. The shape error indicated by such point-to-surface distance is shown in Figure 4(e).

Figure 4.

Calculating the shape error caused by gaps: (a) human model H_i; (b) deforming $\bar{R}$ to simulate H_i; (c) filling gaps by triangulating the boundaries of patches; (d) simulation result; and (e) shape error between the simulation result and the ground truth.

Design refinement

To simulate various 3D scanned human bodies, $\bar{R}$ is deformed accordingly. For each vertex on $\bar{R}$ , the times of interferences in the simulation results are calculated and visualized, as shown in Figure 5(b). In the figure, the redder the color of a local region, the more likely it is to be interfered in simulating human bodies. To reduce the interference, it is reasonable to remove these “dangerous” local regions from the patches, which is achieved by shrinking the boundaries of the patches inward, as shown in Figure 5(c). Consequently, it effectively reduces the interference occurrences, as resulting in Figure 5(d), which makes the robot able to simulate more small-sized human bodies. However, it also enlarges the initial gaps between adjacent patches, which will increase the shape error in simulations. A balance between the interference occurrences and the shape accuracy in simulations can be set. In our experiments, the threshold point-to-surface shape error is set at 5 mm. Subsequently, the human shape space is classified into several sub-spaces for robotic mannequin design, as shown in Figure 2(f).

Figure 5.

Optimizing the design of the skin layer: (a) initial design; (b) visualization of interference times; (c) re-designing patches with the assistance of interference visualization; and (d) interference visualization of the new design.

Design of the skeleton layer and muscle layer

The skeleton layer controls the patches’ displacements and is designed into two parts. One is the shoulder part, and the other is the part below the shoulder.

For the shoulder, its skeletal shape is determined by neck circumference, shoulder width, shoulder thickness, its vertical slope and its horizontal slope. As illustrated in Figure 6, the neck circumference is controlled by the endpoints of four radially distributed racks. These racks are paired with gears and are driven by the stepper motor ①. The thickness of the shoulder is controlled by four linear actuators which are made of screw pairs and controlled by four stepper motors: motors ② and ⑥. Similarly, the width of the shoulder is controlled by motors ③; the left and right vertical slopes of the shoulder are controlled by motors ④, and the horizontal slopes of the shoulder are controlled by motors ⑤. The horizontal slopes and the vertical slopes of the shoulder are reflected by the ridge bars on the top of the shoulder which are hinged to the base by ball joints.

Figure 6.

Skeletal structure of shoulder: (a) structure of the front part; (b) structure of the back part; and (c) physical prototype (the left shows the front part, the right shows the back part). ① one motor for neck circumference; ② two motors for front shoulder thickness; ③ two motors for shoulder width; ④ two motors for vertical shoulder slope; ⑤ two motors for horizontal shoulder slope; ⑥ two motors for back shoulder thickness.

The skeleton part below the shoulder is composed of chest section, waist section and hip section, as shown in Figure 1(a). On each section, radially distributed bones are attached to the vertical pole. Each bone is a linear actuator with its direction computed as $t_{j}^{-}$ in equation (1). The impelling quantity of the linear actuators changes the overall shape of the robotic mannequin.

The main task of the muscle layer is control the bending of the patches on the skin layer. Therefore, rather than designing the muscle layer with a similar appearance to the physiological muscles, in this paper, the muscle layer is functionally designed to make it capable of adjusting the curvatures of patches. The muscle layer is designed as a composition of elastic bars and flexible belts. Both of them can be bent but are hard to stretch. As shown in Figure 1(a), each flexible belt envelops several elastic bars, and the elastic bars are hinged to the linear actuators on the skeleton layer. The design of the rest of the shapes and physical characteristics of the elastic bars follows the work of Li et al.,⁶ and the elastic bars are materialized by 3D printing. Briefly, each elastic bar is designed as a curved bar with its crossing sections being rectangles. These rectangles have the same length but their width changes along the middle axis of the elastic bar, so as to make the elastic bar have the expectant bending properties. The design of the middle axis is based on a large number of feature curve segments which are extracted from 3D scanned human bodies provided by Pishchulin et al.³⁸ The shape of the middle axis is designed as the enveloping curve of these feature curve segments, so that these feature curve segments can be simulated by bending the elastic bar under the tension force of a flexible belt. Each flexible belt is rolled upon a motor-controlled drum to adjust its length and tension force. The bent elastic bars reversely shape the flexible belt. However, the work of Li et al.⁶ lacks the shape control method of the flexible belts, which is solved in the section “Inverse simulation of the robotic mannequin.”

Inverse simulation of the robotic mannequin

Anatomically, a shoulder part is mainly determined by its underlying skeleton, and the shape of the part below the shoulder is mostly reflected by the muscle or fat structures. Therefore, to simulate a human body’s shoulder, the relative linear actuators are repositioned to the intersection points between the target human body and the rays along with the linear actuators. Comparatively, the shape control of the part below the shoulder is more complicated and is representatively detailed as follows.

Algorithm framework

To each crossing section of the chest, waist and hip, its shape is simulated by a flexible belt enveloping several elastic bars jointed on the linear actuators, as shown in Figure 1(a). By controlling the length of the flexible belt and the impelling quantities of the linear actuators, the elastic bars are bent under the tension force of the flexible belt, which conversely shapes the flexible belt. As shown in Figure 7, the flexible belt’s shape is composed of the curve segments that envelop the elastic bars and the straight segments linking the curve segments. The proper positions of the linear actuators’ endpoints are computed in three steps. Firstly, they are initialized as the intersections between the target crossing section and the rays along the linear actuators. Secondly, the elastic bars are virtually bent to fit the target crossing sections. Thirdly, the shape and length error of the simulation result are computed to optimize the linear actuators’ ending positions. The detailed algorithm is presented in Table 2.

Figure 7.

Shaping the flexible belt by linear actuators: (a) flexible belt envelops the elastic bars; (b) equilibrium state of the elastic bars; and (c) elastic bars are bent under the tension force of the flexible belt.

Table 2.

Algorithm for solving the ending positions of linear actuators

Step 1. Computing the curve length of the target crossing section, L₀. Initializing the positions of the linear actuators’ endpoints as the intersections between the rays along with the linear actuators and the target crossing section. Step 2. Setting the tension force on the flexible belt, f, as an infinite decimal near to zero, δ, and then computing the elastic bars’ postures. Step 3. For each elastic bar, to fit the target crossing section, computing the ideal tension force on the flexible belt. Among them, obtain the maximal one, F, and set △f = F/n as step length (n = 30 in our case). for(i = 1; i < n; i++) { Step 4. Let f = δ + i*△f. for(j = 0; j < m; j++) /* m = 15 in our experiments*/ { Step 5. Compute each elastic bar’s bending. Step 6. Update the posture of each elastic bar. if (the maximal pose change is smaller than a threshold (1.5° in our case)) break; } Step 7. Compute shape difference between the elastic bars and the target crossing sections, η _j . if(η _j >

η

_j _-1) break; } Step 8. Computing the enveloping curve length of the elastic bars, L. Scale the impelling quantities of the linear actuators by r = L/L₀.

In the algorithm, the key steps are computing the balance postures (in Step 2 and Step 6) and the bending of the elastic bars (in Step 3 and Step 5). These two aspects are coupled tightly and nonlinearly. To make it solvable, they are decoupled and computed alternatively by keeping one constant when computing another, which will be detailed in the sections “Computing the balance poses of the elastic bars” and “Computing the bending deformation of the elastic bars.”

Computing the balance poses of the elastic bars

To compute the balance of the elastic bars, the force distribution of the elastic bars is illustrated in Figure 7(b), where $P_{i}^{1}$ , $P_{i}^{2}$ and J_i are the two endpoints and the joint point of the ith elastic bar, respectively, $F_{i}^{k}$ is the tension force on $P_{i}^{k}$ , $θ_{i}^{k} = ∠ O J_{i} P_{i}^{k}$ and $L_{i}^{k} = ∥ J_{i} P_{i}^{k} ∥$ , (k = 1,2). The equilibrium equation is

\begin{matrix} F_{i}^{2} \times (P_{i}^{2} - J_{i}) = F_{i}^{1} \times (P_{i}^{1} - J_{i}), F_{i}^{1} = f \frac{P_{i - 1}^{2} - P_{i}^{1}}{∥ P_{i - 1}^{2} - P_{i}^{1} ∥}, \\ F_{i}^{2} = f \frac{P_{i + 1}^{1} - P_{i}^{2}}{∥ P_{i + 1}^{1} - P_{i}^{2} ∥} \end{matrix}

(2)

subject to

P_{i}^{k} = ((J_{i})_{x} \pm L_{i}^{k} cos (θ_{i}^{k})

(J_{i})_{y} \pm L_{i}^{k} sin (θ_{i}^{k})

θ_{i} = θ_{i}^{1} + θ_{i}^{2}

where f is the value of tension force on the flexible belt,

θ_{i} = ∠ P_{i}^{1} J_{i} P_{i}^{2}

is constant in adjusting the poses of the elastic bars, and

(J_{i})_{x} and (J_{i})_{y}

are the x and y coordinates of J _i .

From equation (2), { $θ_{i}^{k}}$ can be computed, thus the pose of each elastic can be determined. Equation (2) is solved by using Newton’s iteration method. To make a quick convergence, $θ_{i}^{k}$ is initialized by rigidly registering the outside contour of each elastic bar to its counterpart on the target crossing section.

Computing the bending deformation of the elastic bars

To bend an elastic bar to simulate its target curve segment, as shown in Figure 7(c), the required bending moment on each section orthogonal to the neutral axis can be computed as

M (s) = \int_{- h / 2}^{h / 2} E ɛ yb (s) dy, ɛ = \frac{(ρ_{1} (s) - ρ_{2} (s)) y}{(ρ_{1} (s) + y) ρ_{2} (s)}

(3)

where s is the curve length from the current section to the end section of the elastic bar, y is the distance from a point to the neutral layer, ɛ is the strain on the point,

ρ_{1} (s)

and

ρ_{2} (s)

are the curvature radii of the neutral layer before and after bending, respectively, and b(s) is the width of the section.

On the other hand, the bending moment on the section can also be computed as

M = F_{i}^{k} \times Δ r (s), Δ r (s) = r (s) - r (0)

(4)

where r(s) = (x(s), y(s), z(s)) is the parametric curve representation of the outside contour of the elastic bar.

Applying equations (3) and (4) to each section on the elastic bar, the ideal tension force of the flexible belt can be computed. As the tension force should be equal along the flexible belt, the expectational tension force is set as the average. Consequently, under the expectational tension force, the curvature radius $ρ_{2} (s)$ along each elastic bar’s neutral layer can be computed by equation (3).

To compute the shape of deformed elastic with its curvature $ρ_{2} (s)$ , the middle axis of the neutral layer is set as a cubic curve

y = {ax}^{3} + {bx}^{2} + cx

(5)

where a, b, c are the coefficients to be determined.

When these coefficients are calculated, the shape of the deformed elastic bar can be obtained. The curvature along this curve can be calculated as

\frac{1}{ρ_{2} (s)} = \frac{d^{2} y / {dx}^{2}}{(1 + (dy / dx)^{2})^{3 / 2}} = \frac{6 ax}{(1 + (3 {ax}^{2} + 2 bx + c)^{2})^{3 / 2}}

(6)

Equation (6) is nonlinear to a, b and c. To make the equation solvable, it is rewritten as

\frac{1}{ρ_{2} (s)} = \frac{6 a_{2} x}{(1 + (3 a_{1} x^{2} + 2 bx + c)^{2})^{3 / 2}} subject to a_{1} = a_{2}

(7)

The coefficients in the numerator and denominator of equation (7) are optimized alternatively and iteratively. The detailed algorithm is described in Table 3.

Table 3.

Algorithm for computing the bending deformation of an elastic bar

Step 1. Coefficients a, b, c are computed by fitting the cubic curve in equation (5) to the middle axis curve of the elastic bar in its rest state; Step 2. Substitute a, b, c into equation (7), let a₁ = a, compute a₂ as

a_{2} = \frac{(1 + (3 a_{1} x^{2} + 2 bx + c)^{2})^{3 / 2}}{6 x ρ_{2} (s)}

(8) Step 3. Let a = a₁ = a₂, rewrite equation (7) as a linear function of b and c,

2 bx + c = ((6 a_{2} x ρ_{2} (s))^{2 / 3} - 1)^{\frac{1}{2}} - 3 a_{2} x^{2}

(9) By sampling the middle axis curve of the neutral layer, b and c can be computed using the method of least squares. Step 4. Go to Step 2 and keep iterating until the differences of the resulting coefficients between two sequential iterations are smaller than the thresholds.

Experimental results

Prototyping the robotic mannequin

A physical prototype of the robotic mannequin has been made, as shown in Figure 8.

Figure 8.

Prototype of the robotic mannequin: (a) three-dimensional (3D) patches of the skin layer; (b) flattening the 3D patches to get two-dimensional patterns and cutting the flexible material; (c) skeleton and muscle layer; (d) attaching the skin layer onto the muscle layer and skeleton layer by using the hinged platform (the top) and claw-shaped groove (the bottom); and (e) assembled robotic mannequin (the top) is covered by an elastic tight garment to form a continuum robotic surface(the bottom).

For the skin layer, the patches can be ideally materialized by injection modeling to precisely get the 3D geometric shape. However, due to its high costs, in the current experimental stage, two-dimensional (2D) flexible material is used to form the patches, as shown in Figure 8(b). The surfaces of 3D patches are flattened into 2D patterns, and then they are used to cut the material. On the areas where the Gaussian curvatures are high, cuts are inserted on the 3D patches to reduce the stretch deformation in the flattening results, as shown in the top image of Figure 8(b). For the female breasts, their surfaces are highly curved, and the breasts are produced by 3D printing to obtain smooth surfaces. As shown in Figure 8(c), the linear actuators on the skeleton layer are designed as screw pairs and are driven by stepper motors. Certainly, other kinds of linear actuators also can be adopted. The elastic bars on the muscle layer are produced by 3D printing and are hinged on the linear actuators. The skin layer is attached to the muscle layer with installation parts, as shown in Figure 8(d). Flexible belts pass through these parts freely and force the patches to bend.

Simulating individuals by the robotic mannequin

With the method proposed in the section “Computational design of the skin layer,” before physical simulation, a virtual simulation is carried out to predict whether a human body is suitable to be simulated by a certain sized robotic mannequin. As shown in Figure 9, the patches on the robotic mannequin tend to interfere with each other when simulating a very slim person (the first human body in the first row in Figure 9), and the shape error tends to be too big when simulating a large-sized human body (the last human body in the second row in Figure 9). Such cases are selected out and simulated by other sized robotic mannequins. It should be pointed out that for the typesetting, the images in the figure are scaled, thus the sizes of patches look different between the various simulation results, but actually they are the same.

Figure 9.

Predicted robotic mannequin simulation, where “√” and “×” indicate if the human body is suitable for simulation or not. The colored patches are the predicted simulation results; the gray region linking the colored patches are used to simulate the elastic tights covering the gaps; the color bar indicates the shape error in the simulation.

In simulation, feature curves on the human body are extracted to guide the shape control of the flexible belts with the method described in the section “Inverse simulation of the robotic mannequin.” Taking the chest curves for instance, as shown in Figure 10, the device simulation ability is evaluated by computing the shape differences between the flexible belt contours (the red curves) and the target chest curves (the blue curves). The shape differences are evaluated with the average point-to-curve distances. Statistically, the average distance is about 3 mm. The range of shape simulation is mostly determined by the impelling quality of the linear actuators. These actuators are designed as screw pairs as shown by the white components distributed radially in Figure 10. When the motors drive the screw rods to rotate clockwise or anticlockwise, the white screw nuts move forward or backward, which span the elastic bars and flexible belts to simulate the target human body feature curves, as shown in the second row in Figure 10. The largest movement of the middle front and middle rear screw nuts is 22 mm, and that of the middle left and middle right screw nuts is 33 mm; the movements of the oblique screw nuts change accordingly. As a result, the simulation range of the chest girth is between 783 mm and 1008 mm.

Figure 10.

Simulations of chest curves.

Figure 11 demonstrates the case of simulating a real person. To do so, firstly, the body dimensions of the target (Figure 11(a)) are measured and used to reconstruct the 3D virtual human model by adopting the method proposed by Li et al.,³⁸ as shown in Figure 11(b); secondly, the feature curves on the human model are extracted to guide the deformation of the elastic bars and flexible belts, which further drive the deformation of the patches. For comparison, the robotic mannequin is also dressed with the same shirt as the target, as shown in Figure 11(c). More simulation results are given in Figure 11(d) to (h); from the results, it can be found that the dressing effect on the robotic mannequin is very close to that on the target human bodies.

Figure 11.

Simulating the shape of a real person: (a) real person; (b) three-dimensional human model of the person; (c) driving the robotic mannequin by the human model; (d) another person; (e) robotic mannequin simulation result; and (f) to (g) driving the robotic mannequin to simulate more human bodies.

To evaluate the simulation effectiveness, as shown by the target model and the deformed robotic mannequin in Figure 12(a) and (b), respectively, eight RGB-D cameras are evenly positioned around the robotic mannequin to obtain the depth data, and then the 3D robotic mannequin model is reconstructed by using the implicit surface method,³⁹ as shown in Figure 12(c). As the postures of the robotic mannequin and the target human model have a slight difference, the skeleton driven method⁴⁰ is adopted to align the 3D human model to the robotic mannequin as shown in Figure 12(d). Before the alignment, the skeleton of the robotic mannequin is automatically built based on the positions of the linear actuators, as shown in Figure 12(c). Although there are small shape differences in local areas, the robotic mannequin simulates the target human body well on the whole (Figure 12(e)). Statistically, the shape difference of the average point-to-surface distance is about 5 mm. For visualizing the shape difference, the torso part is trimmed from the target human model, as shown in Figure 12(e), and the surface-to-surface distances between the torso and the robotic mannequin are computed and rendered to show the shape error, as shown in Figure 12(f), where the gray color indicates a small surface-to-surface distance and the red color indicates the bigger shape difference. Compared with the high precision of the feature curve simulation, as pointed out above, the surface shape error is relatively high. This is mainly due to the high Gaussian curvature variation between the target and the robotic mannequin, especially on the breast area. Such shape error is to be reduced in future work by designing a special shape-changing device to flexibly simulate the regions with high curvature variation.

Figure 12.

Simulating an individual human body by the robotic mannequin and geometrizing the garment on the robotic mannequin: (a) target human model; (b) robotic mannequin and garment deformation result; (c) three-dimensional reconstruction result of the robot and its skeleton; (d) aligning the poses of the target human model and the robot model with the guidance of skeleton; (e) comparing the torso part of the target human model with the robot model; (f) visualizing the shape difference between the target human’s torso and the robot mannequin; (g) rendering the human model and garment photo in a same image; (h) triangulating the garment image into a mesh; and (i) projecting the human model onto garment mesh (the yellow points are the projections).

It is reasonable to assume that when putting the garment on the target human body, it deforms similarly as it deforms on the robotic mannequin that simulates the target human body shape. By taking the feature curves on the 3D target human model and the flexible belts on the robotic mannequin as the shape correspondence, the garment image and the 3D human model are re-rendered in the same image. As a result, the garment well fits the human model, as shown in Figure 12(g). The garment image paired with a 3D human model is called garment-human-model-paired (GHMP) data.

In GHMP data, the geometric relation between the garment image and the 3D human model is further set in two steps. Firstly, the garment image is triangulated into a mesh, as shown in Figure 12(h); secondly, the 3D human model is projected onto this mesh, as shown in Figure 12(i), where the yellow dots are the projections of the vertices of the human model. Each projection is parameterized as (t_j, b _j ) where t_j is the triangle on the garment image mesh containing the projection, and b _j is the barycentric coordinates of the projection on t_j. The applications of GHMP data will be detailed in the section “Applications on virtual try-on.”

Applications on virtual try-on

Fitting a garment onto individuals

By driving the robotic mannequin to simulate various human bodies, a collection of GHMP data can be built (see the examples in Figures 11 and 13).

Figure 13.

A garment in different states. The first row shows deforming the garment by driving the robotic mannequin. Their corresponding human body shapes are shown in the second row.

In online applications, when a customer wants to view a garment’s dressing effect, they are asked to submit their body feature sizes and a picture. Accordingly, our system estimates their 3D body shape in the picture, as shown in Figure 14(b), and chooses the GHMP data whose human model shape is closest to the customer’s body shape. For clarity, the human models of GHMP and the customer are denoted as H^G and H^C, respectively, and the garment image relative to H^G is denoted as I^G. Because H^G and H^C are close in shape, it is reasonable to assume that the garment dressing effect on H^C is close to I^G. Therefore, I^G can be transferred onto H^C by slightly altering I^G to meet the shape of H^C. It is implemented by projecting H^C onto the image plane and setting the projections as the target positions of {(t_j, b _j )} in GHMP data, and then minimizing the following energy

{min}_{v_{i}} (λ_{1} \sum_{k} α v_{j, 0} + β v_{j, 1} + γ v_{j, 2} - p_{j}^{2} + λ_{2} \sum_{i} L (v_{i}) - δ_{i}^{2})

(11)

where b _j = (α,β,γ); v _j _,0, v _j _,1, and v _j _,0 are three vertices’ coordinates of triangle t_j;

p_{j}

is the target position of (t_j, b _j ); L(v_i) and δ _i are the Laplacian operator and Laplacian coordinate of vertex v_i on the mesh of the garment image in GHMP, M^G.

Figure 14.

Transferring a garment image to a customer: (a) frontal image of a user; (b) human body shape estimation; (c) deforming the garment image by body shape difference; (d) the deformed garment image on the customer; and (e) overlaying the garment image on the customer’s image to show the virtual try-on result.

As a result, the garment image is deformed, as shown in Figure 14(c), and well fits the customer’s body shape, as shown in Figure 14(e).

Comparison with existing image-based virtual try-on approaches

Most of the existing image-based virtual try-on approaches follow the strategy of deforming one garment image to fit all customers regardless of their body shape varieties.^34,35 In such cases, the dimensions of garments are neglected, which makes it difficult to simulate the sense of a ready-made garment that is loose for a small-sized customer or tight for a large-sized one. In our approach, the robotic mannequin is used to simulate various shapes of human bodies, thus the dressing effects of a garment on different human bodies can be simulated, which makes our approach more realistic and reliable.

For evaluation, a garment is fitted onto different shapes of human bodies. To do so, the images of the garment on the robotic mannequin in different shapes are taken, as shown in the first row in Table 4 (also shown in Figure 13). These garment images are fitted onto the images of three human bodies that have slim, normal and plump figures by adopting equation (11). The human bodies are shown in the first two columns in Table 4. Consequently, nine results are obtained, as shown in the last three columns in the table. Each column gives the results of fitting one garment image to different human bodies, which are similar to most of the outcomes from existing image-based approaches.^34,35 Accordingly, transferring the garment image taken when the robot is in a small-sized shape to a plump customer, the result will look too loose and less realistic, as shown in the bottom row in the third column. Conversely, transferring the garment image taken when the robot is in a plump state to a slim customer, the result will appear too tight, as shown in the second row in the fifth column. As the robotic mannequin simulates the shape of each customer accordingly, it makes the dressing effect simulation results more realistic, as shown in the diagonal elements in the last three rows and three columns each of which is flagged with “√√√.”

Table 4.

Transfer garment in different deformed states onto human bodies

Human body images	Estimated 3D human models	Garment transfer: state I	Garment transfer: state II	Garment transfer: state III

The greater the number of symbol “√,” the more realistic the virtual try-on result.

3D: three-dimensional.

Conclusion

This paper has designed a simulation predictable and controllable shape-changing robotic mannequin in three layers. The skeleton layer and muscle layer are designed as linear actuators controlled tensioned flexible belts enveloping the elastic bars; the flexible belts serve as feature curves to constrain the shape of the skin layer, which is composed of several patches. The robotic mannequin is computationally designed and evaluated based on a large number of 3D scanned human models, which endows the robotic mannequin with the capability of simulating a large range of human body shapes. Methods for predicting and controlling the robotic mannequin simulation have been proposed.

Our robotic mannequin can be used to simulate the dressing effects of ready-made garments on different human bodies, based on which exemplary applications of image-based augmented virtual try-on have been presented. Our approach provides a possible solution to helping customers to perceive dressing effects of garments when shopping online.

More effort is needed to improve the performances of our robotic mannequin. Firstly, to enhance the robotic mannequin’s ability to simulate shape details of the targets, more elastic bars and patches are to be installed, especially in the areas with high Gaussian curvature variations, such as the belly and hip. However, due to the physical sizes of the components such as the motors and linear actuators, as well as the limited inner space of the robotic mannequin, it is hard to install more elastic bars and patches. To address this issue, linear actuators with smaller sizes and higher power are expected to be designed and used. Secondly, only the torso part has been designed; the structures of limbs are yet to be designed. Thirdly, the current robotic mannequin can only simulate the human body in the upright state; structures and devices that enable the robotic mannequin to adjust its posture are to be designed and integrated, so that the robot can simulate the human body in natural postures.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported in part by the National Key Research and Development Program of China (2018YFB1700700), the National Natural Science Foundation of China under Grants 61732015 and 51575481, and Zhejiang Provincial Natural Science Foundation of China under Grant LY18F020004, and the Fundamental Research Funds for the Central Universities under Grant 2019QNA4001, and Research Funding of Zhejiang University Robotics Institute.

ORCID iD

Jituo Li

References

Sturdee

Alexander

. Analysis and classification of shape-changing interfaces for design and application-based research. ACM Comput Surv 2018; 51: 1–31.

Abels

Kruusmaa

. Shape control of an anthropomorphic tailoring robot mannequin. Int J Hum Robot 2013; 10: 1350002, 1–16.

Guo

Zhang

, et al. The design of a form-changing female fitting robot. J Adv Mech Des Syst Manuf 2016; 10: 1–9.

Guo

Zhang

Feng

, et al. Structural design and optimization of a panel-based fitting robot. J Adv Mech Des Syst Manuf 2018; 12: 1–13.

Allan

. A new paradigm in apparel design and fitting: an interactive robotic mannequin, “I.DummyTm”. J Fash Technol Text Eng 2015; 3: 3–3.

Weng

, et al. Design of robotic mannequin formed by flexible-belt-net. Comput Aided Design 2019; 110: 1–10.

Alexander J, Roudaut A, Steimle J, et al. Grand challenges in shape-changing interface research. In: Proceedings of the 2018 CHI conference on human factors in computing systems. 2018, pp.1–14.

Lee

Kim

, et al. Soft robot review. Int J Control Autom Syst 2017; 15: 3–15.

Cheng

Chen

Martin

, et al. Parametric modeling of 3D human body shape—a survey. Comput Graph 2018; 71: 88–100.

10.

Wang

, et al. Fitting 3D garment models onto individual human models. Comput Graph 2010; 34: 742–755.

11.

Jiang

Sun

, et al. Transferring and fitting fixed-sized garments onto bodies of various dimensions and postures. Comput Aided Design 2019; 106: 30–42.

12.

Hardy J, Weichel C, Taher F, et al. ShapeClip: towards rapid prototyping with shape-changing displays for designers. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM: Seoul, Republic of Korea, 2015, pp.19–28.

13.

Follmer S, Leithinger D, Olwal A, et al. inFORM: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of the 26th annual ACM symposium on user interface software and technology. ACM: St. Andrews, Scotland, United Kingdom, 2013, pp.417–426.

14.

Jang S, Kim S, Tanner K, et al. Haptic edge display for mobile tactile interaction. In: Proceedings of the 2016 CHI conference on human factors in computing systems. 2016, pp.3706–3716.

15.

Leithinger D, Follmer S, Olwal A, et al. Sublimate: state-changing virtual and physical rendering to augment interaction with shape displays. In: Proceedings of the SIGCHI conference on human factors in computing systems. 2013, pp.1441–1450.

16.

Leithinger D, Lakatos D, DeVincenzi A, et al. Direct and gestural interaction with relief: a 2.5D shape display. In: Proceedings of the 24th annual ACM symposium on user interface software and technology. ACM, 2011, pp.541–548.

17.

Rus

Tolley

. Design, fabrication and control of soft robots. Nature 2015; 521: 467–475.

18.

Calisti

Giorelli

Levy

, et al. An octopus-bioinspired solution to movement and manipulation for soft robots. Bioinspir Biomim 2011; 6: 036002–036002.

19.

Seok

Onal

Cho

, et al. Meshworm: a peristaltic soft robot with antagonistic nickel titanium coil actuators. IEEE-ASME Trans Mechatron 2013; 18: 1485–1497.

20.

Laschi

Cianchetti

Mazzolai

, et al. Soft robot arm inspired by the octopus. Adv Robotics 2012; 26: 709–727.

21.

Mosadegh

Polygerinos

Keplinger

, et al. Pneumatic networks for soft robotics that actuate rapidly. Adv Funct Mater 2014; 24: 2163–2170.

22.

Wakimoto

Suzumori

Ogura

. Miniature pneumatic curling rubber actuator generating bidirectional motion with one air-supply tube. Adv Robotics 2011; 25: 1311–1330.

23.

Shen

Trabia

Stalbaum

, et al. A multiple-shape memory polymer-metal composite actuator capable of programmable control, creating complex 3D motion of bending, twisting, and oscillation. Sci Rep 2016; 6: 24462–24462.

24.

Firouzeh A, Salerno M and Paik J. Soft pneumatic actuator with adjustable stiffness layers of multi-dof actuation. In: IEEE/RSJ international conference on intelligent robots and systems. 2015, pp.1117–1124M.

25.

Kasap

Magnenat-Thalmann

. Skeleton-aware size variations in digital mannequins. Vis Comput 2011; 27: 263–274.

26.

Hsiao

Chen

. A study of surface reconstruction for 3D mannequins based on feature curves. Comput Aided Design 2013; 45: 1426–1441.

27.

Hasler

Stoll

Sunkel

, et al. A statistical model of human pose and body shape. Comput Graph Forum 2009; 28: 337–346.

28.

Koo

Park

Choi

, et al. Example-based statistical framework for parametric modeling of human body shapes. Comput Ind 2015; 73: 23–38.

29.

Chu

Tsai

Wang

, et al. Exemplar-based statistical model for semantic parametric design of human body. Comput Ind 2010; 61: 541–549.

30.

. Modeling 3D garments by examples. Comput Aided Design 2014; 49: 28–41.

31.

Bartle

Sheffer

Kim

, et al. Physics-driven pattern adjustment for direct 3D garment editing. ACM Trans Graph 2016; 35:Article 50–Article 50.

32.

. Customizing 3D garments based on volumetric deformation. Comput Ind 2011; 62: 693–707.

33.

Sheffer

Grinspun

, et al. FoldSketch: enriching garments with physically reproducible folds. ACM Trans Graph 2018; 37: 1–13.

34.

Shilkrot

Cohen-Or

Shamir

, et al. Garment personalization via identity transfer. IEEE Comput Graph Appl 2012; 33: 62–72.

35.

Hauswiesner

Straka

Reitmayr

. Virtual try-on through image-based rendering. IEEE Trans Vis Comput Graph 2013; 19: 1552–1565.

36.

Han X, Wu Z, Wu Z, et al. VITON: an image-based virtual try-on network. In: IEEE conference on computer vision and pattern recognition. 2018, pp.7543–7552.

37.

Pishchulin

Wuhrer

Helten

, et al. Building statistical shape spaces for 3D human modeling. Pattern Recognit 2017; 67: 276–286.

38.

, et al. Measuring and modeling human bodies with a novel relocatable mechatronic sensor-net. Text Res J 2019; 89: 4131–4147.

39.

Izadi S, Kim D, Hilliges O, et al. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology. ACM: 2011, pp.559–568.

40.

. Skeleton driven animation based on implicit skinning. Comput Graph 2011; 35: 945–954.