Traction control of space tubular shaped mandrel and detection of preform braiding angle

Abstract

The braiding quality of composite preforms has a significant influence on their mechanical properties. To address the difficulties in the design of the processing technology and the stability of the surface quality of the fabric during the braiding process of the spatial special-shaped mandrel, a robot control method was proposed for prefabrication. Regarding branding of the formed body and inspection of the braiding angle of the fabric by machine vision for quality control, in this study the spatial special-shaped mandrel was discretized, the position and attitude of the tool center point were calculated, and the robot pulling speed was calculated. The calculation was performed to realize the automatic compensation of the braiding angle and, finally, the braiding angle of the fabric surface was detected by machine vision. The experimental results showed that the control method could effectively reduce the braiding angle error of the preform within ±2° through the detection of the visual inspection algorithm and could improve the quality of the preform and the detection speed of the braiding angle in actual production.

Keywords

Complex preform trajectory normalization dynamic compensation braiding angle visual inspection

In recent years, owing to their good specific strength and specific stiffness, high-temperature resistance, corrosion resistance, and light weight, composite materials have widely replaced traditional metals in the fields of aerospace, automobile manufacturing, and medical equipment.¹

Braiding is an important process in composite material preforms. Braiding errors, such as the braiding angle and coverage errors during the braiding process, directly affect the mechanical properties of composite materials. The braiding angle is the angle formed by the fiber yarn and axis of the mandrel and is usually caused by the fluctuation of the braiding speed or traction speed, contact and friction between yarns,² and change in the mandrel section. Whether the composite material can have the desired mechanical properties, such as tensile strength and compressive strength, depends on the size of the braiding angle error³ and the surface coverage.⁴ The production of braided composites is a multi-step manufacturing process. Production efficiency and part quality can be improved by improving the manufacturing process and detection methods, and the need for labor can be reduced.⁵

The following is a study on the braiding process of a spatially curved special-shaped mandrel; the braiding angle on the surface of the preform after braiding is completed is also detected. Previous research has focused on the direction of the cored imitation surface shape. Monnot et al.⁶ established a braiding model suitable for a mandrel with a centrally curved rectangular cross-section and used an industrial manipulator to pull the mandrel to complete the braiding of the preform; however, this braiding model was not suitable for a mandrel with a circular cross-section. Du et al.⁷ developed a theoretical mathematical model to describe the braiding process of a circular section mandrel and related the mandrel pulling speed to the surface braiding angle of the part. The mathematical model could be used to predict the braiding angle but was only suitable for straight-line mandrels with an axisymmetric cross-section. Other scholars used the finite element method to analyze the forces in a braiding process. Zhuo et al.⁸ highlighted that a three-dimensional braiding product has better mechanical properties than those of the two-dimensional circular braiding, analyzed the displacement of the core mold caused by the yarn tension disturbance during the braiding process through finite element technology, and then compensated for the end of the traction robot. However, it is difficult to perform offset compensation based on coordinate system transformation for spatially curved, special-shaped structure mandrels. Gondran et al.⁹ proposed discretizing the mandrel by changing the cross-section into multi-segment cylinders and using iterative feedback through the forward and inverse kinematics solution to obtain a better robot traction speed but did not consider the changes in the yarn collection area. Martinec et al.¹⁰ analyzed the process of robot-end clamping the mandrel through a braiding machine during the fiber braking process and obtained the trajectory of the robot during operation through theoretical calculations; however, they did not consider the constant change in the mandrel section and the three-dimensional space of the centerline of the mandrel. Hans et al.¹¹ used a finite element software program to model the two-dimensional overlapping braiding process of an industrial robot holding a mandrel to simulate the entire process of braiding. Spatially shaped mandrels are widely used for braiding products with complex geometries, such as robotic arms, prosthetics, and hockey sticks.¹² However, there are few studies in the literature on the braiding process of a mandrel with a changing cross-section and spatial special shape; therefore, this article mainly focuses on the research on the processing of the special-shaped braid structure.

Boisse et al.¹³ proposed a Carrier delay-based method for development of the tracks for the transitions between patterns on 3D braiding machines with continuous rotating horn gears. Kamble and Behera¹⁴ have reported the geometric model of a four-directional 3D braided preform developed on a four-step 3D braiding machine which consists of even and an equal number of yarn carriers in the rows and columns, respectively, on machine bed. Huber et al.¹⁵ have proposed a 3D rotary braiding process that can be successful fully applied to man-made cellulose fibers. without causing significant damage to the fibers. In Bilisik’s study,¹⁶ 3D braided fabrics, methods and techniques were reviewed. Biaxial and triaxial 2D braided fabrics have been widely used as simple and complex shaped structural composite parts in various technical areas.

Hajrasouliha et al.¹⁷ proposed a model to predict the braid angle on an arbitrary, constant cross-section and used image processing to detect the braid angle. Hunt and Carey⁵ obtained dynamic and static preform surface images, cropped the region of interest, and then performed a frequency domain analysis to obtain specific braiding angles. Some studies have proposed measuring the surface braiding angle of composite material preforms based on the Hough transform; however, only a single braiding angle could be obtained, which does not represent the braid quality of the entire fabric. Lian et al.¹⁸ developed a fully automated machine vision system, identified the necessary image-processing steps, and successfully implemented a scanning algorithm for braiding angle extraction. Wan et al.¹⁹ applied spectroscopic analyses to three-dimensional (3D) braided composite preform surface images. This method has been shown to be suitable for measuring the surface braiding angle of 3D composites. In this study, a new braiding angle algorithm is proposed based on machine vision and image processing technology to improve detection accuracy.

In this article, a method for controlling the traction trajectory and speed of a space-curved-shaped mandrel and an algorithm for discrete angle mean grayscale visual inspection are proposed to measure the braiding angle on the surface of the preform. This trajectory and speed control method uses a plane perpendicular to the fixed braiding point when calculating the trajectory of the traction robot and does not need to consider the change in the convergence length. By compensating for the braiding angle error caused by the change in the section, the method can effectively reduce the error when braiding the core mold of the special-shaped structure thereby improving the mechanical properties of the preform. After the preform was braided, a surface image was collected, and the discrete angle mean grayscale algorithm was used to detect the surface braiding angle to evaluate the mechanical properties of the preform.

Aiming at this type of mandrel traction problem, this article proposes a method for traction trajectory and speed control of a space-curved-shaped mandrel. This trajectory and speed control method uses a plane perpendicular to the fixed braiding point when calculating the trajectory of the traction robot and does not need to consider the convergence length changes. Furthermore, it establishes a mathematical model of the pulling speed during the braiding process, and then controls the pulling speed to compensate for the braiding angle error caused by the section change. After the preform was braided, a surface image was collected, and the discrete angle mean grayscale algorithm was used to detect the surface braid angle to evaluate the mechanical properties of the preform. To address the problem of braiding angle measurement, a method to measure the surface braid angle of a braided composite material preform is proposed. The method includes image preprocessing, image binarization, edge detection, two-dimensional discrete Fourier transformation, and a discrete angle mean grayscale braiding angle detection algorithm, which is proposed and applied to calculate the preform surface braiding angle.

Trajectory planning and speed control of traction robot in braiding process

Figure 1 shows the braiding preform of a robot pulling a mandrel. The coordinate system where the robot base is located is usually called the base coordinate system or world coordinate system. The external conditions, such as the position of the braiding machine and braiding ring, are described in the base coordinate system, and the shape of the mandrel is described in the tool coordinate system. After planning the trajectory and speed, inputting them into the command library to write the trajectory and speed enables the robot to pull the mandrel.²⁰

Figure 1.

Braiding machine and traction robot.

Trajectory planning of traction robots

Simplified braiding model

The braiding process of a special-shaped space-bending mandrel is shown in Figure 2. The trajectory of the mandrel pulled by the robot must ensure that the mandrel always passes vertically through the plane where the braiding point is located. The coordinates of the center point of the braiding plane in the base coordinate system are S $= [x_{s}, y_{s}, z_{s}]$ , and the vector perpendicular to the plane through the braid point is $s$ .

Figure 2.

Schematic diagram of the braiding process.

The mandrel is represented by n discrete points $P_{i} (i = 0, 1, \dots, n)$ along the centerline.⁹ The discrete approximation of the mandrel is shown in Figure 3. A small piece of mandrel was taken arbitrarily, and the mandrel shape between the plane where $P_{i}$ is located and the plane where $P_{i - 1}$ is located is approximately regarded as a cylinder, and the length of this cylinder is $L$ . The cylinder radius ( $r_{i}$ ) is determined by the section radii at both ends of the discrete core segment. The calculation method is shown in Equation (1). Using this method does not only approximate the real situation of braiding, but also meets the continuity requirements of the braiding process and paves the way for the subsequent derivation.

r_{i} = \frac{r P_{i - 1} + r P_{i}}{2}

(1)

where

{r P}_{i - 1}

and

{r P}_{i}

represent the core radii of the sections where

P_{i - 1}

and

P_{i}

are located, respectively.

Figure 3.

Mandrel discrete approximation process diagram.

The calculation of the average braid convergence length ( $\bar{h})$ , shown in Figure 2, will be derived in subsequent chapters, and subsequent research should be carried out under the following conditions:

The relative coordinates between all discrete $P_{i}$ points should be known;

The number of discrete segments (n) should satisfy the accuracy requirements for describing the shape of the mandrel;

The approximate length (L) of each segment of the cylinder after discretization should ensure that the robot satisfies the response of each segment.

Trajectory planning of the end of the traction robot

At present, the known manipulators pull the mandrel to assist the process of braiding the preform, most of which obtain the trajectory such that the robot moves through the plane of the braiding ring vertically and uniformly at all times. Bending the mandrel affects the braiding angle, resulting in a braiding angle error. The robot clamps the mandrel and passes through the braiding ring vertically. Owing to the convergence distance (h), the centerline of the mandrel at this moment cannot be perpendicular to the plane of the braiding point, as this causes unequal braiding angles on all sides of the mandrel. Therefore, for a special-shaped mandrel, in the braiding process, a mandrel perpendicular to the plane of the braiding point should be the basic theoretical model. Figure 4 is the flow chart of the robot terminal trajectory planning. The left side of the figure is the sequence structure. The sequence has little effect on the planning, and the sequence cannot be changed for the cyclic structure. If it is changed, the correct trajectory cannot be calculated.

Figure 4.

Traction planning flow chart of traction robot. TCP: tool center point.

The trajectory of the robot end is determined according to the pose data of the tool center point (TCP). In this study, the mandrel is divided into a total of $n + 1$ discrete points, that is, $i = 0, 1, \dots, n$ ; two discrete points are set; and there are three transformations of the TCP (the TCP is a homogeneous transformation matrix) between points. To distinguish between the braiding step and discrete mandrel points, the transformation step of the mandrel of the i-th segment ( $P_{i} \to P_{i + 1}$ ) is the i-th step:

That is, point $P_{i}$ is perpendicular to the braiding plane and coincides with center point O of the braiding plane, as shown in Figure 5(a). It transforms to point $P_{i + 1}$ , which is perpendicular to the braiding plane and coincides with center point O of the braiding plane, as shown in Figure 5(d). ${TCP}_{1} (i) (i = 0, 1, \dots, n - 1)$ represents the pose data after the rotation and change in the translation angle (α). When braiding the i-th segment, it is necessary to ensure that vector $\vec{P_{i} P_{i + 1}}$ , that is, the direction of the vector points from $P_{i + 1}$ to $P_{i}$ , rotates to coincide with vector s while simultaneously ensuring that point $P_{i}$ coincides with center point O of the braiding plane, thereby ensuring that this segment of the mandrel passes vertically through the plane where the braiding point is located. To calculate ${TCP}_{1} (i)$ , the pose data in ${TCP}_{3} (i - 1)$ must be used. ${TCP}_{2} (i) (i = 0, 1, \dots, n - 1)$ represents the pose data after the translational movement at a speed of $V_{i}$ , and the translational distance is $L_{1}$ . ${TCP}_{3} (i) (i = 0, 1, \dots, n - 1)$ is the pose data after the translational movement at the speed of $V_{i}$ ; similarly, the translational distance is $L_{2}$ . The relationship between the moving distances ( $L_{1}$ and $L_{2}$ ) and $V_{i}$ is explained in detail in the section below on speed and time control during braiding.

Figure 5.

Schematic diagram of discrete mandrel pose change. (a) TCP₃(i−1); (b) TCP₁(i); (c) TCP₂(i) and (d) TCP₃(i).

For the clamped mandrel, the relative position of the mandrel and the tool coordinate system are determined; that is, ${}^{T}{P_{i}} (i = 0, 1, 2 \dots, n)$ is known. First, the representation of vector $\vec{P_{i} P_{i + 1}}$ in tool coordinate system {T} is calculated and then converted into the base coordinate system, as shown in Equation (2).

\vec{{{}^{B}P}_{i} P_{i + 1}} =_{B}^{A} T \cdot \vec{{{}^{B}P}_{i} P_{i + 1}} = T C P_{3} (i - 1) \cdot (^{T} P_{i} -^{T} P_{i + 1})

(2)

where

{}^{T}{\vec{P_{i} P_{i + 1}}}

denotes the vector (

\vec{P_{i} P_{i + 1}}

) description in tool coordinate system {T} and

{}^{B}{\vec{P_{i} P_{i + 1}}}

denotes the description of the vector (

\vec{P_{i} P_{i + 1}}

) in tool coordinate system {B}.

As shown in Figure 6, the angle (φ) between vector $\vec{P_{i} P_{i + 1}}$ and the vertical braiding plane direction vector ( s ) is calculated using Equation (3):

φ = arc cos (\frac{\vec{{{}^{B}P}_{i} P_{i + 1}} \cdot^{B} s}{|{{}^{B}P}_{i} P_{i + 1}| \cdot |{}^{B} s|})

(3)

where

{}^{B}s

represents the direction vector (s) in the base coordinate system, that is,

{}^{B}s = s

. We transform

Rot (K, φ)

through the general rotation transformation equation, that is, Equation (4).

Rot (K, φ)

= [\begin{matrix} k_{x} k_{x} v φ + c φ & k_{x} k_{y} v φ - k_{z} s φ & k_{x} k_{y} v φ + k_{y} s φ & 0 \\ k_{x} k_{y} v φ + k_{z} s φ & k_{y} k_{y} v φ + c φ & k_{x} k_{x} v φ - k_{x} s φ & 0 \\ k_{x} k_{z} v φ - k_{y} s φ & k_{y} k_{z} v φ + k_{x} s φ & k_{z} k_{z} v φ + c φ & 0 \\ 0 & 0 & 0 & 1 \end{matrix}]

(4)

where

c φ = \cos φ

s φ = \sin φ

v φ = 1 - \cos φ

, and

K = {[k_{x}, k_{y}, k_{z}]}^{T}

. φ is determined using the right-hand rule.

^{T} K = \frac{\vec{{{}^{T}P}_{i} P_{i + 1}} \times^{T} s}{|\vec{{{}^{T}P}_{i} P_{i + 1}} \times^{T} s|}

(5)

The rotation axis (K) is determined using Equation (5) and is relative to tool coordinate system {T}. The i-th mandrel is rotated around the ${}^{T}{K_{i}}$ axis by the pose shown in Figure 6(a). Subsequently, the pose shown by the dotted line in Figure 6(b) is obtained. The pose shown by the dotted line and the $P_{i}^{'}$ point are the only positions used in the derivation process; the traction robot does not actually move to this pose. There is a gap between the point and the origin of the braiding plane according to vector $c_{i}$ ; therefore, it is necessary to perform a shift transformation on the rotation transformation matrix such that point $P_{i}^{'}$ is moved to coincide with the origin of the braiding plane.

Figure 6.

Schematic diagram of discrete mandrel angle change.

The derivation and calculation of vector $c_{i}$ are shown in Figure 7, where vector $a_{i}$ represents the position of the braiding center point relative to the base coordinate system and vector $b_{i}$ represents the rotation angle (φ) of $P_{i}$ when point $P_{i}^{'}$ is reached. Point $P_{i}^{'}$ is relative to the base coordinate system. The position of the calculation is obtained through Equation (6), where vectors $a_{i}$ , $b_{i}$ , and $c_{i}$ are all descriptions in base coordinate system {B}.

[\begin{matrix} a_{i} \\ b_{i} \\ c_{i} \end{matrix}] = [\begin{array}{l} ^{B} O \\ T C P_{3} (k - 1) \cdot Rot (^{T} K_{i}) \\ a_{i} - b_{i} \end{array}]

(6)

Figure 7.

Schematic of compensation vector $c_{i}$ .

According to the homogeneous transformation theory, the rotation around the tool coordinate system should be multiplied by the rotation transformation matrix to the right, and the movement along the base coordinate system should be multiplied by the transformation matrix to the left, as shown in Equation (7), where $Transl (s, L_{1})$ and $Transl (s, L_{2})$ represent the distance of moving $L_{1}$ and $L_{2}$ along vector s .

[\begin{matrix} T C P_{1} (i) \\ T C P_{2} (i) \\ T C P_{3} (i) \end{matrix}] = [\begin{array}{l} T ransl (c_{i}) \cdot T C P_{3} (i - 1) \cdot Rot (^{T} K_{i}, φ) \\ T ransl (s, L_{1}) \cdot T C P_{1} (i) \\ T ransl (s, L_{2}) \cdot T C P_{2} (i) \end{array}]

(7)

${TCP}_{1} (i)$ , ${TCP}_{2} (i)$ and ${TCP}_{3} (i)$ represent the pose information after the third transformation of the i-th mandrel, which is the theoretical basis for path planning of the center point of the robot tool.

For any homogeneous transformation matrix T, the pose information can be obtained from Equations (8) and (9). The 3 × 1 vector (p) represents the position in the coordinate system, and the 3 × 3 rotation matrix (R) is an orthogonal matrix; therefore, although the rotation matrix (R) has nine elements, there are only three independent elements. Therefore, three parameters are usually used to represent an attitude, and the representation method usually uses fixed angles or Euler angles; these two methods correspond to each other, such as the XYZ fixed angle and the ZYX Euler angle, where α, β, and γ are calculated as shown in Equation (9).

T = [\begin{matrix} R & p \\ 0 & 1 \end{matrix}] = [\begin{matrix} r_{11} & r_{12} & r_{13} & p_{x} \\ r_{21} & r_{22} & r_{23} & p_{y} \\ r_{31} & r_{32} & r_{33} & p_{z} \\ 0 & 0 & 0 & 1 \end{matrix}]

(8)

[\begin{matrix} β \\ α \\ γ \end{matrix}] = [\begin{array}{l} \arctan 2 (- r_{13}, \sqrt{r_{11}^{2} + r_{21}^{2}}) \\ \arctan 2 (r_{21} / c β, r_{11} / c β) \\ \arctan 2 (r_{32} / c β, r_{33} / c β) \end{array}]

(9)

where

\arctan 2 (y, x)

is the arctangent function of a two-parameter variable.

It can be seen from the equations above that for an arbitrary pose homogeneous transformation matrix, the current position and attitude can be represented by six parameters: $p_{x}$ , $p_{y}$ , $p_{z}$ , $α$ , $β$ , and $γ$ . The command of the traction robot can also be planned and controlled using these six parameters, and the calculated TCP coordinates can be input into the robot command. According to the aforementioned mandrel trajectory planning process, the traction robot can be precisely controlled to ensure that each segment of the mandrel passes vertically through the plane of the braiding point during the braiding process.

Speed and time control during braiding

Ideal hypothesis and braiding theory

The proposed method is based on the following assumptions for the braiding model of a spatially curved profiled mandrel:

Since the thickness of the single-layer yarn is too small compared with the diameter of the mandrel, it is ignored.

The effect of friction between the yarn and braided guide ring is ignored; therefore, the yarn is not bent in the convergence area.

During the braiding process, the yarn and mandrel do not slide relative to each other, and the yarn and the surface of the mandrel are completely fitted.

The effects of motor-induced vibrations during braiding are ignored.

It can be deduced from the two-dimensional braiding theory that the braiding process is controlled by the kinematic relationship,¹² and there is a mutual kinematic relationship among the rotation speed (ω) of the yarn carrier, steady-state pulling speed (V) of the mandrel, radius (r) of the mandrel, and steady-state braiding. Angle θ is related⁷ to these parameters by Equations (10) and (11):

\tan θ = \frac{r ω}{V} = \frac{\sqrt{R^{2} - r^{2}}}{h}

(10)

Equation (11) can be derived from Equation (10):

V = V (r) = \frac{r ω h}{\sqrt{R^{2} - r^{2}}}

(11)

It can be seen from Equation (12) that there is a theoretical braiding distance (h), that is, the straight-line distance from the center point of the convergence area to the center point of the guide ring, in the braiding process of any segment of the mandrel. The theoretical value of the i-th segment is as follows. As shown in Equation (12), because it is inconvenient to adjust h by translation during the braiding process, it is very important to choose an appropriate actual braiding distance, and this is done using Equations (12) and (13), through which coordinates of ${}^{B}O$ can be determined.

h_{i} = \frac{\sqrt{R^{2} - {r_{i}}^{2}}}{\tan θ_{e}}

(12)

\bar{h} = \frac{1}{n} \sum_{i = 1}^{n} h_{i}

(13)

where

\bar{h}

is the theoretical braiding distance of each section,

h_{i}

represents the theoretical braiding distance of the i-th segment, and

θ_{e}

represents the expected braiding angle.

Adjust braiding speed to compensate braiding angle

The process of using a robot to adjust the convergence length in real time is complicated because the rotary motor of the braiding machine works continuously and cannot be interrupted during the braiding process; thus, the traction robot adjusts the braiding distance (h) by translation before braiding each section. Therefore, the braiding angle can only be compensated by changing the traction speed of the robot.

From Equation (11), it can be seen that the robot traction speed ( $V$ ) can be regarded as a function of $r$ , $V (r)$ , the function of which can be calculated. The recursive equation the speed $V$ can be obtained by discretization, represented by Equations (14) and (15):

\frac{d V}{d r} = \frac{ω h R^{2}}{(R^{2} - r^{2})^{3 / 2}}

(14)

V_{i} = V_{i - 1} + \frac{d V}{d r} \cdot (r_{i} - r_{i - 1}) = V_{i - 1} + \frac{ω \bar{h} R^{2} (r_{i} - r_{i - 1})}{(R^{2} - r^{2})^{3 / 2}}

(15)

where

r = \frac{r_{i} + r_{i - 1}}{2}

and

h = \bar{h}

According to the braiding theory, the pulling speed can be decomposed into two speeds: the pulling speed of the ongoing yarn ( $V_{a}$ ) and the moving speed ( $V_{r})$ of the braiding point. Thus, the speed relationship of any segment of the mandrel is as shown in Equations (16) and (17):

V_{i} = V_{a i} + V_{r i}

(16)

L_{1} = \int_{0}^{t_{1}} V_{r i} d t = \int_{0}^{t_{1}} V_{i} - V_{a i} d t = V_{i} \cdot t_{1} + (h_{1} - h_{0}) \cdot (e^{- \frac{r ω t_{1}}{\sqrt{R^{2} - r^{2}}}} - 1)

(17)

The derivation of in Equation (17) is too cumbersome and will not be presented in this article, similar to Equation (18).

The braiding process transitions from an unstable state to a stable state. Therefore, at this time, $e^{- \frac{r ω t_{1}}{\sqrt{R^{2} - r^{2}}}} \approx 0$ , in combination with other constraints of the braiding process, resulting in four equations, four unknowns, and one constraint inequality, and the four unknowns ( $L_{1}$ , $L_{2}$ , $t_{1}$ , $t_{2})$ can be obtained as follows:

{\begin{array}{l} L_{1} = V_{i} \cdot t_{1} + h_{i - 1} - h_{i} \\ t_{1} = \frac{r ω}{\sqrt{R^{2} - r^{2}}} \times \ln \frac{\tan^{- 1} θ_{s} - \tan^{- 1} θ_{e}}{\tan^{- 1} (0.05 θ_{s} + 0.95 θ_{e}) - \tan^{- 1} θ_{e}} \\ L_{2} = V_{i} \cdot t_{2} \\ \begin{array}{l} L = L_{1} + L_{2} \\ L_{1} \leq L \end{array} \end{array}

(18)

Considering the assumptions for the previous model, this constraint is justified. However, when the actual situation does not fully meet the assumption, that is, when the condition of this segment of the mandrel does not satisfy constraint condition $L_{1} \leq L$ , the speed of this segment of the mandrel should be planned separately.

Braiding angle detection of fabric surface based on machine vision

Braiding angle visual inspection process

For the collected fabric surface images, choosing an appropriate image processing method can reduce the measurement error of the braiding angle and improve detection accuracy. Figure 8 shows a flowchart of the visual inspection method for the braiding angle.

Figure 8.

Braiding angle visual inspection flow chart.

Image interpolation expansion

Since composite braiding mandrels are tubular, the three-dimensional surface profile of these tubular braids must be considered when measuring braid angles from captured images. Because the distance between the camera acquisition plane (i.e. the imaging plane) and the outer surface of the cylinder is not constant, the fiber orientation in the image is distorted, resulting in projection errors.

To reduce the projection error, Hunt and Carey⁵ performed a two-step reduction of the perspective error. The first step is to crop and remove the severely distorted area of the image and obtain the region of interest, and the second step is to interpolate the image of the region of interest to restore the image to a state close to the real state.

Perspective error increases significantly with distance from the centerline of the mandrel, and by removing these parts of the image, the accuracy of the braided angle in the image can be improved.

There are three main methods for image interpolation: nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. Bilinear interpolation is a commonly used method, in which the four nearest neighbor positions are used to estimate the gray level of a given position. Let $(x, y)$ be the coordinates of the position we want to assign, and let $v (x, y)$ represent the grayscale value. For bilinear interpolation, the gray value assignment is obtained using Equation (19):

v (x, y) = a x + b y + cxy + d

(19)

The four coefficients of the above equation can be determined by the unknown equation written using the four nearest neighbors of point $(x, y)$ .

To complete the image unwrapping, the geometric relationship shown in Figure 9 was used, which relates the projected position of point d on the tubular surface and its unwrapped arc length (l) to the radius (r) and circumferential angle (α) of the tubular surface.

Figure 9.

Tubular braid surface projection.

The key parameter in this method is the circumference angle of the tubular surface, which determines the number of unwrapped surfaces. As this angle increases, a large amount of data must be interpolated to smooth the sharp edges in the image. By comparing the distance between the projected position (d) and its corresponding unwrapped arc length (l), the pixel can be replaced with the difference between the projected and unwound lengths. This method provides better computational frequency-domain performance compared to nearest-neighbor interpolation and reduces the computational time compared to bicubic interpolation.

n = \frac{l}{d} = \frac{r \cdot α}{r \cdot \sin α} = \frac{r \cdot arc \sin (\frac{d}{r})}{d}

(20)

As shown in Figure 9, r is the radius of the mandrel and α is the included angle of the cropped region of interest. Equation (20) is the derivation of the horizontal interpolation multiple, (l) is the actual arc length, (d) is the image projection distance, and (n) is the horizontal interpolation multiple, which is the ratio of the actual arc length to the image projection distance.

Figure 10 shows the effect of selecting the region of interest from the braided picture and cropping it to 50% of the original width. Figure 10(a)–(c) images show the picture display of this step; Figure 10(d) passes the region of interest through the resulting plot after linear interpolation expansion.

Figure 10.

Interpolation method to expand the region of interest process. (a) Braid pictures; (b) region of interest; (c) crop region of interest and (d) interpolation expansion.

Edge detection

After practice and analysis, edge detection was chosen to remove the influence of interfering fibers. Edge detection is a common method used for segmenting images based on grayscale mutations.

The edge of the image is the most significant part of the local intensity change in the image, and it also refers to the set of pixels around the image that have a step-like change in the grayscale of the pixels. This feature reflects a discontinuity in the image characteristics. The key step in the image preprocessing step is edge detection, and the detection result directly affects the subsequent image feature extraction and image processing. Common edge detection operators are the Roberts, Sobel, Prewitt, log, and Canny operators.

Using the different operators of the above image edge detection, we used MATLAB to perform edge detection on the braided image in the spatial domain and to analyze the results. After observation and analysis, it was found that the results obtained by the Sobel, Roberts, and Prewitt operators as edge detection operators are more in line with the image required for measuring the braiding angle; the Canny, Gaussian Canny, and log operators were used as edge detection operators. The image obtained by the detection operator cannot highlight the edge lines required by the braided corner because of the large number of details. Because the Sobel, Roberts, and Prewitt operators are all judged and identified based on the first derivative, partial derivative, or gradient, while the Canny, Gaussian Canny, and Log operators are more advanced operators and are identified based on the second derivative and partial derivative. The above situation shows that edge detection that is too detailed for the detection of the braiding angle will lead to the failure to find the edge lines required for measuring the braiding angle, whereas the edge detection operator based on the first derivative can better find the edge lines required for measuring the braiding angle. Therefore, it is the area to be studied to propose a better edge detection operator suitable for braiding angle detection.

The calculation speeds of the Sobel, Roberts, and Prewitt operators are not significantly different, but the Sobel operator template can better suppress noise. Based on this characteristic, the Sobel operator was finally selected for edge detection. In order to make the image easier to observe, grayscale inversion transformation was performed on the grayscale image after edge detection, and the final grayscale transformation result is shown in Figure 11(a).

Figure 11.

Comparison of edge detection using different edge detection operators. (a) First derivative operator and (b) second derivative operator.

Subsequently, a two-dimensional discrete Fourier transform (2D-DFT) transformation was performed on the spatial domain image after edge detection to obtain the corresponding frequency domain image. After the actual operation comparison, it was found that the operation effect significantly improved.

Fiber braiding angle detection method based on 2D-DFT

2D-DFT was used as the main image processing technique to extract the information of the fiber orientation angle from the acquired images. The physical meaning of the Fourier transform is to transform the gray distribution function of the fiber preform fabric image into the corresponding frequency distribution function. This function is transformed into a gray distribution function. The image frequency represents the gradient and direction of pixel changes in a spatial image; high frequencies represent sudden changes in pixel gray values, and low frequencies represent regions with constant gray values. For a square spectrogram, the angular orientation of the edges in the spatial-domain image corresponds to the angular orientation of the features in the frequency domain. Using this property of the 2D-DFT of image processing, the braiding angle can be measured.

Equations (21) and (22) the forward and inverse transform equations for discrete Fourier transforms that allow the transformation of an image of size M×N with pixel intensity values $f (x, y)$ into spectral data matrix $F (u, v)$ , and the value of $f (x, y)$ depends on the bit depth of the image sensor, while the value of $F (u, v)$ is rescaled to $[0, 1]$ . The coordinate system is defined by spatial coordinates x and y and frequency coordinates u and v in the x- and y-directions, respectively. The transformation between spatial domain $f (x, y)$ and frequency domain $F (u, v)$ of a braided image is shown in Figure 12. It also shows the defined starting point of each array, where (0,0) is the starting point of the frequency components, and its position is at the center of the spectrogram. This method has been used in the composite and textile industries for image analysis applications, including feature recognition and fault detection.

F (u, v) = \sum_{x = 0}^{M - 1} \sum_{y = 0}^{N - 1} f (x, y) e^{- j 2 π (\frac{u x}{M} + \frac{v y}{N})}

(21)

f (x, y) = \frac{1}{M N} \sum_{u = 0}^{M - 1} \sum_{v = 0}^{N - 1} F (u, v) e^{j 2 π (\frac{u x}{M} + \frac{v y}{N})}

(22)

Figure 12.

The relationship of the angle between the spatial domain (a) and frequency domain images (b).

It is a key step to intercept a square space domain image and perform a 2D-DFT transform to obtain a frequency-domain image. Figure 12(a) shows the spatial image and its braiding angle, and Figure 12(b) shows the frequency-domain image obtained after 2D-DFT of the image in the spatial domain and its annotated braiding angle.

As shown in Figure 12(b), the two optical fiber straight lines with a cross angle of 2θ are the key lines for calculating the braiding angle, whereas the horizontal and vertical optical fiber straight lines are the interference straight lines that affect the calculation of the braiding angle. The impact of the braid-angle calculation is critical.

Discrete angle mean grayscale braiding angle detection algorithm

Two lines in the same direction in real space coincide in the frequency domain image, regardless of how far apart they are. Because the fiber orientation is the only consideration in this algorithm, it is a critical feature.

Figure 12 shows a 2D-DFT image of the braided structure. As shown in the 2D-DFT image, the two main fiber bundle orientations in real space were transformed into two lines passing through the center of the frequency-domain image of the 2D-DFT, each perpendicular to the original corresponding fiber orientation. The lines are dispersed, which is related to the distribution of the fiber directions.

To solve these problems, this study combines the theory of Hunt and Carey^5,21 to propose a straight-line average gray-level algorithm to obtain the discrete angle mean grayscale level distribution and then obtain the braiding angle. A line segment was drawn to connect the center pixel of the 2D-DFT image with one of its edge pixels, and the average pixel intensity was calculated for all the pixels touching this line segment. With one end of the line fixed at the center, the line segment was rotated and swept through 180° (as shown in Figure 13) until half a cycle was completed (owing to the symmetry of the 2D-DFT image, only half a cycle must be scanned). The angle between the maximum values of the two obtained average gray levels was twice the braiding angle, as shown in Figure 12(b).

Figure 13.

Schematic diagram of discrete angle mean grayscale algorithm.

The maximum value in the gray distribution of the mean value of each angle corresponds to the pulling direction of the main fiber bundle in the actual space. The scanning line width and each scanning increment are two important parameters for representing the calculation accuracy of the braiding angle, while for the gray scale, the average value is the weighted average value of the gray values of the pixel points within the line segment where each angle is located. Because multiple fiber directions are associated with the braided structure, the rotation angle average grayscale algorithm searches for the specified direction and sequence. It searches for the discrete mean grayscale of all angles and repeats this process until all the specified fiber bundle directions are obtained, determines the two maxima (first and second maxima), and calculates the regional average braid angle. No human intervention is required in this image-processing step because of the use of the relative mean grayscale.

To calculate the average gray value in the straight-line area, it was necessary to traverse 180°, and the angle that increased each time the average gray value of the straight line was calculated is called the angular resolution. As shown by $Δ θ$ in Figure 13, the angular resolution has a significant influence on the accuracy of the algorithm; therefore, a smaller angular resolution is usually selected for calculation, and 180° is divided into n lines to calculate the sum, as shown in Equation (23), where n is the resolution.

Δ θ = \frac{π}{n} \times 18 0^{} \circ

(23)

Another important point to note is that the coordinate system in the image is not the same as that used to calculate the average grayscale of the line. The coordinate system of the image is shown in the $xOy$ coordinate system in Figure 13; the coordinates of the line and rotation angle are generated. The system is shown as $x^{'} O^{'} y^{'}$ , which must be considered in the algorithm.

The Bresenham line-drawing algorithm is the most widely used line-generation algorithm in computer graphics. In the process of calculating the best compression point of the line, all integer operations are performed; thus, the calculation speed can be greatly improved. Assuming a straight line from the starting point $(x_{s}, y_{s})$ to the endpoint $(x_{e}, y_{e})$ , the equation of the straight line can be expressed as $y = k x + b$ . Then, the equation of the straight line can be expressed as $y = k x + b$ , where $b = y_{s} - k x_{s}$ and $k = (y_{e} - y_{s}) / (x_{e} - x_{s})$ .

Taking quadrant 1a as an example, the remaining seven quadrants can be generated by centrosymmetric and axisymmetric methods on the straight line in quadrant 1a. As shown in Figure 14, when the line is rasterized, x is increased by one unit each time, that is, $x_{i + 1} = x_{i} + 1$ ;

Figure 14.

Eight quadrant distinction of square spectrogram.

The corresponding increase in y should be less than 1. As shown in Figure 15, to find the gray value of the pixel point closest to the straight line, $y_{i + 1}$ can only choose $y_{i + 1} = y_{i}$ or $y_{i + 1} = y_{i}$ as one of two positions.

Figure 15.

Step i + 1 error calculation and vertical coordinate selection.

The principle of selecting the pixel position is determined by the precise values of the distances among y and $y_{i}$ , $y_{i + 1}$ , $d_{1}$ , and $d_{2}$ . The relationships are governed by Equations (24)–(26).

y = k (x_{1} + 1) + b

(24)

d_{1} (x_{i + 1}) = y - y_{i}

(25)

d_{2} (x_{i + 1}) = y_{i} + 1 - y

(26)

If $d_{1} (x_{i + 1}) - d_{2} (x_{i + 1}) > 0$ , then $y_{i + 1} = y_{i} + 1$ ; otherwise, $y_{i + 1} = y_{i}$ . Therefore, the key to this algorithm is whether the sign of $d_{1} - d_{2}$ can be easily obtained. We substitute Equations (24), (25), and (26) into $d_{1} (x_{i + 1}) - d_{2} (x_{i + 1})$ ;

d_{1} (x_{i + 1}) - d_{2} (x_{i + 1}) = 2 \frac{d y}{d x} (x_{i} + 1) - 2 y_{i} + 2 b - 1

(27)

For Equation (27), multiplying the left and right sides of the equation by $d x$ does not change the signs on both sides of the equation; let $e_{i} = d x (d_{1} - d_{2})$ , and substitute this into the above equation.

e_{i} = 2 x_{i} d y - 2 y_{i} d x + 2 d y + d x (2 b - 1)

(28)

$d_{1} (x_{i + 1}) - d_{2} (x_{i + 1})$ is the error used to judge the symbol. Because it is in the first quadrant and the slope is less than 1, $d x > 0$ , the error $e_{i}$ is used to judge the symbol, and $e_{i + 1}$ is obtained through Equation (28).

e_{i + 1} = {\begin{array}{l} e_{i} + 2 d y, e_{i} \leq 0 \\ e_{i} + 2 d y - d x, e_{i} > 0 \end{array}

(29)

Substituting $x_{1}$ , $y_{1}$ , and $b$ into Equation (28) to obtain the initial value ( $e_{1}$ ), we obtain

e_{1} = 2 x_{1} d y - 2 y_{1} d x + 2 d y + d x (2 b - 1) = 2 d y - d x

(30)

Since the area represented by the pixels of a line found is too small, it is necessary to widen the width of the line segment, and the square brush method for generating the line width is selected to collect the average gray value of the line generated above and its surrounding area.

The concept of the average grayscale calculation algorithm in the 1a quadrant is shown in Figure 16.

Figure 16.

Flowchart of discrete mean grayscale algorithm.

After the above process, the average gray value in the 1a quadrant can be obtained, and it should be noted that owing to the use of the square brush method, the boundary range of the scan must be limited, and the accumulation should be stopped when the x and y coordinates reach the boundary.

The braiding angle can be obtained by calculating the average gray value using linear scanning in the range of 180° and plotting this value with the corresponding angle, as shown in Figure 17. The braided angle can be calculated by determining the angle (in degrees) between the two maximum values. If the wire brush and angular frequency are not properly adjusted in the algorithm, the calculation accuracy may be affected or the braiding angle calculation will not be achieved.

Figure 17.

Average grayscale distribution after frequency domain map scanning.

Results and discussion

To verify the applicability of the method proposed in this study, taking a cylindrical mandrel of a special-shaped bending structure as an example, Table 1 shows the coordinates of the center points of each segment of the discrete mandrel in robot tool coordinate system {T}, where the mandrel is a variable cross-sectional elbow, the cross-sectional radius of the head end of the mandrel is 72.5 mm, and the cross-sectional radius of the tail end is 81.5 mm. The centerline of the mandrel is located in the same plane. The radius values are listed in Table 2. The equipment used in the experiment included a radial braiding machine, a six-degree-of-freedom industrial robot, and image-acquisition equipment. The radial braiding machine is an 88-gear single-ring braiding machine that includes 176 spindles, and each spindle carries a yarn bobbin for braiding. The motion system of the braiding machine included a servo motor, two vibration motors, and a controller. During the braiding process, the rotating speed of the host was 600 r/min, angular velocity (ω) of the spindle movement was 0.07 rad/s, frequency of the vibration motor was 46 Hz, image acquisition device was a CMOS sensor with a maximum exposure time of 3 ms, focal length of the lens was 50 mm, and pixels were 50 lp/mm. The on-site image-acquisition equipment is shown in Figure 18.

Table 1.

Discrete mandrel points in tool coordinates system

Discrete point number	x/mm	y/mm	z/mm
0	1500.41	135.81	1570.15
1	1500.41	135.81	1605.65
2	1504.67	135.80	1637.67
3	1511.67	136.02	1675.45
4	1529.17	135.90	1705.33
5	1546.67	137.13	1735.21
6	1560.67	136.00	1764.98
7	1565.83	135.90	1797.5
8	1561.83	137.21	1834.89
9	1542	136.62	1858.86
10	1523.33	135.83	1881.21

Table 2.

Radius size of each segment of discrete mandrel

1-th/mm	2-th/mm	3-th/mm	4-th/mm	5-th/mm	6-th/mm	7-th/mm	8-th/mm	9-th/mm	10-th/mm
72.5	70	67.5	67.5	70	74.5	75.5	77	79	81.5

Figure 18.

Braiding site image acquisition equipment construction.

The traction trajectory and speed of the robot ( $V_{i}$ ) were calculated using the algorithm proposed in this article, as shown in Figure 19, and the duration of each traction is shown in Table 3, which changes with the change in the cross-sectional size of the mandrel.

Figure 19.

Comparison diagram of traction speed (V) and ideal braiding speed (Vs).

Table 3.

Traction time of each segment of the discrete mandrel

1-th/s	2-th/s	3-th/s	4-th/s	5-th/s	6-th/s	7-th/s	8-th/s	9-th/s	10-th/s
5.1312	5.6105	5.8437	5.5847	5.1032	4.4358	4.6967	4.5988	4.2870	4.1070

Table 1 lists the coordinates of the center point of the mandrel after the mandrel in the robot tool seat is discretized using the simplified braiding model introduced in the section on trajectory planning of the end of the traction robot. Table 2 lists the radius of each mandrel after discretizing 10 cylinders. Knowing the data of the discrete mandrel, the traction trajectory of the robot can be planned according to the traction robot trajectory planning algorithm proposed above. When the section size changes, the macroscopic geometric profile directly affects the offset or error between the braid angle of the machined surface and the target braid angle, which is caused by the change in the section radius. The size of the error is determined by the gradient of the section-size change. The larger the change rate, the larger the deviation; the smaller the size change, the smaller the error. After discretizing the core mold, the rate of change of the section radius was reduced, and within an acceptable range, the subsequent speed compensation was combined to meet the target requirements.

Figure 18 shows the pulling speed of each segment of the trajectory of the mandrel by the robot and compares it with the steady-state braiding speed of each segment of the mandrel. Table 3 lists the duration of the traction speed for each segment. The robot traction control is controlled by traction speed and time. First, the braiding speed was adjusted to the vicinity of the desired braiding speed, and then the distance was adjusted during the passing time. By calculating and controlling the pulling speed and pulling time, the braiding angle of the compensating fabric could be satisfied without affecting the braiding of the next mandrel, thereby obtaining a preform with a uniform braiding angle. Using the control method proposed in this study to braid the preform, the obtained product is shown in Figure 20. It can be observed that the yarn on the preform is relatively tight, and the braiding angle is constant.

Figure 20.

Preforms braided using the control algorithm proposed in this paper.

Figure 21 is a picture of the fabric surface of the same preform at three different positions, obtained using the braiding angle detection algorithm and process proposed above to measure the braiding angle. Figure 22 is the average grayscale comparison of the three positions; the braid at the three positions had angles of 60.044°, 61.124°, and 61.213°, respectively. For other measurement algorithms (Hough transform, etc.) or measurement methods (manual measurement, etc.), the error value is within the range of 2°–5°. After using the abovementioned control methods for speed and traction time to adjust the position, attitude, and the traction speed of the robot, the space curve variable cross-section mandrel was accurately braided, and the braiding angle of the special-shaped structure mandrel was 2° above and below the desired value. It floats inside to improve braiding efficiency, and under the same conditions, it can improve the mechanical properties of the composite material. In addition, for complex-shaped mandrels, the algorithm can still be used to obtain ideal results, as long as the acquired image is a proper Region of Interest(ROI).

Figure 21.

Surface pictures of the same preform at three different positions.

Figure 22.

Braiding angle detection average grayscale comparison chart.

Conclusion

In this study, a method for controlling the trajectory and speed of the mandrel in the process of robot-assisted braiding of composite fibers was proposed, and the braiding angle on the surface of the preform was detected by image processing.

This study proposes a trajectory planning method for a traction robot in the braiding process of a spatially curved special-shaped core mold, which avoids the time spent in real-time control of the robot trajectory. The proposed algorithm theory can accurately locate the robot trajectory and improve braiding accuracy.

An adjustment method for the pulling speed of the mandrel was proposed. The pulling speed of the mandrel was adjusted discretely and segmentally, and the relationship between the braiding speed of the fabric and its relative sliding speed during the braiding process was analyzed. The expected braiding angle can be reduced to within ±2° by planning the path and speed of the mandrel pulling trajectory.

To detect the braiding angle on the surface of the braided preform, an algorithm for detecting the braiding angle based on the image processing of machine vision is proposed, which can effectively calculate the average braiding angle on the surface of the tubular fabric.

The results show that the traction robot control method and machine vision detection algorithm proposed in this study meet the requirements of actual production accuracy and quality inspection and can be used as a theoretical and technical basis for manufacturing high-performance composite preform products.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project is supported by National Natural Science Foundation of China (Grant No. 51905088).

ORCID iDs

Fan Jiyong

Zhang Yujing

References

Zhou

, et al. Transverse impact performance and finite element analysis of three dimensional braided composite tubes with different braiding layers. Compos Struct 2017; 168: 345–359.

Xiao

Lanceron

Wang

, et al. Mechanical and thermal behaviors of ultra-high molecular weight polyethylene triaxial braids: The influence of structural parameters. Text Res J 2019; 89: 3362–3373.

Boisse

Colmars

Hamila

, et al. Bending and wrinkling of composite fiber preforms and prepregs. A review and new developments in the draping simulations. Compos Part B-Eng 2018; 141: 234–249.

Heieck

Hermann

Middendorf

, et al. Influence of the cover factor of 2D biaxial and triaxial braided carbon composites on their in-plane mechanical properties. Compos Struct 2017; 163: 114–122.

Hunt

Carey

JP.

A machine vision system for the braid angle measurement of tubular braided structures. Text Res J 2019; 89: 2919–2937.

Monnot

Levesque

Lebel

LL.

Automated braiding of a complex aircraft fuselage frame using a non circular braiding model.

Compos Part A-Appl S 2017; 102: 48–63.

Popper

Analysis of a circular braiding process for complex shapes.

J Text I 1994; 85: 316–337.

Zhuo

Lingling

Jianqiu

, et al. Prediction method for offset compensation on three-dimensional mandrel with spatial irregular shape. J Ind Text 2021; 50: 1205–1224.

Gondran

Abdin

Gendreau

, et al. Automated braiding of non-axisymmetric structures using an iterative inverse solution with angle control. Compos Part A-Appl S 2021; 143.

10.

Martinec

Mlynek

Petru

Calculation of the robot trajectory for the optimum directional orientation of fibre placement in the manufacture of composite profile frames.

Robot Cim-Int Manuf 2015; 35: 42–54.

11.

Hans

Cichosz

Brand

, et al. Finite element simulation of the braiding process for arbitrary mandrel shapes. Compos Part A-Appl S 2015; 77: 124–132.

12.

Ayranci

Carey

2D braided composites: A review for stiffness critical applications.

Compos Struct 2008; 85: 43–58.

13.

Glessner

Kyosev

Carrier delay-based method for development of the tracks for the transitions between patterns on 3D braiding machines with continuous rotating horn gears.

Text Res J 2021; 91: 2833–2845.

14.

Kamble

Behera

BK.

A novel geometric model of four-directional 3D braided preforms.

J Ind Text 2022; 51: 1701–1715.

15.

Huber

Kuckhoff

Gries

, et al. Three-dimensional braiding of continuous regenerated cellulose fibres. J Ind Text 2016; 45: 707–715.

16.

Bilisik

Three-dimensional braiding for composites: A review.

Text Res J 2013; 83: 1414–1436.

17.

Hajrasouliha

Nedoushan

Sheikhzadeh

, et al. Theoretical and experimental study of braid pattern in mandrels with arbitrary cross-sections. J Compos Mater 2018; 52: 4009–4022.

18.

Lian

Jiang L Mcgrath

, et al. Quantitative determination of morphological features of triaxially braided composites by the use of machine vision. Compos Sci Technol 2000; 60: 159–166.

19.

Wan

JL.

Measurement research of parameter on three-dimensional braided composite material preform surface.

J Compos Mater 2004; 38: 435–448.

20.

Sciavicco

Siciliano

Modelling and control of robot manipulators. Springer Science and Business Media, Berlin, Germany. 2011.

21.

Hunt A and Carey J. Towards the development of a machine vision system for braided composite materials. In: Proceedings of the international SAMPE technology conference, SAMPE 2016, Long Beach CA, 23-26 May 2016, pp. 15.