Data-driven affective product design using complete three-dimensional surface data

Abstract

As the main part of design display and evaluation, product three-dimensional (3D) form is the core object in affective product design. However, previous research has not yet addressed the development of technical models and method involving complete 3D surface data, and thus cannot guarantee the quality of affective product design. By using the techniques of triangular mesh model, spherical harmonic and conditional variational auto-encoder, this paper proposes a data-driven affective product design method composed of several technical models using complete 3D surface data. These models include: mathematical model for quantifying 3D form, recognition model for recognizing customer’s affective responses, and generative model for generating new 3D forms. For affective product design, the mathematical model achieves the acquisition and processing of complete 3D surface data, the recognition model improves the objectivity and accuracy of recognition by integrating the 3D form data into the calculation process of emotion recognition, and the generative model realizes the automatic generation of new 3D forms in response to emotional data based on the recognition results. Each model provides technical support for realizing the acquisition, processing and generation of complete 3D surface data of product form, and ensures the systematicness and completeness of the proposed method for the affective product design involving 3D form innovation. The feasibility of the method is verified by an example of car design, and the results show that it is an effective affective product design method involving 3D form innovation.

Keywords

Affective product form design complete three-dimensional surface data emotion recognition spherical harmonic conditional variational auto-encoder

1 Introduction

Affective product design has a decisive influence on consumers’ emotional preference and choice for products [1 –3]. By integrating customers’ cognitive behavior and emotional needs into product form design, it shows better applicability and higher practical value and attracts more and more scholars’ attention [4 –6]. Data is the basis and key of ensuring the scientificity and efficiency of design, and data-driven design has become an inevitable trend in the development of affective product design. As an integrated technical system, the data-driven affective product design method usually includes design processes such as product form quantification, emotion recognition and product form generation, and current research mainly focuses on these processes and their corresponding technical models. Focusing on the technical research status and problems involved in these different technical processes, the related work is presented in three parts below.

(1) Product form quantification

Product form quantification is the basis and prerequisite for data-driven affective product design. It mainly involves quantifying product form by using mathematical model to obtain the complete data reflecting product form feature. Considering the complete data availability and analytical convenience, two-dimensional (2D) form is the main research object of current research and its mathematical model. Compared with 2D form, three-dimensional (3D) form is the actual carrier for consumers to perceive products. It represents shapes more comprehensively and authentically, and delivers richer emotional information. Therefore, 3D form is a more necessary and valuable research object for affective product design. However, because it is difficult to obtain and process its complete surface data, current research mainly focuses on constructing mathematical models for incomplete surface data acquisition of 3D form. Hsiao et al. [7] studied the basic dimension of affective response triggered by 3D form based on the mathematical model constructed by morphological analysis technique. Chang and Chen [8] also used a mathematical model established by morphological analysis to realize the incomplete quantification of steering wheel’s 3D form and the determination of consumer’s affective preferences. However, because the morphological analysis technique only gets the classification data of partial surface of the 3D form, rather than the complete surface data, the mathematical model based on this technique obviously cannot guarantee the diversity and flexibility of the 3D form innovation in affective product design.

Among the possible techniques used in other fields for building mathematical models that can obtain complete 3D surface information, point cloud [9] and volumetric grid [10] are more representative, but both have problems that are not applicable to data-driven affective product design. Point cloud data obtained is discrete and irregular [11], its amount is large and it is not easy to be processed. Volumetric grid data has low output accuracy and introduces extra computation cost due to computational and spatial complexity, which restricts it to be applied on complex 3D forms [12]. Different from these two techniques, triangular mesh model has the applicability for affective product design. It can get the complete surface data of complex 3D forms in the form of mesh data, which has good convertibility with the common Non-Uniform Rational B-Spline (NURBS) surface data in 3D form design, and has the potential to simplify the data analysis process through data conversion and dimensionality reduction [13, 14]. In the authors’ previous studies [15, 16], we have proved that elliptical Fourier technique can process the 2D coordinate data of product form into harmonic coefficient with better form analyzability. As an extension of this technique to 3D objects, spherical harmonic (SPHARM) can process mesh data into SPHARM coefficient [17, 18]. After further extraction of data features, the dimension-reduced SPHARM coefficient is more suitable for quantitative analysis of affective product design. In this process, conditional variational auto-encoder (CVAE), a representative technology of deep learning with excellent big data processing and feature mining functions, is used to complete the processing of SPHARM coefficient. Therefore, a mathematical model that combines the triangular mesh model, SPHARM and CVAE is formed to achieve product form quantification and obtain complete 3D surface data in this study.

(2) Emotion recognition

Emotion recognition mainly involves the quantification of consumers’ affective responses to product forms. It uses several adjectives and their evaluation data that represent the type and degree of emotion as the output result of emotion recognition. The quantitative results provide design goals and data sources for data-driven affective product design, which is of great significance [19]. Many scholars have studied the construction techniques of emotion recognition model of 2D form. Shieh et al. [20] employed factor analysis to construct an emotion recognition model for the 2D car shape. Guo et al. [21] completed the emotion recognition of the 2D camera form through expert interview and cluster analysis, and in the follow-up study, Guo et al. [22] further built an emotion recognition model for the 2D car form by combining the event related potential method with physiological measurement. Sutono [23] proposed a similar emotion recognition model based on questionnaire survey, semantic difference method and cluster analysis. However, these emotion recognition models urgently need to break through the limitations of 2D objects and complete the expansion to 3D form. The modeling ideas and techniques of these models separated from the actual form data are easy to cause recognition problems such as inaccurate recognition and incomplete coverage of key emotions. These problems need to be improved in the construction of 3D form’s emotion recognition model. In the authors’ previous studies, we have used 2D form data to participate in the recognition process, and improved the accuracy and completeness of 2D form’s emotion recognition. Therefore, this research will further explore the related techniques of using complete 3D surface data to participate in emotion recognition, so as to establish a more effective emotion recognition model for 3D form.

(3) Product form generation

Emotion-oriented form generation is the final technical process of affective product design, and the generative model is the key to determining the outcome of the process. Current studies often build generative models based on hybrid algorithms. Hsiao et al. [7] established a generative model based on quantification theory type I and genetic algorithm, and verified the feasibility of the model through the shape generation of the coffee machine. Deng and Wang [24] adopted neural network and interactive genetic algorithm to construct the generative model. Lin and Zhao [25] combined neural network and particle swarm algorithm to build the emotion-oriented form generative model of cultural creative products. Although generative models based on hybrid algorithms have certain performance advantages, these studies are still in the stage of constructing 2D generative models rather than 3D generative models. In addition, there are already many design processes of affective product design, the introduction of generative model based on hybrid algorithm in the product form generation process will make the operation and debugging of the overall method more complex, and it is more susceptible to multiple constraints of performance bottlenecks of different single algorithms. For example, quantification theory type I and shallow neural network are not the best choice for mining data features and ensuring generation accuracy, so the hybrid algorithm composed of them cannot give the best performance to the generative model.

As mentioned above, CVAE provides the technique required to construct a mathematical model for complete quantification of 3D form. At the same time, it also provides favorable conditions for constructing an emotion-oriented 3D generative model. As a representative deep learning technology that uses multi-layer neural networks, CVAE has a complete algorithm structure and can build a widely used generative model with good generalization ability without mixing other algorithms [26, 27]. Its multi-layer structure gives it the advantage of more effective expression of complex functional relationships, and enables it to more efficiently extract data features from a large amount of product form and emotional data. Therefore, CVAE has the ability to generate new 3D form data in response to emotional data. To reduce the technical complexity of the method that combines different models, in view of the multiple roles of CVAE in different design processes of affective product design, this study proposes a generative model based on this technique.

In summary, by combining the above models in different design processes, a data-driven affective product design method using complete 3D surface data is proposed. Figure 1 shows the overall research framework of the method, which includes three parts. In the first part, the complete quantification of 3D form is realized by using the techniques of expert interview, triangular mesh model and SPHARM, and the reduced-dimensional SPHARM coefficient matrix is obtained by using the CVAE technique. In the second part, emotion recognition and quantification are achieved by using the techniques of expert interview, questionnaire survey, recognition model, cluster analysis and factor analysis. In the last part, new 3D forms that respond to emotional data are generated by using the generative model.

Fig. 1

Research framework of the proposed method.

The innovations of this paper are summarized as follows:

A mathematical model for complete quantification of 3D form is first constructed, which achieves quantitative description, data acquisition and processing of 3D form. This laid a solid foundation for the formation of quantitative research technique of affective product design involving 3D form innovation.

A more effective emotion recognition model for 3D form driven by its complete surface data is proposed, which forms a systematic and effective emotion recognition technique for 3D form.

It innovatively proposes the 3D form generation and evaluation technique based on CVAE and effect evaluation indexes, and realizes the generation of rich and innovative novel 3D form design schemes that meet design goals.

The rest of this study is organized as follows. Section 2 provides a theoretical framework and background overview of the related theories and techniques. Section 3 details the implementation procedures of the proposed method, and Section 4 presents some discussions of the results. Finally, Section 5 demonstrates some brief conclusions.

2 Theoretical framework and background

2.1 Mathematical model construction

2.1.1 3D mesh data acquisition

NURBS is an advanced 3D surface modeling method. NURBS surface is a free-form surface, which is equivalent to a thin square rubber sheet with infinite flexibility. Designers usually use it to carry out product 3D modeling in actual design operations. After obtaining the picture samples of the target product, use NURBS surface modeling software-Rhinoceros 3D to complete the 3D form modeling of the samples, and then save their triangle mesh models in the OBJ file format through the data format conversion. Figure 2 shows the process of acquiring 3D mesh data for a certain product.

Fig. 2

3D mesh data acquisition process.

2.1.2 SPHARM technique

SPHARM technique is well suitable for general surface manipulation and analysis applications of triangle mesh model [28]. As a parametric technique for surface representation, SPHARM takes the angular space coordinates of 3D surface points and the radius from an internal center point as parameters to expand the surface into SPHARM functions, and hierarchically describes the comprehensive and local form properties of 3D surface through spatial frequency components [29]. It first defines a 3D form by three SPHARM functions, then converts them into three sets of SPHARM coefficients by Fourier transform [30] to complete the morphological information transformation from time domain to frequency domain and get the standard spectrum information that can be used to quantitatively analyze the 3D form. The whole process is generally composed of three steps, which are (1) spherical parameterization, (2) spherical Fourier transform, and (3) SPHARM coefficients normalization.

Spherical parameterization is the process of obtaining a bijective mapping between each point p on a 3D surface and a pair of unit spherical coordinates ω (polar coordinate) and Φ (azimuthal coordinate) [31], which is expressed as $P (ω, Φ) = (u (ω, Φ), v (ω, Φ), w (ω, Φ))^{T}$ (1)

Where ω ∈ [0, π], Φ ∈ [0, 2π], and u² (ω, Φ) + v² (ω, Φ) + w² (ω, Φ) =1

Since any continuous function P (ω, Φ) defined on a spherical surface can be expanded by SPHARM function, then a 3D surface is expanded into a complete SPHARM function by spherical Fourier transform as $P (ω, Φ) = \sum_{l = 0}^{\infty} \sum_{m = - l}^{l} c_{l}^{m} Y_{l}^{m} (ω, Φ)$ (2)

Where $Y_{l}^{m}$ is a Fourier basis function defined on a sphere, its degree is l and order is m. SPHARM coefficient, $c_{l}^{m} = (c_{xl}^{m}, c_{yl}^{m}, c_{zl}^{m})^{T}$ , can be expressed as follows: $c_{l}^{m} = \int_{0}^{2 π} \int_{0}^{π} P (ω, Φ) Y_{l}^{m} (ω, Φ) sin (ω) d ω d Φ$ (3)

To perform group analysis and have comparability between different samples, the SPHARM coefficients of each sample need to be normalized to remove the effects of scaling, rotation, and translation. First order ellipsoid [32] technique is adopted to obtain the normalized SPHARM coefficients of each sample, and the SPHARM coefficient matrix of the overall sample is established based on the coefficients.

2.1.3 SPHARM coefficient matrix dimensionality reduction

CVAE is further used to perform dimensionality reduction and data feature extraction on SPHARM coefficient matrix. Its network structure is shown in Fig. 3, which includes a recognition network (encoder) and a generative network (decoder). The output result of the encoding process of the encoder realizes the dimensionality reduction of the SPHARM coefficient matrix and assists in the completion of emotion recognition, and the output result of the generation network achieves the generation of new 3D form data in combination with external conditions (emotional data). By using the multi-layer neural network of the encoder, CVAE can extract the low-dimensional deep structural features of the high-dimensional input data as the representation of the data. The lowest dimensional vector x_n of the latent space is the output data after dimensionality reduction.

Fig. 3

Network structure of CVAE.

After the network coding of CVAE, the SPHARM coefficient matrix with the total sample size of s will be output as a t-dimensional data matrix as $C = [\begin{matrix} C_{11} & . . . & C_{1 t} \\ ⋮ & ⋱ & ⋮ \\ C_{s 1} & \dots & C_{st} \end{matrix}]$ (4)

2.2 Recognition model construction

The steps of using the reduced-dimensional SPHARM coefficient matrix to further participate in the construction of the recognition model are as follows:

After collecting and screening adjectives, a focus group composed of industry experts is invited to select the relevant adjectives;

Questionnaires are designed by combining representative 3D form samples with relevant adjectives and conducted to get the evaluation scores of all test subjects, which is given as $T = [\begin{array}{l} \begin{matrix} T_{_{11}}^{(k)} & T_{_{12}}^{(k)} & ... & T_{_{1 j}}^{(k)} \\ T_{_{21}}^{(k)} & T_{_{22}}^{(k)} & ... & T_{_{2 j}}^{(k)} \\ ⋮ & ⋮ & ⋮ & ⋮ \end{matrix} \\ \begin{matrix} T_{_{i 1}}^{(k)} & T_{_{i 2}}^{(k)} & ... & T_{_{i j}}^{(k)} \end{matrix} \end{array}]$ (5)

Where: i is the sample number; j is the number of relevant adjective; $T_{ij}^{(k)}$ is the image evaluation score of the j-th product image adjective of the i-th sample by the k-th subject.

The evaluation mean score (EMS) matrix $\bar{T}$ of all test subjects and standard deviation σ of the evaluation data are calculated as $\bar{T} = \sum_{k = 1}^{K} T_{ij}^{(k)} / K$ (6)

$σ = {[\frac{1}{K - 1} \sum_{k = 1}^{K} (T - \bar{T})^{2}]}^{1 / 2}$ (7)

Where: K is the total number of test subjects.

As a method of eliminating the abnormal value from data set, Pauta criterion is used to determine the validity of the subjects’ evaluation data. The method uses the unbiased estimator of the standard deviation of the statistical data calculated by the Bessel formula as the residual σ. It states that the sample values are almost completely within 3σ, so highly abnormal sample values exceeding 3σ are deleted. Consider the evaluation data of individual test subjects that meet the condition of $| T^{(k)} - \bar{T} | > 3 σ$ as gross errors and remove them from the matrix.

The correlation analysis was conducted to get the correlation coefficient matrix R, and assign a multiplier coefficient α (α = 0, 1, 2) to different significance levels of the correlation coefficient in R, and construct the recognition model as $S_{u} = \sum_{v = 1}^{v max} (| R_{uv} | \times α), α = {\begin{matrix} 2, P_{uv} < 0.01 \\ 1, 0.01 ⩽ P_{uv} < 0.05 \\ 0, P_{uv} ⩾ 0.05 \end{matrix}$ (8)

Where: u is the number of relevant adjective; v is the coefficient number; v_max is the maximum number of v; R_uv and P_uv are the correlation coefficient and significance level between the u-th adjective and the v-th coefficient number, respectively; S_u is the final emotional score (FES).

Representative adjectives and their rankings are determined according to the score, and the FESs of representative adjectives constitute the recognition result;

Factor analysis and clustering analysis are performed to get the sample emotional label.

2.3 Generative model construction

CVAE is currently one of the most advanced methods of deep learning generative models. Its technical principle is based on the variational auto encoder (VAE). VAE can generate output data with the same data distribution as the input data but with different numerical values, but lacks a certain orientation in data generation, so it cannot realize the generation of new 3D form data matching with the specified target emotion required by affective product design. By introducing label data as external information, CVAE makes up for the deficiency of VAE in the application of affective product design. It has the ability to generate output data in a specified direction and is an ideal generative model technique. Therefore, to take into account the technical needs of both the mathematical model and the generative model, the generative model is constructed with CVAE technique. As shown in Fig. 3, the decoder structure enables CVAE to have the data generalization capability. By encoding the probability density distributions of input data and the sample emotional label data $Y = {y_{i}}_{i = 1}^{n}$ with one-hot encoding, CVAE can output data with the same distribution but different values from the input data, and therefore it has the function of using the emotional label data to generate 3D form data in response to emotional data. In its decoder structure, z is a low-dimensional unobserved latent variable, φ and θ are constraint parameters of two networks, and μ and σ are mean and variance. For its further technical details, please see Sohn et al. [26].

3 Implementation procedures

Cars are typical products with complex 3D surfaces and display a variety of emotional types, the feasibility and effectiveness of the proposed method are illustrated in detail by an example of car form design in the following sections.

3.1 3D form quantification

In consideration of the brand and visual differentiation, we extensively collected 300 pictures of car samples with three-box shape through websites, car magazines and brochures. Each sample was displayed its 3D form features with three-view drawings for sample screening. A focus group of nine experts (five men and four women) with more than two years of car design experience was established. The main functions of the focus group are sample screening and adjective screening. To better accomplish this task, the experts come from the academic and industrial circles of car form design, of which 6 are teachers from universities majoring in industrial design, and 3 are senior designers from car design companies. They all have rich practical experience in car form design, and have a keen insight into the similarity and emotion recognition of car form. These provide a guarantee for accurately selecting representative samples and adjectives. Then, the focus group conducted a sample screening based on visual similarity, and 256 samples were retained as the research objects. The 3D solid modeling of each sample was completed to build the overall sample database. The specific 3D forms of all samples are showed in Fig. 4.

Fig. 4

3D forms of all samples.

The 3D forms of all samples were first preprocessed to obtain their mesh data containing vertices and faces. In this process, the surface of each sample need to be closed first, so as to meet the requirement of SPHARM that its parameterized object can only be genus zero surfaces [31]. To normalize the morphological data of different samples for data analysis, the number of vertices and faces was set to be close to the value of 20,000 vertices and 40,000 faces, on the premise of guaranteeing the product form features of each sample. Each vertex is given an identifying number, and each face is marked by three vertices with different identifying numbers. Figure 5 manifests the result of the data preprocessing of sample NO. 1. Then, the mesh data of all samples were extracted by using SPHARM software. Table 1 demonstrates the mesh data of sample NO. 1, including 19923 vertices and 39842 faces.

Fig. 5

Data preprocessing of sample NO. 1.

Table 1

Mesh data of sample NO. 1

Vertex	x	y	z	Face	Identifying number
1	1.9686	0.6174	1.1166	1	17115	15307	15077
2	1.0484	0.4242	–1.9130	2	13922	18721	18017
3	–1.2311	2.2114	4.0791	3	1960	5968	9083
4	–0.5539	3.1631	–2.2680	4	848	12633	12942
5	–2.1348	1.2922	–5.6823	5	14223	7928	19576
10	0.3745	2.9396	–3.1928	10	624	2582	8868
50	2.2984	1.4343	4.5280	50	15193	15196	17898
100	1.3315	0.7530	–5.8086	100	476	884	10573
500	–1.6634	2.6012	–4.0360	500	3047	10083	4114
1000	–1.1689	1.7990	–5.9915	1000	755	7805	4856
5000	–1.2242	0.4374	5.4813	5000	2649	4715	10144
10000	1.0950	0.5174	–4.3316	10000	1803	1079	5002
15000	–0.7836	3.2214	0.2632	20000	1356	14140	10002
19923	–0.4236	0.4074	–0.9581	39842	19778	19920	19923

Then, all samples were subjected to spherical parameterization using Equation (1). The surface mesh and unit sphere surface mesh of sample NO. 1 after spherical parameterization are manifested in Fig. 6. After spherical parameterization, the spherical vertex coordinates mapped to the unit sphere of all samples were obtained. Table 2 shows the coordinates of sample NO. 1.

Fig. 6

Spherical parameterization of sample NO. 1.

Table 2

Spherical vertex coordinates of sample NO. 1

Spherical vertex	u	v	w
1	0.9397	0.1584	0.3032
2	0.8911	–0.2685	–0.3660
3	–0.6547	0.0131	0.7558
4	–0.6792	0.6657	–0.3090
5	–0.4367	–0.1898	–0.8794
10	–0.2321	0.8663	–0.4424
50	0.4217	0.3193	0.8486
500	–0.7327	0.2759	–0.6221
1000	–0.2165	0.0804	–0.9730
5000	–0.0099	–0.5061	0.8624
10000	0.6273	–0.0974	–0.7727
15000	–0.8534	0.5110	0.1027
19923	0.4828	–0.8628	–0.1502

Spherical Fourier transform was performed on the spherical vertex coordinates of each sample using Equations (2) and (3) to obtain its SPHARM coefficients. In this process, we tested the visual effect of 3D form reconstruction for each sample under different degrees, and determined that the ideal degree l suitable for representing the main features of different samples was 30. Figure 7 displays the 3D form reconstruction of sample No.1 under different degrees. With the increase of degree, the surface features obtained by reconstruction are more and more abundant and detailed. When l = 30, the reconstructed visual effect is fine enough to represent the main form features, and the shape noise is well controlled. Consequently, the SPHARM coefficients of all samples were obtained at this degree. The SPHARM coefficients of sample No.1 are manifested on the left column of Table 3, and its normalized SPHARM coefficients are demonstrated on the right column of Table 3. Subsequently, the SPHARM coefficient matrix of the overall sample as shown in Table 4 was established.

Fig. 7

3D form reconstruction of sample No. 1 under different degrees.

Table 3

SPHARM coefficients of sample No. 1

	SPHARM coefficients			Normalized SPHARM coefficients
	$c_{xl}^{m}$	$c_{yl}^{m}$	$c_{zl}^{m}$	$c_{xl}^{' m}$	$c_{yl}^{' m}$	$c_{zl}^{' m}$
1	5.42 + 1.28e-16i	0.37-9.72e-15i	-1.23 + 5.05e-18i	0.08-0.68i	-0.63-0.68i	-0.85-0.68i
2	2.31 + 1.07e-14i	4.34e-16 + 3.60i	-1.69e-15 + 1.11e-15i	-0.36-0.68i	-0.68-0.18i	-0.68-0.68i
3	1.67e-16 + 4.63e-16i	3.33e-16 + 1.19e-15i	12.04-1.58e-16i	-0.68-0.68i	-0.68-0.68i	1.00-0.68i
4	-2.31 + 1.00e-14i	1.22e-15 + 3.60i	1.96e-15 + 2.97e-15i	-1.00-0.68i	-0.68-0.18i	-0.68-0.68i
5	0.062-0.05i	0.13-0.10i	0.08 + 0.04i	-0.67-0.69i	-0.66-0.70i	-0.67-0.67i
10	-0.32 + 0.01i	-0.02-0.16i	0.00- 0.01i	-0.72-0.68i	-0.68-0.70i	-0.68-0.68i
50	0.03 + 0.01i	-0.00-0.00i	0.00 + 0.00i	-0.67-0.68i	-0.68-0.68i	-0.68-0.68i
100	0.00-0.00i	0.00 + 0.02i	0.00-0.00i	-0.68-0.68i	-0.68-0.68i	-0.68-0.68i
200	0.00 + 0.00i	-0.00 + 0.00i	-0.00 + 0.00i	-0.68-0.68i	-0.68-0.68i	-0.68-0.68i
500	-0.00 + 0.00i	-0.00 + 0.00i	-0.00-0.00i	-0.68-0.68i	-0.68-0.68i	-0.68-0.68i
900	1.52e-05 + 2.11e-05i	-3.82e-05 + 3.31e-05i	-0.00 + 0.00i	-0.68-0.68i	-0.68-0.68i	-0.68-0.68i

Table 4

SPHARM coefficient matrix of the overall sample

Sample NO.	Coefficient NO.
	1	2	3	4	5	...	5396	5397	5398	5399	5400
1	0.077	–0.628	–0.850	–0.357	–0.679	...	–0.679	–0.679	–0.679	–0.678	–0.676
2	–0.295	–0.460	0.564	–0.339	–0.670	...	–0.670	–0.670	–0.669	–0.670	–0.668
3	–0.728	–0.664	–0.591	–0.329	–0.665	...	–0.665	–0.665	–0.663	–0.663	–0.662
4	–1.000	–0.393	–0.326	–0.091	–0.368	...	–0.368	–0.368	–0.368	–0.368	–0.366
5	0.366	–0.875	–1.000	–0.211	–0.531	...	–0.531	–0.531	–0.531	–0.531	–0.528
10	–1.000	–0.362	–0.266	–0.084	–0.363	...	–0.364	–0.362	–0.361	–0.364	–0.361
50	–0.813	–0.657	–0.592	–0.287	–0.644	...	–0.644	–0.643	–0.643	–0.643	–0.642
100	0.160	–0.717	–0.502	–0.333	–0.666	...	–0.667	–0.667	–0.667	–0.665	–0.663
150	0.259	–0.567	–0.477	–0.263	–0.632	...	–0.632	–0.631	–0.632	–0.631	–0.629
200	0.197	–0.671	–0.507	–0.229	–0.614	...	–0.614	–0.615	–0.614	–0.614	–0.611
256	0.191	–0.696	–0.543	–0.312	–0.656	...	–0.656	–0.657	–0.655	–0.654	–0.653

After using CVAE for data feature extraction, the dimension of SPHARM coefficient matrix was reduced from 5400 to 8 (Table 5).

Table 5

Reduced-dimensional SPHARM coefficient matrix

Sample NO.	Coefficient NO.
	C₁	C₂	C₃	C₄	C₅	C₆	C₇	C₈
1	–4.0066	0.0143	–0.2700	–0.1452	–0.0496	–0.0153	0.0520	–0.0318
2	–3.3435	–0.5264	0.8910	0.4479	0.3634	–0.0438	0.0522	0.0449
3	–2.9698	–0.7730	–0.0587	–0.1070	–0.1830	0.0419	–0.0770	0.0495
4	18.7934	–0.7683	0.1790	–0.1320	–0.0857	0.0734	–0.0069	–0.0595
5	6.8230	0.5112	–0.3925	–0.0468	–0.3207	0.0194	–0.0672	0.0266
10	19.1842	–0.7733	0.2251	–0.1082	–0.0395	0.0480	0.0238	–0.0901
50	–1.4260	–0.8314	–0.0535	–0.1305	–0.1959	–0.0725	0.0491	0.0521
100	–3.1145	0.0774	0.0101	0.0987	–0.0461	0.1052	–0.0463	0.0510
150	–0.5562	0.1868	0.0101	0.0906	0.1245	–0.0110	0.0444	–0.0540
200	0.7041	0.1570	–0.0317	0.1457	–0.0042	0.1266	0.0289	–0.0135
256	–2.3369	0.1223	–0.0212	0.0720	–0.0353	0.1026	–0.0617	0.0584

3.2 Emotion recognition and label generation

3.2.1 Emotion evaluation and recognition

Firstly, the 36 adjectives identified in Wang et al. [15] that are closely related to the emotional attributes of cars with three-box shape are taken as the preliminary relevant adjectives in this study (Table 6).

Table 6
Preliminary relevant adjectives

Adorable Complex Expensive Futuristic Hard Male Organic Simple Sturdy

Avant-garde Contemporary Exquisite Generous Lightweight Modern Passionate Smooth Tough

Classic Dynamic Fahionable Geometric Lively Natural Rational Soft Unique

Common Elegant Formal Harmonious Luxurious Neutral Round Steady Wild

Adorable	Complex	Expensive	Futuristic	Hard	Male	Organic	Simple	Sturdy
Avant-garde	Contemporary	Exquisite	Generous	Lightweight	Modern	Passionate	Smooth	Tough
Classic	Dynamic	Fahionable	Geometric	Lively	Natural	Rational	Soft	Unique
Common	Elegant	Formal	Harmonious	Luxurious	Neutral	Round	Steady	Wild

To allow the focus group to do the omni-directional observation of samples’ 3D forms from multiple perspectives, a standardized observation video was made for each sample. Figure 8 shows multiple screenshots of the video for sample No. 1.

Fig. 8

Screenshots of video for sample No. 1.

All videos were produced in a uniform standard, with the same length and shooting angle. Each sample was set to rotate once around the same rotational axis that coincides with the central axis of its shape within the same unit time (10 seconds). Focus group experts were invited to browse videos of 50 randomly selected samples from all samples, and then select the adjectives that best represented the emotion of the sample’s 3D form from 36 preliminary related adjectives. After assembling all expert opinions, the adjectives were sorted by occurrence frequency and the top 15 adjectives were retained as relevant adjectives. Subsequently, 100 senior college students studying in industrial design major (50 males and 50 females) were invited to conduct a questionnaire survey on the 3D form emotions of the overall sample. Due to the large sample size, to reduce the burden on test subjects and improve the evaluation accuracy, the sample and test subjects were randomly divided into 4 groups, respectively. Twenty-five test subjects in each group (men and women at the same proportion) evaluated 64 samples using 7-point semantic differential method (-3-3), where -3 is the lowest score and 3 is the highest for adjectives. The Z - score normalization method was used to check whether there were abnormal data in each group. After data verification, the EMS of all samples were calculated (Table 7).

Table 7

EMS matrix

Sample NO.	Relevant adjectives
	Steady	Generous	Elegant	Luxurious	Modern	Organic	Avant-garde	Dynamic	Smooth	Unique	Geometric	Concise	Common	Lightweight	Formal
1	2.06	1.95	1.54	1.91	–1.10	1.43	–1.04	–0.95	–1.56	2.01	1.05	–1.00	–1.96	–2.07	–0.95
2	2.08	1.47	0.92	1.41	1.06	–1.06	–0.97	–0.98	0.95	–0.97	–0.97	1.60	–1.06	–0.92	1.98
3	2.43	2.54	2.01	2.42	2.00	–1.06	1.03	1.05	1.01	0.51	–1.00	–1.00	–2.47	–1.97	2.07
4	2.58	2.43	1.51	2.06	2.01	–1.09	0.42	0.53	1.05	0.09	–1.05	–0.46	–2.01	–1.57	2.09
5	2.03	1.54	–0.43	1.59	–0.93	–2.08	–1.52	–1.58	–1.03	–0.93	2.09	–1.52	–2.01	–0.99	1.98
10	1.59	1.24	–0.50	0.92	0.09	1.10	1.07	1.94	–0.46	2.44	–1.09	–2.08	–0.99	–1.25	–2.00
50	1.03	0.27	0.22	0.25	–1.03	–0.47	–1.43	–1.10	0.25	–2.02	0.99	0.75	–0.29	–0.86	0.08
100	–0.07	0.27	0.40	0.12	0.06	–0.19	0.04	0.16	0.98	0.95	0.43	0.54	0.14	0.01	–1.00
150	–1.53	–2.02	–1.44	–1.97	–1.46	–1.08	–1.47	–1.44	1.07	–2.44	0.91	1.37	2.04	1.96	0.02
200	2.08	–0.95	–1.98	0.97	1.57	–0.94	1.99	1.97	–0.95	2.07	1.45	1.76	–0.95	–1.06	–1.59
256	–0.58	–1.57	–1.45	–0.97	–1.54	–0.92	–1.95	–1.50	0.59	–1.97	0.94	1.48	1.02	0.46	0.59

Then, the correlation coefficient matrix were further calculated (Table 8), and the recognition model was used to get the FES and the rankings of relevant adjectives (Table 9). Finally, a total of 12 adjectives with a score bigger than 1 were further selected as representative adjectives.

Table 8

Correlation coefficient matrix

Relevant adjectives	C₁	C₂	C₃	C₄	C₅	C₆	C₇	C₈
Steady	–0.149^*(0.017)	–0.100(0.109)	0.103(0.101)	–0.082(0.193)	0.107(0.087)	–0.139^*(0.026)	–0.123^*(0.050)	–0.072(0.252)
Generous	–0.313^**(0.000)	–0.239^**(0.000)	0.241^**(0.000)	–0.075(0.229)	0.265^**(0.000)	–0.301^**(0.000)	–0.278^**(0.000)	–0.064(0.307)
Elegant	–0.318^**(0.000)	–0.239^**(0.000)	0.271^**(0.000)	–0.038(0.549)	0.270^**(0.000)	–0.338^**(0.000)	–0.302^**(0.000)	–0.071(0.255)
Luxurious	–0.269^**(0.000)	–0.219^**(0.000)	0.201^**(0.000)	–0.133^*(0.034)	0.337^**(0.000)	–0.335^**(0.000)	–0.292^**(0.000)	–0.071(0.258)
Modern	–0.158^*(0.011)	–0.118(0.060)	0.062(0.325)	–0.178^*(0.004)	0.285^**(0.000)	–0.309^**(0.000)	–0.292^**(0.000)	–0.005(0.933)
Organic	–0.072(0.249)	–0.062(0.322)	0.019(0.763)	–0.136^*(0.030)	0.191^**(0.002)	–0.222^**(0.000)	–0.223^**(0.000)	0.040(0.528)
Avantgardee	–0.179^*(0.004)	–0.147^*(0.019)	0.104(0.096)	–0.127^*(0.042)	0.252^**(0.000)	–0.309^**(0.000)	–0.286^**(0.000)	0.003(0.966)
Dynamic	–0.091(0.145)	–0.069(0.271)	0.007(0.907)	–0.162^*(0.010)	0.196^**(0.002)	–0.239^**(0.000)	–0.232^**(0.000)	0.033(0.604)
Lightweight	–0.107(0.086)	–0.077(0.221)	0.097(0.122)	0.009(0.884)	0.002(0.969)	–0.088(0.162)	–0.078(0.215)	0.035(0.576)
Unique	–0.192^**(0.002)	–0.195^**(0.002)	0.150^*(0.016)	–0.072(0.253)	0.159^*(0.011)	–0.231^**(0.000)	–0.181^*(0.004)	–0.015(0.816)
Geometric	0.099(0.114)	0.079(0.210)	–0.059(0.346)	0.100(0.109)	–0.199^*(0.001)	0.239^**(0.000)	0.236^**(0.000)	–0.024(0.699)
Concise	0.365^**(0.000)	0.349^**(0.000)	–0.298^**(0.000)	–0.039(0.531)	–0.190^*(0.002)	0.261^**(0.000)	0.177^**(0.005)	0.154^*(0.014)
Common	0.292^**(0.000)	0.242^**(0.000)	–0.221^**(0.000)	0.145^*(0.020)	–0.255^**(0.000)	0.353^**(0.000)	0.208^**(0.000)	0.091(0.145)
Smooth	–0.149^*(0.017)	–0.176^**(0.005)	0.187^**(0.003)	0.194^**(0.002)	–0.168^**(0.007)	0.052(0.408)	0.079(0.208)	–0.057(0.366)
Formal	0.154^*(0.014)	0.183^**(0.003)	–0.152^*(0.015)	–0.023(0.715)	–0.049(0.438)	0.128^*(0.041)	0.076(0.227)	0.033(0.600)

Note: The value of significance level P is the data in bracket, * indicates P < 0.05 and significantly correlated. ** indicates P < 0.01 and very significantly correlated.

Table 9

FES and rankings of representative adjectives

Rankings	Representative adjectives	FES
1	Elegant	3.476
2	Luxurious	3.439
3	Generous	3.274
4	Concise	3.244
5	Common	3.187
6	Avant-garde	2.147
7	Modern	2.108
8	Unique	1.726
9	Smooth	1.599
10	Dynamic	1.496
11	Organic	1.408
12	Geometric	1.149

Since the reduced-dimensional SPHARM coefficient matrix extracts the 3D form data features of the overall sample, the correlation coefficient and FES objectively represent the quantitative relationship between 3D form features and its emotional attributes, so the proposed recognition model has higher objectivity and accuracy. In addition, it quantitatively calculates and ranks the importance of all representative adjectives, fully retains the key emotional information, and helps to set scientific and flexible 3D form design goals.

3.2.2 Emotional label generation

Based on the recognition of representative adjectives, factor analysis was performed on the EMS matrix of 12 representative adjectives by SPSS statistics software. In the light of the variance contribution rate in the result of factor analysis manifested in Table 10, four principal factors were determined, whose cumulative variance contribution rate reached 83.78%.

Table 10
Total variance of interpretation

Factor number Initial eigenvalue Extracting square sum loading Rotation square sum loading

Total Proportion (%) Cumulative (%) Total Proportion (%) Cumulative (%) Total Proportion (%) Cumulative (%)

1 5.36 44.71 44.71 5.36 44.71 44.71 3.47 28.89 28.89

2 2.50 20.82 65.52 2.50 20.82 65.52 3.17 26.41 55.31

3 1.11 9.24 74.77 1.11 9.24 74.77 1.78 14.80 70.11

4 1.08 9.01 83.78 1.08 9.01 83.78 1.64 13.67 83.78

Factor number	Initial eigenvalue	Extracting square sum loading	Rotation square sum loading
1	5.36	44.71	44.71	5.36	44.71	44.71	3.47	28.89	28.89
2	2.50	20.82	65.52	2.50	20.82	65.52	3.17	26.41	55.31
3	1.11	9.24	74.77	1.11	9.24	74.77	1.78	14.80	70.11
4	1.08	9.01	83.78	1.08	9.01	83.78	1.64	13.67	83.78

According to the maximum factor loading, as shown in Table 11, the four principal factors were named as “high-grade”, “dynamic”, “organic” and “unique” respectively, and they were set as sample emotional labels of this study.

Table 11

Rotated component matrix

Representative adjectives	Factor number
	Factor 1	Factor 2	Factor 3	Factor 4
Generous	0.9258	0.1095	–0.1187	0.1526
Elegant	0.9069	0.0433	0.1937	–0.1596
Luxurious	0.8599	0.3205	–0.0863	0.2816
Common	–0.8272	–0.3117	0.0801	–0.3147
Dynamic	0.0838	0.9322	0.1940	0.0874
Avant–garde	0.2534	0.9040	0.1294	0.1660
Modern	0.2969	0.8744	0.1506	0.1625
Organic	0.0478	0.3579	0.8120	0.1996
Geometric	–0.0904	–0.4245	–0.7829	–0.1811
Smooth	–0.3783	–0.2821	0.5806	–0.1633
Concise	–0.2231	–0.0057	–0.0579	–0.8219
Unique	0.0322	0.3422	0.1455	0.7598

Then, k-means clustering analysis was performed on the scores of the four principal factors (left column of Table 12) extracted after the factor analysis, and all samples were classified into four clusters according to the four emotional labels, so that each sample had its own emotional label number (i.e., cluster number, right column of Table 12). These numbers are the label data, which together with the SPHARM coefficient matrix make up the input data of CVAE. The total number of samples in each cluster is shown in Table 13.

Table 12

Scores of principal factors and result of k-means clustering analysis

Sample number	Scores of four principal factors				Cluster member
	Factor 1	Factor 2	Factor 3	Factor 4	Cluster number	Distance
1	1.8918	–1.6254	0.7371	1.8048	1	1.7814
2	1.4877	–0.3897	0.7218	–1.3985	2	1.7340
3	2.1863	0.5150	0.1021	0.3994	2	1.8351
4	1.9317	0.2404	0.2915	0.1761	2	1.5160
5	1.1626	–1.7668	–1.4561	1.4778	4	2.2685
10	0.2056	0.2181	1.2135	2.5550	1	2.5984
50	0.6141	–1.1234	0.2050	–0.9028	2	1.7036
100	0.1319	–0.2344	0.7845	–0.1247	2	0.9139
150	–1.4660	–1.2473	1.0172	–1.5302	3	1.7568
200	–1.0761	2.2588	–1.5543	–0.2665	4	2.7345
256	–0.9421	–1.3155	0.3862	–0.7823	3	1.0790

Table 13

Sample size and proportion in four clusters

Cluster	Size	Proportion (%)
1	22	8.5938
2	117	45.7031
3	65	25.3906
4	52	20.3125

3.3 3D form generation based on generative model

3.3.1 Model structure and parameter

After comparison and analysis, the encoder and decoder structure of the CVAE with three hidden layers were affirmed finally (Fig. 9).

Fig. 9

Network structure of CVAE.

As manifested in Fig. 9, the sample emotional label data was input into the second fully connected layer (FCL) of the encoder and the third FCL of the decoder, respectively. Except that the output layer uses the Sigmoid function, all other activation functions adopt the Relu function, and the Adam optimization algorithm is used as the model’s optimizer. The specific dimension number of hidden layers and other parameters of the generative model were illustrated in Table 14.

Table 14

Network structure and parameters of generative

Type	Attribute
Network Structure	5400⟶200⟶8⟶2⟶8⟶200⟶5400
Input data	SPHARM coefficients and sample emotional label data
Output data	SPHARM coefficients of newly generated 3D forms
Batch size	10
Learning rate	0.001
Epoch	20
Number of training	204
samples
Number of test	52
samples

3.3.2 Novel 3D form schemes generation and evaluation

Then, the generative model was independently run on a ThinkStation P410 (Xeon E5-1620 v3) with windows 10 operating system and implemented by using the Python and Keras API according to the above parameters. After inputting the data into the generative model, SPHARM coefficients of 8 novel 3D form schemes (2 schemes were generated for each emotional label) were generated. Figure 10 displays the 3D forms reconstructed from the SPHARM coefficients of these schemes. All test subjects were invited to evaluate these schemes using four labels, and then their EMSs were calculated (Table 15). It can be seen that these schemes perform well on the score of the emotional label to which they belong, which indicates that the generative model achieves the generation of novel schemes that conform to the target emotion.

Fig. 10

Novel schemes reconstructed from newly generated SPHARM coefficients.

Table 15

EMSs of 8 novel schemes

NO.	High-grade	Dynamic	Organic	Unique
1	2.115	1.520	1.485	–0.425
2	1.840	1.255	1.805	–0.210
3	1.520	2.335	–0.575	1.970
4	1.655	2.520	–0.710	1.925
5	1.205	1.265	2.165	1.055
6	1.755	2.145	1.990	1.565
7	1.645	1.720	–0.435	2.270
8	1.580	1.565	1.965	1.855

4 Discussions

The following is a further discussion of the core techniques, technical parameters and design effects of the proposed method.

4.1 Values of SPHARM

SPHARM is the core technique that forms the 3D form quantitative function of the method. Its most important value is to realize the conversion of multi-type and difficult-to-process mesh data to single-type and easy-to-process SPHARM coefficient data, which greatly enhances the analyzability of 3D form data and the applicability of affective product design. SPHARM coefficient shows good properties for data feature extraction and feature reconstruction for 3D forms, which is helpful to construct the recognition model with 3D form data participation and the generative model based on a single high-performance algorithm. The variance contribution rate in Table 16 illustrates that most data features of SPHARM coefficient can be extracted by a small number of principal components. The reduced-dimensional SPHARM coefficient after feature extraction improves the objectivity and accuracy of emotion recognition and the generative quality of 3D form. In addition, the SPHARM coefficient endows the method with high operational flexibility and applicability for the affective product design involving 3D form innovation of different types of products. It can be flexibly adjusted to reconstruct 3D forms according to actual engineering needs, using high frequency harmonics to represent complex and irregular surfaces, and using low frequency harmonics to describe simple and continuous surfaces. Meanwhile, It has the advantages of fixed interpolation and precise scaling for 3D forms and is suitable for data compression and statistical modeling [31, 33].

Table 16
Variance contribution rate of each PC

PC number Proportion (%) Cumulative (%)

PC₁ 99.6275 99.6275

PC₂ 0.1694 99.7969

PC₃ 0.0839 99.8808

PC₄ 0.0503 99.9311

PC₅ 0.0288 99.9599

PC₆ 0.0081 99.9680

PC₇ 0.0061 99.9742

PC₈ 0.0033 99.9774

PC number	Proportion (%)	Cumulative (%)
PC₁	99.6275	99.6275
PC₂	0.1694	99.7969
PC₃	0.0839	99.8808
PC₄	0.0503	99.9311
PC₅	0.0288	99.9599
PC₆	0.0081	99.9680
PC₇	0.0061	99.9742
PC₈	0.0033	99.9774

4.2 Network structure of CVAE

Due to its global role, CVAE is a key factor in determining the overall performance of the method, and the reasonable selection of its network structure is crucial. There are many factors influenced the determination of its network structure. Firstly, to observe the data distribution of latent space, CVAE usually sets the dimension of bottleneck layer as 2. After testing, it was found that when the number of hidden layers in the model’s encoder and decoder is 3, the calculation precision and efficiency can be guaranteed at the same time, and further increasing the number of hidden layers does not enhance the model’s convergence effect, but adds the computational complexity and time, so the number of hidden layers in both networks of the model was set to be 3. Then, to extract a low-dimensional SPHARM matrix to participate in the generation of sample emotional label, an appropriate low dimension of the second FCL was determined in front of the bottleneck layer. Through the test of principal component analysis, it was found that when there were 4–12 principal components, enough original data information can be extracted. This dimension range is also helpful to reduce the burden of data processing for the recognition of emotional label. Therefore, we tested the appropriate number of neurons in the second FCL in this range. In this process, a total loss index was used to describe the overall performance of the model. This index introduces mean squared error to measure the reconstruction error of new data to original data on the one hand, and on the other hand uses KL divergence to represent the difference between the distribution of latent variables and the distribution of recognition network q_φ (z|x). It takes the sum of the reconstruction error and KL divergence divided by the number of harmonic features as its index value. The smaller the index value is, the better the model’s overall performance will be. Figures 11 and 12 reflect the changes in the mean and variance of the loss after 30 independent runs when the first and second FCLs take different numbers of neurons, respectively. Among them, the translucent area around the polyline reflects the change range of the loss’s variance. By observing the change trend in Fig. 11, the number of second FCL neurons was first determined as 8. Figure 12 shows further test results of different structures in the first FCL, which proves that the effect is best when the number of neurons in this layer is 200. Therefore, the optimal structure of CVAE was finally determined as 5400-200-8. Compared with other structures, this structure can reduce the mean loss of the model to the global optimal value rapidly and stably in 20 iterations, and minimize loss’s variance under multiple independent runs, which proves it has the best convergence and stability.

Fig. 11

Trend chart of the mean and variance of loss for different structures of the second FCL.

Fig. 12

Trend chart of the mean and variance of loss for different structures of the first FCL.

Emotion-oriented form generation is a crucial characteristic of affective product design, so it is necessary to establish a close relationship between form data and emotional data. Figure 13 reflects a change in mean and variance of the loss after running the CVAE independently for 30 times with different emotional labels. As can be seen from the figure, with the increasing of iterations, various losses demonstrate a gradual downward trend, and all reach a certain value within acceptable error range. The result proves that the selected network structure has the accuracy and stability for the generation of 3D form adapted to different emotional labels.

Fig. 13

Trend chart of the mean and variance of loss under different emotional labels.

4.3 Design effect of the proposed method

The novel schemes generated by the four emotional labels shown in Fig. 10 and the EMSs obtained from the evaluation results of all test subjects recorded in Table 15 have shown to a certain extent that each scheme fits well with its target emotion. To further observe the design effect of the method, two indexes of form diversity and emotion matching degree were introduced. These indexes are calculated as $\begin{matrix} D_{(i)} = \frac{S (D_{(i)})}{S_{(i)}} \\ M_{(i)} = \frac{S (M_{(i)})}{S_{(i)}} \end{matrix}$ (9)

Where: D_(i) and M_(i) are the form diversity and emotion matching degree of the design schemes generated under the i-th emotional label; S_(i) is the total number of the schemes under the i-th emotional label; S (D_(i)) is the number of schemes with visual differences in S_(i), and S (M_(i)) is the number of schemes in S_(i) that conform to the target emotion.

Then, CVAE was run independently using one of the four emotional labels in turn to generate 10 design schemes, respectively, and all test subjects were invited to evaluate the affective response and visual similarity of the schemes. The statistical results were further used to calculate the index values (Table 17).

Table 17

Design effect evaluation index of method

Index	High-grade	Dynamic	Organic	Unique
Form diversity	0.7	0.9	0.7	0.9
Emotion matching degree	0.8	1	0.8	0.9

Refer to the sample size of each cluster (Table 13) and index values, the effect of the method to different emotional labels can be further observed. As shown in Table 17, the “dynamic” emotional label has a high index value, which reflects that the sample size is an important factor affecting the generative quality, and relatively comprehensive and abundant sample form information make the method have better feature mining and generalization ability. In addition, although the sample sizes of “organic” and “unique” clusters are close, their index values are different. Compared with the sample forms in the “organic” cluster, the sample forms in the “unique” cluster have more different overall form features, but fewer partial form features. The index values of “unique” are higher than that of “organic”, which shows that the method has a better design effect for database samples with obvious differences in overall form features. The overall index values under the four emotional labels indicate that the method has established a 3D form generation mechanism that conforms to different emotional labels, and can generate 3D forms that are in line with the target emotion. It actually realizes the distribution fitting of the 3D form data of multiple emotions at the same time. By mastering the distribution of 3D form data of the samples with the same emotional label, novel 3D form design schemes matching the target emotion can be randomly sampled and generated, which increases the possibility of obtaining abundant and various schemes and reduces the workload of designers.

5 Conclusions

To improve the design quality and make up for the obvious shortcomings of existing design methods involving 3D form innovation in affective product design, this paper proposes a data-driven affective product design method composed of several technical models using complete 3D surface data. It completes the construction of the mathematical model for quantifying 3D form based on the triangular mesh model, SPHARM and CVAE, which realizes the acquisition of the complete surface data of 3D object and lays a foundation for the objective and accurate emotion recognition and generation of 3D form. In addition, by using complete 3D surface data to participate in the emotion recognition process, it establishes a new recognition model for determining the customer’s affective response, which improves the objectivity and accuracy of recognition. Finally, it further uses CVAE to build a single algorithm generative model that accurately responds to emotional data, so as to achieve efficient generation of novel 3D form design schemes that meet design goals.

In general, this study proposes a new concept of data-driven affective product design using complete 3D surface data, and forms a systematic and effective method to make up for the deficiencies of existing design methods in affective product design involving 3D form innovation. The method is in line with the future development trend of product design and its technical requirements in the context of Industry 4.0. Industry 4.0 is based on intelligent design, intelligent manufacturing and intelligent operation. Data-driven digital twins and other technologies play an important role in it [34], and the research work of this paper fully reflects this point. It is based on the product model in the real world, then the model is mapped to the image of the virtual space and converted into data, and then the improvement of the real product is realized through data-driven intelligent design, which is exactly what Industry 4.0 should be. The key performances of the method are summarized as follows:

The mathematical model for the complete quantification of 3D form solves the problem that 3D form is difficult to be complete quantified and analyzed. It uses l = 30 to achieve a fine reconstructed visual effect of 3D form and obtain its complete surface data.

Compared with the traditional method that completely separates from the form data to complete emotion recognition, the emotion recognition model achieves the quantitative calculation of emotion based on the correlation between the complete surface data information of 3D form and the emotional data, which can identify and sort target emotions more objectively and comprehensively. Based on the model, four emotion labels were accurately determined.

The generative model based on CVAE and effect evaluation indexes well realizes the generation and evaluation of 3D forms. The key design effect evaluation index values of the proposed method, form diversity and emotion matching degree, are not less than 0.7 and 0.8, which can effectively ensure the efficiency and quality of the actual design.

However, it still has some problems. The preprocessing of 3D forms still requires manual intervention. The efficiency of this process is not high, and the accuracy needs to be further improved. Since the method cannot generate 3D forms that respond to multiple emotions and contain color information, further developing a multi-objective affective product design method that integrates 3D form and color will be a focus of follow-up research. In addition, because human emotions are vague and uncertain, future research can use fuzzy evaluation methods to better adapt to these features. At the same time, the actual product form design should not only meet emotional requirements, but also be constrained by product function and performance. Therefore, how to comprehensively consider these factors to obtain a form design result that is more in line with actual requirements is a problem that needs to be studied in depth in the future.

Footnotes

Acknowledgments

This work is supported by the National Natural Science Foundation of China (Grant NO. 71661023) and Jiangxi Province Humanities and Social Science Research Projects of Universities (Grant NO. YS19234). We thank the editors and anonymous reviewers for their valuable comments and suggestions. Furthermore, we thank all test subjects for their participation and assistance in the experimental study and Dr. Qu Min from Nanchang University for her help in providing language. Readers interested in the original data can contact the author via email.

Appendix

References

Wang

, A hybrid Kansei engineering design expert system based on grey system theory and support vector regression, Expert Systems With Applications 38(7) (2011), 8738–8750.

Kang

X.H.

, Porter

C.S.

and Erik

, Using the fuzzy weighted association rule mining approach to develop a customer satisfaction product form, Journal of Intelligent & Fuzzy Systems 38(4) (2020), 4343–4357.

Ding

, Bai

Z.H.

, Zhang

J.Z.

and Huang

X.G.

, Dynamic color design for multimodal industrial products based on genetic algorithm, Journal of Intelligent & Fuzzy Systems 38(1) (2020), 293–302.

Lotoshynska

, Izonin

, Nazarkevych

and Fedushkoc

, Consumer-centered design of the secondary packaging for industrial pharmaceuticals[J], CIRP Journal of Manufacturing Science and Technology 32 (2021), 257–265.

Shieh

M.D.

, Li

Y.F.

and Yang

C.C.

, Comparison of multi-objective evolutionary algorithms in hybrid Kansei engineering system for product form design, Advanced Engineering Informatics 36 (2018), 31–42.

Y.F.

, Shieh

M.D.

and Yang

C.C.

, A posterior preference articulation approach to Kansei engineering system for product form design, Research in Engineering Design 30(1) (2019), 3–19.

Hsiao

S.W.

, Chiu

F.Y.

and Lu

S.H.

, Product-form design model based on genetic algorithms, International Journal of Industrial Ergonomics 40(3) (2010), 237–246.

Chang

Y.M.

and Chen

C.W.

, Kansei assessment of the constituent elements and the overall interrelations in car steering wheel design, International Journal of Industrial Ergonomics 56 (2016), 97–105.

C.R.

, Su

, Mo

and Guibas

L.J.

, PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, computer vision and pattern recognition, IEEE Conference on Computer Vision and Pattern Recognition (2017), 652–660.

10.

, Song

, Khosla

, Yu

, Zhang

, Tang

and Xiao

, 3D ShapeNets: A deep representation for volumetric shapes, IEEE Conference on Computer Vision and Pattern Recognition (2015), 1912–1920.

11.

Feng

Y.F.

, You

H.X.

, Zhao

X.B.

and Gao

, MeshNet: Mesh Neural Network for 3D Shape Representation, National Conference on Artificial Intelligence 33 (2019), 8279–8286.

12.

Park

, Yang

, Yumer

, Ceylan

and Berg

A.C.

, Transformation-Grounded Image Generation Network for Novel 3D View Synthesis, IEEE Conference on Computer Vision and Pattern Recognition (2017), 3500–3509.

13.

Zhang

S.W.

, Pang

M.Y.

, Wang

G.C.

and Lu

Z.P.

, Designing Automobile Covering Parts with Mesh Subdivision Technique, Journal of Engineering Graphics 3 (2007), 67–72.

14.

Gao

, Zhang

, Feng

, Savchenko

, Hagiwara

and Zheng

, Flexible mesh morphing in sustainable design using data mining and mesh subdivision, Future Generation Computer Systems (2017), 1–8.

15.

Wang

, Liu

W.D.

, Yang

M.L.

and Han

D.J.

, A Multi-Objective Evolutionary Algorithm Model for Product Form Design Based on Improved SPEA2, Applied Sciences 9 (2019), 1–26.

16.

Wang

, Liu

W.D.

, Yang

M.L.

and Han

D.J.

, Research on product image form design based on elliptic Fourier, Computer Integrated Manufacturing System 26(2) (2020), 481–495.

17.

Ege

Y.C.

, Foth

, Baum

, Wirkner

C.S.

and Richter

, Adapting spherical-harmonics-based geometric morphometrics (SPHARM) for 3D images containing large cavity openings using ambient occlusion: a study with hermit crab claw shape variability, Zoomorphology (2020), 1–12.

18.

Shen

, Farid

and Mcpeek

M.A.

, Modeling Three-Dimensional Morphological Structure Using Spherical Harmonics, Evolution 63(4) (2009), 1003–1016.

19.

Yang

C.C.

, A classification-based Kansei engineering system for modeling consumers’ affective responses and analyzing product form features, Expert Systems With Applications 38(9) (2011), 11382–11393.

20.

Shieh

M.D.

, Li

Y.F.

and Yang

C.C.

, Product form design model based on multiobjective optimization and multicriteria decision-making, Mathematical Problems in Engineering (2017), 1–15.

21.

Guo

, Liu

W.L.

, Liu

F.T.

, Wang

and Wang

T.B.

, Emotional design method of product presented in multi-dimensional variables based on Kansei Engineering, Journal of Engineering Design 25(4) (2014), 194–212.

22.

Guo

, Qu

, Nagamachi

and Duffy

V.G.

, A proposal of the event-related potential method to effectively identify kansei words for assessing product design features in kansei engineering research, International Journal of Industrial Ergonomics 76 (2020), 1–10.

23.

Sutono

S.B.

, Selection of Representative Kansei Adjectives using Cluster Analysis: A Case Study on Car Design, International Journal of Advanced Engineering, Management and Science 2(11) (2016), 1885–1891.

24.

Deng

and Wang

G.H.

, Application of EEG and Interactive Evolutionary Design Method in Cultural and Creative Product Design, Computational Intelligence and Neuroscience 2019 (2019), 1–16.

25.

Lin

and Zhao

Y.Z.

, Optimal Design of Product Culture Image Modeling Based on FNN Model and PSO Algorithms, IOP Conference Series: Materials Science and Engineering 573 (2019), 1–12.

26.

Sohn

, Yan

and Lee

, Learning structured output representation using deep conditional generative models, Neural Information Processing Systems (2015), 3483–3491.

27.

Wang

S.X.

, Chen

H.W.

, Li

X.P.

and Shu

, Conditional Variational Automatic Encoder Method for Stochastic Scenario Generation of Wind Power and Photovoltaic System, Power System Technology 42(6) (2018), 1860–1869.

28.

Shen

and Makedon

, Spherical mapping for processing of 3D closed surfaces, Image and Vision Computing 24(7) (2006), 743–761.

29.

Erturk

and Dennis

T.J.

, 3D model representation using spherical harmonics, Electronics Letters 33(11) (1997), 951–952.

30.

Shen

, Kim

and Saykin

A.J.

, Fourier method for large-scale surface modeling and registration, Computers & graphics 33(3) (2009), 299–311.

31.

Athanasiadis

, Fudos

, Nikou

and Stamati

, Feature-based 3D morphing based on geometrically constrained spherical parameterization, Computer Aided Geometric Design 29(1) (2012), 2–17.

32.

Gerig

, Styner

, Jones

, Weinberger

and Lieberman

, Shape analysis of brain ventricles using spharm, IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (2001), 171–178.

33.

Zhang

D.S.

and Lu

G.J.

, Review of shape representation and description techniques, Pattern Recognition 37(1) (2004), 1–19.

34.

Jiang

Y.C.

, Yin

, Li

, Luo

and Kaynak

, Industrial applications of digital twins, Philosophical Transactions of the Royal Society A-Mathematical Physical and Engineering Sciences 379(2207) (2021), 20200360.

Data-driven affective product design using complete three-dimensional surface data

Abstract

Keywords

1 Introduction

2.1 Mathematical model construction

2.1.1 3D mesh data acquisition

3 Implementation procedures

3.1 3D form quantification

3.2.1 Emotion evaluation and recognition

3.3.1 Model structure and parameter

4.1 Values of SPHARM

Table 16 Variance contribution rate of each PC PC number Proportion (%) Cumulative (%) PC1 99.6275 99.6275 PC2 0.1694 99.7969 PC3 0.0839 99.8808 PC4 0.0503 99.9311 PC5 0.0288 99.9599 PC6 0.0081 99.9680 PC7 0.0061 99.9742 PC8 0.0033 99.9774

Footnotes

Acknowledgments

Appendix

References

Table 16
Variance contribution rate of each PC

PC number Proportion (%) Cumulative (%)

PC₁ 99.6275 99.6275

PC₂ 0.1694 99.7969

PC₃ 0.0839 99.8808

PC₄ 0.0503 99.9311

PC₅ 0.0288 99.9599

PC₆ 0.0081 99.9680

PC₇ 0.0061 99.9742

PC₈ 0.0033 99.9774