A prospective analysis of the engineering design discipline evolution based on key influencing trends

2.3.1 Natural language processing and KG

As far as needs or requirements are expressed in natural language, computational linguistics methods form a technique of choice to analyse and manage them. Natural Language Processing (NLP) approaches have been utilized to highlight different relationships between pieces of language. The NLP methods are used in three main specific contexts, first for comparison between pieces of language, second for classification of pieces of language into predefined or dynamically defined categories. Finally, the most challenging task for NLP methods is understanding language meaning. This can be used for example for detecting errors in requirements or detecting contradictions between requirements. To summarize, the purpose of NLP methods can be separated between labelling part of speech, extracting meaning from sentences, comparing language parts. The algorithms employed to achieve those tasks belong to the three fundamental categories of supervised, semi-supervised or unsupervised algorithms. The key techniques used in the literature currently are transformers, SVD decomposition applied to the frequency of terms. This technique is also named Latent Semantic Analysis (LSA). Different adaptations of LSA methods are also available in the literature (Landauer et al., 2008).

To be more precise, the classification tasks can be handled via a broad variety of methods belonging to the field of machine learning (ML). The labels or clusters can be predefined or learned via clustering algorithms. Classical algorithms such as Kmeans or DBscan can be interesting for each specific situation (Pelleg et al., 1999) (Ester et al., 1996). For comparison purposes, the comparison between pieces of language can be used for example similarity metrics. Semantic similarity measures can be provided by using vector space measures or probabilistic measures (Lamar, 2009). If probabilistic data are not present, then vector space measures can be used for similarity analysis. One common vector space measure is the cosine similarity method, which is widely used for measuring the distance between documents or sentences (Lamar et al., 2010). In their research work (Christophe et al., 2014), Christophe et al. used the cosine metric together with a logistic function to find out the similarity between requirements. Lash used a computational linguistic approach to identify the relationships between requirements. Along with this, he compared different requirements analysis methods (Lash, 2013). Among the computational linguistic methods, Singular Value Decomposition (SVD) or one of its variants is commonly used to identify and relate the semantics of requirements. SVD is used to develop an NLP approach called Latent Semantic Analysis (LSA) (Landauer et al., 2008). LSA can also be seen as a dimension reduction technique. It takes a set of semantic objects that exist in a high-dimensional space and represents them in a lower-dimensional space (Lamar, 2009) in which the main dimensions are represented. LSA is widely used to find out the relations between sentences. This approach is applied, for example, in software industries to propose titles for books that are related to the keywords used in a search query. LSA and SVD are also commonly used in collaborative filtering. This is a method of choice employed for example to suggest tv programs on some well-known online platforms. This method has been validated for its accuracy in different applications (Landauer et al., 2008). Four key features of LSA are listed below.

Latent meaning: the truncated result of SVD analysis (low-dimensional matrix); the correspondence between words and contexts indicates similarities between contexts (in our case sentences).

In the context of comparing and analysing requirements, the work of J. Misra is significant. He used LSA to prioritize requirements within clusters (Misra et al., 2014). Although this helps engineers to prioritize requirements, it does not provide an understanding of the interactions between requirements. Presenting those interactions will lead to easier requirements management for engineers.

One type of interaction important for requirement engineering is the ability to understand and detect contradictions. For a large and complex project, identifying the interaction (or relations) between requirements texts is a key issue to consider because it can provide significant benefits when later developing causal ordering and early reasoning.

Apart from requirements, contradictions can occur in a normal text, when two sentences are extremely unlikely to be true at the same time. In this area, the works of (Harabagiu et al, 2006) and (de Maneffe et al., 2008) are significant. Compared to Harabagiu’s work, de Marneffe’s refinement method for finding contradictions between texts is easier to implement in computer systems. Table 1 below summarizes different types of contradictions that are listed in those works.

De Marneffe mentioned that some of the contradictions could be detected using computational linguistic approaches. Other contradictions, for example, factive world knowledge structure-related contradictions, are difficult to detect using NLP approaches and the results are error-prone.

More advanced methods such as transformers can become potential solutions to detect the most challenging types of contradictions present in the text (Vaswani et al., 2017).

Requirement’s quality metrics are also covered by NLP methods. Table 2 from Mokammel et al. 2018, summarizes different metrics considered in the requirement engineering literature.

Such type of contradiction detection approach was developed for a general text corpus. Applying this approach to requirements texts was proposed by Mokammel et al. in (Mokammel et al., 2018). This generic overview of different uses of NLP methods has highlighted three types of possible knowledge that can be extracted from text. In an engineering context, requirements can be directly and automatically extracted from text. The quality of those requirements can be evaluated, and automatic corrections can be proposed to requirement structures. Last, it is possible to detect links between requirements and to detect potential contradictions between requirements. Those requirements can also be classified into specific predefined or automatically defined categories. The natural move from the requirements’ network is causal relations and causal reasoning. This is the topic of the next section.

A central aim of the scientific activity is to develop methods to systematically unveil cause-effect relationships between variables of a problem. This is also particularly relevant to the engineering design and manufacturing discipline. The discovery of potential confounding variables has been a major source of investigation (VanderWeele et al., 2013). In statistics, a confounding variable is a variable that influences both the dependent and independent variables. This is the source of a false association between variables. Confounding is a concept related to cause-effect relationships and this should not be mixed with the concepts of correlations or relations. The existence of confounders is explaining why correlation does not imply always a causal relationship. An important variety of approaches have been developed to generate causal ordering, but for this article, the focus of the authors is on methods using graphs and equations directly coming from physics or other scientific disciplines. Those approaches are limiting the scope of the analysis, but they have demonstrated their effectiveness and robustness in several practical engineering contexts and problems. They can become particularly relevant for the future of the engineering design disciplines as presented later in this article. Methods such as Bond graph causal ordering (Gawthrop et al., 1996) or causal ordering of equations via the direct use of Iwasaki and Simon algorithm (Iwasaki et al., 1994) or its extensions (Travé-Massuyès et al., 1997), as well as work done in qualitative physics (Bhashkar et al., 1990), are few examples of causal ordering methods belonging to such a classification. Another approach applied more specifically to the social science context is also interesting for this limited literature review. This is a method searching causes and effects relationships via models using structural equations modelling (SEM) (Pearl, 2000; Spirtes et al., 2000). SEM derives from path analysis which was developed by the biologist S. Wright in the 1920 s and has been often applied to analyse causal relations of non-experimental data using an empirical approach. Path analysis extends regression analysis by analysing simultaneously many endogenous and exogenous variables of a system or problem. Path analysis was combined with factor analysis and latent variables to form the SEM approach to causal analysis. In path analysis, variables can be endogenous or exogenous. Relations between variables can be represented by a one-sided arrow when a relationship exists between variables and a two-sided arrow when a correlation exists between variables as presented in Fig. 2. Different path tracing rules have been defined in the path analysis approach such as the following key rules. It is possible to trace the following type of relations between arrowheads, heads-tails, or tails-heads, but not heads-heads. It is possible to pass through each variable only once in a chain of paths and finally no more than one bi-directional arrow can be included in each path chain. Factor analysis is integrated with latent variables in the SEM approach. Factor analysis is a statistical method used to describe variability among observed, correlated variables with the central idea that the variation of observed variables can also be reflecting the variations of unobserved variables name latent variables. The observed variables are modelled as linear combinations of the potential factors plus “error” terms. Factor analysis may help to deal with data sets where there are large numbers of observed variables that are thought to reflect a smaller number of underlying/latent variables (Harmann, 1976). However, SEM is of confirmatory nature and researchers have first to model the true causal relationships based on other background knowledge before collecting or analysing data (Goldberger, 1972). This task is difficult, especially at the beginning of a design process because of the lack of background initial knowledge. A method presented below and named DACM for Dimensional Analysis Conceptual Modelling (Coatanéa, 2014) aims at building initial causal relationships from the available background knowledge using a framework integrating engineering design concepts such as functions, organs, elementary organs laws, variables’ classification, and dimensional analysis theory.

OPEN IN VIEWER

Fig. 2

Example of SEM diagram.

Consequently, using SEM alone, it is often difficult to unveil the initial causal structure. Additionally, SEM suffers from another limitation because SEM assumes normality of the statistical distribution and only uses the covariance structure and cannot find causal direction between two highly correlated variables because the models produced will be equivalent using the SEM approach. Nevertheless, the SEM method constitutes the conceptual basis of several algorithms that are successful in the discovery of causal structure, especially in complex systems studied by sciences such as economics and sociology. This has also been used in medicine and more specifically in epidemiology.

The fundamental concepts of formalism are summarized in the following manner by Halpern (Halpern, 2000). The construction of structural equations (SE) models requires three key steps. First, the problem being studied is represented by a finite set of variables, corresponding to the features of the problem. This can also be the function of a system to be designed as a solution for a specific design problem. There are two sorts of variables. Endogenous variables are such that their values are determined by other variables within the model, whereas the values of exogenous variables are determined in a way that is independent of other variables of the system. The structural equations describe the functional dependence of the endogenous variables on other variables (endogenous and exogenous) in the model (Kaplan, 2008).

SEM use graphs to represent relations between observed variables and latent variables. Latent variables are not observed. Latent variables can be also residues or errors in the modelling and measurement process. Software specifically developed for SEM modelling are existing to form the model. An SEM model can take the following form presented in Fig. 2. Each of the generic situations presented above must form each time a specific model. The β is the element of a correlation matrix. The λ is the coefficient allowing to model quantitatively the different properties in Fig. 2.

The dimensional homogeneity of models is one property required to support reasoning. A second property is the right causal ordering of the variables. This was discussed above.

The DACM method is merging SEM with Dimensional Analysis (DA) and qualitative physics and reasoning (Mokhtarian et al., 2017). This is an approach of choice for bringing automatic reasoning capabilities to the early design stages. Additionally, discovering contradictions as defined in TRIZ (Savransky, 2000) and especially physical contradictions can be computed automatically using DACM. When physical contradictions are detected, it is becoming possible to apply transformation rules to the causal graphs in DACM implementing in simple terms the separation principles of TRIZ. The graph transformations are separation of nodes, change of node types for expansion or limitation of systems scope, addition, or removal of links between nodes. Causal graphs can find usage in surrogate modelling too. This is the purpose of the next section.

Surrogate modelling and MDO are two disciplines of great importance for the engineering design process. Usually, they take place during the embodiment and detail design stages and consequently their initial role in early design stages is limited. Nevertheless, with the changes affecting the design process and especially the availability of massive data collection presented in a previous section, it is possible to also consider the early use of surrogate modelling and MDO in a design process.

Traditionally surrogate modelling has been seen as a solution to cope with the fundamental contradiction in requirements imposing on one side to explore the design space to find optimal solutions and on the other side the prohibitive exploration time associated with different high-fidelity simulations. This has been coupled with MDO strategies to gain time by exploring not the entire design space but only some zones of potentially bigger interest. A significant effort has been dedicated in the MDO discipline to the resolution of the fundamental contradiction between exploration of design space and exploitation in terms of global optimization search. Usually, a trade-off is considered between those two criteria, but a potential paradigm shift due to the future introduction of Quantum computing might provide a change since both objectives might become compatible.

Models and simulations, from different fidelity levels, are an important part of product design used to evaluate the product performance for different design parameters. Product performance evaluation through high fidelity model simulations is a shared approach across various engineering fields ranging from automobile, aerospace to civil engineering and medical engineering. Since prototyping is not always a viable approach for product development, due to potentially unacceptable costs, simulations are providing an alternative in form of virtual prototyping. Simulations play an important role in facilitating an efficient design process through sensitivity analysis, multi-domain optimizations and at early design stages, analysis of risk and uncertainty resulting from limited knowledge. However, evaluating product performance using high fidelity models are time-intensive and require costly computer simulations. Thus, the development of surrogate models also known as metamodels can facilitate design space exploration via fast simulations and scenario analyses.

Surrogate or metamodels are approximation models with a measurable level of accuracy that closely model the outputs of a system of interest. The approximation of the model from a high-fidelity physics-based simulation model to a simpler input-output behaviour of the system alleviates the burden of complex computations and enables cheaper simulations. Metamodels are often black-box models as they do not model the inner workings of the system and are developed solely based on data. Several surrogate and metamodeling methods have been developed ranging from traditional approaches such as polynomial regression, kriging, and radial basis functions to name a few, to more data advanced methods such as spatial mapping, artificial neural networks, and Bayesian networks. The development of a surrogate model can be streamlined to the following steps, 1) data sampling through the design of experiments, 2) assembling of the training dataset, 3) surrogate model selection, choosing the right type of model to represent the input-output relationships of the collected dataset, 4) model construction, development of the model using the chosen modelling approach, and 5) active learning through data enrichment, i.e., adding more data to retrain the model to achieve the required accuracy. All surrogate models do not contain all of those five steps. Active learning is in our viewpoint specific to models that can be qualified as machine learning models. This active learning can also take place during steps 1, 2, 3 and 4 described above.

In recent years, metamodels have been used to facilitate fast simulations for various fields and applications. Cai et al., 2021 studied three ML-based surrogate modelling approaches for parameter estimation of left ventricular myocardium from clinical data. The authors investigated the use of K-nearest neighbour (KNN), XGBoost, and multi-layer perceptron (MLP) approaches to perform modelling. Their research showed that the three ML approaches were able to learn the relationships between the properties of interest and found that parameter estimation could be performed in a matter of minutes. They found that XGBoost and MLP approaches performed better with lesser uncertainties than KNN. The authors concluded that ML-based surrogate modelling was able to predict left ventricular diagnostic dynamics and estimate passive parameters with an acceptable level of accuracy with future work focusing on emulating cardiac pump function in a multi-physics and multi-scale framework. Zhang et al., 2017 evaluated the effectiveness of advanced hybrid surrogate modelling with different sampling techniques and sizes for four different complex engineering systems namely, wind farm power generation, product platform planning, three-pane window eat transfer, and onshore wind farm cost estimation. They found that the sampling techniques had a higher impact on the accuracy of the surrogate modelling approach. Increasing sample size alone did not present a higher level of accuracy rather choosing the right sampling technique alongside increasing sampling size was better suited for improving model accuracy. The authors concluded that adaptive hybrid surrogate modelling provided good accuracy in representing complex system behaviour and that choosing the right sampling method and the sample size was essential in improving the accuracy of the models. Kuya et al., 2011 presented a multi-fidelity surrogate modelling approach combining experimental and computational aerodynamics datasets. A multi-fidelity cokriging regression model was used for the study. The study showed that low fidelity computation data can contribute to improving the surrogate model with limited high-fidelity data from experiments. The authors also examined the impact of characteristics of the sampling design for low fidelity data on the final surrogate models. They found that the low fidelity data provided a priori reduce the need for a large quantity of high-fidelity data.

Metamodeling has been increasingly used as a solution to the long-standing problem of developing fast methods to estimate the properties of a system from data. However, despite their vast advantages, metamodels can be limited by their accuracy, inability to generalize over a large design data space, and inability to accommodate data deviating from the modelled limits. The black-box nature of metamodels makes it difficult to generalize a phenomenon observed in the model, and high-fidelity data from costly simulations are required to enrich the training dataset to facilitate active learning and to improve model accuracy. Thus, it is advantageous to focus on the development of a modelling approach that can combine knowledge from different forms to facilitate a grey box modelling approach in the form of oriented knowledge graphs. The knowledge graphs integrate knowledge from functional and non-functional requirements of the design combined with collected data to develop an accurate model of the system. The use of knowledge graphs in future design tools is discussed next.