Kriging,Polynomial Chaos Expansion,and Low-Rank Approximations in Material Science and Big Data Analytics

Abstract

In material science and engineering, the estimation of material properties and their failure modes is associated with physical experiments followed by modeling and optimization. However, proper optimization is challenging and computationally expensive. The main reason is the highly nonlinear behavior of brittle materials such as concrete. In this study, the application of surrogate models to predict the mechanical characteristics of concrete is investigated. Specifically, meta-models such as polynomial chaos expansion, Kriging, and canonical low-rank approximation are used for predicting the compressive strength of two different types of concrete (collected from experimental data in the literature). Various assumptions in surrogate models are examined, and the accuracy of each one is evaluated for the problem at hand. Finally, the optimal solution is provided. This study paves the road for other applications of surrogate models in material science and engineering.

Introduction

Human lives are full of artificial materials; however, concrete is the most common man-made material used in construction. Three basic ingredients of concrete are water, cement, and aggregate. However, many natural and artificial additives are used to impose specific characteristics on concrete, such as high strength or durability. Determination of the impact of various ingredients on the properties of concrete was always one of the challenges in material science. This research field is typically supported by large experimental programs coupled with data analysis and postprocessing the findings.

More specifically, predicting the mechanical properties of concrete mixes can help improve our understanding of their behavior and may lead to the development of robust design codes and standards. Developing such predictive models is not new, and the application of simple statistical models in terms of linear and nonlinear regression analysis goes back to a hundred years.¹ These models aimed to provide an analytical formula to determine the unknown parameters affecting the relationship between concrete strength and its ingredients or environmental variables.

With the development of machine learning (ML) algorithms, many researchers have adapted one or more techniques for concrete strength prediction. The two main objectives were as follows: (1) develop a practical relation that can assist practitioners in analysis and design, and (2) compare the accuracy and performance of various ML algorithms. This study will not dive into the pool of hundreds of articles on the application of ML in concrete mix and concrete structures. Multiple state-of-the-art review articles have discussed this topic more in detail.^2–4 Several researchers have compared and contrasted ML algorithms in prediction of concrete strength: Young et al.⁵ compared artificial neural network (ANN), decision trees, and support vector machines (SVM); DeRousseau et al.⁶ compared regression trees, random forest (RF), and boosted trees; Cook et al.⁷ applied ANN, SVM, RF, and several hybrid models; and Abuodeh et al.⁸ used deep ML techniques.

With too many applications of ML in the prediction of concrete compressive strength, there is not yet any application of polynomial chaos expansion (PCE),^† Kriging, and their family in this field. PCE is capable of capturing the stochastic relation for complex and nonlinear systems using homogeneous orthogonal polynomial basis functions,⁹ and thus is a good candidate to be used in material science. Berveiller et al.¹⁰ used PCE to update the long-term creep strains in concrete structures.

Kriging (also known as Gaussian process modeling/regression)¹¹ is an efficient surrogate model for problems with high nonlinearity. Hoang et al.¹² used Kriging for modeling compressive strength of high-performance concrete with only seven input parameters and compared the results with ANN and SVM. Verma et al.¹³ compared several kernel-based methods, including Kriging, to predict the compressive strength in a small data set of only 50 samples and 4 input parameters. Afshoon et al.¹⁴ proposed a combined method of Kriging with U-learning function and K-means clustering to predict the concrete fracture energy using a limited number of inputs. Asteris et al.¹⁵ compared Kriging with ANN, multivariate adaptive regression splines, and minimax probability machine regression in the prediction of concrete compressive strength. The input variables include only seven variables, and more than a thousand experimental data have been used.

Aside from application-oriented articles, several major studies compared the performance of surrogate models, including PCE¹⁶ and Kriging. Hadigol and Doostan¹⁷ provided a state-of-the-art review on least-squares PCE with various sampling strategies. Luthen et al.¹⁸ provided a comprehensive survey on sparse PCE methods, including benchmark problems.

On the contrary, due to fast development of information technology, a large amount of information has been collected in many branches of material science, which necessitates the application of big data management. The term “big data” has been used for in 1970s in an article related to atmospheric and oceanic soundings to point out to large amounts of collected data for that project. However, its concept has been altered in the past two decades, and nowadays it mainly refers to data that surpass the processing capacity of conventional data management systems and software tools to capture, store, manage, and analyze. The big data research can be tackled in different ways¹⁹: (1) collection, storage, and management, (2) data analytics, and (3) data sharing and collaboration. Material science data analytics to develop surrogate models sometimes is beyond the capacity of a single computer, and thus, parallel computing and collective mining are required.

Omran and Chen²⁰ conducted a comprehensive research on the application of big data in various aspects of construction-related research area. They reported that the big data has a low percentage of articles related to concrete and construction materials, and thus, this research field requires more attention.

Torre et al.²¹ discussed the PCE as an ML regression and its application for big data analytics. Grasedyck et al.²² and Kishore Kumar and Schneider²³ discussed various low-rank approximation (LRA) techniques, and Udell and Townsend²⁴ explained the effectiveness of low-rank models in big data science. On the contrary, big data is always challenging to be used in conjunction with ML and surrogates models.²⁵ van Stein et al.²⁶ proposed an optimally weighted cluster Kriging for big data regression. In this approach, several Kriging models built on disjoint subsets of the data are properly weighted for the predictions. Kleijnen and van Beers²⁷ proposed a simple one-shot design founded on the nearest neighbors of the new database to handle the hyperparameters in Kriging models of big data.

According to the above-discussed literature survey, the application of PCE and Kriging is immature in material science. In addition, the available research is mainly simple applications on various databases with a limited number of input variables. Such a database is not challenging for PCE and Kriging, as their central promise is to handle complex and nonlinear models. Therefore, this study aims to provide a comprehensive application of PCE, Kriging, their combination (i.e., polynomial chaos Kriging), as well as canonical LRA in material science. More specifically, we explore the application of these four surrogate models to predict the compressive strength of concrete mixes with normal aggregates and reclaimed aggregates with a large number of input variables. To the best of the author's knowledge, such research has not been conducted before.

The Theory of Surrogate Models section provides a short and high-level review on the theoretical underpinning of the applied surrogate models. The Description of Database section describes the database and the basic relationship among the input parameters. The Results and Discussion section discusses the results of surrogate models, and finally, the Conclusions section summarizes the research findings.

Theory of Surrogate Models

Surrogate models (i.e., the meta-models) are models used to approximate the actual response of the models (either analytical or numerical). The commonly used meta-models are PCE,^28,29 Kriging,^30,31 canonical LRAs,^32,33 and high-dimensional model representation.^34,35 This section provides a high-level review of the underpinning theory of these methods.

Polynomial chaos expansion

The fundamental idea behind the PCE is to expand the model response onto basis consisting of multivariate polynomials, which are orthogonal with respect to the joint distribution of the input variables.³⁶ Consider an M dimensional random vector with independent components $X = \{X_{1}, X_{2}, \dots, X_{M}\}$ described by the joint probability density function $f_{X_{i}}, i = 1, 2, \dots, M$ . Thus, the scalar output (or quantity of interest) resulted from this system is also a random variable, denoted $Y = ℳ (X)$ . By using PCE, the model Y is approximated with a polynomial expansion $ℳ^{P C E}$ as follows³⁷:

where $ψ_{α} (X) = \prod_{i = 1}^{M} ϕ_{α_{i}}^{(i)} (x_{i})$ are multivariate polynomials orthonormal with respect to f_X, among them, $ϕ_{α_{i}}^{(i)}$ is the univariate orthogonal polynomial in the $i^{t h}$ variable of corresponding polynomial degree $α_{i}$ . $α \in ℕ^{M}$ is a multi-index that identifies the components of the multivariate polynomials $ψ_{α}$ and the $β_{α} \in ℛ$ are the expansion coefficients to be determined. Moreover, $A \in ℕ^{M}$ is the truncation set of multi-indices of cardinality P. There are two main truncation schemes, that is, standard and hyperbolic.³⁸ The former one corresponds to all polynomials in the M input variables of total degree less than or equal to p: $A^{M, p} = \{α \in ℕ^{M} : | α | \leq p\} c a r d A^{M, p} \equiv P = \frac{(M + p)!}{p! M!}$ (2)

The standard scheme can be modified in the form of the hyperbolic truncation scheme by using the parametric q to define the truncation:

where by using q = 1, the hyperbolic truncation yields to the standard truncation scheme in Eq. (2). For $q < 1$ , hyperbolic truncation includes all the high-degree terms in each single variable, but high-order interaction terms should be avoided to the extent possible. An illustration presents a set of two-dimensional hyperbolic truncation with varying p and q found in Hariri-Ardebili and Sudret.³⁹

The initial computational model, $ℳ$ , (or Y) can be rewritten as a sum of truncated version of infinite series in Eq. (1), and a residual (truncation error), $ε_{P}$ :

where the error $ε_{P}$ can be estimated using the leave-one-out (LOO) metric.

The expansion coefficients can be computed using multiple techniques, such as the least angle regression (LAR), least-squares regression, orthogonal matching pursuit (OMP), and Bayesian compressive sensing (BCS).⁴⁰ The LAR algorithm uses low rank truncation schemes and aims to find coefficient vectors with only a few nonzero entries (i.e., sparse solutions), while the other coefficients are set to zero.⁴¹ The LAR algorithm can be formulated by expanding the least-square minimization and adding a penalty term $λ ∥ β ∥_{1}$ as follows:

where $∥ \hat{β} ∥_{1} = \sum_{α \in A} |β_{α}|$ is the regularization term that forces the minimization to favor low-rank solutions.

On the contrary, the OMP is an iterative algorithm that minimizes the approximation residual at each iteration by solving this equation⁴⁰:

where R_n is the approximation residual for a polynomial basis with n elements. Once the basis element $ψ_{α_{n} + 1}$ has been added to the active set of regressors, the polynomial coefficients $β_{α}$ are updated.

Last but not least, in the BCS algorithm, a vector $γ$ of coefficient variances is to be maintained. For the nonzero $γ_{i} > 0$ , the coefficient y_i does exist, and the associated terms in the basis are active. In each iteration, one regressor is added to the basis, or deleted from the basis, or reassessed. The objective function $ℒ$ is the logarithm of $p (γ, λ, Y)$ , and can be solved numerically. In this hierarchical framework, the $γ_{i}$ s are i.i.d. following an exponential distribution with shared parameter $λ$ : $p (γ_{i}, λ) = E x p (γ_{i} | λ ∕ 2)$ .

Kriging

Kriging was introduced by Sacks et al.³¹ in which the model response $ℳ^{K} (x)$ is a realization of the Gaussian process indexed by x ⁴²: $Y^{K} = ℳ^{K} (x) = β^{T} f (x) + σ^{2} Z (x, ω)$ (7)

where $β^{T} f (x)$ consists of N bias functions $f (x)$ and regression coefficients $β$ . The second term consists of variance of the Gaussian process $σ^{2}$ and a zero-mean stationary Gaussian process $Z (x, ω)$ . The parameter $ω$ is defined in terms of a correlation function $R (x_{i}, x_{j}; θ)$ , which describes the “similarity” between two observations with hyperparameters $θ = {[θ_{1}, \dots, θ_{n}]}^{T}$ .

The common types of Kriging include simple, ordinary, and universal. The first two are special cases of universal Kriging. The universal Kriging aims to find the best linear unbiased predictor while minimizing the mean square error of the prediction. For $Y = {\{ℳ (x^{(1)}), \dots, ℳ (x^{(N)})\}}^{T}$ , which is assumed to follow a multivariate Gaussian distribution, the unknown Kriging parameters $γ = (β, σ^{2}, θ)$ can be estimated by maximizing the likelihood function as follows: $ℒ (γ; Y) = \frac{{(d e t C)}^{- 1 ∕ 2}}{{(2 π)}^{N ∕ 2}} exp [- \frac{1}{2} {(Y - F β)}^{T} C^{- 1} (Y - F β)]$ (8)

where the covariance matrix $C = σ^{2} R + Σ_{n}$ sums up the covariance matrix of the Gaussian processes and noisy response; and $F = {[p (x_{1}), \dots p (x_{N})]}^{T}$ is the regression matrix.

Taking the partial derivative of the log-likelihood function with respect to $β$ and $σ^{2}$ to zeros, the hyperparameters $θ$ can be obtained from solving the optimization problem in Eq. (9). This optimization problem can be solved using the covariance matrix adaptation-evolution strategy. This is a derandomized stochastic search algorithm introduced by Hansen and Ostermeier.⁴³

The correlation function (i.e., kernel or covariance function), $R (x_{i}, x_{j}; θ)$ , is a crucial ingredient for a Kriging model, since it not only contains the assumptions about the approximation function, but also controls the smoothness of the Kriging model. Lataniotis et al.⁴² introduced some typical one-dimensional correlation functions such as linear, exponential, Gaussian (or squared exponential), and Matérn.

Canonical LRAs

Using the same notation as of PCE in the Polynomial Chaos Expansion section, the canonical rank (i.e., a rank-one function of X ) is first defined as $w (X) = \prod_{j = 1}^{M} v^{(j)} (X_{j})$ , where $v^{(j)}$ presents a univariate function in the jth dimension. This function can be expanded to a rank-R approximation by assuming that the number R of rank-one terms is small. Using a polynomial basis that is orthonormal, a canonical LRA take the form as follows:

where $ϕ_{k}^{(j)}$ denotes the kth degree univariate polynomial in the jth input variable, p_j is the maximum degree of $ϕ_{k}^{(j)}$ , $z_{k, i}^{(j)}$ is the coefficient, and b_i the weight factors.

Description of Database

Various types of concrete have different applications and properties, such as normal strength, reinforced, high strength, roller compacted, and asphalt concrete. In this study, two databases are examined, that is, DB1: conventional concrete with additives, and DB2: reclaimed asphalt pavement aggregate concrete.⁴⁴ DB1 is traditionally used for many ML applications and includes the main ingredients of concrete with natural aggregate and few additives. The main reason to use DB2 is that a reclaimed aggregate is different from the virgin/natural one by having an extra layer of asphalt around it. This layer restrains, forming a perfect intermolecular bond between the aggregate and the mortar paste.

DB1 is primarily adapted from Yeh⁴⁵ with 425 samples (a subset of original database), with 7 input parameters, and one output (i.e., compressive strength), $Y_{425 \times 1} .$ The input matrix, $X_{425 \times 7}$ , includes the following input parameters: $X_{1} :$ water [kg/m³], $X_{2} :$ cement [kg/m $^{3}$ ], $X_{3} :$ fine aggregate [kg/m $^{3}$ ], $X_{4} :$ coarse aggregate [kg/m $^{3}$ ], $X_{5} :$ fly ash [kg/m $^{3}$ ], $X_{6} :$ blast furnace slag [kg/m $^{3}$ ], and $X_{7} :$ superplasticizer [kg/m $^{3}$ ]. The ratio of the number of samples to input parameters is about 60. Such a high ratio is in favor of many ML algorithms. Figure 1a illustrates the relationship between the 7 input parameters and the output.

FIG. 1.

Input–output scatter plot.

On the contrary, the second database (i.e., DB2) includes 128 samples with up to 18 input parameters and one output (i.e., compressive strength), $Y_{128 \times 1}$ . The input matrix, $X_{128 \times 18}$ , is not a full matrix, and there are multiple missing data. Before using this database in surrogate models, it is fully processed, and the missing cells in the input matrix are replaced with an appropriate value. There are multiple techniques to do so⁴⁶ such as moving average and using the mean/median of the corresponding data column. However, we first developed simple (i.e., first and second degree) polynomial models between each of the 18 input parameters and the output (using the existing data). Then we predicted the missing data using these simple relationships. This database has been collected from various sources^47–58

The selected 18 input parameters are: $X_{1} :$ water [kg/m³], $X_{2} :$ cement [kg/m³], $X_{3} :$ coarse reclaimed aggregate (CRA) [kg/m $^{3}$ ], $X_{4} :$ specific gravity of CRA [−], $X_{5} :$ percent of water absorption in CRA [%], $X_{6} :$ percent of asphalt content in CRA [%], $X_{7} :$ percent of replacement of CRA [%], $X_{8} :$ coarse virgin aggregate (CVA) [kg/m $^{3}$ ], $X_{9} :$ specific gravity of CVA [−], $X_{10} :$ percent of water absorption in CVA [%], $X_{11} :$ fine reclaimed aggregate (FRA) [kg/m³], $X_{12} :$ specific gravity of FRA [−], $X_{13} :$ percent of water absorption in FRA [%], $X_{14} :$ percent of asphalt content in FRA [%], $X_{15} :$ percent of replacement of FRA [%], $X_{16} :$ fine virgin aggregate (FVA) [kg/m $^{3}$ ], $X_{17} :$ specific gravity of FVA [−], and $X_{18} :$ percent of water absorption in FVA [%]. Figure 1b illustrates the relationship between the 18 input parameters and the output.

Results and Discussion

This section discusses the application of PCE, Kriging, and LRA surrogate models on two databases reviewed in the previous section.

Database 1 ( $X_{425 \times 7}$ )

Figure 2 presents the developed PCE surrogate models using two techniques: LAR and OMP. In each case, the predicted output, $Y^{P C E}$ , is plotted versus the initial experimental data, $Y^{E X P}$ . The scatter plots are based on the entire database. For all the surrogate models, 20% of the data are used for testing, while 80% of the data are used to train the meta-model. The following hyperparameters are assumed for meta-models: for both algorithms, an early stop criterion to stop adding regressors after the LOO error is above its minimum value for at least 10% of the maximum number of possible iterations.⁴⁰

FIG. 2.

Developed surrogate models for concrete DB1 using PCE including nonzero coefficients. LAR, least angle regression; OMP, orthogonal matching pursuit; PCE, polynomial chaos expansion.

In LAR, the polynomial degrees are selected by a degree-adaptive polynomial chaos method, which varies from 3 to 15. A same method is used in OMP with degree varying from 5 to 95. In both LAR and OMP, the hyperbolic truncation scheme corresponding to $q = 0.75$ is used, and the maximum rank truncation is set to 10 to limit the basis terms.

There are multiple metrics to evaluate the performance of a surrogate model. Sudret¹⁶ proposed the LOO metric to be used for PCE and Kriging, which is formulated as follows: $L O O = \frac{\sum_{i = 1}^{N} {(ℳ (x^{(i)}) - ℳ^{m e t a ∖ i} (x^{(i)}))}^{2}}{\sum_{i = 1}^{N} {(ℳ (x^{(i)}) - {\hat{μ}}_{Y})}^{2}}$ (11)

where ${\hat{μ}}_{Y} = \frac{1}{N} \sum_{i = 1}^{N} ℳ (x^{(i)})$ is the sample mean of the experimental design response. First, only a single point is retained at a time, and a meta-model $ℳ^{m e t a ∖ i}$ is constructed based on a reduced experimental design $χ ∖ x^{(i)} = \{x^{(j)}, j = 1, \dots, N, j \neq i\}$ . The error is calculated by comparing its prediction on the excluded point $x^{(i)}$ with the real value $y^{(i)}$ .

On the contrary, Daneshvar et al.⁵⁸ collected more than 10 error metrics that can be used to compare the initial database and the predicted one. It includes mean bias error, root mean-squared error (RMSE), mean absolute percentage error (MAPE), coefficient of determination, and Willmott's index.

Some of these error metrics are computed for all the surrogate models based on DB1 and are tabulated in Table 1. As seen, the error metrics from the OMP algorithm are twice better than LAR. This is also intuitive from scatter data in Figure 2. This figure also shows the distribution of the expansion coefficients, which are eventually used to develop the meta-model. The LAR-based model has only 18 nonzero (NNZ) coefficients, while the OMP-based one includes 78 coefficients. This indeed increases the accuracy of the OMP-based model for this specific database.

Table 1.

Comparison of the relative error for all surrogate models based on DB1

Metric	PCE (LAR)	PCE (OMP)	Kriging (exponential)	Kriging (Gaussian)	LRA	Evaluation criteria
MBE	8.6E-15	1.2E-13	−3.3E-13	4.9E-13	−6.9E-02	Close to zero
MAE	5.32	2.56	0.21	2.79	5.43	Small
RMSE	6.88	3.39	0.55	3.81	6.81	Small
MAPE	16.40	7.74	0.56	7.98	17.81	Small
R2	0.781	0.947	0.999	0.933	0.785	Large
WI	0.935	0.986	1.000	0.982	0.936	Large
LOO	0.258	0.081	0.097	0.139	—	Small

LAR, least angle regression; LOO, leave-one-out; LRA, low-rank approximation; MAE, mean absolute error; MAPE, mean absolute percentage error; MBE, mean bias error; OMP, orthogonal matching pursuit; PCE, polynomial chaos expansion; R2, coefficient of determination; RMSE, root mean-squared error; WI, Willmott's index.

Figure 3 presents the results of Kriging using the two-correlation family: exponential and Gaussian. The correlation function (also called kernel function), $R (x_{i}, x_{j}; θ)$ , is already defined in Eq. (7) and presents how similar the points are, depending on the distance between them. For a pair of one-dimensional input x and $x'$ and characteristics length scale $θ$ , the exponential and Gaussian correlation functions are defined as $R (x, x'; θ) = exp (- \frac{|x - x'|}{θ})$ and $R (x, x'; θ) = exp (- \frac{1}{2} {(\frac{|x - x'|}{θ})}^{2})$ , respectively. According to Figure 3, the Gaussian model presents less correlation compared with the exponential model. As seen, the performance of the exponential model is much better than Gaussian, and this can be quantitatively observed by Table 1. While the LOO error of exponential is only slightly less than the Gaussian model, all other error metrics are greatly in favor of the exponential model (e.g., mean absolute error, RMSE, and MAPE of exponential model are about 1/10 of the Gaussian model).

FIG. 3.

Developed surrogate models for concrete DB1 using Kriging including correlation matrix.

We have adapted the ordinary Kriging in these simulations, and thus, $β^{T} f (x) = β_{0}$ is a constant yet unknown value. Its value is estimated to be 34.2 and 28.2 based on the exponential and Gaussian models, respectively.

Figure 4 presents the results of the surrogate models for DB1 using the LRA. The required hyperparameters for this method are as follows: the range of rank selection is set to be 1–20; and the range of the polynomial degrees is set to 6–20. Moreover, the rank and degree adaptation strategy are used in LRA. The accuracy of this surrogate model is less than PCE and Kriging, and it shows a larger dispersion. The cross-validation (CV) error (based on 10-fold CV) is 3.19 (not shown in the Table 1), and all other error metrics in Table 1 are worse than PCE and Kriging. This figure also illustrates the natural logarithm of $z_{k, i}^{(j)}$ coefficients, as discussed in Eq. (10). This matrix includes $p + 1$ rows corresponding to the polynomial degrees and M columns corresponding to the dimensions. Those cells with a black dot indicate the negative values.

FIG. 4.

LRA-based surrogate models for concrete DB1 including the univariate polynomial coefficients. LRA, low-rank approximation.

Database 2 ( $X_{128 \times 18}$ )

This section presents the results of surrogate models using database 2, which includes a larger set of input variables with a smaller sample size. In this experimental program, the ratio of the number of samples to input parameters is about 7 (which is one-tenth of the previous example, DB1). The hyperparameters in database 2 are similar to those reported for database 1.

Figure 5 presents the developed PCE surrogate models using three techniques: LAR, OMP, and BCS. Again, $Y^{P C E}$ versus $Y^{E X P}$ scatter points are illustrated, including the NNZ expansion coefficients. The LAR algorithm is a linear regression tool that iteratively moves the regressors from a candidate set to an active set. However, OMP is a greedy algorithm that iteratively retrieves the polynomial basis elements that are most correlated with the current approximation residual.⁴⁰ Finally, the BCS is a completely different algorithm in which the regression problem is reformulated in a Bayesian framework.

FIG. 5.

Developed surrogate models for concrete DB2 using PCE including nonzero coefficients. BCS, Bayesian compressive sensing.

Table 2 summarizes all the error metrics associated with DB2 and three surrogate models. Again, the error metrics from the OMP algorithm are two to three times better than LAR and BCS. More specifically, the LOO error for LAR is 0.034, while it is 0.201 for both OMP and BCS. The scatter data points in Figure 5 also confirm this conclusion. The distribution of the expansion coefficients shows that the OMP-based model has 43 NNZ coefficients, while the LAR-based and BCS-based models have 9 and 21 coefficients, respectively. This is indeed one of the objectives of the LAR algorithm to develop the surrogate models with minimum possible polynomial coefficients. One should note that the range of $α$ coefficients reaches 1–6 in the OMP model (with a maximal degree of 14), while it is in the range of a few hundreds for two others (with a maximal degree of 3–4).

Table 2.

Comparison of the relative error for all surrogate models based on DB2

Metric	PCE (LAR)	PCE (OMP)	PCE (BCS)	Kriging (linear)	Kriging (exponential)	Kriging (Gaussian)	Kriging (Matérn)	LRA
MBE	−2.9E-16	4.2E-14	3.7E-02	−9.0E-15	7.2E-15	1.4E-15	2.7E-13	−1.9E-02
MAE	3.57	1.04	2.53	0.32	0.30	0.82	0.64	1.16
RMSE	4.44	1.70	3.20	0.78	0.71	1.62	1.48	1.70
MAPE	17.40	4.96	12.42	1.47	1.37	3.89	3.01	6.31
R2	0.825	0.974	0.909	0.995	0.996	0.977	0.981	0.974
WI	0.950	0.993	0.975	0.999	0.999	0.994	0.995	0.993
LOO	0.201	0.034	0.210	0.138	0.104	0.066	0.072	—

BCS, Bayesian compressive sensing.

Figure 6 illustrates the results of Kriging using the four-correlation family: exponential, Gaussian, linear, and Matérn-5/2, in which the first two models have been explained already. The linear correlation function is defined as follows: $R (x, x'; θ) = max (0, 1 - \frac{|x - x'|}{θ})$ , while the simplified function for Matérn-5/2 is formulated as follows:

FIG. 6.

Developed surrogate models for concrete DB2 using Kriging including correlation matrix.

R (x, x'; θ, v = 5 ∕ 2) = [1 + \sqrt{5} \frac{|x - x'|}{θ} + \frac{5}{3} {(\frac{|x - x'|}{θ})}^{2}] exp (- \sqrt{5} \frac{|x - x'|}{θ})

(12)

where v is a shape parameter and is considered 5/2 in this study. For $v = 1 ∕ 2$ , Matérn kernel yields to the exponential correlation, while for $v \to \infty$ it tends toward the Gaussian correlation function.

According to Figure 6, the linear model presents less correlation compared with the other three, while the Matérn-5/2 presents a higher correlation. Based on Table 1, the performance of various correlation function models depends on the interpretation of the error metrics. The classical metrics point to linear and exponential functions as the superior models, while the LOO presents contrarily (i.e., better performance for Gaussian and Matérn-5/2 models). This can be explained by checking Figure 6a–d. The majority of data points have nearly exact predictions based on the Gaussian and Matérn-5/2 models, while there are few outliers with larger errors.

On the contrary, the surrogate models based on linear and exponential models do not have such outliers, but their overall prediction is a bit less than the two others. The classical error metrics are based on the majority vote, and thus, they prioritize the linear and exponential models. Finally, for the ordinary Kriging models, $β_{0}$ ranges from 23 to 25 for the four correlation models.

Figure 7 illustrates the results of the surrogate model for DB2 using the LRA. While the accuracy of this surrogate model is less than PCE and Kriging, there is no large gap as we observed for the previous example (Fig. 4). The CV error is 11.3 (not shown in the Table 1) and all other error metrics in Table 1 are worse than PCE and Kriging (except BCS-based PCE). The natural logarithm of $z_{k, i}^{(j)}$ coefficient matrix is also shown in this figure for $M = 18$ and $p + 1 = 7$ .

FIG. 7.

LRA-based surrogate models for concrete DB2 including the univariate polynomial coefficients.

Conclusions

In this study, several surrogate models were trained to investigate their capability in predicting the mechanical properties of materials (in our case, the compressive strength of normal and reclaimed aggregate concrete admixtures). For this purpose, three methods, including the PCE, Kriging (i.e., Gaussian process), and canonical LRA, are used to find the optimal architecture for the meta-models. The two databases used in this study are essentially different in the number of samples and input random variables. The first one (DB1) is a low-input, large-sample size database, while the second one (DB2) is high-input, small-sample size. A total of seven error metrics were also used to compare the performance of the surrogate models.

For both databases, the Kriging model outperformed PCE and LRA. It is found that the performance of the Kriging model has a direct relation to the selected correlation function. For DB1, the exponential function provided better performance considering all the error metrics. However, in DB2, decision-making really depends on the selected error metric. This is evident from Figure 8, which proposes a coupled decision-making criterion using parallel plots.

FIG. 8.

Coupling of various error metrics in decision-making toward the optimal surrogate model. LOO, leave-one-out; MAE, mean absolute error; MAPE, mean absolute percentage error; MBE, mean bias error; R2, coefficient of determination; RMSE, root mean-squared error; WI, Willmott's index.

The accuracy of LRA method is less than the PCE and Kriging methods. Specifically for DB1, it causes a relatively large dispersion in the predicted data. Finally, comparing different PCE algorithms, it shows that OMP outperforms both the LAR and BCS methods. However, it has more expansion coefficients compared with the other two methods.

In conclusion, it is recommended to consider the Kriging model as the primary choice of surrogate model when it comes to problems in material science, including the prediction of the mechanical properties of concrete and asphalt mixtures. This study is based on only two databases for concrete, and further research is required to generalize the findings in this study. The performance of PCE, Kriging, and LRA can be compared with other conventional ML algorithms such as ANN, SVM, and RF. A similar idea can be also used in nano- and microscales to characterize the impact of material ingredients on their response behavior. For the larger databases, it is beneficial to compare the computational time from different meta-models with various assumptions as hyperparameters.

Footnotes

Authors' Contributions

G.M. and M.A.H.-A.: Conceptualization (equal), writing—original draft (equal), formal analysis (equal), and writing—review and editing (equal).

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

Abbreviations Used

References

Wright

PJF.

Statistical methods in concrete research. Mag Concr Res, 1954; 5(15):139–149.

Chaabene

, Flah

, Nehdi

. Machine learning prediction of mechanical properties of concrete: Critical review. Constr Build Mater, 2020; 260:119889.

Chong

, Othman

, Jaya

, et al. Design of experiment on concrete mechanical properties prediction: A critical review. Materials, 2021; 14(8):1866.

Nunez

, Marani

, Flah

, Nehdi

. Estimating compressive strength of modern concrete mixtures using computational intelligence: A systematic review. Constr Build Mater, 2021; 310:125279.

Young

, Hall

, Pilon

, et al. Can the compressive strength of concrete be estimated from knowledge of the mixture proportions?: New insights from statistical analysis and machine learning methods. Cem Concr Res, 2019; 115:379–388.

DeRousseau

, Laftchiev

, Kasprzyk

, et al. A comparison of machine learning methods for predicting the compressive strength of field-placed concrete. Constr Build Mater, 2019; 228:116661.

Cook

, Lapeyre

, Ma

, Kumar

. Prediction of compressive strength of concrete: Critical comparison of performance of a hybrid machine learning model with standalone models. J Mater Civil Eng, 2019; 31(11):04019255.

Abuodeh

, Abdalla

, Hawileh

. Assessment of compressive strength of ultra-high performance concrete using deep machine learning techniques. Appl Soft Comput, 2020; 95:106552.

Wiener

The homogeneous chaos. Am J Math, 1938; 60(4):897–936.

10.

Berveiller

, Le Pape

, Sudret

, Perrin

. Updating the long-term creep strains in concrete containment vessels by using markov chain monte carlo simulation and polynomial chaos expansions. Struct Infrastruct Eng, 2012; 8(5):425–440.

11.

Schöbi

Surrogate models for uncertainty quantification in the context of imprecise probability modelling. IBK Bericht, 2019; 505.

12.

Hoang

, Pham

, Nguyen

, Pham

. Estimating compressive strength of high performance concrete with Gaussian process regression model. Adv Civil Eng, 2016;2016; doi: 10.1155/2016/2861380

13.

Verma

, Thirumalaiselvi

, Rajasankar

. Kernel-based models for prediction of cement compressive strength. Neural Comput Appl, 2017; 28(1):1083–1100.

14.

Afshoon

, Miri

, Mousavi

. Combining kriging meta models with u-function and k-means clustering for prediction of fracture energy of concrete. J Build Eng, 2021; 35:102050.

15.

Asteris

, Skentou

, et al. Predicting concrete compressive strength using hybrid ensembling of surrogate machine learning models. Cem Concr Res, 2021; 145:106449.

16.

Sudret

Global sensitivity analysis using polynomial chaos expansions. Reliab Eng Syst Saf, 2008; 93(7):964–979.

17.

Hadigol

, Doostan

. Least squares polynomial chaos expansion: A review of sampling strategies. Comput Methods Appl Mech Eng, 2018; 332:382–407.

18.

Luthen

, Marelli

, Sudret

. Sparse polynomial chaos expansions: Literature survey and benchmark. SIAM/ASA J Uncert Quant, 2021; 9(2):593–649.

19.

Wactlar

Big data R&D initiative. National Science Foundation; 2012.

20.

Omran

, Chen

. Trend on the implementation of analytical techniques for big data in construction research (2000–2014). In: Construction Research

Congress 2016

, May 31–June 2, 2016, San

Juan

, Puerto

Rico

. 2016; pp. 990–999.

21.

Torre

, Marelli

, Embrechts

, Sudret

. Data-driven polynomial chaos expansion for machine learning regression. J Comput Phys, 2019; 388:601–623.

22.

Grasedyck

, Kressner

, Tobler

. A literature survey of low-rank tensor approximation techniques. GAMM Mitteilungen, 2013; 36(1):53–78.

23.

Kishore Kumar

, Schneider

. Literature survey on low rank approximation of matrices. Linear Multilinear Algebra, 2017; 65(11):2212–2244.

24.

Udell

, Townsend

. Why are big data matrices approximately low rank?. SIAM J Math Data Sci, 2019; 1(1):144–160.

25.

Pourkamali-Anaraki

, Becker

. Preconditioned data sparsification for big data with applications to PCA and K-means. IEEE Trans Inform Theory, 2017; 63(5):2954–2974.

26.

van Stein

, Wang

, Kowalczyk

, et al. Optimally weighted cluster kriging for big data regression. In: International Symposium on Intelligent Data Analysis. Springer; 2015; pp. 310–321.

27.

Kleijnen

JPC

, van Beers

WCM

. Prediction for big data through kriging: Small sequential and one-shot designs. Am J Math Manag Sci, 2020; 39(3):199–213.

28.

Xiu

, Karniadakis

. The wiener–askey polynomial chaos for stochastic differential equations. SIAM J Sci Comput, 2002; 24(2):619–644.

29.

Blatman

, Sudret

. Adaptive sparse polynomial chaos expansion based on least angle regression. J Comput Phys, 2011; 230(6):2345–2367.

30.

Krige

DG.

A statistical approach to some mine valuation and allied problems on the Witwatersrand: By DG Krige. PhD Thesis, University of the Witwatersrand, 1951.

31.

Sacks

, Welch

, Mitchell

, Wynn

. Design and analysis of computer experiments. Stat Sci, 1989; 4:409–423.

32.

Chevreuil

, Lebrun

, Nouy

, Rai

. A least-squares method for sparse low rank approximation of multivariate functions. SIAM/ASA J Uncert Quant, 2015; 3(1):897–921.

33.

Konakli

, Sudret

. Polynomial meta-models with canonical low-rank approximations: Numerical insights and comparison to sparse polynomial chaos expansions. J Comput Phys, 2016; 321:1144–1169.

34.

Chowdhury

, Rao

. Assessment of high dimensional model representation techniques for reliability analysis. Probabilistic Eng Mech, 2009; 24(1):100–115.

35.

Liu

, Hussaini

, Ökten

. Accurate construction of high dimensional model representation with applications to uncertainty quantification. Reliab Eng Syst Saf, 2016; 152:281–295.

36.

Amini

, Abdollahi

, Hariri-Ardebili

, Lall

. Copula-based reliability and sensitivity analysis of aging dams: Adaptive kriging and polynomial chaos kriging methods. Appl Soft Comput, 2021; 109:107524.

37.

Soize

, Ghanem

. Physical systems with random uncertainties: Chaos representations with arbitrary probability measure. SIAM J Sci Comput, 2004; 26(2):395–410.

38.

Fajraoui

, Marelli

, Sudret

. Sequential design of experiment for sparse polynomial chaos expansions. SIAM/ASA J Uncert Quant, 2017; 5(1):1061–1085.

39.

Hariri-Ardebili

, Sudret

. Polynomial chaos expansion for uncertainty quantification of dam engineering problems. Eng Struct, 2020; 203:109631.

40.

Lüthen

, Marelli

, Sudret

UQLab user manual—Polynomial chaos expansions. Technical report, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2021. Report UQLab-V1.4-104.

41.

Efron

, Hastie

, Johnstone

, et al. Least angle regression. Ann Stat, 2004; 32(2):407–499.

42.

Lataniotis

, Wicaksono

, Marelli

, Sudret

UQLab user manual—Kriging (Gaussian process modeling). Technical report, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2019. Report UQLab-V1.3-105.

43.

Hansen

, Ostermeier

. Completely derandomized self-adaptation in evolution strategies. Evol Comput, 2001; 90(2):159–195.

44.

Yeh

IC.

Modeling of strength of high-performance concrete using artificial neural networks. Cem Concr Res, 1998; 28(12):1797–1808.

45.

Hariri-Ardebili

, Pourkamali-Anaraki

. Matrix completion for cost reduction in finite element simulations under hybrid uncertainties. Appl Math Modell, 2019; 69:164–180.

46.

Huang

, Shu

, Li

. Laboratory investigation of portland cement concrete containing recycled asphalt pavements. Cem Concr Res, 2005; 35(10):2008–2013.

47.

Hossiney

, Tia

, Bergin

. Concrete containing rap for use in concrete pavement. Int J Pavement Res Technol, 2010; 3(5):251.

48.

Ibrahim

, Mahmoud

, Khodair

, Patibandla

. Fresh, mechanical, and durability characteristics of self-consolidating concrete incorporating recycled asphalt pavements. J Mater Civil Eng, 2014; 26(4):668–675.

49.

Modarres

, Hosseini

. Mechanical properties of roller compacted concrete containing rice husk ash with original and recycled asphalt pavement material. Mater Design, 2014; 64:227–236.

50.

Katkhuda

, Shatarat

, Hyari

. Effect of silica fume on mechanical properties of concrete containing recycled asphalt pavement. Struct Eng Mech, 2017; 62(3):357–364.

51.

Fakhri

, Amoosoltani

. The effect of reclaimed asphalt pavement and crumb rubber on mechanical properties of roller compacted concrete pavement. Constr Build Mater, 2017; 137:470–484.

52.

Khodair

, Raza

. Sustainable self-consolidating concrete using recycled asphalt pavement and high volume of supplementary cementitious materials. Constr Build Mater, 2017; 131:245–253.

53.

Singh

, Ransinchung

, Kumar

. Laboratory investigation of concrete pavements containing fine rap aggregates. J Mater Civil Eng, 2018; 30(2):04017279.

54.

Abraham

, Ransinchung

GDRN

. Influence of rap aggregates on strength, durability and porosity of cement mortar. Constr Build Mater, 2018; 189:1105–1112.

55.

Debbarma

, Ransinchung

, Singh

. Feasibility of roller compacted concrete pavement containing different fractions of reclaimed asphalt pavement. Constr Build Mater, 2019; 199:508–525.

56.

Boussetta

, El Euch Khay

, Neji

. Experimental testing and modelling of roller compacted concrete incorporating rap waste as aggregates. Eur J Environ Civil Eng, 2020; 24(11):1729–1743.

57.

Shi

, Grasley

, Hogancamp

, et al. Microstructural, mechanical, and shrinkage characteristics of cement mortar containing fine reclaimed asphalt pavement. J Mater Civil Eng, 2020; 32(4):04020050.

58.

Daneshvar

, Moradi

, Amooie

, et al. Response of low-percentage frc slabs under impact loading: Experimental, numerical, and soft computing methods. Structures, 2020; 27:975–988.

Kriging,Polynomial Chaos Expansion,and Low-Rank Approximations in Material Science and Big Data Analytics

Abstract

Introduction

Theory of Surrogate Models

Polynomial chaos expansion

Kriging

Canonical LRAs

Description of Database

Results and Discussion

Database 1 ( X 4 2 5 × 7 )

Database 2 ( X 1 2 8 × 1 8 )

Conclusions

Footnotes

Authors' Contributions

Author Disclosure Statement

Funding Information

Abbreviations Used

References

Database 1 ( $X_{425 \times 7}$ )

Database 2 ( $X_{128 \times 18}$ )