Liang Zhao, K.K. Choi, Ikjin Lee, David Gorsich

[Steve Yenisch]

Paper: www.mae.ufl.edu/nkim/ISSMO/LiangZhaoPaper.pdf
Presentation: www.mae.ufl.edu/nkim/ISSMO/LiangZhaoPresentation.pptx

Sequential-Sampling-Based Kriging Method with Dynamic Basis Selection
Liang Zhao, K.K. Choi and Ikjin Lee
Department of Mechanical & Industrial Engineering
College of Engineering
The University of Iowa, Iowa City, IA 52242, U.S.A.
Email: [e-mail][/email,

ABSTRACT
Metamodeling has been widely used in engineering applications when a true experiment is not feasible or is extremely hard to obtain due to high computational expense. A surrogate model is desirable for representing the true model when only a limited number of experiments need to be evaluated. Researchers have been investigating the methods for generating the surrogate model based on limited samples. A number of methods, such as the least square regression, moving least square regression and radial basis functions, have been developed in recent decades. Recently, the Kriging method has gained broad interest due to its capability of dealing with highly nonlinear model. In the Kriging method, the response of the model is considered as two parts: the mean structure and the residue. The ordinary Kriging (O-Kriging) assumes this mean structure part is zero or a constant on the entire domain. The universal Kriging (U-Kriging) considers the mean structure as first- or second-order polynomials, which are obtained from a generalized least square regression. However, in practical use of these methods, a problem has been discovered that neither the ordinary Kriging nor the universal Kriging maximally uses the information from the evaluated samples due to the fixed form of the mean structure. Therefore, a new method that can automatically adjust the mean structure and maximally use the information based on current samples would be desirable. In this paper, we propose a new method to dynamically determine the mean structure of the Kriging model by applying a feature-selection process based on a new proposed criterion. By introducing this dynamic basis selection procedure, the proposed D-Kriging method can fit the nonlinear problem more accurately compared with the universal Kriging.
Another crucial issue of metamodeling is the sampling strategy. The Latin hypercube sampling method (LHS) has been applied in metamodeling. The method attempts to occupy the entire design domain most evenly and gain as much information about the true model as it can. However, it is not a problem-specified method, which means that no matter what the true response is, it always give us a similar sample profile that occupies the entire domain evenly. This could cause problem if the distribution of the highly nonlinear area is aggregating in certain part of the domain. Another sampling technique, importance sampling, samples around the limit state area and predicts the response accurately around the limit state. This importance sampling method also only gives a good local surrogate model around the limit state area and usually does not represent the true model accurately enough in other areas of the domain. In this paper, we propose a new sequential sampling strategy integrated with the proposed D-Kriging method. By coupling the sampling method with the D-Kriging method, the efficiency and accuracy can be simultaneously achieved. Mathematical examples provide promising results of the surrogate model constructed by this sequential-sampling-based Kriging method with dynamic basis selection.
Key Words: Response Surface Method (RSM), Kriging Method, Dynamic Basis Selection, Prediction Interval, Sequential Sampling Method

References

• C. Kim, S. Wang and K.K. Choi, Efficient response surface modeling by using moving least-squares method and sensitivity, AIAA Journal, 43(11), 2404-11, 2005.
  
• A. Forrester and A. Keane, Recent advances in surrogate-based optimization, Aerospace Sciences, 45(1-3), 50-79, 2009
  
• J.P. Chiles and P. Delfiner, Geostatistics: Modeling Spatial Uncertainty, Wiley, New York, 1999.
  

[Felipe Chegury Viana]

Dear authors,

First of all, congratulations on the very interesting work.

My first question is about slide #4: you say that K is the number of testing points. Does it mean that you have to run extra expensive simulations? The first impression that I had is that you were doing a integration of the ratio d/yhat. In this case, neither d (see slide #3) nor yhat require the extra simulations.

My second question is still about the dynamic basis selection: whenever you have a new individual of the GA, do you perform the expensive optimization of the kriging correlation parameters?

My third question is about slide #8: I expect IC to be multi-modal; if so, have you tried to find multiple samples to be added at a time (like a collection of local optima of IC)?

My fourth question: have you compared the maximization of IC with the maximization of the expected improvement (like in the EGO algorithm)?

Last but not least: any words on the computational cost of the D-kriging and on the curse of dimensionality?


Sorry if I have too many questions, but that is the result of such interesting work.


Thanks very much,
Felipe A.C. Viana

[Ikjin Lee]

I may answer some of your questions.

1. K is the number of testing points, which means that after we generate response surface, we will use K points to check which point is the most inaccurate and this point will be used as the next sample insertion point. So, there is no need of running expensive simulation. All K function estimations are from the generated response surface.

2. We use GA to find the best combination of basis and to save the computational cost. After finding the best combination of basis function, we generate response surface using Kriging method. Hence, we do perform the optimization of the kriging correlation parameters only once for the given design. And the optimization is not that expensive compared to real fatigue analysis.

3. Using K test points, we try to find a point where IC becomes maximum. So, there exists only one point that has the maximum IC value unless two points have the exactly same value. So, we only consider adding a sample one by one based on IC.

5. For large dimensional problem, you may want to see a roadarm example in the presentation material. In terms of efficiency, the proposed method still works. But, if the problem is high-dimensional, then, we need more testing points, K, to search the maximum IC point. For example, for a 2-D example, we can use 100 points along each axis. Then, we can use K=10000 to search the maximum IC points. However, for a 12-D example, we cannot use 100 points along each axis. So, to have the same K=10000, we can only use about 2 or 3 points along each axis. To resolve this problem, we use very small area near the interesting point not whole domain to check IC.

Thank you.

May 18, 2009 11:04:25 GMT -5 Felipe Chegury Viana said:

Dear authors,

First of all, congratulations on the very interesting work.

My first question is about slide #4: you say that K is the number of testing points. Does it mean that you have to run extra expensive simulations? The first impression that I had is that you were doing a integration of the ratio d/yhat. In this case, neither d (see slide #3) nor yhat require the extra simulations.

My second question is still about the dynamic basis selection: whenever you have a new individual of the GA, do you perform the expensive optimization of the kriging correlation parameters?

My third question is about slide #8: I expect IC to be multi-modal; if so, have you tried to find multiple samples to be added at a time (like a collection of local optima of IC)?

My fourth question: have you compared the maximization of IC with the maximization of the expected improvement (like in the EGO algorithm)?

Last but not least: any words on the computational cost of the D-kriging and on the curse of dimensionality?


Sorry if I have too many questions, but that is the result of such interesting work.


Thanks very much,
Felipe A.C. Viana