Felipe A. C. Viana, Victor Picheny, R.T. Haftka

[Steve Yenisch]

Paper: www.mae.ufl.edu/nkim/ISSMO/FelipeVianaPaper.pdf
Presentation: www.mae.ufl.edu/nkim/ISSMO/issmo-online-felipe.htm

CONSERVATIVE PREDICTION VIA safety margin: DESIGN THROUGH Cross-validation and benefits of Multiple Surrogates
Felipe A. C. Viana^*, Victor Picheny, and Raphael T. Haftka

ABSTRACT
Using surrogate models for learning or optimization creates a risk associated to the fitting error that must be accounted for. Conservative surrogates are metamodels designed to safely estimate the actual response of the system. In this work we use safety margins to generate conservative surrogates. Given a desired level of conservativeness (percentage of safe predictions), we propose the use of cross-validation for estimating the required safety margin. We also explore how multiple surrogates and cross-validation can be used to minimize the loss of accuracy inherent in conservative surrogates. The approach was tested on two algebraic examples for ten basic surrogates including different instances of kriging, polynomial response surface, radial basis neural networks and support vector regression surrogates. For these examples we found that cross-validation (i) is effective for selecting the safety margin; and (ii) allows us to select a surrogate with the best compromise between conservativeness and loss of accuracy.

* Research assistant and author of correspondence, Phone: [phone], Fax: [phone], Email: [e-mail]

[Nam Ho Kim]

Felipe:

I think you need to divide Eq. (8) by the volume (area) of D. Isn't it?

[Nam Ho Kim]

In Figure 2, why the y_hat + s curve passes data point at x=0 and x=6.2?

[Stratos]

May 14, 2009 11:27:07 GMT -5 Steve Yenisch said:

Paper: www.mae.ufl.edu/nkim/ISSMO/FelipeVianaPaper.pdf
Presentation: www.mae.ufl.edu/nkim/ISSMO/issmo-online-felipe.htm

CONSERVATIVE PREDICTION VIA safety margin: DESIGN THROUGH Cross-validation and benefits of Multiple Surrogates
Felipe A. C. Viana^*, Victor Picheny, and Raphael T. Haftka

ABSTRACT
Using surrogate models for learning or optimization creates a risk associated to the fitting error that must be accounted for. Conservative surrogates are metamodels designed to safely estimate the actual response of the system. In this work we use safety margins to generate conservative surrogates. Given a desired level of conservativeness (percentage of safe predictions), we propose the use of cross-validation for estimating the required safety margin. We also explore how multiple surrogates and cross-validation can be used to minimize the loss of accuracy inherent in conservative surrogates. The approach was tested on two algebraic examples for ten basic surrogates including different instances of kriging, polynomial response surface, radial basis neural networks and support vector regression surrogates. For these examples we found that cross-validation (i) is effective for selecting the safety margin; and (ii) allows us to select a surrogate with the best compromise between conservativeness and loss of accuracy.

* Research assistant and author of correspondence, Phone: [phone], Fax: [phone], Email: [e-mail]

1. The logic of the method of conservative surrogates is at odds with the concept of probabilistic analysis, in my opinion. In probabilistic analysis, all uncertainties are modeled by random variables. In the conservative surrogate a deterministic safety margin s is added to a surrogate model in order to account for the error. The estimation of this safety margin is done in an ad hoc manner. Different users of the method will estimate the safety margin in a different way.
2. Please explain why using a safety margin to account for the error in a model is better than using a random variable.
3. In the tank design problem, is the safety index approximated by a surrogate model as a function of the design variables or the stress?