Validation Tools for the Assessment of Modelling Errors

Assessing the modelling errors of mathematical/computational models (Validation) used in Engineering is as important as developing new simulation tools, because results of modelling and simulation are being used to take decisions. The purpose of this document is to illustrate the Validation techniques presented in the ASME V&V20 Standards ASME V&V20 [2009], ASME VVUQ20.1 [2025]. The basic technique presented in ASME V&V20 [2009] requires an experimental measurement D and a computational result S for the same geometry, boundary conditions, material properties and heat transfer coefficients that define a set-point. The Validation procedure produces an interval centered at the comparison error E = S − D with a width that depends on the experimental u_D, numerical u_num and input parameters u_input standard uncertainties.
The point-wise assessment is usually performed for multiple quantities of interest and/or set-points and so it is useful to have a technique to make a global assessment of the differences between experiments and simulations. A Multivariate metric that performs such assessment taking into account possible shared uncertainties at the multiple quantities of interest / set-points is presented in ASME VVUQ20.1 [2025]. The result of the Multivariate metric E_mv is compared to a reference value E_ref obtained from the number of quantities of interest / set-points. If the ratio E_mv / E_ref is smaller than 1, the discrepancies between experiments and simulations are globally within the validation uncertainty. On the other hand, a ratio E_mv / E_ref larger than 1 indicates that the differences between experiments and simulations are globally larger than the validation uncertainties. Logically, the level of mismatch between simulations and experiments depends on E_mv / E_ref and on the level of validation uncertainty obtained at the multiple quantities of interest / set-points.
In practical problems, simulations are many times performed for conditions of an application point that is not covered by the set-points included in the validation space ASME [2019]. Therefore, there is no experimental data at the application point and so the V&V20 point-wise technique cannot be applied. Nonetheless, an estimate of the interval that contains the modelling error at the application point can be obtained using regression techniques applied to the validation data obtained at the set-points of the validation space Peltier et al. [2025].
This document illustrates the use of all these Validation techniques for the simulation of a two-dimensional flow of an incompressible, single-phase, Newtonian fluid around the Eppler 387 airfoil. Experimental data is available in the open literature for a Reynolds number of 3 × 10⁵ McGhee et al. [1988]. The quantities of interest are the lift coefficient C_L, drag coefficient C_D and the pitching moment coefficient around the location at 25% of the chord C_M25 for nine angles of attack α between α = −1° and α = 9°.
Sets of five multiblock geometrically similar grids are generated for each angle of attack to allow the estimation of the numerical uncertainty using the techniques presented in Eça et al. [2023]. The only input parameter that is assumed to be uncertain is the angle of attack that is supposed to be measured with an accuracy of ±0.2°. Therefore, to enable the calculation of the input parameters uncertainties using sensitivity coefficients ASME V&V20 [2009], extra grids are generated with ±0.2° for each of the nine angles of attack.
Simulations are performed with the Reynolds-averaged Navier-Stokes (RANS) solver ReFRESCO using the two-equation, eddy-viscosity, k − ω Shear-Stress Transport (SST) model Menter et al. [2003], which is not appropriate for the selected low Reynolds number of 3 × 10⁵. Therefore, a second set of simulations was made using the SST turbulence model in addition with the Local Correlation Transition Model (LCTM) based on γ and Re_θ, as proposed by Langtry and Menter [2009]. In all simulations, the shear-stress at the wall is calculated from its definition, which means without wall functions.
The point-wise Validation metric is applied to the three quantities of interest for the nine angles of attack selected. The Multivariate variate metric is applied to each quantity of interest, C_L, C_D and C_M25 separately and to all the quantities of interest simultaneously. Polynomial regression is applied to a Validation space of six angles of attack (α = −1°, α = 1°, α = 3°:, α = 5°, α = 7° and α = 9°) to estimate the interval that contains the modelling error at α = 0°, α = 4° and α = 8°. The results obtained with polynomial regression are compared with the point-wise metric results for these three angles of attack. Detailed results of this study are presented by Eça et al. [2025].