Contenido

Existe un gran número de enfoques para realizar un análisis de sensibilidad, muchos de los cuales se han desarrollado para abordar una o varias de las limitaciones comentadas anteriormente. También se distinguen por el tipo de medida de sensibilidad, ya sea basada en (por ejemplo) descomposiciones de varianza"), derivadas parciales o efectos elementales. En general, sin embargo, la mayoría de los procedimientos se ciñen al esquema siguiente:.

En algunos casos, este procedimiento se repetirá, por ejemplo en problemas de gran dimensión en los que el usuario tenga que descartar variables sin importancia antes de realizar un análisis de sensibilidad completo.

Los distintos tipos de "métodos básicos" (que se examinan más adelante) se distinguen por las diversas medidas de sensibilidad que se calculan. Estas categorías pueden solaparse de algún modo. Se pueden dar formas alternativas de obtener estas medidas, bajo las restricciones del problema.

One at a time (OAT)

Main article: Single factor method.

One of the simplest and most common approaches is to change one factor at a time (OAT), to see what effect it has on the result.[13]​[14]​[15]​ OAT usually involves:.

Sensitivity can be measured by monitoring changes in the outcome, for example, using partial derivatives or linear regression. This seems a logical approach, since any observed change in the result will be unequivocally due to the single changed variable. Additionally, by changing one variable at a time, you can keep all the others fixed at their central or reference values. This increases the comparability of the results (all "effects" are calculated with reference to the same central point in space) and minimizes the chances of computer program crashes, which are more likely when several input factors are changed simultaneously. OAT is often preferred by modellers for practical reasons. In case of model failure, the modeller immediately knows which input factor is responsible for the failure.

However, despite its simplicity, this approach does not fully explore the input space, as it does not take into account the simultaneous variation of the input variables. This means that the OAT approach cannot detect the presence of interactions between input variables and is not suitable for non-linear models[16].

The proportion of input space left unexplored by an OAT approach grows superexponentially with the number of inputs. For example, a parameter space of 3 variables that is explored one at a time is equivalent to taking points along the x, y, and z axes of a cube centered at the origin. The convex hull that bounds all these points is an octahedron that has a volume of only 1/6 of the total parameter space. More generally, the convex hull of the axes of a hyperrectangle forms a hyperoctahedron that has a volume of . With 5 inputs, the explored space already drops to less than 1% of the total parameter space. And even this is an overestimate, since the off-axis volume is not being sampled at all. Compare with random sampling of space, in which the convex hull approaches the entire volume as more points are added.[17] Although in theory OAT sparsity is not a problem for linear models, true linearity is rare in nature.

Local methods based on derivatives

Methods based on the local derivative consist of taking the partial derivative of the output Y with respect to an input factor X:.

where the subscript x indicates that the derivative is taken at a fixed point in the input space (hence the "local" in the class name). Adjoint modeling[18][19]​ and automated differentiation[20]​ are methods of this class. Like OAT, local methods do not attempt to fully explore the input space, as they examine small perturbations, typically one variable at a time. It is possible to select similar samples of sensitivity based on derivatives using Neural Networks and perform uncertainty quantification.

One of the advantages of local methods is that it is possible to make a matrix to represent all the sensitivities of a system, thus providing an overview that cannot be achieved with global methods if there are a large number of input and output variables.[21]​.

Regression analysis

Regression analysis, in the context of sensitivity analysis, involves fitting a linear regression to the model response and using standardized regression coefficients as direct measures of sensitivity. The regression must be linear with respect to the data (i.e., a hyperplane, without quadratic terms, etc., as regressors) because otherwise the normalized coefficients are difficult to interpret. Therefore, this method is more suitable when the model response is linear; Linearity can be confirmed, for example, if the coefficient of determination is large. The advantages of regression analysis are its simplicity and low computational cost.

Variance-based methods

Main article: Variance-based sensitivity analysis.

Variance-based methods[22]​ are a class of probabilistic approaches that quantify input and output uncertainties as probability distributions, and decompose the output variance into parts attributable to input variables and combinations of variables. The sensitivity of the output to an input variable is measured by the amount of variance in the output caused by that input. They can be expressed as conditional expectations, that is, considering a model Y = f(X) for X = {X, X, ... X}, a sensitivity measure of the i-th variable X is given by,.

where "Var" and "E" denote the variance and expected value operators respectively, and X denotes the set of all input variables except X. This expression essentially measures the contribution of X alone to the uncertainty (variance) in Y (averaged over variations in other variables), and is known as the first-order sensitivity index or main effect index. It is important to note that it does not measure the uncertainty caused by interactions with other variables. Another measure, known as the total effect index, gives the total variance in Y caused by X and its interactions with any of the other input variables. Both quantities are usually normalized by dividing them by Var(Y).

Variance-based methods allow a complete exploration of the input space, taking into account interactions and non-linear responses. For these reasons they are widely used when it is feasible to calculate them. Typically, this calculation involves the use of Monte Carlo methods, but as this can involve many thousands of model runs, other methods (such as emulators) can be used to reduce the computational overhead where necessary.

Variographic analysis of response surfaces (VARS)

One of the main shortcomings of the above sensitivity analysis methods is that none of them consider the spatially ordered structure of the response/output surface of the Y=f(X) model in the parameter space. Using the concepts of directional variograms and covariograms, variogram analysis of response surfaces (VARS) addresses this weakness by recognizing a spatially continuous correlation structure at the values ​​of Y, and therefore also at the values ​​of.

.[23][24]​.

Basically, the greater the variability, the more heterogeneous the response surface along a particular direction/parameter, at a specific perturbation scale. Consequently, in the VARS framework, directional variogram values ​​for a given perturbation scale can be considered as a complete illustration of sensitivity information, by linking variogram analysis to the concepts of perturbation direction and scale. As a result, the VARS framework takes into account the fact that sensitivity is a scale-dependent concept, and therefore overcomes the scale problem of traditional sensitivity analysis methods.[25]​ More importantly, VARS is able to provide relatively stable and statistically robust estimates of parameter sensitivity with much lower computational cost than other strategies (around two orders of magnitude more efficient).[26]​ Importantly, it has been shown that there is a theoretical link between the VARS framework. and variance-based and derivative-based approaches.

Se han desarrollado varios métodos para superar algunas de las limitaciones comentadas anteriormente, que de otro modo harían inviable la estimación de las medidas de sensibilidad (la mayoría de las veces debido a los gastos de cálculo). En general, estos métodos se centran en el cálculo eficiente de medidas de sensibilidad basadas en la varianza.

Emulators

Emulators (also known as metamodels, surrogate models, or response surfaces) are data modeling/machine learning approaches that involve constructing a relatively simple mathematical function, known as an emulator, that approximates the input/output behavior of the model itself.[27]​ In other words, it is the concept of "modeling a model" (hence the name "metamodel"). The idea is that although computer models can be a very complex series of equations that can take a long time to solve, they can always be considered as a function of their inputs Y = f(X). By running the model at a series of points in the input space, it may be possible to fit a much simpler emulator η(X), such that η(X) ≈ f(X) within an acceptable margin of error.[28] Sensitivity measures can then be calculated from the emulator (either with Monte Carlo or analytically), which will have negligible additional computational cost. Importantly, the number of model runs required to tune the emulator can be orders of magnitude smaller than the number of runs required to directly estimate sensitivity measures from the model[29].

Clearly, the crux of an emulator approach is to find a η (emulator) that is a close enough approximation to the f model. This requires the following steps:

Model sampling can often be done with low discrepancy sequences, such as the Sobol sequence - due to mathematician Ilya M. Sobol - or Latin hypercube sampling, although random designs can also be used, with some loss of efficiency. The selection of emulator type and training are intrinsically linked, since the training method will depend on the class of emulator. Some types of emulators that have been used successfully for sensitivity analysis are:

Using an emulator introduces a machine learning problem, which can be difficult if the model response is very non-linear. In all cases, it is useful to check the accuracy of the emulator, for example by cross-validation.

High Dimensional Model Representations (HDMR)

A high-dimensional model representation (HDMR)[35]​[36]​ (the term is due to H. Rabitz[37]​) is essentially an emulator approach, which consists of decomposing the output of the function into a linear combination of input terms and interactions of increasing dimensionality. The HDMR approach takes advantage of the fact that the model can usually be well approximated by neglecting higher order interactions (second or third order and higher). The terms of the truncated series can then be approximated by, for example, polynomials or splines (REFS) and the response can be expressed as the sum of the main effects and interactions up to the order of truncation. From this perspective, HDMRs can be seen as emulators that neglect high-order interactions; The advantage is that they are capable of emulating models with higher dimensionality than full-order emulators.

Fourier amplitude sensitivity test (FAST)

The Fourier amplitude sensitivity test (FAST) uses the Fourier series to represent a multivariate function (the model) in the frequency domain, using a single frequency variable. Therefore, the integrals necessary to calculate the sensitivity indices become univariate, which represents computational savings.

Others

Methods based on Monte Carlo filtering.[38] These are also based on sampling and the objective here is to identify regions in the space of input factors corresponding to particular values ​​(e.g. high or low) of the output.

A sensitivity analysis of a model-based study may be intended to support an inference and certify its robustness, in a context where the inference feeds into a policy or decision-making process. In these cases, the framing of the analysis itself, its institutional context, and the motivations of its author can take on great importance, and a pure sensitivity analysis - with its emphasis on parametric uncertainty - can be considered insufficient. The emphasis on framing may derive, among other things, from the relevance of political study to different interest groups that are characterized by different norms and values ​​and, therefore, by a different story about “what the problem is” and, above all, about “who is telling the story.” In most cases, the framework includes more or less implicit assumptions, which may be political (e.g. which group needs to be protected) or technical (e.g. which variable can be treated as a constant).

To properly take these concerns into account, AS instruments have been expanded to provide an assessment of the entire knowledge and model generation process. This approach has been called a "sensitivity audit." It is inspired by NUSAP,[39]​ a method used to qualify the value of quantitative information with the generation of 'Pedigrees' of numbers. The sensitivity audit has been specially designed for a contentious context, in which not only the nature of the evidence, but also the degree of certainty and uncertainty associated with it, will be the subject of partisan interests.[40]​ The sensitivity audit is recommended in the European Commission's guidelines for impact assessment,[41]​ as well as in the Scientific Advisory Council of the European Academies report.[42]​.

Contenido

Existe un gran número de enfoques para realizar un análisis de sensibilidad, muchos de los cuales se han desarrollado para abordar una o varias de las limitaciones comentadas anteriormente. También se distinguen por el tipo de medida de sensibilidad, ya sea basada en (por ejemplo) descomposiciones de varianza"), derivadas parciales o efectos elementales. En general, sin embargo, la mayoría de los procedimientos se ciñen al esquema siguiente:.

En algunos casos, este procedimiento se repetirá, por ejemplo en problemas de gran dimensión en los que el usuario tenga que descartar variables sin importancia antes de realizar un análisis de sensibilidad completo.

Los distintos tipos de "métodos básicos" (que se examinan más adelante) se distinguen por las diversas medidas de sensibilidad que se calculan. Estas categorías pueden solaparse de algún modo. Se pueden dar formas alternativas de obtener estas medidas, bajo las restricciones del problema.

One at a time (OAT)

Main article: Single factor method.

One of the simplest and most common approaches is to change one factor at a time (OAT), to see what effect it has on the result.[13]​[14]​[15]​ OAT usually involves:.

Sensitivity can be measured by monitoring changes in the outcome, for example, using partial derivatives or linear regression. This seems a logical approach, since any observed change in the result will be unequivocally due to the single changed variable. Additionally, by changing one variable at a time, you can keep all the others fixed at their central or reference values. This increases the comparability of the results (all "effects" are calculated with reference to the same central point in space) and minimizes the chances of computer program crashes, which are more likely when several input factors are changed simultaneously. OAT is often preferred by modellers for practical reasons. In case of model failure, the modeller immediately knows which input factor is responsible for the failure.

However, despite its simplicity, this approach does not fully explore the input space, as it does not take into account the simultaneous variation of the input variables. This means that the OAT approach cannot detect the presence of interactions between input variables and is not suitable for non-linear models[16].

The proportion of input space left unexplored by an OAT approach grows superexponentially with the number of inputs. For example, a parameter space of 3 variables that is explored one at a time is equivalent to taking points along the x, y, and z axes of a cube centered at the origin. The convex hull that bounds all these points is an octahedron that has a volume of only 1/6 of the total parameter space. More generally, the convex hull of the axes of a hyperrectangle forms a hyperoctahedron that has a volume of . With 5 inputs, the explored space already drops to less than 1% of the total parameter space. And even this is an overestimate, since the off-axis volume is not being sampled at all. Compare with random sampling of space, in which the convex hull approaches the entire volume as more points are added.[17] Although in theory OAT sparsity is not a problem for linear models, true linearity is rare in nature.

Local methods based on derivatives

Methods based on the local derivative consist of taking the partial derivative of the output Y with respect to an input factor X:.

where the subscript x indicates that the derivative is taken at a fixed point in the input space (hence the "local" in the class name). Adjoint modeling[18][19]​ and automated differentiation[20]​ are methods of this class. Like OAT, local methods do not attempt to fully explore the input space, as they examine small perturbations, typically one variable at a time. It is possible to select similar samples of sensitivity based on derivatives using Neural Networks and perform uncertainty quantification.

One of the advantages of local methods is that it is possible to make a matrix to represent all the sensitivities of a system, thus providing an overview that cannot be achieved with global methods if there are a large number of input and output variables.[21]​.

Regression analysis

Regression analysis, in the context of sensitivity analysis, involves fitting a linear regression to the model response and using standardized regression coefficients as direct measures of sensitivity. The regression must be linear with respect to the data (i.e., a hyperplane, without quadratic terms, etc., as regressors) because otherwise the normalized coefficients are difficult to interpret. Therefore, this method is more suitable when the model response is linear; Linearity can be confirmed, for example, if the coefficient of determination is large. The advantages of regression analysis are its simplicity and low computational cost.

Variance-based methods

Main article: Variance-based sensitivity analysis.

Variance-based methods[22]​ are a class of probabilistic approaches that quantify input and output uncertainties as probability distributions, and decompose the output variance into parts attributable to input variables and combinations of variables. The sensitivity of the output to an input variable is measured by the amount of variance in the output caused by that input. They can be expressed as conditional expectations, that is, considering a model Y = f(X) for X = {X, X, ... X}, a sensitivity measure of the i-th variable X is given by,.

where "Var" and "E" denote the variance and expected value operators respectively, and X denotes the set of all input variables except X. This expression essentially measures the contribution of X alone to the uncertainty (variance) in Y (averaged over variations in other variables), and is known as the first-order sensitivity index or main effect index. It is important to note that it does not measure the uncertainty caused by interactions with other variables. Another measure, known as the total effect index, gives the total variance in Y caused by X and its interactions with any of the other input variables. Both quantities are usually normalized by dividing them by Var(Y).

Variance-based methods allow a complete exploration of the input space, taking into account interactions and non-linear responses. For these reasons they are widely used when it is feasible to calculate them. Typically, this calculation involves the use of Monte Carlo methods, but as this can involve many thousands of model runs, other methods (such as emulators) can be used to reduce the computational overhead where necessary.

Variographic analysis of response surfaces (VARS)

One of the main shortcomings of the above sensitivity analysis methods is that none of them consider the spatially ordered structure of the response/output surface of the Y=f(X) model in the parameter space. Using the concepts of directional variograms and covariograms, variogram analysis of response surfaces (VARS) addresses this weakness by recognizing a spatially continuous correlation structure at the values ​​of Y, and therefore also at the values ​​of.

.[23][24]​.

Basically, the greater the variability, the more heterogeneous the response surface along a particular direction/parameter, at a specific perturbation scale. Consequently, in the VARS framework, directional variogram values ​​for a given perturbation scale can be considered as a complete illustration of sensitivity information, by linking variogram analysis to the concepts of perturbation direction and scale. As a result, the VARS framework takes into account the fact that sensitivity is a scale-dependent concept, and therefore overcomes the scale problem of traditional sensitivity analysis methods.[25]​ More importantly, VARS is able to provide relatively stable and statistically robust estimates of parameter sensitivity with much lower computational cost than other strategies (around two orders of magnitude more efficient).[26]​ Importantly, it has been shown that there is a theoretical link between the VARS framework. and variance-based and derivative-based approaches.

Se han desarrollado varios métodos para superar algunas de las limitaciones comentadas anteriormente, que de otro modo harían inviable la estimación de las medidas de sensibilidad (la mayoría de las veces debido a los gastos de cálculo). En general, estos métodos se centran en el cálculo eficiente de medidas de sensibilidad basadas en la varianza.

Emulators

Emulators (also known as metamodels, surrogate models, or response surfaces) are data modeling/machine learning approaches that involve constructing a relatively simple mathematical function, known as an emulator, that approximates the input/output behavior of the model itself.[27]​ In other words, it is the concept of "modeling a model" (hence the name "metamodel"). The idea is that although computer models can be a very complex series of equations that can take a long time to solve, they can always be considered as a function of their inputs Y = f(X). By running the model at a series of points in the input space, it may be possible to fit a much simpler emulator η(X), such that η(X) ≈ f(X) within an acceptable margin of error.[28] Sensitivity measures can then be calculated from the emulator (either with Monte Carlo or analytically), which will have negligible additional computational cost. Importantly, the number of model runs required to tune the emulator can be orders of magnitude smaller than the number of runs required to directly estimate sensitivity measures from the model[29].

Clearly, the crux of an emulator approach is to find a η (emulator) that is a close enough approximation to the f model. This requires the following steps:

Model sampling can often be done with low discrepancy sequences, such as the Sobol sequence - due to mathematician Ilya M. Sobol - or Latin hypercube sampling, although random designs can also be used, with some loss of efficiency. The selection of emulator type and training are intrinsically linked, since the training method will depend on the class of emulator. Some types of emulators that have been used successfully for sensitivity analysis are:

Using an emulator introduces a machine learning problem, which can be difficult if the model response is very non-linear. In all cases, it is useful to check the accuracy of the emulator, for example by cross-validation.

High Dimensional Model Representations (HDMR)

A high-dimensional model representation (HDMR)[35]​[36]​ (the term is due to H. Rabitz[37]​) is essentially an emulator approach, which consists of decomposing the output of the function into a linear combination of input terms and interactions of increasing dimensionality. The HDMR approach takes advantage of the fact that the model can usually be well approximated by neglecting higher order interactions (second or third order and higher). The terms of the truncated series can then be approximated by, for example, polynomials or splines (REFS) and the response can be expressed as the sum of the main effects and interactions up to the order of truncation. From this perspective, HDMRs can be seen as emulators that neglect high-order interactions; The advantage is that they are capable of emulating models with higher dimensionality than full-order emulators.

Fourier amplitude sensitivity test (FAST)

The Fourier amplitude sensitivity test (FAST) uses the Fourier series to represent a multivariate function (the model) in the frequency domain, using a single frequency variable. Therefore, the integrals necessary to calculate the sensitivity indices become univariate, which represents computational savings.

Others

Methods based on Monte Carlo filtering.[38] These are also based on sampling and the objective here is to identify regions in the space of input factors corresponding to particular values ​​(e.g. high or low) of the output.

A sensitivity analysis of a model-based study may be intended to support an inference and certify its robustness, in a context where the inference feeds into a policy or decision-making process. In these cases, the framing of the analysis itself, its institutional context, and the motivations of its author can take on great importance, and a pure sensitivity analysis - with its emphasis on parametric uncertainty - can be considered insufficient. The emphasis on framing may derive, among other things, from the relevance of political study to different interest groups that are characterized by different norms and values ​​and, therefore, by a different story about “what the problem is” and, above all, about “who is telling the story.” In most cases, the framework includes more or less implicit assumptions, which may be political (e.g. which group needs to be protected) or technical (e.g. which variable can be treated as a constant).

To properly take these concerns into account, AS instruments have been expanded to provide an assessment of the entire knowledge and model generation process. This approach has been called a "sensitivity audit." It is inspired by NUSAP,[39]​ a method used to qualify the value of quantitative information with the generation of 'Pedigrees' of numbers. The sensitivity audit has been specially designed for a contentious context, in which not only the nature of the evidence, but also the degree of certainty and uncertainty associated with it, will be the subject of partisan interests.[40]​ The sensitivity audit is recommended in the European Commission's guidelines for impact assessment,[41]​ as well as in the Scientific Advisory Council of the European Academies report.[42]​.