Measuring relative value of independent variables in neural networks

Setting all other variables to zero while varying one at a time is a common and generally sound method for assessing the relative contribution of individual input variables in a trained neural network model. This technique is often referred to as sensitivity analysis or feature importance analysis.

Here’s a breakdown of why it works and some important considerations:

Why it works:

• Isolating Impact: By setting all other variables to zero (or a baseline value), you isolate the effect of the variable you’re changing on the model’s output.

• Relative Comparisons: You can compare the changes in the model’s output for each variable to understand their relative importance.

Important Considerations:

• Baseline Values:

  • Setting other variables to zero might not always be the best approach, especially if zero is outside the realistic range of those variables.

  • Consider using the mean or median of each variable as a baseline value. This can provide a more realistic assessment of their impact.

• Interactions:

  • Neural networks can capture complex interactions between variables. Your method focuses on individual effects, and may not fully capture the effects of interactions between variables.

  • To get a better understanding of interactions you would need to test combinations of variables.

• Non-Linearity:

  • Neural networks are highly non-linear. The importance of a variable might vary depending on the values of other variables.

• Range of Variation:

  • When varying each variable, consider the realistic range of values. Don’t extrapolate beyond the range of data the model was trained on.

• Multiple Runs:

  • Run the analysis multiple times with different baseline values or ranges of variation to ensure the results are consistent.

• Permutation Importance:

  • An alternative to setting variables to zero is permutation importance. This involves randomly shuffling the values of each variable and measuring the impact on the model’s performance. This approach can be more robust to interactions between variables.

• SHAP Values:

  • SHAP (SHapley Additive exPlanations) values are a more advanced method that can provide a more accurate and interpretable assessment of feature importance, taking into account interactions between variables.

In summary:

This method is a good starting point for understanding the relative contributions of individual variables.