Effectiveness of Various Variable Importance Methods in A Case Study of Highly Correlated Predictors

Variable importance is a common decision-making problem to public health (healthcare) to determine the key risk factors (quality measures) that contribute more to the outcome. However, it can be a very challenging statistical problem when predictors are strongly correlated and no clear clue to remove the redundancy since it’s hard to isolate the unique contribution.

Traditional methods such as correlation and OLS regression are probably two most popular methods used by public health professionals but they both have the inherent limitations. Correlation only measures the marginal effect between a predictor and the outcome without considering the other predictors. OLS takes multiple predictors into consideration but the estimates (standard beta or partial correlation) become unstable when multicollinearity is present.

Three advanced methods that are developed more recently to address this inter-correlation issue are: relative weight analysis (RW), variable importance in projection (VIP), and correlation-adjusted regression (CAR).  RW and VIP share similar framework where the importance is measured on the transformed and independent principal components and then transformed back through the loading matrix. CAR adjusts the standard beta with the correlation structure without transformation.

Although these methods are theoretically more advanced their effectiveness was rarely studied before. In this small case study with a clear-cut correlation structure the true importance is possibly available as the benchmark through a sensible dimension reduction regression and assumption of similar marginal effect within a reduced dimension. RW is the only method completely consistent with the benchmark rank while PLS similar to OLS and CAR to correlation.