Model Transparency
IA believes that there are two critical components to Model Transparency:
- No black boxes. Clients should have unlimited access to information about how the model works. No detail should be considered “proprietary.”
- Verification of Results. Back testing and confidence interval analysis are both critical components in understanding when a model works and when it doesn’t.
Not only does Investor Analytics provide an array of risk analyses for your portfolio, but we also show you exactly how we arrived at these analyses and how accurate they are. The result: You always know what tools and data were used to calculate your portfolio's risk profile.
- For position-based portfolios, we perform a bottom-up analysis each day.
- For returns-based portfolios, we use a top-down approach.
We share the methodologies' details and the calculation of each analytic we provide, and, where possible, we show you diagnostics on how well the models actually work in practice.
The chart below shows the IA Value-at-Risk (VaR) methodology tested against a portfolio’s actual returns. In all VaR models, the 95% VaR limit should actually be exceeded about 5% of the time. The chart below shows an actual portfolio's returns (green bars) and the daily 95% VaR (red line) over all of 2008. The VaR number was exceed 15 times out of 252 trading days, or 5.8% of the time, a very solid result.
Daily Returns (green bars) and daily VaR (red line) shown for an actual portfolio for all of 2008. This graph shows that for this portfolio, IA's 95% VaR model does very well on backtesting - with 15 exceptions out of 252 trading days, equivalent to 5.8%.
Our clients have full access to each statistic’s “fit” accuracy measures, so you can see how well the models perform. Our proprietary Re
turns-based non-linear regressions take your portfolio’s actual performance and determine the best basket of market risk factors.
IA also provides numerical assessments of the goodness-of-fit: R2 is targeted at levels above .6 or .7—one of the highest in the risk analysis industry. Durbin-Watson is targeted as close to 2 as possible—to help identify and eliminate sources of autocorrelation.
Additionally, IA performs a principal component analysis:
Performing this analysis at the outset is a good start, but IA goes farther: We provide daily measures, so you can monitor how well the model continues to work in different market conditions.
