INTRODUCTION

In ‘The Battle for Investment Survival’ Loeb (2007) claims that ‘Diversification might be necessary where no intelligent supervision is likely’. Diversification is not an issue for the intelligent investor who is able to perfectly forecast market returns. In reality however and particularly today it is difficult to estimate risk premia (de Laguiche and Pola, 2012) because: (i) many financial variables are in uncharted territory, (ii) rising macroeconomic volatility makes it difficult to apply equilibrium-based approaches, and (iii) the unconventional monetary policy implemented by central banks impact the risk premia durably.

Awareness has grown over the eighties and nineties, that making incorrect assumptions in a portfolio construction can be dramatic (Jobson and Korkie, 1980, Best and Grauer, 1991 and Chopra and Ziemba, 1993). The evidence has led researchers and practitioners to investigate more robust portfolio construction schemes that can reduce the estimation risk in the process. The most popular approaches rely on Bayesian theory, for example, the Black and Litterman (1991) model, and Robust Asset Allocation models (Meucci, 2007).

In the last decade the financial industry moved further, designing investment processes that can completely ignore assumptions on expected returns. The most popular are the Minimum Variance-, the Most Diversified- and the Risk-Parity portfolios (see Choueifaty and Coignard, 2008; Clarke et al, 2013; Martellini, 2008; Maillard et al, 2010). These approaches allow the investor to allocate assets according to a risk model alone. The need for diversification led the financial industry to explore new directions (see Ilmanen, 2011), complementing traditional asset classes (bonds and equity) with commodities, real estate, investment styles (value, trend, carry and volatilities), and factors (growth, inflation, illiquidity and tail risks).

The subject of this article is portfolio diversification. For a review on the mathematical formulation of the metrics, we refer the interested reader to Roncalli (2013) who gives a comprehensive review, or to Pola (2013) for a shorter version. In this article, we discuss certain aspects that seem to have been neglected in the literature:

  • how to quantify a lack of diversification in Markowitz-optimal portfolios?

  • what is the relationship between risk and diversification?

  • how to improve the diversification of portfolios on the efficient frontier?

  • how to reconcile a pure risk-parity (high-diversification) portfolio with a fundamental-based (high-conviction) approach?

This article is organised as follows. The section ‘Diversification measures’ offers an introduction to the most relevant diversification measures. In the section ‘Entropy as a measure of portfolio diversification’ we focus on entropy. From the section ‘How to quantify the lack of diversification in a Markowitz-optimal portfolio?’ to ‘How can a fully diversified (risk-parity-like) portfolio be reconciled with a fundamental-based (high-conviction) approach to portfolio management?’ we address the above-mentioned questions. In the section ‘Conclusion’ we conclude with some final remarks. The Appendix contains details on the database and on risk-return hypotheses.

DIVERSIFICATION MEASURES

Diversification is a well-known concept in finance. A portfolio is diversified if it is not heavily exposed to individual shocks (Meucci, 2009), individual shocks referring to events such as default, a sharp drawdown, a macroeconomic shock (for example, unexpected rising inflation), a market crash (for example, Black Monday in October 1987 or the Lehman crash in September 2008).

The notion of portfolio diversification is closely related to that of asset segmentation. The taxonomy of assets can be approached via:

  1. i)

    the similarity of asset types (for example, nominal bond, equity, …),

  2. ii)

    risk arguments on security level (volatility, VaR and correlation),

  3. iii)

    the similarity of assets’ sensitivity to macroeconomic- and stress factors.

In line with this portfolio construction can be regarded as an allocation over (i) capital, (ii) risk budget, or (iii) factors, giving three ways of looking at portfolio diversification. Diversification can be expressed in terms of

  1. i)

    portfolio weights,

  2. ii)

    risk contributions,

  3. iii)

    economic scenario dependence.

We briefly review the main metrics deployed in the literature for each category.

Diversification metrics in terms of portfolio weights do not require information on asset risk properties; they depend on portfolio weights only and hence do not suffer from estimation risk. They are adequate for quantifying the diversification of a portfolio composed of assets that are similar in terms of risk (for example, stock portfolios or bond buy-and-hold portfolios). The most popular in the literature are the Herfindahl index, the Lorentz curve (see Pola, 2013 for a formal definition), the Gini coefficient, and the entropy-based diversification measure, discussed in the section ‘Entropy as a measure of portfolio diversification’.

Diversification measures in terms of risk contribution incorporate information on asset risk, which is in practice usually obtained through historical evaluation. They typically rely on the historical covariance matrix, and hence suffer from estimation risk. Those measures provide a good indication of diversification in an investment universe characterised by a rather large volatility spectrum (for example, diversified portfolios invested in assets that range from short-term bonds to highly volatile stocks). Common measures are entropy-based (expressed in terms of asset volatility, asset risk contribution and principal components; Meucci, 2009), the diversification ratio (Choueifaty and Coignard, 2008) and the metrics mentioned in the previous category augmented by a risk contribution (the Herfindahl index, Lorentz curve and the Gini coefficient). Meucci et al (2014) introduced the minimum-torsion bet as a generalisation of the ‘marginal contribution to risk’ in traditional risk-parity portfolios.

As to diversification by economic scenario dependence, the atoms of asset allocation are factors. Asset price behaviour is considered as a mere manifestation of the underlying (hidden) macroeconomic- or stress factors. In this case prior information might be particularly useful in identifying relevant factors; additional information on asset volatilities and correlations are sometimes taken into account as well.

In the next section we review the entropy approaches. We refer the interested reader to Pola (2013) for more technical details and analytical properties of the diversification metrics reviewed in this section.

ENTROPY AS A MEASURE OF PORTFOLIO DIVERSIFICATION

The link between portfolio diversification and entropy lies in the notion of uncertainty. Entropy is a measure that originates from Physics, but is now applied in many disciplines such as Computer Science, Sociology, Economics, Medicine, Mathematics and Finance (Bera and Park, 2008).

In Information Theory, entropy indicates the degree of predictability of a stochastic dynamical system (Cover and Thomas, 1991): the higher the entropy, the less predictable the system. Entropy decreases when additional information becomes available. On the same token portfolio diversification is desirable when information lacks or the financial markets are uncertain.

Let us suppose that a physical system can be described by N discrete states, {p i }i=1, …, N representing the probability associated to each state. The Shannon entropy (Cover and Thomas, 1991) is defined as follows:

where ln represents the natural logarithm. The average number of relevant states (Meucci, 2009) is defined as η = exp(H). It can be proven that:

  1. i)

    entropy is maximal (η = N) in a completely unpredictable physical system, where all states are equally likely {p i =1/N} i=1, …, N ;

  2. ii)

    entropy is minimal (η = 1) in a fully deterministic system where the probability of one state is equal to 1;

  3. iii)

    if probabilities {p i } i=1, …, N are equally likely on M (s.t. M<N) states; then η = M.

The analytical properties support the interpretation of η as the average number of relevant states. We shall now review the application of entropy as a tool to quantify portfolio diversification.

The entropy measure in portfolio weights (H w ) is defined by replacing the probability set {p i }i=1, …, N with the portfolio weights {w i }i=1, …, N. H w is maximal for the equally weighted portfolio. η w = exp(H w ) ranges from 1 (maximal concentration) to N (maximal diversification). The measure is well-specified for long-only portfolios only with weights that sum up to 1.

The entropy measure in asset volatility (H vol ) is a function of asset volatilities. It is maximal (ln N) for the naïve risk-parity portfolio (i.e. weights are inversely proportional to the asset volatilities). The measure is defined by replacing the probability set {p i }i=1, …, N with the quantities {w i 2 σ i 2 Z}i=1, …, N, where σi represents asset volatility and Z is a normalisation constant to ensure all adds up to 1, that is,

In this case η vol = exp(H vol ) represents the average number of relevant assets in the risk space, assuming zero correlation among assets. η vol ranges from 1 to N.

The entropy measure in risk contribution (H w risk) is a function of asset risk contributions. It is maximal (ln N) for the risk parity portfolio. The probability set is replaced by the set of risk contributions in an analogous way, though account must be taken of possible negative correlations. Pola (2013) suggests a generalisation of the metrics proposed in the literature. He suggests rescaling the risk contributions {θ i }i=1, …, N as follows

where ρij refers to correlation between asset i and j. In this case represents the average number of relevant assets in the risk space. It is worth stressing that in order to avoid degenerate cases, absolute values were to be introduced; that corroborates with the evidence that (rare) extremely negative risk contributions are certain to reduce portfolio diversification.

The entropy measure in principal components () can be deployed when a portfolio is represented in terms of principal components. It is maximal for the risk-parity portfolio defined in the principal component space. Principal Component Analysis allows us to rewrite the covariance matrix CM as

where E and Λ respectively correspond to the eigenvector matrix and to a diagonal matrix including the eigenvalues. The portfolio exposure w can be projected onto the principal components (Meucci, 2009) as

The risk exposure to the principal components is given by the following terms

for i = 1, …., M, where M is the number of relevant principal components within a given confidence level. {h}i=1, …, M is referred to in the literature as the variance concentration curve (Meucci, 2009). The probability set {p i }i=1, …, N in the entropy formula is replaced by the normalised variance concentration curve (see Pola, 2013 for details), which is defined as follows

represents the average number of relevant principal components in the portfolio. ranges from 1 to M. Different measures of entropy defined in terms of principal components can be found in Meucci (2009), Rudin and Morgan (2006) and Partovi and Caputo (2004).

We briefly review the diversification ratio (Choueifaty and Coignard, 2008) in this section. The diversification ratio (DR) is specified as the ratio of the weighted sum of asset volatilities on the portfolio’s total volatility. DR is maximal for the Most Diversified or Maximum Sharpe Portfolio. The square of DR can be interpreted as the portfolio’s degree of freedom (see Choueifaty and Coignard, 2008). In the case of independent assets and portfolio weights inversely proportional to asset volatilities (naïve risk-parity portfolio), DR reaches its maximum (N1/2), N being the number of assets in the portfolio.

We conclude this section by briefly discussing the so-called duplication invariance property that deals with identical assets (asset correlated to one) in a portfolio construction problem. As detailed in Choueifaty and Coignard (2008), ‘consider a universe where an asset is duplicated (for example, because of multiple listings of the same asset). An unbiased portfolio construction process should produce the same portfolio, regardless of whether the asset was duplicated’. Among all the diversification measures discussed in this section, only the diversification ratio verifies this property. In Pola and Zerrad (2014), we present a numerical illustration (see Exhibit 4). Indeed, the drawback of fulfilling the duplication invariance property is a higher sensitivity to correlation: indeed the portfolio construction of the Most Diversified portfolio scales quadratically with the correlation matrix. This might lead to a more pronounced exposure to estimation risk on correlation and hence to more unstable allocations in case of uncertainty in asset correlation.

HOW TO QUANTIFY THE LACK OF DIVERSIFICATION IN A MARKOWITZ-OPTIMAL PORTFOLIO?

Despite its limitations, asset allocation is in practice usually based on the Markowitz (1952) model. The lack of diversification in the efficient portfolios is well documented in the literature. Bayesian and robust approaches have been proposed to alleviate the problem and to limit the instability of optimal solutions as well because of the uncertainty in the input parameters (Black and Litterman, 1991; Meucci, 2007).

In this section we introduce the concept of available diversification in a specific risk budget, to then evaluate the lack of diversification in Markowitz-optimal portfolios. We consider a simple case study.

Let an investment universe be composed of 12 risky assets, 4 bonds, 4 equities and 4 alternatives, over a period running from June 2005 to May 2015:

  • Bonds: JP Morgan US all maturity bond (B1), Barclays US inflation-linked bond (B2), Merrill Lynch US Investment Grade (B3), Merrill Lynch US High Yield (B4);

  • Equities: S&P 500 (E1), Russell 2000 (E2), MSCI World (E3), MSCI Emerging (E4);

  • Alternatives: US Private Equity (A1), Global Infrastructure (A2), Global Real Estate (A3), Commodity CRB (A4).

In Appendix are the details on the time series and risk-return assumptions. We computed volatilities and correlations according to the EWMA technique (last 3 years half-life on monthly observations). Expected returns have been set according to a normative approach based on the constant Sharpe ratio hypothesis (see de Laguiche and Pola, 2012).

Figure 1 (top panel) displays the Markowitz-optimal portfolios. Note that the portfolios are not very diversified. Despite a neutral assumption in terms of expected Sharpe ratio, only seven out of twelve assets have been allocated.

Figure 1
figure 1

Lack of diversification in the Markowitz frontier.

Full size image

In Figure 1 (bottom panel) we introduce the diversification curves. They correspond to the level of diversification (entropy) plotted against the portfolio volatility; indeed diversification curves are a function of portfolios. We define available diversification as the maximum diversification that can be achieved for a specific risk target under the standard budget and long-only constraints (that is, portfolio weights positive numbers and sum up to 1). Entropy has been calculated in terms of portfolio weights in the chart. In order to quantify the lack of diversification in the Markowitz-efficient portfolios, we also plot the diversification curve of those. Diversification (Y-axis) is measured in terms of ηw, that is, the average number of relevant assets.

The available diversification peaks at 12 for a volatility target of 11.5 per cent, corresponding to the equally weighted portfolio. Conversely, the Markowitz portfolios deliver a diversification of 5.8 at most, about half of the available diversification.

WHAT IS THE RELATIONSHIP BETWEEN RISK AND DIVERSIFICATION?

Figure 2 displays the diversification curves for the efficient portfolios, as calculated in terms of portfolio weights, asset volatility, risk contribution, principal components and by taking the square of the diversification ratio.

Figure 2
figure 2

Diversification curves of the efficient portfolios.

Full size image

The diversification curve expressed in portfolio weights (solid line) is copied from Figure 1. It peaks at about 10 per cent portfolio volatility indicating that, in this risk region, asset classes are maximally represented. The value of the curve is close to 6, meaning that on average six assets are present. Note that the curve converges to 1 for a high-risk profile thus indicating a concentration on only one asset.

The diversification curve calculated in asset volatility (dashed line) peaks at a lower volatility level than the one calculated in portfolio weights. This result corroborates with the analytical result of Maillard et al (2010) stating that risk-parity portfolios tend to be less risky than equally weighted allocations. Again the right part converges to 1.

The dotted line represents the diversification in asset risk contribution. The line is almost always below the dashed line, because of the fact that it takes the (positive) correlations into account which reduces the diversification potential. Its pattern is similar to the dashed line.

The curve with plus marker type refers to the entropy in terms of principal components. Note that in this definition the diversification potential is much lower. Interestingly the curve does not converge to 1 for the high-volatile one-asset portfolio (on the right). This is because a one-asset investment can lead to an exposure to more than one principal component. In our test the riskiest portfolio is 100 per cent invested in US Private Equity and is exposed (on average) to 1.5 principal components.

The line with circle market type corresponds to the square of the diversification ratio. This ratio is maximal in correspondence with the tangency portfolio under the constant Sharpe ratio hypothesis (Choueifaty and Coignard, 2008).

In all the diversification curves show a complex pattern between portfolio volatility and diversification. Note that higher diversification does not always imply lower risk. The evidence suggests that the portfolio diversification and the risk budget should be managed separately; while the first should be leveraged in the event of uncertainty and conversely reduced if convictions on asset returns are strong, the second is proportional to the overall exposure to asset risk premia. It should therefore only be reduced if fund managers believe that risk premia have increased significantly in the markets. Moreover, the risk level of an asset allocation is usually set ex-ante according to the investor’s risk aversion.

HOW TO IMPROVE DIVERSIFICATION OF THE EFFICIENT FRONTIER?

In this section, we propose a simple remedy that can produce more diversified allocations at negligible cost in terms of portfolio risk-return efficiency. The main idea is to consider a class of mean-variance sub-optimal portfolios that are situated just below the efficient frontier on the risk-return grid, and to optimise them with respect to a given diversification measure. Figure 3 gives the results.

Figure 3
figure 3

Diversification curves of optimally diversified portfolios.

Full size image

More formally, portfolios are obtained by maximising the diversification measures (the entropy in portfolio weights, asset volatility, asset risk contribution, principal components and the diversification ratio), under the following constraints:

  • long-only and budget constraints (portfolio weights are positive numbers that sum up to 1);

  • delta (penalty) between the portfolio’s expected return and the return of an efficient portfolio with the same volatility is bounded (in absolute terms) by 10 bps.

By comparing Figure 3 (top panel) with Figure 2, one can see that a negligible cost in terms of portfolio return (10 bps at most) can produce a huge increase in portfolio diversification according to all the metrics. Figure 3 (bottom panel) shows the optimally diversified portfolios with respect to the entropy in asset risk contribution. The asset allocation in this chart seems much more coherent with a neutral view on asset returns as expressed by the constant Sharpe ratio hypothesis. Indeed, all 12 assets are being allocated.

HOW CAN A FULLY DIVERSIFIED (RISK-PARITY-LIKE) PORTFOLIO BE RECONCILED WITH A FUNDAMENTAL-BASED (HIGH-CONVICTION) APPROACH TO PORTFOLIO MANAGEMENT?

Up to this point we have relied on the constant Sharpe ratio hypothesis to obtain unbiased expected returns for a strategic asset allocation. We now add tactical views to the portfolio construction process. We consider an investor who is bullish on equity markets, assigning a Sharpe ratio for equities equal to 1 as opposed to 0.25. Figure 4 displays three portfolios:

  • Markowitz SAA, the strategic portfolio corresponding to the efficient allocation with respect to the (unbiased) strategic views on asset returns (all Sharpe equal);

  • Markowitz TAA, the tactical portfolio determined by assuming a bullish view on equities;

  • RP, the diversified portfolio obtained by maximising the entropy in asset risk contribution.

Figure 4
figure 4

The effect of a bullish view on equity in the portfolio.

Full size image

In order to better compare the three portfolios, we consider the same risk budget determined by the investor’s given risk aversion (5 per cent annualised volatility). Not surprisingly the investment in equities increases significantly (moving from 18.6 per cent in the strategic allocation to 28.5 per cent in the tactical portfolio). Even if the tactical portfolio is coherent with the investor’s tactical views, it is very concentrated (only four assets are significantly allocated). Moreover, despite the same view on all the equity markets only two equities have been significantly allocated, while exposure to E1 is negligible and to E3 is 0.

The TAA and the RP represent two extreme cases: while the TAA was built on the basis of a strong conviction regarding the equity markets (high-conviction portfolio), the RP is fully forecast-free (full risk-parity portfolio). In Table 1, we report the in-sample risk-return figures of the two portfolios and the diversification analysis (entropy in portfolio weights, asset risk contribution and principal components) as well.

Table 1 Comparing the high-conviction portfolio (TAA) to the diversified one (RP)
Full size table

The high-conviction portfolio delivers a return of 8.4 per cent per annum; it is poorly diversified (only three assets in the risk space are significant). The diversified portfolio is less performing in the example, but it is more diversified according all metrics.

We propose a way to reconcile the two extreme allocations. We maximise the portfolio diversification while controlling for a performance delta with respect to the high-conviction portfolio. More formally, we conduct an optimisation process that maximises the entropy in terms of asset risk contribution under the following constraints:

  • Standard budget and long-only constraints (portfolio weights are positive numbers and sum up to 1);

  • ex-ante annualised volatility is 5 per cent;

  • (the absolute value of) the difference between the portfolio return and the return of the high-conviction portfolio (8.4 per cent) is bound by Δ, Δ corresponding to a penalty in terms of return that we accept in exchange for some additional diversification.

In Figure 5 (top panel) we show how the allocation evolves (Y-axis) with the Δ penalty (X-axis). The portfolio on the left corresponds to high conviction, the one on the right is fully diversified. In order to add more elements to the investment decision process we provide the diversification figures, Figure 5, low panel. Note that the entropy in terms of asset risk contribution increases monotonically with the performance penalty. Also note that the diversification in terms of principal components lags behind when the asset diversification increases.

Figure 5
figure 5

Matching high-conviction with the full risk-parity portfolio.

Full size image

We point out that a minor penalty increases the diversification in terms of asset risk contribution significantly, while the portfolio seems more coherent with the investor’s bullish view on the equity market.

CONCLUSION

Risk diversification is a sensible objective when navigating in uncertain financial conditions. Belief in the need for diversification has driven many asset managers to launching forecast-free investment products such as Minimum Variance, Most Diversified Portfolios and Risk Parity. As they are based on a risk model only, they are supposed to be more robust, especially in situations of uncertainty in the financial markets. There is growing awareness that high-conviction-or value-based-approaches tend to concentrate portfolio bets on few assets.

The rise in diversification-focused investment products is perhaps not surprising today in a time of high macroeconomic volatility and the central banks applying unconventional monetary policy that may cause asset prices to deviate from their fundamentals. Many financial variables are in uncharted territory; whether they will revert to the mean or stabilise to new levels is not clear. In such market conditions, diversification can help to stabilise portfolio performance.

Portfolio diversification is an intuitive concept; however, there is no consensus in the literature on how to measure it. Probably this reflects the fact that many aspects should be taken into account, and hence diversification should be approached from many angles. Diversification measures can be divided into three main categories: (i) metrics in portfolio weights, (ii) measures in risk contribution (either in the asset space or in the principal component space) and (iii) the economic scenario approaches. Among those, the second class suffers most from estimation risk.

Through an empirical portfolio construction example we present a practical means to boost the diversification in a Markowitz-optimal portfolio at negligible cost in terms of risk-return and producing more coherent allocations with respect to the asset return hypotheses.

We introduced a framework to balance a fully diversified portfolio with a fundamental-based high-conviction portfolio. We believe that risk control and generating excess returns are complementary objectives, which in a unified approach can be effective in producing robust allocations.

On a more conceptual note, we show that higher diversification does not always imply lower risk. The implication of that finding is that diversification and risk (volatility) control should be managed separately; while portfolio diversification should be managed according to the uncertainty in the macro variables, portfolio risk should be modulated according to the performance opportunity in the market.

We believe that the ideas presented in this work can contribute to the ongoing research on portfolio diversification, and to provide practical hints to improve standard approaches in asset allocation. The applications presented in this article should be considered for illustrating the methodology. The views expressed in this work are those of the author and do not necessarily correspond to those of ANIMA SGR.