Abstract
We examine mutual funds that appeared in the Wall Street Journal’s SmartMoney Fund Screen column from September 2004 through July 2009. We find that the majority of funds listed do not have Morningstar’s highest five star rating. Regardless of Morningstar rating, the average prepublication performance of the funds is significantly higher than the benchmarks used to measure performance. Post publication, fund performance declines, and the decline is statistically significant across our performance measures. However, additional tests indicate that the SmartMoney funds which have a three or four star rating from Morningstar are better investment values than corresponding five star Morningstar funds with the same prospectus objective and expense ratio.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Recent academic papers have established that media coverage and advertising heavily influence investors’ stock and mutual fund choices (e.g., Jain and Wu 2000; Cronqvist 2006; Barber and Odean 2008; Gallaher, Kaniel, and Starks 2009), and work by Tetlock (2007) emphasizes the influence that the Wall Street Journal has on the investment community. Late in 2004, the WSJ began to publish a column entitled “SmartMoney Fund Screen,” which provides a list of mutual funds based on specific criteria such as fund objective, historical returns, and expense ratios. The majority of the published funds are diversified domestic stock funds, but 40 % of the funds represent other mutual fund classifications such as sector funds, international funds, hybrid funds, and fixed income funds. The column does not explicitly recommend that individuals invest in the listed funds. But, it is likely that many investors view the list as implicitly recommending funds that are most likely to perform the best in the future. At a minimum, with so many mutual funds available to investors, the list serves to attract media attention to the funds and to reduce consumers’ search costs.
In this study, we examine whether the SmartMoney Fund Screen is able to provide value to investors by correctly forecasting which mutual funds will outperform over the subsequent year. We first examine the pre and post publication performance of the funds using various benchmarks that are appropriate given the objective of the published funds. As a second step, we examine the forecasting performance of the SmartMoney Fund Screen relative to the performance of the Morningstar star rating system. As demonstrated by DelGuercio and Tkac (2008), the Morningstar rating system is extremely popular and influential among individual and institutional investors, and previous studies provide evidence of the positive forecasting ability of the Morningstar system. Specifically, Morey and Gottesman (2006) provide evidence that funds which have Morningstar’s highest rating of five stars generate superior future performance relative to lower rated funds, and DelGuercio and Tkac (2008) find that following a trading strategy of investing in only five star rated funds produces positive risk adjusted returns out of sample. Thus, given Morningstar’s dominance in the field, we examine the value of the SmartMoney Fund Screen recommendations in both absolute terms and in terms of their value as an alternative to the Morningstar star system.
We examine 1485 mutual funds published in the WSJ’s “SmartMoney Fund Screen” column from September 7, 2004 to July 14, 2009 when the column was discontinued in the newspaper but remained available to subscribers through the SmartMoney.com website. We find that despite the availability of enough five star funds to compose their entire listings, only 35 % of the SmartMoney funds have a Morningstar five star rating. The largest percentage of funds (42 %) has a four star rating while 18 % of funds have a three star rating.
We use daily fund returns and three different measures to estimate fund performance. We find that on average, the published funds generate positive and significant performance measures during the year before publication which is consistent with viewing the SmartMoney screen as providing implicit recommendations of funds for investors. Excess returns relative to a market wide benchmark average 5.75 % while the average annualized fund alpha is 2.61 %. In the year after publication, there is a statistically significant decline in performance. Excess returns fall on average by −5.38 % while average annualized alpha turns negative. These results hold in absolute terms and conditional on the fund’s Morningstar rating. In addition, these results hold across fund classifications.
We then directly compare the SmartMoney funds to Morningstar. Since investors could choose a five star Morningstar fund instead of the SmartMoney fund, for each SmartMoney fund, we select a five star Morningstar fund that has the same prospectus objective and the closest expense ratio in absolute terms. We then compare the post publication performance of the SmartMoney fund to its corresponding Morningstar fund. Given the results of Morey and Gottesman (2006), we would expect that the Morningstar funds would outperform lower rated funds selected by SmartMoney.
For the five star SmartMoney funds, we find no significant difference in performance between the funds and their matched Morningstar fund. However, we find that the three and four star SmartMoney funds provide better investment value as they outperform their corresponding funds based on each of our performance measures. The average alpha of the three star funds is positive and significantly better than the matched funds while both the three and four star funds generate significantly higher excess returns than their counterparts. Further tests find that this result is consistent across types of funds, and the result is not driven by the methodology used to match the funds. As a robustness test, we find that the results are not driven by fund performance during the 2008 financial crisis.
2 The SmartMoney fund screen sample
The SmartMoney Fund Screen column debuted in the Wall Street Journal on September 7, 2004, as part of an expansion of the Personal Journal section of the newspaper. Typically, the column is published on a weekly basis with few exceptions. Each article focuses on either a specific style of funds (e.g., mid cap funds, emerging market funds), funds with specific characteristics (e.g., low minimum investment funds, funds with long time managers), or funds that fit a specific theme (e.g., funds for retirees, funds for IRAs). The column provides background information on the past and current performance of the fund group on which it is focused. Then, in the last paragraph of the article, the column describes the screens used to select the funds that are listed at the end of the article. For the listed funds, each article always provides the fund name, ticker, and Lipper fund category. Depending on the theme of the article, additional information such as average past return, expense ratio, or total net assets is also provided.
The screens used to determine which funds to list are typically based on the following five criteria: historical returns, expense ratios, minimum required initial investment, net assets, and fund load. The screen used for each criterion varies depending on the theme of the article but has several consistent features. Historical returns must be in the top quarter or half of all funds. Depending on the article’s theme, the screen may use one, three, five, or 10 year historical returns as the criterion or some combination of the returns. Current expense ratios must be in the bottom half of funds, and several screens require expenses to be in the bottom quarter. Typically, the minimum initial investment is required to be less than or equal to $5000. Some screens use a minimum investment value as low as $500 or as high as $25,000. Fund total net assets are required to be at least $50 million unless the screen is specifically focused on smaller mutual funds. Load funds are always eliminated. Additional criteria are used if dictated by the article’s theme.
The SmartMoney screens can be considered less sophisticated than Morningstar’s rating system as it does not explicitly consider risk. A fund’s Morningstar rating is based on various factors including the fund’s risk adjusted return. The ten percent of funds with the highest risk adjusted return within their category receive the highest rating of five stars. Depending on how long a fund has been in existence, their overall star ratings will be a weighted average of its star ratings over the previous three, five, and 10 year period.Footnote 1
The screen was discontinued in the print version of the Wall Street Journal after the last article appeared on July 14, 2009. However, investors who are interested in the articles could continue to access them in real time with a subscription to the SmartMoney.com website or with an approximate 1 month lag at no cost. Given the broad influence that the Wall Street Journal has on the investment community, our study only focuses on the Wall Street Journal articles and thus ends with the July 14, 2009 list of funds.
Given the various themes of the articles, several funds are listed multiple times over the course of a given year. To ensure that these multiple fund listings do not bias the overall results, we create two fund samples. The first fund sample, which we refer to as the full sample, does not include a fund more than once in a calendar quarter. Much of the analysis that follows involves each fund’s Morningstar star rating. We use the quarterly Morningstar Principia Pro fund CDs to obtain the rating. Given that the ratings available to us do not change during a calendar quarter, we only include a fund’s first appearance during a quarter in our sample. Our second sample is more restrictive. As detailed in the next section, we calculate a fund’s performance for a 1 year period before its appearance in the newspaper and for a 1 year period after its listing. Multiple listings of the same fund are only included in the second sample if there is no overlap between the pre and post listing return series used to analyze the fund’s performance. The only funds that meet the above criteria yet are excluded from our analysis are funds listed in eight articles where the screen is explicitly designed to either identify funds that investors should consider selling or the screen is focused on pure index funds.Footnote 2
Our full sample is composed of 1485 mutual funds listed across 214 SmartMoney articles while our subsample is composed of 944 funds. All of our analysis is performed for both the full sample and the subsample. Panel A of Table 1 provides descriptive statistics for the full sample of SmartMoney funds while Panel B provides statistics for the subsample. All data is obtained from the quarterly Morningstar Principia Pro Cds. Panel C provides statistics for the universe of mutual funds that share the same prospectus objectives as the SmartMoney funds. The data for the Morningstar universe of mutual funds is based on the Morningstar Cds at the end of each year covered by our SmartMoney sample.Footnote 3
We highlight three specific aspects of the data. First, when comparing the full fund sample to the Morningstar universe, we find that the sample has a much lower expense ratio (1.13 vs. 2.24 %), is larger in terms of net assets ($3398.2 million vs. 384.04 million), has a lower turnover ratio (81.91 vs 103.91 %), and is much older in years (16.78 vs. 7.38). The results are similar for the subsample of funds. All of these results are consistent with the characteristics of the SmartMoney fund screens discussed previously.
Second, given that Morningstar is the dominant mutual fund data provider and is used by the majority of mutual fund investors, we partition our SmartMoney sample based on the listed fund’s Morningstar rating. The data in Table 1 indicate that the sample does not represent the best funds as determined by Morningstar. Based on the quarterly Morningstar star rating assigned to the fund at the time the fund is listed in the SmartMoney column, only 35.1 % (521 out of 1485) of the SmartMoney funds have a five star rating. The largest number of funds have a four star rating (42.2 %, 626 out of 1485) while 17.7 % of funds (263 out of 1485) have a three star rating. It is possible that the selection of such a large number of lower rated funds is due to the lack of five star funds that exist with the same prospectus objective as the SmartMoney fund. We check this possibility and find this to be true in only 65 of the 889 selections. Thus, it is most likely that the selection of three and four stars funds is due to the restrictive nature of the SmartMoney fund screen as no other five star funds meet the fund screen criteria.
Third, we note the strong similarity of the characteristics of the SmartMoney funds regardless of the Morningstar rating. The mean expense ratio for the three, four, and five star funds falls in a narrow range of 1.09 to 1.13 %. The same holds for average net assets ($3287.17 million to $3890.19 million), turnover ratio (69.07 to 81.30 %), and fund age (16.27 years to 18.21 years). This strong relationship across the star ratings highlights the greater weight that the fund screen places on these fund characteristics relative to the Morningstar star rating system.
Given the differences in the star ratings of the SmartMoney funds, we examine the pre and post publication performance of the SmartMoney funds in both absolute terms and conditional on the fund’s Morningstar star rating. In the last section of this study, we test whether the three and four star funds selected by SmartMoney can outperform comparable five star funds from Morningstar.
3 Empirical methodology: measuring fund performance
To measure performance, we use raw annual returns of the funds and performance relative to three separate benchmarks. Although raw returns do not reflect relative performance nor any adjustment for risk, we include this as a performance measure since some investors using the SmartMoney list are most concerned about absolute returns regardless of risk.
Our second measure is the fund’s excess return relative to a market wide benchmark. Given that our sample includes funds from five major mutual fund groups (domestic stock, domestic sector, international stock, fixed income, and hybrid), we select appropriate market wide benchmarks for each of these classifications. For the domestic stock and domestic sector funds, the benchmark is the value weighted market index from the Center of Research and Securities Prices (CRSP). For the international stock funds, the benchmark is the Morgan Stanley Capital International (MSCI) World excluding US index. For the fixed income funds, we use the Barclay’s US Aggregate Bond Index. Our hybrid fund benchmark is a 60/40 combination of the CRSP stock index and the Barclay’s bond index. The 60/40 combination is based on the work of Comer, Larrymore, and Rodriguez (2009) which indicates that the average stock allocation of their sample of hybrid mutual funds was 59.2 %.
Our third measure is the fund’s Sharpe ratio. We define each fund’s daily Sharpe ratio as
where rd is the average daily excess return of the fund and σr is the standard deviation of the daily excess return. In calculating the Sharpe ratio, we follow Morningstar’s methodology and the ratios we report in our table have been annualized. We consider the Sharpe ratio as an important measure given that our sample is composed of several different categories of funds.
Our final measure is the fund’s alpha. Given that our funds may hold domestic stocks, international stocks, or fixed income securities, we use the following ten factor performance model to measure alpha:
where r p,d is the excess return of fund p on day d, and r ds,d represents the daily returns to a set of domestic stock market factors, r is,d represents the daily returns to a set of international stock market factors, and r b,d represents the daily returns to a set of bond market factors.
There is no widely accepted asset pricing model designed to measure the performance of both domestic and international funds which hold both stocks and bonds. Our model choice reflects a combination of factors which have been used to measure the performance of specific categories of mutual funds. For the three domestic stock market factors, we use the following three factors of Fama and French (1993): 1) the excess return of the market portfolio, 2) the size factor, and 3) the book to market factor. To control for funds’ exposure to international equities, we use the MSCI indices to compute international factors analogous to the Fama French factors. The excess return of the international market portfolio is defined as the return of the MSCI World excluding US index minus the same risk free rate used to calculate the Fama French excess market return. The international size factor is defined as the difference between the MSCI World excluding US Small Cap Index and the MSCI World excluding US Large Cap Index. Similarly, the international book to market factor is defined as the difference between the MSCI World excluding US Value Index and the MSCI World excluding US Growth Index. Comer and Rodriguez (2012) provide evidence that these international factors capture the majority of variation in the returns of world and foreign mutual funds.Footnote 4 The four bond factors are from the hybrid fund model designed by Comer, Larrymore, and Rodriguez (2009). They use the following four bond indices from Barclays as part of the bond maturity model: intermediate maturity government/credit index, long maturity government/credit index, mortgage index, and high yield index.Footnote 5
In the following sections, we use our performance measures to examine the pre and post listing performance of the SmartMoney fund sample.
4 Fund performance
For each fund listed in our sample, we obtain a daily return series from the CRSP database. We follow Comer, Larrymore, and Rodriguez (2009) and use daily return data which provides greater degrees of freedom than monthly data given the use of a ten factor model to measure performance.
Each fund’s daily return series is defined as follows:
where nav p,d is the net asset value of fund p on day d and div p,d ex div dividends of fund p on day d. The pre publication period begins 1 year prior to the day before the fund is listed in the column while the post listing period includes the day of publication and extends 1 year afterwards.Footnote 6 As a specific example, if the fund is listed in the January 4, 2005 SmartMoney column, the pre publication period is defined as January 4, 2004 to January 3, 2005, while the post publication period covers January 4, 2005 to January 3, 2006.Footnote 7
We use the methodologies described in the previous section to estimate the performance of each fund during the pre and post publication periods. In the results that follow, we report the average of the performance measure across all funds in the sample or subsample. The statistical significance of the measure is determined using a t-statistic which is based on the cross sectional standard deviation of all the funds in the sample. For the Sharpe ratio and ten factor alpha, all reported measures have been annualized. For all regressions, we correct for potential heteroskedasticity using the White correction.
4.1 Pre-publication performance
In Table 2, we report the average performance measures during the pre publication period for the entire sample and subsample. The results provide strong evidence that the SmartMoney Fund Screen identifies funds that generate positive performance during the year before publication and thus the screen can be viewed as an implicit recommendation of mutual funds for investors. The average excess fund return is 5.75 %, the Sharpe ratio is 0.62, and the ten index alpha is 2.61 %. The subsample, which only includes funds whose pre and post publication returns series do not overlap, has similar results (5.49 % excess return, 0.65 Sharpe ratio, and 2.50 % alpha) suggesting no significant differences in the composition of the two fund samples. All three performance measures are statistically significant at the one percent level. At the individual fund level, we find that 70.9 % of the funds have a positive excess return, 72.7 % of funds have a positive Sharpe ratio, and 60.0 % of funds have an alpha greater than zero. The large percentage of funds generating a positive alpha is particularly significant given that fund returns are measured after expenses while no expenses have been deducted from the benchmarks and factors included in the index model.
We partition our results by the funds’ Morningstar star rating and report these results in Table 2. Across the five categories, the excess return measure ranges from 4.48 to 9.50 % with performance statistically significant across all classifications. Similarly, all five classifications have statistically significant Sharpe ratios ranging from 0.52 to 0.83. For these two performance measures, no specific category appears to be an outlier.
For the funds’ ten factor model alpha, we do see a distinct pattern that is consistent with the way Morningstar calculates its ratings. Since Morningstar ratings are based on risk adjusted past performance, we would expect higher rated funds to have better pre publication risk adjusted performance, and we do observe a monotonic relationship between the SmartMoney funds’ Morningstar star rating and alpha. The two star SmartMoney funds have the lowest Morningstar rating of any funds in the sample and also have the smallest alpha of 0.97 % which is not statistically significant. Alpha increases to a statistically significant 2.13 % for the three star funds and then there is a slight increase to 2.15 % for the four star funds. The five star funds, the highest rated Morningstar funds in the sample, have the best alpha as their average performance is 3.53 %. This pattern holds for the subsample and is more pronounced as one moves across the star ratings.
Although our focus is on the SmartMoney funds Morningstar ratings, we also take a look at performance by fund type to better understand if specific groups of funds are driving the results. Our overall sample is composed as follows: 59.5 % domestic stock funds, 15.5 % sector funds, 13.9 % international stock funds, 8.9 % hybrid funds, and 2.2 % fixed income funds. One of the reasons that we do not see significant differences in the descriptive statistics of the three, four, and five star funds in Table 1 is that the distribution of funds by type is consistent across the star classifications. Across the categories, domestic stock funds make up between 52.1 and 64.1 % of the sample. The corresponding ranges for the sector funds (13.2 to 16.7 %) international stock funds (11.9 to 20.2 %), hybrid funds (6.6 to 9.8 %) and fixed income funds (1.5 to 4.6 %) are also relatively stable across Morningstar ratings.
Although not reported, the performance results by type indicate that all fund groups have strong pre publication performance.Footnote 8 Excess return relative to the benchmark is positive and significant for all five groups ranging from 2.11 % (hybrid) to 9.70 % (international stock). All Sharpe ratios are also positive and significant ranging from 0.48 (hybrid) to 1.06 (international stock). All alphas are positive except for the hybrid fund category. The alphas for the domestic stock, international stock, and sector category are statistically significant ranging from 2.11 % (domestic stock) to 5.27 % (sector).
Overall, our results strongly suggest that the SmartMoney Fund Screen is able to identify funds that have superior risk-adjusted performance in the year before publication although the screen does not explicitly use any risk adjustment technique as part of the actual fund screen. In the next section, we examine whether this superior performance persists in the post publication period.
4.2 Post publication performance
In Table 3, we report the average performance measures during the post publication period, and we also include the change in the measure from the pre to the post publication period. As we did in the previous section, we report overall results and results partitioned by the funds’ Morningstar star ratings.
Our overall results indicate that there is a significant decline in fund performance during the post publication period. The annual excess return relative to the fund’s benchmark declines −5.38 %. The average Sharpe ratio is −0.14 lower. The average alpha from the ten factor model falls −2.66 %. All of these changes are statistically significant at the one percent level, and the results are consistent for the subsample of funds. At the individual fund level, we find that in the post period only 44.2 % of funds have a positive alpha and only 47.2 % generate a positive excess return. Although 65.9 % of funds still have a positive Sharpe ratio, the percentage is lower than the 72.7 % during the pre publication period. Over 60 % of funds saw their alpha, Sharpe, and excess return estimates decline from the pre publication period.
When we look at the performance changes based on the Morningstar star ratings, we find the decline in performance is consistent across all Morningstar categories. All categories show a decline in excess returns with the changes ranging from −2.16 to −8.44 %. All are statistically significant. The Sharpe ratio falls for four of the five categories with only the five star category showing a slight increase in the Sharpe ratio of 0.02. The decline in alpha is also consistent across all five categories. Only the three star SmartMoney funds maintain a positive alpha, but their post publication performance drops by 35 basis points. The declines for the other categories are larger in magnitude ranging from −2.30 to −6.57 %. Again, the results for the subsample are consistent with the results for the overall sample.
Although not reported, we also find that the performance decline is consistent across fund type. Excess return relative to the benchmark falls for all fund types except for the fixed income group which makes up the smallest portion of the fund sample. The Sharpe ratio falls for all five categories ranging from −0.01 (hybrid) to −0.31 (sector). There are similar declines in alpha across the five groups ranging from −0.14 % (hybrid) to −3.52 % (international stock).
These results suggest that the strong pre publication performance of the SmartMoney sample does not persist during the year after publication. This result raises the question of the value the SmartMoney Fund Screen provides to investors. Given that Morey and Gottesman (2006) and Del Guercio and Tkac (2008) provide evidence that funds which have Morningstar’s highest rating of five stars generate superior future performance relative to lower rated funds, the question is whether investors would have been better off using Morningstar’s rating system to select funds rather than using the SmartMoney lists.
As a first step in analyzing this question, we look at the relationship between the post publication performance of the five star SmartMoney funds and the remaining funds in the sample. If the Morningstar star rating system has strong predictive abilities, we should see evidence of superior performance among these funds relative to the other funds in our sample. Because the funds that are not rated and funds with a two star rating compose a small portion of our sample, we focus on a comparison of the five star funds to the three and four star funds which make up the majority of the sample.
The evidence from Table 3 does not indicate that the five star funds are superior performers. Compared to the three and four star funds, we find that the five star funds have a negative excess return during the post publication period (−0.59 %) while the excess returns for the three star (2.32 %) and four star funds (0.50 %) is positive. The decline in the excess return from the pre publication period to the post publication period is −8.23 % which is much greater than that of the three and four star funds (−2.16 and −4.00 % respectively). Similar poorer performance holds for the ten factor model alpha. The overall alpha for the five star funds is lower during the post publication period (−0.34 % compared to 1.78 % for the three star funds and −0.14 % for the four star funds) and the decline in alpha from the pre to the post publication period is also greater (−3.87 % compared to −0.35 and −2.30 % respectively). Only when measuring performance based on the Sharpe ratio is there evidence of the five star funds performing better as their ratio of 0.55 is greater than the three and four star funds.
Thus the preliminary evidence suggests that within the SmartMoney sample, the Morningstar star rating does not provide any value, and investors would have been marginally better off by ignoring the five star SmartMoney funds that were listed. In the next section, we conduct a more rigorous test of the relative value of the SmartMoney fund screen by comparing the post publication performance of the SmartMoney sample to the performance of an alternative sample of funds composed only of five star Morningstar funds.
5 Post publication performance: SmartMoney vs. five star Morningstar funds
Our goal is to compare the post publication performance of the SmartMoney funds to a comparable sample of the highest rated Morningstar funds. Based on the work by Morey and Gottesman (2006), we would expect the Morningstar funds to outperform the lower rated Morningstar funds selected by SmartMoney. Our test will provide a measure of the value provided to investors by the SmartMoney funds.
We use the following process to identify a matching Morningstar fund for each SmartMoney fund in our sample. Given that the SmartMoney list is designed for retail investors, we eliminate all institutional funds from the population of Morningstar funds. For each SmartMoney fund, the pool of potential Morningstar funds includes all non institutional funds included on the quarterly Morningstar Principia Pro Mutual Fund CD previous to the publication date of the SmartMoney fund.Footnote 9 We obtain each SmartMoney fund’s prospectus objective and expense ratio as reported by Morningstar. We then begin the matching procedure by sorting the SmartMoney funds by expense ratio and finding a match for the fund with the lowest expenses. The selected Morningstar fund meets the following criteria: 1) it has a five star Morningstar star rating, 2) it has the same prospectus objective as the SmartMoney fund, 3) the fund has the closest expense ratio to the SmartMoney fund in absolute terms, and 4) during the quarter, neither that Morningstar fund or a different share class of the same Morningstar fund has been previously selected as a matching Morningstar fund for another SmartMoney fund.
The use of the fund’s prospectus objective is designed to create a sample such that the results are not driven by funds that have a significant difference in investment objective and style from the SmartMoney funds. Our use of the expense ratio is to ensure differences in performance are not driven by the costs of the funds. Finally, our limit on the use of different share classes of the same fund ensures that our results are not driven by a specific fund that is used repeatedly during a given quarter.
We handle special situations that occur in the following manner. If there are no five star funds available that meet the criteria, we then select a four star Morningstar fund which meets the criteria. This occurs 96 times (6.5 %) across our sample. If there are multiple funds that meet criteria one and two above and have the same expense ratio, we then select the fund that is closest in size, as measured by total net assets, to the SmartMoney fund. If the absolute difference in expense ratio is the same for two potential matching funds, we select the fund with the lowest expense ratio. These last two situations occur less than one percent of the time in our fund sample.
We obtain daily fund returns for the matching funds from CRSP. We are unable to obtain publicly reported daily net asset values or returns for fourteen of the matching Morningstar funds. Since these were the actual funds chosen given the criteria, we do not replace them with alternative funds but choose to exclude them from the analysis.Footnote 10 Thus, the sample of funds being analyzed for this test differs slightly from the full SmartMoney sample examined in the earlier sections.
Table 4 compares the descriptive statistics of the SmartMoney funds and the matched sample of Morningstar funds. Although not reported, we also compare the characteristics of the subsample of SmartMoney funds and we find them similar. Given that the results of the full sample and the subsample have been similar throughout the paper, for the sake of brevity, we focus only on the full sample for the remainder of the paper.
We find that our matching procedure provides a Morningstar sample that has a slightly higher expense ratio (1.20 vs. 1.13 %), is smaller ($1737 million vs. $3342.38 million), has lower turnover (72.51 vs 82.21 %) and is younger (9.85 vs. 16.76 years). The same relationships hold if you examine the median rather than mean characteristics. An analysis by SmartMoney Morningstar star rating indicates that the same relationships also hold for the three, four, and five star funds with the only exception being that the four star SmartMoney funds have a lower turnover ratio than their matched funds. In unreported results, we also examine the 96 four star Morningstar funds that are included in the matched sample, and we find that their characteristics fit the profile discussed above. After we complete our performance analysis, we will return to these characteristics and test whether any of the differences drive our performance results.
Using the matched sample, we calculate the benchmark excess return, Sharpe ratio, and ten factor model alpha for each of the funds and the difference in each performance measure between each SmartMoney fund and its matched fund. We report the results of a matched pairs t-test to determine if any of the differences are statistically significant.
Table 5 contains results for the entire sample and by the SmartMoney funds’ Morningstar star rating. We find across the entire sample that the SmartMoney funds outperform the matched sample under all three performance measures. The SmartMoney funds generate a positive average excess return that is 0.78 % greater than the negative excess return generated by the Morningstar funds. This superior performance is statistically significant at the five percent level. The SmartMoney funds’ Sharpe ratio and ten factor model alphas are also greater than their counterparts although the differences of 0.002 for the Sharpe ratio and 0.15 % for alpha are small in magnitude and not statistically significant.
By looking at each of the SmartMoney Morningstar ratings groups, we find that the superior performance of the overall sample is primarily driven by superior performance of the three star SmartMoney funds and to a lesser extent by the four star funds. Based on the excess return measure, the three star funds outperform the matched funds by a statistically significant 1.69 %. When we look at performance at the individual fund level, we find that 56.1 % of the three star funds have a greater excess return. Results are similar for performance based on alpha. The three star funds have an average alpha of 1.79 % and they outperform their counterparts by an average of 1.94 % which is statistically significant at the five percent level. We find that 144 funds (55.0 %) have a positive alpha and 148 funds (56.5 %) have an alpha greater than their matched fund. The greater performance of the three star funds also holds for the Sharpe ratio as their average ratio is 0.03 larger than the matched sample. Although this difference is not statistically significant, the difference is the largest positive difference among the five Morningstar star rating groups in our sample.
Results are similar for the four star group but much weaker in magnitude. The four star funds generated a positive excess return of 0.53 % which is 1.04 % greater than the negative excess return (−0.50 %) generated by the matched sample. As in the case of the three star funds, this difference is statistically significant at five percent. The Sharpe ratio of the four star funds is only slightly greater than that of the matched sample (0.48 vs. 0.47). The average alpha of the four star funds is 0.28 % greater than the alpha of the matched sample, but we find that this difference is not statistically significant, and we note both the Smartmoney funds and the matched Morningstar funds generate negative alphas on average.
These results provide evidence that the SmartMoney recommendations are valuable for investors relative to comparable five star Morningstar funds.Footnote 11 By matching funds on prospectus objective, the above test directly controls for differences in performance related to fund type. An additional factor that may be driving these results is systematic differences in the characteristics of the matched funds versus the SmartMoney funds. As we noted earlier in this section, on average, the matched Morningstar funds have a slightly higher expense ratio, are smaller, have lower turnover, and are younger. We test to see if differences between the two samples can explain the differential performance we see between the three and four star funds and their corresponding Morningstar funds.
For each of our performance measures, we calculate the difference between the SmartMoney fund and its matched fund. The difference in the performance measure serves as our dependent variable. The independent variables are the differences between the expense ratio, net assets, fund age, and turnover ratio. We estimate the regression separately for the three star funds and the four stars fund. If the independent variables are statistically significant, this would provide evidence that systematic differences in the funds’ observable characteristics are driving the average performance differences.
We report the results of the regressions in Table 6. The regressions provide no evidence that our matching procedure is driving the performance differences. The regressions have virtually no explanatory power as the adjusted r-squareds range from 0.0007 to 0.009. None of the differences in fund characteristics is statistically significant. We find that the value of the intercept terms in each regression closely relates to the value of the difference in the performance measures reported in Table 5. For the three star funds, we find that the intercept term for the excess return regression is 1.65 % which is extremely close to the 1.69 % difference in performance found in Table 5. The same holds for the excess return regression for the four star funds (1.10 % compared to 1.04 % in Table 5) and the alpha regression for the three star funds (2.06 % compared to 1.94 % in Table 5).
The results of our test provide strong evidence that the three and four star Morningstar funds listed by SmartMoney are a better investment value than a comparable five star Morningstar fund with the same prospectus objective and corresponding expense ratio.
6 Robustness: the 2008 financial crisis
Our sample period includes the 2008 financial crisis which brought unprecedented turmoil to the world’s economy and financial markets. We want to examine whether our main results are consistent across our entire sample or whether the results are heavily influenced by the performance of our fund sample during the crisis. We repeat the tests run in the previous sections of this study and separately analyze all funds published in 2008 and the remainder of the sample. There were 281 funds published during 2008 which represents 18.9 % of the entire sample of 1485 funds published from September 2004 through July 2009. 41.8 % of the funds had a five star rating from Morningstar, 40.9 % had a four star rating, while 12.1 % had a three star rating. Footnote 12
In Section 4.1, we presented evidence that the SmartMoney funds have strong performance during the year before publication. In Table 7, we report results for the pre publication performance of the 2008 funds and the remainder of the sample. We find strong evidence that the pre publication results are not strictly a function of the 2008 funds. The pre publication excess return (7.62 %) and the pre publication alpha (5.47 %) of the 2008 funds are much greater in magnitude than for the remainder of the sample (5.31 % excess return and 1.95 % alpha). But the excess return and alpha estimates are statistically significant for both groups of funds. The Sharpe ratio for the 2008 funds is negative (−0.29) which is contrary to the finding that the remainder of the sample generates a significant positive ratio. But this result for the 2008 funds is not surprising given that virtually all assets classes had negative returns during the crisis.
In Section 4.2, we presented evidence that the pre publication performance of the SmartMoney funds does not persist and performance on average declines significantly. The results in Table 7 confirm this result for both samples. The excess returns decline by −7.02 % for the 2008 sample and by −5.00 % for the remainder of the sample. Both of these declines are statistically significant. Performance as measured by alpha falls by −5.58 % for the 2008 funds and by −1.98 % for the remainder of the sample. Again, both declines are statistically significant. Both samples also show a drop in the Sharpe ratio. The decline for the 2008 sample is not significant though the decline is statistically significant for the remainder of the sample.
Our final result presented in Section 5 showed that the three and four star Morningstar funds published by SmartMoney are a better investment value than a comparable five star Morningstar fund which has the same prospectus objective and a similar expense ratio. Table 8 reports results of the tests where we compare the performance of the three and four star funds during the 2008 period to the performance of the three and four star funds for the remainder of the sample. The results again confirm that the performance of funds during 2008 is not the main driver of our results. Focusing on the three star funds, we find that the difference in excess return between the SmartMoney funds and the matched Morningstar funds is 6.31 % for the 2008 funds and 1.01 % for the remainder of the sample. The difference in alpha across samples is 6.00 % for the 2008 funds and 1.33 % for the remaining funds. Although in both cases the difference is much greater in magnitude for the 2008 funds, we find all differences are statistically significant. Results are similar for the four star funds although the differences in performance are not statistically significant. The one exception is that the difference in excess returns is significant for the sample which excludes the 2008 funds reinforcing the point that the 2008 sample is not the main driver of the results presented in the earlier sections.
Overall, there is no evidence that our main findings are a result of the performance of our fund sample during the financial crisis.
7 Conclusion
For a 5 year period, the Wall Street Journal published a list of mutual funds in a column entitled “SmartMoney Fund Screen.” It is likely many investors viewed this list as a list of recommended mutual funds. In this study, we examine the value to investors of the mutual fund lists provided in the weekly column. The value of these columns to investors is of particular interest given that research has established that media coverage heavily influences investors’ mutual fund choices. In addition, the column serves as an alternative to Morningstar’s star rating system which is extremely popular among investors.
We find that only 35 % of SmartMoney funds have a Morningstar’s top five star rating, We then examine the value of the SmartMoney fund lists both in absolute terms and in terms of their value as an alternative to Morningstar. We find that in the year before publication, the SmartMoney funds have strong positive performance that is statistically significant across most of our performance measures regardless of the funds’ Morningstar rating. Post publication, performance declines, with only the SmartMoney funds which have a three star rating from Morningstar demonstrating significant positive performance.
Given that investors could choose a five star Morningstar fund instead of the SmartMoney fund, we create a matched sample of five star Morningstar funds that have the same prospectus objective and expense ratio as each SmartMoney fund. We then compare the post publication performance of these two fund samples. We find that the three and four star SmartMoney funds outperform their corresponding funds based on each of our performance measures indicating some value provided by SmartMoney. Our robustness tests indicate that these results are not driven by the methodology used to match the fund nor are the results heavily influenced by the 2008 financial crisis.
Although the Morningstar star rating system is extremely influential and popular among retail investors, our results highlight weaknesses in the performance of the rating system and suggests that despite its own weakness, the SmartMoney fund screen is an additional tool that mutual fund investors may want to consider when making their mutual fund decisions.
Notes
More details concerning the Morningstar star rating procedure can be found at www.morningstar.com or in Morey and Gottesman (2006).
The dates of these articles are July 12, 2005 (Underperformers), May 30, 2006 (Index Funds), November 11, 2006 (Index Funds), November 28, 2006 (Lackluster Funds), April 10, 2007 (Funds That Stumble), November 27, 2007 (Poor Performers), June 10, 2008 (Funds That Stumble), and July 1, 2008 (Losers).
In order to be comparable to the SmartMoney sample, the Morningstar universe is determined as follows. We start with all funds available from the December 2004, 2005, 2005, 2007, and 2008 Morningstar Principia Pro Cds. Since SmartMoney funds are designed for retail investors, we eliminate all funds whose share class is institutional. We also excluded funds with the following prospectus objectives as no fund screen in our sample analyzed these types of funds: corporate bond high yield, government bond adjustable rate mortgage, government bond mortgage, multi sector bond, municipal bond national, municipal bond single state, and money market. We calculated the descriptive statistics based on the remaining funds.
An alternative to these factors are the factors used by Hou, Karolyi, and Kho (2011). However, these factors are not available on a daily basis as is required for our analysis in the next section.
One potential weakness of the model is that we do not include a momentum indices for the domestic stock, international stock, or fixed income portions of the model. However, given that neither SmartMoney nor Morningstar explicitly account for momentum in the screens or ratings combined with the lack of available daily momentum indices for international stocks and fixed income securities, we do not include any momentum factors in the model.
Our use of a one year pre and post publication window is motivated by characteristics of the data sample. On average, SmartMoney runs specific screens every 10 to 12 months, thus implicating indicating the screen effective horizon. Our one year pre and post publication window is also identical to that used by Jain and Wu (2000).
We include the actual publication in the post publication period because we want to capture any changes made by the manager in response to the listing in the post publication period. For example, a manager who is aware that the fund appears in the WSJ on January 4 may make trades on the 4th and these trades would be reflected in changes in the net asset value (and thus the return) of the fund on the 4th.
Results are available upon request.
The quarterly Morningstar Cds provide data as of March 31, June 30, September 30, and December 31. Thus, the pool of available funds for a fund published a column dated November 1 would come from the September 30 Cd of that year.
An analysis of the alternative funds indicates that their inclusion would not change the reported results.
We note that this result is consistent with the result found by Blake and Morey (2000) which led to a change in the Moringstar ratings system in 2002. However, our sample begins in 2004 and this result is inconsistent with what would be expected based on Morey and Gottesman (2006). But, Morey and Gottesman only analyze data from 2002 to 2005 while our sample covers 2004 to 2009. Thus our period only has a small overlap with theirs.
There are several potential definitions of the financial crisis period. We choose the entire 2008 calendar year rather than a shorter time period as it provides us with a sufficient number of funds to analyze performance during the crisis period.
References
Barber B, Odean T (2008) All that glitters: the effect of attention and news on the buying behavior of individual and institutional investors. Rev Financ Stud 21:785–818
Blake C, Morey M (2000) Morningstar ratings and mutual fund performance. J Financ Quant Anal 35:451–483
Comer G, Rodriguez Javier (2012) International Mutual Funds: MSCI Benchmarks and Portfolio Evaluation, working paper
Comer G, Larrymore N, Rodriguez J (2009) Controlling for fixed income exposure in portfolio evaluation: evidence from hybrid mutual funds. Rev Financ Stud 22:481–507
Cronqvist H (2006) Advertising and Portfolio Choice, working paper
DelGuercio D, Tkac P (2008) Star power: the effect of morningstar ratings on mutual fund flow. J Financ Quant Anal 43:907–936
Fama E, French K (1993) Common risk factors in the returns on stocks and bonds. J Financ Econ 33:3–56
Gallaher S, Kaniel R, and Starks L (2009) Advertising and Mutual Funds: From Families to Individual Funds, working paper
Jain P, Wu J (2000) Truth in mutual fund advertising: evidence on future performance and fund flows. J Financ 55:937–958
Morey M, Gottesman A (2006) Morningstar mutual fund ratings redux. J Invest Consul 8:25–37
Tetlock P (2007) Giving content to investor sentiment: the role of media in the stock market. J Financ 62:1139–1168
Acknowledgments
We would like to thank Norris Larrymore for his work on an earlier version of this paper. We thank Naielia Allen, Jennifer Lopez, Laura McCann, Herminio Romero, and Arthur Wharton for their excellent research assistance. We also thank Jeff Busse, Debra Glassman, Martin Gruber, Prem Jain, Phyllis Keys, Jennifer Koski, Matt Morey, and James Rimbey for their suggestions and comments on earlier versions of this study. In addition, we appreciate the feedback of workshop participants at Georgetown University, the Rising Stars Conference at Rensselaer Polytechnic Institute, Rutgers University Camden, Saint John’s University, University of Arkansas, University of Montevideo, University of Puerto Rico, and University of Washington. Rodriguez would like to thank the University of Washington for their hospitality during the course of this research. Comer would like to acknowledge the financial support provided by the McDonough School of Business Dean’s Research Fellowship.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Comer, G., Rodriguez, J. An empirical analysis of the Wall Street Journal’s SmartMoney fund screen. J Econ Finan 40, 380–401 (2016). https://doi.org/10.1007/s12197-014-9314-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12197-014-9314-2