Introduction

Housing asset returns are of highly growing interest in studies on macroeconomics, asset pricing, and housing market dynamics. The recent U.S. housing boom that started around 1999, and the subsequent bust in the mid 2000s, have attracted numerous studies which attempt to explain key drivers underlying the recent dramatic change in housing prices.

This paper investigates the effect of the important driver of the recent housing boom and bust, people’s expectations, on the U.S. housing asset returns. In particular, this paper extends the volatility feedback model proposed in Kim et al. (2004) (KMN), Turner et al. (1989), and Campbell and Hentschel (1992) to study the relationship between volatility and return in the housing market during the last 50 years, assuming different information sets. Noticeably, this relationship per se is not the whole story in this study. Its novelty lies in insightful investigations into the recent U.S. housing boom and bust in the post-1999 period, applying the associations between housing volatility and asset returns, including realized and expected returns. Motivated by the findings of the extended model, this paper reveals underlying factors which can explain the recent dynamics of housing prices. It investigates the “bubble-like” housing boom-and-bust cycle from a fresh perspective which is not addressed in existing studies.

KMN extend the volatility feedback model by assuming that volatility of the U.S. stock market follows a Markov switching process. In the framework, exogenous volatility feedback occurs when people modify their future expected asset returns as they observe new information about volatility of the asset market during the current period. Particularly, KMN address three main reasons for which a Markov switching (MS) specification is more appropriate to be used to investigate asset return dynamics than alternative frameworks, such as models of AutoRegressive Integrated Moving Average (ARIMA) or AutoRe-gressive Conditional Heteroskedasticity (ARCH). First, the dynamics in Markov-switching models can persist for a longer time than those in ARIMA or ARCH models. As monthly or quarterly data are applied, the dynamics of asset returns can remain in the MS model, not disappear gradually as in ARIMA or ARCH models. Second, a MS specification spotlights the volatility feedback effect, and rules out other possible kinds of interactions between volatility and asset returns (e.g., a leverage effect) while ARCH models cannot provide satisfactory results. Third, taking advantage of the filter of Hamilton (1989), the MS specification addresses the relationship between the asset volatility and return in a less complicated fashion than ARCH-type specifications.

Motivated by KMN, this paper considers two alternative assumptions about information availability to examine the volatility feedback effect: partial revelation and full revelation. The former assumes that people can only observe past returns, while the latter assumes that people are able to identify the previous volatility regime as the trading period starts. The observable realized asset return is composed of people’s expected return, the volatility feedback effect, and the shock (news) to the asset market. Thus, people’s updated expectations have an influence on asset returns.

The model has three characteristics, which facilitate its application to the housing market. First, it emphasizes that volatility feedback is able to capture comprehensive effects of housing market volatility on all future discounted expected housing asset returns. While some related studies only incorporate partial effects of Markov-switching housing market volatility on the contemporaneous expected return, the model considered in this paper does not have this restriction. Thus, it allows a more comprehensive examination of the relationship between housing asset volatility and returns than the existing literature on housing markets. Typically, investors hold housing assets for a longer time period than other varieties of assets. As a result, the framework applied in this study enhances the observations into housing market because it incorporates volatility feedback into the analysis.

Second, high-volatility regimes of housing asset returns delivered by the proposed model highly coincide with the U.S. housing cycles. Noticeably, the 2001 recession is hardly captured in a satisfactory way in the existing empirical studies because it has different characteristics compared to other NBER-dated recessions. It resulted from a sharp cut in business investment, especially in information technology, while other recessions originated from the decline of consumption. The model considered in this paper, which incorporates people’s expectations, is able to capture the 2001 recession due to the volatility feedback effect. Therefore, it proposes an insightful way to characterize the business cycle through cyclical movements in housing prices.

Third, KMN distinguish a one-time permanent structural break from a temporary but persistent Markov-switching component in the U.S. stock market. However, KMN don’t estimate the permanent breakpoint, and follow Campbell and Hentschel (1992) by choosing year 1951 as the structural break. This study adopts the two alternative breakpoints, 1984Q1 or 1999Q1, in order to compare their influences on the role of volatility feedback. Further, it applies Hansen’s (2001) application of Bai and Perron (1998) sequential estimation of breakpoints, and the method supports the two chosen permanent structural breakpoints in the U.S. housing asset returns during the period analyzed. Hansen’s (2001) method allows for the identification of structural changes at unknown dates in the autoregressive process of time series. Under two different criteria, it identifies two significant structural breaks in the U.S. housing market: 1984Q1 and 1999Q1. The breakpoint, 1984Q1, coincides with the break towards stabilization found in the US GDP and many other macroeconomic variables as documented in a vast literature. On the other hand, the breakpoint, 1999Q1, represents a permanent structural change in the U.S. housing market which approximates the beginning of the recent housing boom.

This study discusses four scenarios because four different assumptions regarding information available to agents are adopted to examine the relationship between housing volatility and expected asset returns. Particularly, the result comparisons between the models without taking volatility feedback into account and those incorporating the role of people’s expectations facilitate our investigations into the most recent housing boom-and-bust cycle. The results indicate a strong positive relationship between the U.S. housing market volatility and expected housing asset returns. Interestingly, the volatility feedback exerts the most important effect on the housing market in the post-1999 period among analyzed sub-periods. Further, the high-volatility regime in the proposed model with volatility feedback succeeds in matching all NBER-dated recessions, with the exception of the 1973–1975 recession. The findings imply a strong linkage between people’s expectations and the great uncertainty around the 2001 recession, and support the assumption that people’s expectations about high housing asset returns have played an important role in the U.S. housing boom-and-bust cycle in the early 2000s.

Noticeably, this study provides some fresh implications for the recent U.S housing boom during 2001–2005 through relationships between housing volatility and asset returns, including realized and expected returns. The findings are indicative of a remarkable housing boom in the post-1999 period although the presence of a housing bubble is an on-going debate. While KMN solely focus on the relationship between stock market volatility and the equity premium, this paper further delivers fresh insights into the dynamics of returns to the asset other than the common stocks—the housing asset.

This paper proceeds as follows. “Literature Review” reviews the literature which inspires this study. “Models” introduces models with different assumptions regarding information availability in the formation of people’s expectations. “Breakpoint Determination” discusses the methods used to determine the permanent breakpoint. “Empirical Results” shows the empirical results, including the roles of volatility feedback, the connection between high-housing-volatility regimes and business cycles, and the implications for the recent housing boom and bust through the investigation into the relationship between housing volatility and asset returns. The final section concludes.

Literature Review

There is a vast literature on different aspects of the role of the housing asset in the economy. Because of the recent dramatic housing dynamics, the comparison between housing and stock assets is worth our more analyses. Motivated by the strain of the literature which regards the housing market as another source of asset returns other than the stock market,Footnote 1 this study extends KMN framework to the analysis of housing asset returns. Furthermore, Campbell et al. (2009) adopt a dynamic Gordon growth framework to provide support for the similarity of dynamic patterns between the housing markets and financial markets. Thus, this study contributes to the literature which provides a parallel between housing and financial assets.

Moreover, according to the recent literature, people’s expectations of the housing price play a key role in the U.S. housing market. For example, Glaeser and Gyourko(2007) and Glaeser et al. (2008)Footnote 2 analyze housing markets from the supply side, while Davis and Palumbo (2008), Piazzesi and Schneider (2009) and Sommervoll et al. (2010)Footnote 3 from the demand side.

This paper is partially motivated by the work of KMN to investigate the role of volatility feedback in explaining the U.S. housing market dynamics. KMN assume that volatility of the equity premium follows a two-state Markov-switching process in the prewar and postwar periods, assuming different assumptions regarding available information which people can utilize to revise their expectations about the equity premium. They primary propose that there is a positive relationship between stock market volatility and the equity premium if volatility feedback affects the current stock prices negatively. Regarding methodology, this paper is also motivated by the literature that assumes Markov-switching housing market volatility, such as Roche (2001) and Ceron and Suarez (2006)Footnote 4 which both suggest that housing market volatility is negatively associated with housing price growths. Importantly, the negative relationship between housing market volatility and realized housing asset returns proposed in this paper are highly consistent with the housing market dynamics suggested in the existing empirical studies.

Besides, numerous studies are devoted to capturing the 2001 recession owing to its different causes from other recessions. For instance, Kim et al. (2005a, b) introduce a “bounce-back effect” in the business cycle, and Kim et al. (2007a, b) decompose the business cycle into permanent and transitory components. Motivated by these efforts, the study attempts to provide a new way to identify the 2001 recession.

With respect to the structural break, the related literature which discusses greater stabilization in the U.S. since the mid-1980s was initiated with the work McConnell and Perez-Quiros (2000).Footnote 5 Interestingly, 1984Q1 also represents shifts in other macroeconomic variables, such as in the inflation-output relationship. Concerning breakpoints in the housing sector, Vargas-Silva (2008) suggests that housing starts are less volatile in all of the four U.S. regional housing markets after 1980 because of the deregulation in the housing financial system. Moreover, Kim et al. (2007a, b) investigate the dynamic behavior of monthly returns to Real Estate Investment Trusts (REITs), equity markets, and related macroeconomic variables during 1971 to 2004. They find that the real estate market shows a stronger causal relationship with other variables in the post-1980, which is regarded as a breakpoint, and conclude that REITs play a more important exogenous role in the U.S. economy after 1980.

However, the exact breakpoint of housing markets is hardly discussed in the existing related studies, and the methodological analysis is very thin. The recent literature only approximates the onset of the recent bubble-like housing boom-and-bust cycle. For instance, Wheaton and Nechayev (2008) and Agnello and Schuknecht (2011) state that the recent remarkable run-up of the housing price occurs during 1998–2005. Besides, Shiller (2006) mentions that the US mortgages of the second-home purchase have doubled since 1999, and Shiller (2008) points out that the US housing boom begins in 1998 and starts to spillover since 1999. Goodman and Thibodeau (2008) emphasize that homeownership increases by more than 2% from 1999 to 2005, and Lai and Van Order (2010) argue that the momentum behavior of the US housing price increases after 1999. Noticeably, none of these studies propose a methodology to specify a structural break of the housing market although they implicitly suggest that the post-1999 period displays a quite different pattern of the housing market from the pre-1999 period.

Importantly, while KMN limit its main goal to examine the relationship between stock market volatility and the equity premium by introducing volatility feedback, this study provides insightful investigations into the US housing market by emphasizing the role of revised people’s expectations (volatility feedback) in driving the recent housing boom-and-bust cycle and matching high-housing-volatility states with business cycles.

Models

This section introduces the underlying framework used in this paper to investigate the U.S. housing market with Markov-switching volatility.

The framework in Campbell and Shiller (1988) allows for investigations into the impact of people’s changing expectations on the housing asset return. Let the one-period housing asset return be represented as the log-linear approximation:

$$ {r_{{t + 1}}} = \log \left( {{X_{{t + 1}}} + {D_{{t + 1}}}} \right) - \log \left( {{X_t}} \right) $$
(1)

where X t is the housing price, and D t+1 represents the housing rent. As Meese and Wallace (1990) suggest, the housing rent can be regarded as the dividend on the housing asset. Taking the first-order Taylor expansion of Eq. 1, we obtain:

$$ {r_{{t + 1}}} = \kappa + \rho {x_{{t + 1}}} + \left( {1 - \rho } \right){d_{{t + 1}}} - {x_t} $$
(2)

where lower-case letters denote the log of the series. At time t + 1, the housing asset return r t+1 is the sum of a weighted (ρ and 1-ρ) average of log housing price x t+1 (the percentage change in the housing price) and log housing rent d t+1 (the percentage change in the housing rent). κ is a nonlinear function of ρ, which is defined as the average ratio of the quarterly housing price to the sum of the quarterly housing price and the quarterly housing rent. ρ is very close to one because the percentage change in the housing price contributes to housing asset returns much more than the change in the housing rent. Then ex-ante version of Eq. 2 can be solved forward to obtain the percentage change in the housing price at time t (x t ) in Eq. 3:

$$ {x_t} = \frac{\kappa }{{(1 - \rho )}} + (1 - \rho ){E_t}\left( {\mathop{\sum }\limits_{{{\text{j}} = 0}}^{\infty } \left. {{\rho^j}{d_{{t + 1 + j}}}} \right|{\Psi_t}} \right) - {E_t}\left( {\mathop{\sum }\limits_{{{\text{j}} = 0}}^{\infty } \left. {{\rho^j}{r_{{t + 1 + j}}}} \right|{\Psi_t}} \right) $$
(3)

Applying KMN which extend the framework of Campbell and Hentschel (1992) by decomposing the asset returns into three components—people’s expected returns, a volatility feedback effect, and shocks (news) to the asset market, the proposed model which is to investigate the housing asset dynamics is represented as follows:

$$ E\left[ {\left. {{r_{{t + j}}}} \right|{\Psi_t}} \right] = {\mu_0} + {\mu_1}Pr\left[ {\left. {{S_{{t + j}}} = {1}} \right|{\Psi_t}} \right] = {\mu_0} + {\mu_1}Pr\left[ {{S_t} = {1}} \right] + {\mu_1}{\lambda^j}\left( {Pr\left[ {\left. {{S_t} = {1}} \right|{\Psi_t}} \right] - Pr\left[ {{S_t} = {1}} \right]} \right) $$
(4)
$$ \sigma_{{{S_t}}}^2 = \sigma_0^2\left( {1 - {S_t}} \right) + \sigma_1^2{S_t}, \sigma_0^2 < \sigma_1^2; {S_t} = 0,1 $$
(5)
$$ {r_t} = E\left[ {\left. {{r_t}} \right|{\Psi_{{t - 1}}}} \right] + {f_t} + {\varepsilon_t},\,{\varepsilon_t} \sim N\left( {0,\sigma_{{{S_t}}}^2} \right) $$
(6)

where r t is a realized housing asset return at time t, and it is assumed to consist of three “unobservable” components as shown in Eq. 6. First, \( E\left[ {\left. {{r_{{t + j}}}} \right|{\Psi_t}} \right] \) is the expected housing asset return at time t + j under available information set at the end of time t (Ψ t ). Thus, \( E\left[ {\left. {{r_t}} \right|{\Psi_{{t - 1}}}} \right] \) is the expected housing asset return at time t under available information set at the end of time t-1. Second, f t represents volatility feedback. Third, ε t represents the shock to the housing market. \( \sigma_{{St}}^2 \) is the variance of ε t , which is new information (available during time t) about future housing rent.

Notice that the three components on the right-hand side of Eq. 6 are not observed, what we observe is only the realized housing asset return (r t ) on the left-hand side. μ 0 is the mean housing asset return in the low-volatility regime, and μ 1 represents the volatility-state-dependent housing asset return. S t  = 1 is the high-volatility regime, and S t  = 0 is the low-volatility regime. Both Markov-switching regimes (S t ) are unobserved. Thus, (μ 0   + μ 1 ) is the mean housing asset return in the high-volatility regime as perfect expectations (i.e., \( Pr\left[ {{S_t} = {1}} \right] = {1} \)) occur.

The transition probabilities that govern the evolution of S t are \( Pr\left[ {\left. {{S_t} = 1} \right|{S_{{t - 1}}} = 1} \right] = p \), \( Pr\left[ {\left. {{S_t} = 0} \right|{S_{{t - 1}}} = 0} \right] = q \), and \( \lambda = p + q - {1} > 0 \) reflects the persistence of volatility regimes because λ is the autoregressive coefficient of volatility state S t which is assumed to have a AR(1) strictly stationary stochastic process based on Hamilton (1989).

Volatility feedback, f t , allows revision of future expected housing asset returns due to new information obtained during time t. Housing investors observe the new available information about volatility of housing market through past returns (partial revelation) or through volatility regimes (full revelation)during time t, and update their expectations about future housing asset returns. Thus, by definition, it is represented as the difference between the expected sums of returns under two different information sets:

$$ {f_t} = E\left[ {\sum\nolimits_{{j = 0}}^{\infty } {\left. {{\rho^j}{r_{{t + j}}}} \right|{\Psi_{{t - 1}}}} } \right] - E\left[ {\sum\nolimits_{{j = 0}}^{\infty } {\left. {{\rho^j}{r_{{t + j}}}} \right|\Psi_t^{\prime }} } \right] $$

The information set \( \Psi_t^{\prime } = {\Psi_t} - {r_t} > {\Psi_{{t - 1}}} \). Ψ t is the information set at the end of time t and it contains the realized housing asset return at time t(r t ). In other words, \( \Psi_t^{\prime } \) contains all information of time period t, aside from the realized housing asset return at the end of time t(r t ).

If no volatility feedback is considered, we consider the partial revelation case \( \left( {{\Psi_{{t - 1}}} = \Psi_t^{\prime } = \left\{ {{r_{{t - 1}}},{r_{{t - 2}}}, \ldots } \right\}} \right) \) in which people are only able to observe past housing asset returns. Another case considered is the full revelation case \( \left( {{\Psi_{{t - 1}}} = \Psi_t^{\prime } = \left\{ {{S_t}} \right\}} \right) \) that people are able to recognize the volatility regime of the housing market. The two information sets, Ψ t-1 and \( \Psi_t^{\prime } \), are the same as volatility feedback is not considered. Otherwise, if there is volatility feedback effect, information sets at the beginning and the end of time t are different. \( {\Psi_{{t - 1}}} = \left\{ {{r_{{t - 1}}},{r_{{t - 2}}}, \ldots } \right\} \), \( \Psi_t^{\prime } = \left\{ {{S_t}} \right\} \) are information assumptions for the partial revelation case, and \( {\Psi_{{t - 1}}} = \left\{ {{S_{{t - 1}}}} \right\} \), \( \Psi_t^{\prime } = \left\{ {{S_t}} \right\} \) are those for the full-revelation case.

The news about the housing market ε t can be represented as:

$$ {\varepsilon_t} = E\left[ {\sum\nolimits_{{j = 0}}^{\infty } {\left. {{\rho^j}\Delta {d_{{t + j}}}} \right|\Psi_t^{\prime }} } \right] - E\left[ {\sum\nolimits_{{j = 0}}^{\infty } {\left. {{\rho^j}\Delta {d_{{t + j}}}} \right|{\Psi_{{t - 1}}}} } \right] $$

where the change in the housing rent (Δd t ) is regarded as the proxy of the housing market condition at time t.

Based on Eq. 4, the discounted sum of future expected housing asset returns is:

$$ E\left[ {\sum\nolimits_{{j = 0}}^{\infty } {\left. {{\rho^j}{r_{{t + j}}}} \right|{\Psi_t}} } \right] = \frac{{{\mu_0}}}{{(1 - \rho )}} + \frac{{{\mu_1}}}{{(1 - \rho )}}Pr\left[ {{S_t} = {1}} \right] + \frac{{{\mu_1}}}{{(1 - \lambda \rho )}}\left( {Pr\left[ {{S_t} = {1}|{\Psi_t}} \right] - Pr\left[ {{S_t} = {1}} \right]} \right) $$
(7)

Therefore, volatility feedback is represented as Eq. 8:

$$ {f_t} = \frac{{{\mu_1}}}{{(1 - \lambda \rho )}}\left( {\Pr \left[ {\left. {{S_t} = 1} \right|{\Psi_{{t - 1}}}} \right] - \Pr \left[ {\left. {{S_t} = 1} \right|\Psi_t^{\prime }} \right]} \right) = \delta \left( {\Pr \left[ {\left. {{S_t} = 1} \right|\Psi_t^{\prime }} \right] - \Pr \left[ {\left. {{S_t} = 1} \right|{\Psi_{{t - 1}}}} \right]} \right) $$
(8)

where \( \delta = - \frac{{{\mu_1}}}{{(1 - \lambda \rho )}} \).

Finally, combining Eqs. 6 and 8, the realized housing asset return r t is represented as:

$$ {r_t} = {\mu_0} + {\mu_1}Pr\left[ {\left. {{S_t} = {1}} \right|{\Psi_{{t - 1}}}} \right] + \delta \left( {Pr\left[ {\left. {{S_t} = {1}} \right|\Psi_t^{\prime }} \right] - Pr\left[ {\left. {{S_t} = {1}} \right|{\Psi_{{t - 1}}}} \right]} \right) + {\varepsilon_t} $$
(9)

This paper extends the framework to investigate Markov-switching housing volatility in the U.S. housing market. The relationship between housing volatility and expected housing asset returns is discussed under four different assumptions regarding information available to agents. The four scenarios of the analyzed models are reported as follows:

  1. Model 1:
    $$ \begin{array}{*{20}{c}} {{r_t} = {\mu_0} + {\varepsilon_t};{\varepsilon_t}\sim N\left( {0,\sigma_{{{S_t}}}^2} \right)} \\ {\sigma_{{{S_t}}}^2 = \sigma_0^2\left( {1 - {S_t}} \right) + \sigma_1^2{S_t}, \sigma_0^2 < \sigma_1^2; {S_t} = 0,1} \\ {Pr\left[ {\left. {{S_t} = 1} \right|{S_{{t - 1}}} = 1} \right] = p,Pr\left[ {\left. {{S_t} = 0} \right|{S_{{t - 1}}} = 0} \right] = q.} \\ \end{array} $$
  2. Model 2:
    $$ \begin{array}{*{20}{c}} {{r_t} = {\mu_0} + {\mu_1}Pr\left[ {\left. {{S_t} = 1} \right|{\Psi_{{t - 1}}}} \right] + {\varepsilon_t};{\varepsilon_t}\sim N\left( {0,\sigma_{{{S_t}}}^2} \right)} \\ {\sigma_{{{S_t}}}^2 = \sigma_0^2\left( {1 - {S_t}} \right) + \sigma_1^2{S_t}, \sigma_0^2 < \sigma_1^2;{S_t} = 0,1} \\ {Pr\left[ {\left. {{S_t} = 1} \right|{S_{{t - 1}}} = 1} \right] = p,Pr\left[ {\left. {{S_t} = 0} \right|{S_{{t - 1}}} = 0} \right] = q.} \\ {{\Psi_{{t - 1}}} = \Psi_t^{\prime } = \left\{ {{r_{{t - 1}}},{r_{{t - 2}}}, \ldots } \right\}{\text{in}}\,{\text{the}}\,{\text{partial}}\,{\text{revelation}}\,{\text{case}}.} \\ {{\Psi_{{t - 1}}} = \Psi_t^{\prime } = \left\{ {{S_t}} \right\}{\text{in}}\,{\text{the}}\,{\text{full}}\,{\text{revelation}}\,{\text{case}}.} \\ \end{array} $$
  3. Model 3:
    $$ \begin{array}{*{20}{c}} {{r_t} = {\mu_0} + {\mu_1}Pr\left[ {\left. {{S_t} = 1} \right|{\Psi_{{t - 1}}}} \right] + \delta \left( {Pr\left[ {\left. {{S_t} = 1} \right|\Psi_t^{\prime }} \right] - Pr\left[ {\left. {{S_t} = 1} \right|{\Psi_{{t - 1}}}} \right]} \right) + {\varepsilon_t}} \\ {{\varepsilon_t}\sim N\left( {0,\sigma_{{{S_t}}}^2} \right)} \\ {\sigma_{{{S_t}}}^2 = \sigma_0^2\left( {1 - {S_t}} \right) + \sigma_1^2{S_t}, \sigma_0^2 < \sigma_1^2; {S_t} = 0,1} \\ {Pr\left[ {\left. {{S_t} = 1} \right|{S_{{t - 1}}} = 1} \right] = p,Pr\left[ {\left. {{S_t} = 0} \right|{S_{{t - 1}}} = 0} \right] = q.} \\ {\delta = - \frac{{{\mu_1}}}{{(1 - \lambda \rho )}}{\text{for}}\,{\text{restricted}}\,{\text{volatility}}\,{\text{feedback}};{\Psi_{{t - 1}}} = \left\{ {{r_{{t - 1}}},{r_{{t - 2}}}, \ldots } \right\},\Psi_t^{\prime } = \left\{ {{S_t}} \right\}} \\ \end{array} $$
  4. Model 4:
    $$ \begin{array}{*{20}{c}} {{r_t} = {\mu_0} + {\mu_1}Pr\left[ {\left. {{S_t} = 1} \right|{\Psi_{{t - 1}}}} \right] + \delta \left( {Pr\left[ {\left. {{S_t} = 1} \right|\Psi_t^{\prime }} \right] - Pr\left[ {\left. {{S_t} = 1} \right|{\Psi_{{t - 1}}}} \right]} \right) + {\varepsilon_t}} \\ {{\varepsilon_t}\sim N\left( {0,\sigma_{{{S_t}}}^2} \right)} \\ {\sigma_{{{S_t}}}^2 = \sigma_0^2\left( {1 - {S_t}} \right) + \sigma_1^2{S_t}, \sigma_0^2 < \sigma_1^2;{S_t} = 0,1} \\ {Pr\left[ {\left. {{S_t} = 1} \right|{S_{{t - 1}}} = 1} \right] = p,Pr\left[ {\left. {{S_t} = 0} \right|{S_{{t - 1}}} = 0} \right] = q.} \\ {\delta = - \frac{{{\mu_1}}}{{(1 - \lambda \rho )}}{\text{for}}\,{\text{restricted}}\,{\text{volatility}}\,{\text{feedback}};{\Psi_{{t - 1}}} = \left\{ {{S_{{t - 1}}}} \right\},\Psi_t^{\prime } = \left\{ {{S_t}} \right\}} \\ \end{array} $$

We consider both restricted volatility feedback (\( \left( {\delta = - \frac{{{\mu_1}}}{{(1 - \lambda \rho )}}} \right) \) and the unrestricted (freely-estimated) volatility feedback cases, assuming ρ = 0.997 as suggested in the existing literature.Footnote 6 Model 1 examines whether or not there is Markov-Switching housing market volatility. Model 2 assumes no volatility feedback (i.e., δ = 0), and it examines if there is a significant volatility-state-dependent housing asset return (μ 1  ≠ 0) under two different information availability assumptions (full and partial revelation in information about market volatility). Models 1 and 2 show some disadvantages as volatility feedback is not considered. Thus, Model 3 and Model 4 which incorporate volatility feedback are illustrated to analyze the important role of people’s expectations.

Model 3 assumes the existence of volatility feedback (i.e., δ ≠ 0), and it is capable of investigating the relationship between housing volatility and asset returns due to partial revelation (\( {\Psi_{{t - 1}}} = \left\{ {{r_{{t - 1}}},{r_{{t - 2}}}, \ldots } \right\},\Psi_t^{\prime } = \left\{ {{S_t}} \right\} \)). In the partial revelation case, people are only able to observe past housing asset returns at the beginning of time t. Model 4 investigates the relationship between return volatility and expected asset returns under full revelation (\( {\Psi_{{t - 1}}} = \left\{ {{S_{{t - 1}}}} \right\},\Psi_t^{\prime } = \left\{ {{S_t}} \right\} \)). In this case, people can recognize the previous housing volatility regime at the beginning of time t.

Breakpoint Determination

Data

The U.S. housing price y t is the quarterly Median Sales Prices of House, which is obtained from the U.S. Census Bureau. The data span from 1963Q1 to 2007Q4. The Core Consumer Price Index (CPI for all urban consumers: all items less food & energyFootnote 7) from the Bureau of Labor Statistics is used as the deflator to obtain the real housing price. The realized housing asset return r t is defined as log first difference of the real housing price (i.e., \( {r_t} = {1}00 \times { \log }\left( {{y_t}} \right) - {1}00 \times { \log }\left( {{y_{{t - 1}}}} \right) \)).Because the impact of interest rates has been implicitly embedded in the real housing price, the risk-free rate is not subtracted from the return r t. Thus, what is analyzed in this study is the gross housing asset return as used in some studies, such as Hwang et al. (2006).Footnote 8

Method

Hansen (2001) applies Bai and Perron (1998) sequential estimation test for breakpoints to determine potential structural changes in U.S. labor productivity. This framework is utilized in this paper to determine the breakpoint of the U.S. housing market. The main advantage of this approach over the conventional Chow’s structural change test is that it allows identification of unknown breakpoints.

Let y t represent the housing market return (log first difference of the real housing price), which follows a first-order autoregression AR (1) process:

$$ \begin{array}{*{20}{c}} {{y_t} = \alpha + \theta {y_{{t - 1}}} + {\omega_t}} \hfill \\ {E\omega_t^2 = {\sigma^2}} \hfill \\ \end{array} $$

where ω t is a time series of serially uncorrelated shocks. The breakpoint refers to the date at which at least one of the three parameters (α,θ,σ 2) changes.

The test indicates a complete structural change in 1982Q2 based on the least sum of squared errors (minimum of residual variance). The autoregressive parameter (θ) in the pre-break period is 0.15, while it is −0.31 in the post-break period. Further, applying Quandt-Andrews Sup Test (1993) and Andrews-Ploberger Exp Test (1994), the null hypothesis that there is no break change is rejected (asymptotic p-value is 0.037 and 0.012, respectively). Thus, there is a statistically significant change in the autoregressive coefficient for the series of housing asset returns.

Additionally, the break in the error variance of housing asset returns AR(1) (σ 2) is 2001Q2. In the pre-break period, the standard deviation is 2.64, while in the post-break period, the standard deviation is 3.82. Applying Quandt-Andrews Sup Test (1993) and Andrews-Ploberger Exp Test (1994), we can reject the null hypothesis that there is no break change (asymptotic p-value is 0.014 and 0.0475, respectively).

Bai’s 90% confidence interval suggests the large uncertainty of the exact breakpoint dating which motivates the exploration of possible breakpoints in the interval. As it can be observed in Fig. 1, the housing price soars remarkably around 1998–1999. It suggests that this period might mark important changes in the volatility of the housing market. In addition, given the important findings and implications regarding the break in volatility of the U.S. output in 1984, we also investigate the behavior of the housing market before and after this date. Thus, 1984Q1 and 1999Q1 are chosen as two alternative breakpoints in this study, and the results using these two break points are presented and compared in the following section. The determination of the exact breakpoint in the U.S. housing market entails further confirmative researches. However, for reasons stated above, and as discussed below, these dates, which are very close to the ones found in the breakpoint tests, turn out to be very important in the analysis of the relationship between volatility feedback and housing asset returns.Footnote 9

Fig. 1
figure 1

U.S. Real housing price (1963Q1-2007Q4). Notes: The housing price is the median sales price in the US deflated by Consumer Price Index for All Urban Consumers: All Items Less Food & Energy. The unit is US dollar

Full size image

Empirical Results

Volatility feedback effect on U.S. housing asset returns

Model 1: No Volatility-State-Dependent Housing Asset Returns and Volatility Feedback

This model examines if there exists Markov-switching housing market volatility. When 1984Q1 is used as the breakpoint, there is a significant Markov-switching volatility. In particular, the transition probabilities \( p\left( {Pr\left[ {\left. {{S_t} = {1}} \right|{S_{{t - 1}}} = 1} \right]} \right) \) and \( q\left( {Pr\left[ {\left. {{S_t} = 0} \right|{S_{{t - 1}}} = 0} \right]} \right) \) are significant for both subsamples as shown in Part 3 of Table 1. When 1999Q1 is used as the breakpoint, Markov-Switching volatility is significant (t-statistics of q = 13.2 and t-statistics of p = 3.6) in the pre-1999, but Markov-Switching component in the high-volatility regime in the post-1999 is not significant (t-statistics of p = 0.16). The standard deviation in the post-1999 period is almost the same in the high volatility and low-volatility regimes (\( {\sigma_0} = {\sigma_1} = {3}.{52} \)), and this corresponds to the result of no significant Markov-switching volatility in this period. This result of Model 1 implies that in the post-1999 there is only a permanent structural change in housing asset returns, and there is no temporary Markov-switching volatility.

Table 1 No volatility-state-dependent housing returns and volatility feedback
Full size table

Additionally, in both regimes, the differences of the standard deviation between the pre-period and the post- period are larger for 1999Q1 breakpoint than the 1984Q1 breakpoint. For example, in the case without switching variance, the standard deviation is 2.7 in the pre-1999 and 3.5 in the post-1999 period. The difference is about 0.8, while there is almost no difference between the pre-1984 and the post-1984 periods. This result is similar when Markov-switching variance is considered. For example, in the low-volatility regime, the difference of between the pre-1999 and the post-1999 is about 1.4, while the difference is only about 0.1 as 1984Q1 is used as the breakpoint.

This supports our argument that there can be a permanent structural change in volatility in the post-1999. The housing price surges remarkably, and housing asset returns are more volatile during the post-1999 period. As growing recent studies argue, the post-1999 period is characterized by a “bubble-like” housing boom-and-bust cycle.

Model 2: Volatility-State-Dependent Housing Asset Returns

This model allows investigations into the evidence of volatility-state-dependent housing asset returns (μ 1  ≠ 0) due to full revelation and partial revelation of information about housing market volatility (shown in Table 2).

Table 2 Volatility-state-dependent housing asset returns
Full size table

Only the pre-1984 period with full revelation information has significant volatility-state-dependent housing asset returns. The high-volatility regime of the pre-1984 has a lower “realized contemporaneous housing asset return” since it is associated with a negative mean (μ 1  = −4.92, t-statistic = −2).

This result raises two noticeable points. First, full revelation facilitates the significance of volatility-state-dependent housing asset returns. Second, because no volatility feedback effect is assumed, the underlying reason for the existence of a “negative correlation” between variance and the mean of the housing asset return is uncertain. This is the issue that Models 3 and 4 are able to address.

Model 3: Volatility Feedback Effect Due to Partial Revelation

The existence of volatility feedback for both cases of freely-estimated and restricted volatility feedback due to partial revelation is investigated in Model 3. Volatility feedback reflects people’s adjusted expectations of all future discounted expected housing asset returns because of news about the housing market during the current period of time. In other words, volatility feedback captures the dynamics of people’s expectation which can update over time.

Part1 of Table 3 shows empirical results without considering the permanent breakpoint in order to highlight its influence on the dynamics of housing asset returns. It shows that volatility feedback effects are not significant for both the partial and the full revelation cases at the 5% significance level (the t-statistic of δ is −1.27 for the partial revelation case, and −1.74 for the full revelation case). In addition, volatility-state-dependent asset returns (μ 1 ) are not significant in both cases (the t-statistic of μ 1 is −0.68 for the partial revelation case, and of μ 1 is −0.53 for the full revelation case). On the other hand, if a permanent breakpoint is considered, all the models have significantly negative volatility feedback, except for the post-1984 period (shown in Part 2 of Table 3). This supports the critical role of the permanent breakpoint in the empirical investigations into the volatility feedback effect on the U.S. housing market.

Table 3 Volatility feedback effect due to partial revelation
Full size table

In addition, Table 3 shows that for the scenario of freely-estimated volatility feedback (δ and μ 1 are estimated separately), the post-1999 and the pre-1984 have significant volatility-state-dependent housing asset returns, and all sub-periods have significantly negative volatility feedback effects except the post-1984 (which has negative but insignificant feedback effect).

Model 4: Volatility Feedback Effect Due to Full Revelation

The existence of volatility feedback for both cases of freely-estimated and restricted volatility feedback due to full revelation is investigated in Model 4 (Tables 4 and 5). There are four common findings in Model 4 and Model 3 which are worth our closer observations.

Table 4 Volatility feedback effect due to full revelation (restricted)
Full size table
Table 5 Volatility feedback effect due to full revelation (freely-estimated)
Full size table

First and most importantly, in the case of unrestricted volatility feedback, all sub-periods have significantly negative volatility feedback, with the exception of the post-1984. Hence, this study provides support for a positive linkage between housing market volatility and the expected housing asset return which is indicative of the important role of people’s expectations in the U.S. housing market. Second, for both the restricted and unrestricted volatility feedback, post-break periods have larger standard deviation difference between high- and low-volatility regimes (σ 1 –σ 0 ) than the corresponding pre-periods (i.e., the standard deviation difference of the post-1984 is higher than pre-1984, and the standard deviation difference of the post-1999 is higher than the pre-1999). Third, in the case of restricted volatility feedback, only the post-1999 displays significant volatility-state-dependent housing asset returns (μ 1  = 2.12, t-statistics = 3.22 for partial revelation case, and μ 1  = 1.87, t-statistics = 1.96 for full revelation case). Fourth, freely-estimated volatility feedback is stronger and more significant in the post-1999(δ = −6.76, t-statistics = −11.72), compared to all other sub-periods.

As “Method” analyzes, the method supports that 1984Q1 and 1999Q1 can be chosen as the alternative breakpoints of the sample analyzed since they both lie within Bai’s 90% confidence interval. Besides, 1984Q1 is supported by numerous previous studies on macroeconomic stabilization, and 1999Q1 is chosen as the approximate onset of the recent bubble-like housing boom-and-bust cycle in the US.

Noticeably, some studies argue the housing price started to surge since 1998. Therefore, the robust test of the findings is conducted by using 1998Q1 as the breakpoint in order to compare the results with those of 1999Q1 breakpoint. Table 6 indicates that for the scenario of freely-estimated full revelation, the results of 1998Q1 are qualitatively the same as those of 1999Q1. It reflects that the breakpoint 1999Q1 is empirically and econometrically appropriate to be chosen to extract the implication for the recent housing boom-and-bust cycle.

Table 7 Turning points of Markov-switching regimes: freely-estimated volatility feedback
Full size table

Smoothed Probability of the High-Volatility Regime vs. the Business Cycle

In this section, we address the estimated smoothed probabilities of the high-volatility regime in the U.S. housing market. The first goal is to examine the similarity of the smoothed probability inferences across different proposed models. In particular, the evidence of the volatility feedback effect on asset returns entails exogeneity of market volatility. Although the smoothed probabilities of these two models (full revelation and partial revelation) display different persistence of high-volatility regimes around some recessions, in both cases they lag the 2001 recession and lead the 2007 recession. Thus, the smoothed probability inferences support exogeneity of housing market volatility because the models with full and partial revelations deliver the similarity for both alternative breakpoints (1984Q1 and 1999Q1).

The second goal is to investigate how the U.S. housing cycles empirically associate with business cycles through housing asset returns. Notice that many other papers have failed to detect the 2001 recession, since it is quite different from other NBER-dated recessions. Otherwise, the proposed model overcomes this challenge to some extent. It displays that high-volatility probabilities start to rise before the 2001 recession when the 1984Q1 breakpoint is considered, while the probabilities start to rise with a small lag when the 1999Q1 breakpoint is considered.

The smoothed probabilities of the model with unrestricted volatility feedback due to full revelation and partial revelation are investigated for the two breakpoints-- 1999Q1 (Fig. 2) and 1984Q1 (Fig. 3). Correspondingly, Table 7 shows the beginnings and endings of the high- volatility regimes in the U.S. housing market during 1963–2007, which are regarded as the turning points of the time series of the housing asset returns. As suggested in Hamilton (1989), the turning points indicate the periods whose probabilities of high-housing-volatility regimes are larger than 0.5 (i.e., \( Pr\left[ {\left. {{S_t} = {1}} \right|{S_{{t - 1}}} = {1}} \right] \geqslant 0.{5} \)), and they enable us to observe the persistence of volatility regimes.

Fig. 2
figure 2

Smooth probabilities −1999 Breakpoint. a: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to full revelation – Pre-1999. b: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to full revelation – Post-1999. c: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to partial revelation—Pre-1999. d: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to partial revelation—Post-1999. Notes: The shaded areas are NBER recessions

Full size image
Fig. 3
figure 3

Smooth probabilities- 1984 Breakpoint. a: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to full revelation—Pre-1984. b: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to full revelation—Post-1984. c: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to partial revelation—Pre-1984. d: Smoothed probabilities of high volatility regimes (in solid blue ink) for the model with freely-estimated feedback due to partial revelation—Post-1984. Notes: The shaded areas are NBER recessions

Full size image
Table 6 Robustness of findings: Volatility feedback effect due to full revelation (freely-estimated)
Full size table

The high-volatility regime indicates the high uncertainty, and its smoothed probability rises when the housing market is in a low-return phase. Further, high volatility implies low realized returns during the current period, and high realized returns in the future. Interestingly, interrelationships between volatility and returns (realized and expected) indicate a strong association between the U.S. housing cycle and the business cycle.

In the pre-1999 period, the probability of high volatility goes up when realized housing returns goes down, and this pattern occurs around recessions except the 1973–75 recession.Footnote 10 However, in the post-1999, the probability of high volatility fails to coincide with the 2001 recession. This reflects that the housing market does not experience a low-return phase in this recession. Moreover, the probability of the high-volatility regime continues to be at very high level up to the current recession since 2007Q4. This result implies there is a significant uncertainty in the U.S. housing market around and after the 2001 recession. Overall, the study suggests that the recent housing crisis was associated with a high level of uncertainty or ‘risk’, as measured by the probabilities of high volatility of housing returns. Also, it indicates a strong association between the U.S. housing cycle and the business cycle.

Implications of a Bubble-Like Housing Boom and Bust

This section discusses how interactions between housing volatility and asset returns, including realized and expected returns, reveal a special story of the current housing boom and bust. While the debate on whether the current housing boom and bust can be defined as a housing bubble is still lasting, the recent huge surge of housing prices definitely has become an important issue for many households, investors, researchers and policy-makers. Although the housing-bubble identification is beyond the scope of this study, the proposed model provides some insights into the recent bubble-like housing market dynamics.

First, the association between realized housing asset returns and housing volatility facilitates our investigations into how fundamentals are deviated during the recent housing boom and bust. Before 2001 and after 2004, their negative relationship always holds. Otherwise, during 2001–2004, realized returns are very high along with high volatility-- their negative relationship is broken. As defined by Stiglitz (1990), a housing bubble is the situation which economic fundamentals fail to support high housing price growth. Currently, growing studies address what are the fundamentals in order to examine the presence of the housing bubble, such as Shiller(2005), Himmelberg et al. (2005), Gallin (2006), Smith and Smith (2006), Mikhed and Zemcík (2007), etc. Correspondingly, if the negative relationship between housing realized returns and housing volatility is regarded as one of the fundamentals, their “broken negative relationship” during 2001–2004 signals a possible housing bubble during this period. Interestingly, empirical results of the model highly coincide with the remarkable housing price appreciation in the U.S.

Second and more importantly, expected housing asset returns and their relationship with housing volatility are capable of explaining the current housing boom and bust. As Piazzesi and Schneider (2009) highlight, the Michigan Survey of Consumers separates the recent housing boom into two stages: the early-boom phase during 2002–2003, and the second phase during 2004–2005. During the early-boom period, a large and growing number of households believed that it was a good time to buy a house. Correspondingly, the empirical results of this study support that people’s expectation is the driving forces behind the surge of the U.S. housing price. In the model, high volatility during 2002–2003 implies low realized returns in 2002–2003 and high expected returns afterwards-- during 2004–2005. Thus, the second stage of the housing boom in 2004–2005 is captured by the dynamics of people’s expectations in 2002–2003 through volatility feedback (revised expected returns). As a result, large housing volatility during the slow recovery period after the 2001 recession reflects that revised expectations play a dominant role in the U.S. housing market.

Conclusion

This paper extends a Markov-switching model that incorporates people’s expectations into the housing market to investigate the dynamics of housing asset returns in the last five decades. The analysis is undertaken for the full sample as well as subsamples in order to distinguish permanent structural breakpoints from the temporary Markov-switching component. In particular, two dates are considered---1984Q1 and 1999Q1, which are associated, respectively, with the great moderation and with the start of the recent U.S. boom-and-bust cycle. This study investigates the “bubble-like” housing boom-and-bust cycle from a fresh perspective which is not documented in the previous literature.

The results indicate that the relationship between the U.S. housing market volatility and expected housing asset returns is significantly positive due to the negative volatility feedback effect. Particularly, the important role of people’s expectations on the demand side is strongly supported as the volatility feedback is employed. Also, the significant volatility feedback effect and the smoothed probability inferences indicate a strong association between the housing cycle and the business cycle, as well as a remarkable uncertainty in the U.S. housing market during the post-1999 period.

The U.S. housing market in the post-1999 period is worth our closer investigations because of the remarkable housing boom-and-bust cycle after 1999. This study delivers five main findings of the housing price dynamics in the post-1999. First, if no volatility feedback effect is considered, there is only a permanent structural change; neither significant temporary Markov-switching housing market volatility nor significant volatility-state-dependent housing asset return can be found in the U.S. housing market. Otherwise, if volatility feedback is considered, the latter two characterize the housing market in the post-1999 and other subsamples. Second, the unrestricted volatility feedback effect is of the most significance in the post-1999. Third, the difference between high and low volatility is larger in the post-1999 than the pre-1999. Fourth, the negative relationship between housing volatility and realized returns during 2001–2004 is broken. This broken relationship which can be regarded as a deviated economic fundamental implies a possible housing bubble during this period. Finally and most importantly, volatility feedback is capable of anticipating the recent bubble-like housing boom-and-bust cycle in the U.S. because high volatility during the early-boom stage (2002–2003) implies low realized returns in this period as well as high expected returns in the second phase of the housing boom (2004–2005).