1 Introduction

Annual reports represent one of the most important means of gathering information on companies. To guarantee the usefulness of the information in the annual report, the boundaries of mandatory disclosure have been gradually broadened over time (Alcaniz et al. 2011; Zambon 2011; Page 2014; Singleton-Green 2014). In recent decades, it has been generally recognised that the contents of annual reports should be expanded beyond the traditional financial reporting to include more strategic, forward-looking, and non-financial information (CFA 2007; FASB 2009; CICA 2009; EFRAG 2013; IIRC 2013). As a consequence, many regulators have been focusing attention on the management commentary statement in the annual report, typically referred to as “Management Discussion and Analysis” (MD&A) in the US and Canada, “Strategic Report” in the UK, and “Management Commentary” by the International Accounting Standard Board (IASB).Footnote 1

The regulation of management commentary information is a topic of intense debate (ASB 2006; ICAEW 2009; IASB 2010; EFRAG 2013; FRC 2013, 2014). Generally, the effectiveness of a regulatory intervention depends on the “level of specification” (Diver 1983; Baldwin et al. 2011): an approach based on specific rules may not always be applicable to concrete situations and may burden firms with excessive demands, while an approach based on general principles might be unclear and easily misunderstood by the intended audience. This issue of adequacy of the regulation specification is particularly relevant for management commentary disclosure, which aims to transmit a view of a company’s business “as seen through the eyes of those who manage that business” (SEC 2003). Following the “through the eyes of management” principle, the regulation should allow managers a high degree of discretion in depicting a view of their company. On the other hand, a regulatory system without a sufficient level of specification could result in scarcely useful disclosure. Few studies addressing the regulation of management commentary disclosure have expressed concerns about the effectiveness of the regulatory systems in force, and highlight the limitations of those systems based on loosely specified requirements (Beattie and McInness 2006; Huefner 2007). In this respect, in 2009 the Institute of Chartered Accountants in England and Wales (ICAEW) issued a call for research about the efficacy of broadly stated disclosure requirements for management commentary disclosure (ICAEW 2009).

Our paper contributes to this debate by focusing specifically on performance indicators, which is considered one of the most important pieces of information included in a management commentary statement (AICPA 1994; Eccles and Coleman 1998; AIMR 2000; Hooks et al. 2002; Campbell and Slack 2008). Performance indicator disclosure is generally regulated by loosely specified requirements (AAA 2002; CICA 2002; ASX 2003; ASB 2006; SEC 2008). This approach has also been adopted by the EU, in a rule that is included in the Directive 2003/51/EC, labelled as Modernisation Directive (hereafter Directive). Article 14 simply requires European firms to publish “both financial and, where appropriate, non-financial key performance indicators relevant to the particular business, including information relating to environmental and employee matters”. The issuance of this European intervention offers the opportunity to evaluate the effectiveness of a regulation with a low level of specification in guaranteeing high quality disclosure practice concerning performance indicators.

Our analysis enriches previous evidence on performance indicator disclosure practices, which focuses exclusively on one specific country, the United Kingdom (Black Sun Plc 2007; PwC 2007; Tauringana and Mangena 2009). Here, the national standard setter has been playing an active role in complementing the specification of the Directive’s provision. On the contrary, we have investigated disclosure practices in Italy, a context where, similarly to the majority of European countries, the obligation introduced by the Directive has not been supported and integrated by any national intervention, to date.

In investigating disclosure practices we have not only counted the number of indicators published, but we have also assessed their presentation modalities. For a long time, the leading literature on disclosure measurement has pointed out that the amount of disclosure is not a sufficient measure to assess a multidimensional concept such as disclosure quality (Beattie et al. 2004; Beretta and Bozzolan 2008; Chatterjee et al. 2008; Aripin et al. 2010). This is particularly true concerning performance indicators: in fact, an increase in the number of indicators could paradoxically lead disclosure quality to decline, if certain modalities of presentations are not considered. Therefore, we offer a contribution to the previous literature by developing a presentation modality index that is based on the recommendations issued by leading professional and political bodies (SEC 2003; CICA 2002; G100 2003; IASB 2005; ASB 2006).

Comparing disclosure practices in the three-year period before (2005–2007) and the three-year period after (2008–2010) the adoption of the Directive, we point out that a regulatory intervention, even with loose specification, is associated with an increase in disclosure practices. However, such an increase is more “formal” than “substantial”. In fact, regardless of the implementation of the Directive, companies primarily focus on common financial measures. On the contrary, non-financial indicators are rarely disclosed. High proprietary costs typically associated to strategic information might have influenced our findings. Furthermore, companies are still far from full compliance with the recommendations concerning presentation modalities. Evidently, such disclosure practices do not address the call for a business-reporting model that emphasises more information that is forward-looking and non-financial in nature (Beattie et al. 2004).

Additionally, our results offer some points to be considered for regulators and standard setters involved in the regulatory process. In particular, we identify three main limitations of the European regulation. Firstly, it lacks a framework that states the purposes of this information and the information needs that it should be able to satisfy. Secondly, it does not separate the disclosure of financial and non-financial indicators, as they markedly have different features in terms of information content and proprietary costs. Finally, it does not offer any guidelines on disclosure modalities, which is necessary to ensure the usefulness of each set of information.

The remainder of the paper is organised as follows: Sections two, three, and four review the complex debate on the opportunities and ways to regulate performance indicator disclosure in annual reports. Section five depicts the state of the art concerning the regulatory system for performance indicators in the EU. On the basis of this literature, we set forth our research question. The research design is explained in Section six. The results are presented and discussed in Section seven. Finally, we report our conclusions and outline the limitations of our work.

2 The regulation of management commentary: management discretion versus standardized information

The accounting research has largely focused on the reasons why disclosure regulations exist, although a comprehensive theory of mandatory disclosure is lacking. In their review of existing models of disclosure regulation, Beyer et al. (2010) identify two main approaches to explain why most developed capital markets mandate corporate disclosures. The social value of disclosure underpins the first approach. In this framework, regulatory activity assumes a normative connotation, as it is necessary to reset the balance in the information market and avoid opportunistic behaviours by the insiders (Demski 1974; Verrecchia 1982). In contrast, the second approach identifies rule-making processes as dialectical processes in which law-makers are subject to significant pressure from various political constituents. Several empirical studies confirm that accounting standards and regulations (including their enforcement) are subject to political pressure (Ramanna 2008; Hochberg et al. 2009; Correira 2014).

Regardless of the approach adopted to justify disclosure regulation, one of the most challenging issues related to this topic is the difficulty of designing an appropriate and effective regulatory method (Admati and Pfleiderer 2000; Rutherford 2003); in other words, how to define the adequate level of specification of the mandatory requirements (Diver 1983; Baldwin et al. 2011). In fact, an inadequate specification with few details leads to the failure of regulatory intervention. The consequences of an insufficient level of specification are well illustrated by the case of the 4th and 7th Directives. In issuing these two directives, the European Commission intended to maximise the comparability of European financial reports, especially among listed firms. These rules are characterised by a low level of specification to maximise their applicability in different countries. For this reason, the directives have proven to be poorly effective (Walton 1992; Emenyonu and Gray 1992; Van Hulle 1993; Theunisse 1994; Thorell and Whittington 1994; Herrmann and Thomas 1995). In particular, Van Hulle (1993), as a head of Unit for Financial Reporting and Company Law at the EU Commission, contests the use of the directives to standardise European financial reports. Moreover, Theunisse (1994) holds the permissible alternative accounting measurements provided by the directives responsible for the low level of comparability among European annual reports.

Over time, the debate about corporate disclosure regulation has been enriched by several contributions focused on management commentary disclosure (Larrinaga et al. 2002; Rutherford 2003; Cormier et al. 2005; Beattie and McInnes 2006; Huefner 2007; Beattie et al. 2008; Fraser and Henry 2010). The definition of an adequate level of specification for this type of disclosure becomes more problematic due to its narrative and subjective content (Rutherford 2003). In fact, the management commentary information is asked to represent the firm “through the eyes of management”. Following this principle, management commentary disclosure aims to offer “a context within which to interpret the financial position, financial performance and cash flows of an entity. It also provides management with an opportunity to explain its objectives and its strategies for achieving those objectives” (IASB 2010, §IN3). Similarly, the SEC notes that management commentary “should be a discussion and analysis of a company’s business as seen through the eyes of those who manage that business” (SEC 2003).

These characteristics of management commentary disclosure challenge policymakers to define an adequately specified regulation that balances standardisation with management discretion. An under-specified provision might result in low transparent narrative information, allowing management to omit relevant information. On the other hand, an over-specified system risks reducing managers’ ability to express their views and even overriding those views (see Diver 1983; Baldwin et al. 2011).

The few studies dealing with management disclosure regulation express concerns about the absence of a well-defined framework for management commentary disclosure and note the weakness of current regulatory systems (Beattie and McInness 2006; Huefner 2007). Beattie and McInness (2006) criticise the regulatory system adopted in the UK, which is accused of providing managers with too much discretion. On the other hand, in analysing the quality of the disclosure included in the US MD&A reports, Huefner (2007) criticises the insufficient accuracy of the rules in specifying the information that should be provided in a company’s strategic analysis.

In Europe, certain topics included in a management commentary are regulated by the Modernization Directive (2003/51/EC). This is the case of performance indicators. With a requirement included in Article 14, the European legislator strengthened the relevance of management analysis and required managers to include “both financial and, where appropriate, non-financial key performance indicators relevant to the particular business, including information relating to environmental and employee matters”. The EU has opted for a low level of specification in the regulation governing performance indicator disclosures in management commentary. The European requirement consists of an isolated and loosely specified rule that leaves wide margin of discretion to managers. In fact, the rule lacks any detail about the types of indicators that managers have to convey, the information needs that must be satisfied, and moreover the ways in which this information should be disclosed.

Few studies have investigated the effects of the European requirement on performance indicator disclosure practices. They primarily focus on the amount and the type of performance indicators disclosed by companies, while presentation modalities are marginally investigated. Focusing on media industry, Tauringana and Mangena (2009) examine annual reports over a two-year period (2004–2005) before the implementation of the Directive in the UK and a two-year period (2006–2007) after implementation. They found a significant increase in the number of indicators disclosed by firms that rose from 1.70, in 2004, to 3.19, in 2007. These results align with those reported by the Accounting Standard Board (ASB)’s survey of a sample of 23 UK-listed companies on the FTSE 100 after the implementation of the Directive (ASB 2007). This survey scores as “fair” the disclosure of the performance indicators published by companies and documents an increase in the percentage of firms that use performance in the narrative sections of their annual reports. Similarly, the Black Sun Plc (2007) survey on the FTSE 100 notes that a substantial number of companies significantly improved their reports between 2005 and 2006, providing more disclosure about the market environment, strategy, risk management and business performance through the use of performance indicators. Finally, the survey conducted by PricewaterhouseCooper (PwC) on the FTSE 350 documents an increase in the average number of performance indicators disclosed between 2005 and 2006 (from five to eight) and also an improvement in the way in which those indicators are presented, i.e., why a certain indicator is published and how it is calculated, along with the trend data and targets (PwC 2007).

The majority of these studies are focused on the UK, which represents a unique context within Europe. In the early 1990s, the UK was the first European country to consider management commentary as necessary information to integrate within the annual report (Weetman 2009). Moreover, the national standard setter, the ASB, plays an active role, providing UK firms with integrative specification that complements the general intervention of the European legislator.

Considering other European countries, we can observe that they have not generally added any detail to increase the level of specification of the rule, when they adopted the EU provision concerning performance indicators in national legislation (KPMG 2013). Certain national standard setters have only intervened on a limited basis, providing a few general recommendations or clarifications of the terminology used by the legislator. This is true of the German Accounting Standards Board [GASB, Deutscher Standardisierungsrat (DSR)], which in 2005 issued GAS No. 15 on Management Reporting (Lageberichterstattung). Regarding performance indicator disclosure, the standard highlights the opportunity to include the most common operating and financial indicators such as leverage, profitability, market, debt and liquidity indicators (GAS No. 15, section 45–80 and section 103–114). The standard specifies that non-financial performance indicators should not be limited to information related to environmental and employee matters; they should also contain information concerning the company’s customer base, market shares or human capital (GAS No. 15, section 31–32).Footnote 2 In the same period, the Netherlands Council for Annual Reporting (“Raad voor de Jaarverslaggeving”) developed an explanatory guideline that explicitly recommends reporting non-financial performance indicators (Zandvliet 2011). Other European countries have enacted legislation on specific categories of performance indicators, elaborating on the directive’s requirements. In France, for example, listed companies are required to report on social and environmental issues in their annual reports (Loi sur les Nouvelles Régulations Economiques, NRE). In any case, these interventions do not represent a framework for performance indicator disclosure, leaving companies with a high level of discretion in deciding the extent, contents, and modalities of performance indicator disclosure.

In conclusion, the effects of the EU regulatory intervention concerning performance indicators on companies’ disclosure practices are not clear. First, previous studies focus only on the amount of indicators published by companies to assess disclosure quality. Secondly, they investigate one single country, the UK, where the standard setter has consistently worked to integrate the level of specification of the European prescript, contrary to the majority of other European countries. Taking into consideration that performance indicator disclosure plays a key role in improving the informativeness of corporate reporting, we formulate the following research question: Has the loosely specified regulatory intervention introduced by the European Modernisation Directive been effective in improving performance indicator disclosure in management commentaries?

3 The challenges in regulating performance indicator disclosure

In light of the “decision usefulness” approach, a lot of empirical research has demonstrated the relevance of performance indicators for both risk and return evaluations (Belkaoui 1978; Palepu 1986; Barnes 1990; Tennyson et al. 1990; Smith and Taffler 1995; Abrahamson and Amir 1996; Amir and Lev 1996; Bryan 1997; Behn and Riley 1999; Sorensen 2000; Riley et al. 2003; Kallapur and Kwan 2004; Grunert et al. 2005; Banker and Mashruwala 2007; Ya-wen 2007; Wyatt 2008; Simpson 2010; Coram et al. 2011; Ghosh and Wu 2012). More recently, the importance of indicator disclosure, especially concerning non-financial indicators, is strongly supported by the strands of literature concerning intellectual capital disclosure (Bukh et al. 2005; Li et al. 2008) and business model disclosure in the annual report (Nielsen 2010; Nielsen and Bukh 2013). Researchers state that, although this disclosure is mainly narrative in nature, performance indicators are necessary, especially non-financial, because they help tell a story about the mechanisms by which companies attempt to create and capture value. In particular, the presence of quantitative information contribute to making narratives more reliable (Mouritsen et al. 2001; Holland 2006).

Both academics and professional entities agree that financial and non-financial performance indicators can improve the quality and usefulness of annual reports (AICPA 1994; Eccles and Coleman 1998; AIMR 2000; Hooks et al. 2002; Campbell and Slack 2008). The Jenkins Committee methodically studied users’ information needs, and clearly found that performance indicator disclosure is one of the most important for users, particularly when attempting to carry out a forward-looking analysis (AICPA 1994). Similarly, investigating shareholders’ expectations of the type of information that should be disclosed in corporate annual report, the survey conducted by Hooks et al. (2002) ranked the segmental financial measures, efficiency measures, and market measures as among the most important “information gaps”. Additionally, influential bodies recognise the importance of performance measures. The IASB states that performance indicators can help users of financial reports assess the extent to which goals and objectives are being achieved (IASB 2010, point 37). The International Integrated Reporting Council (IIRC) ascribes a central role to performance indicators in its Integrated Reporting Framework, by specifying that the ability of the organisation to create value can best be reported on through a combination of quantitative and qualitative information (IIRC 2013, §1.11). Similarly, the document issued by the Organisation for Economic Cooperation and Development (OEDC) clearly indicates the relevance of non-financial performance indicators in conveying information about intellectual capital to external users (OECD 2006).

An alternative argument could be made that the financial indicators do not provide significant additional information to external users, because the same information can be easily calculated using financial statements. Nevertheless, the mandatory inclusion of financial indicators in annual reports safeguards unsophisticated users who are not able to autonomously calculate financial ratios (Elliott 2006; Allee et al. 2007; Bhattacharya et al. 2007). Moreover, Watson et al. (2002) maintain that the disclosure of financial ratios can also be important for sophisticated users if it helps them to quickly understand the information and saves their time.

Since 2003, the International Organisation of Securities Commission (IOSCO) has emphasised the importance of a framework on non-GAAP performance indicator disclosure and recommends issuers to use care when presenting non-GAAP measures (IOSCO 2003). In response to this warning, in 2005 the European Securities and Markets Authority (ESMA) issued a Recommendation on non-GAAP measures, in which a set of guidelines concerning the presentation of non-GAAP measures are stated. Among many other recommendations, these guidelines include the need to provide a clear definition of each measure, the basis for calculation, and comparable information for other periods (ESMA 2005). The debate on non-GAAP measures was revived in 2014, when both IOSCO (IOSCO 2014) and ESMA (ESMA 2014) developed two draft guidelines on Non-GAAP Financial Measures, which aim to assist issuers in providing clear and useful disclosure and to reduce the risk that such measures are presented in a way that could be misleading (IOSCO 2014).

Also a lot of national authorities have issued specific recommendations on performance measures, which are often included in more general guidance concerning management commentary disclosure (ASX 2003; CICA 2002, 2009; ASB 2006; FRC 2014). In its guidance on the listing rule 4.10.17, concerning “Review of Operations and Activities”, the Australian Stock Exchange (ASX) prescribes specific presentation for performance indicators: “The Review should define and explain the financial and non-financial measures included in the Review, their sources and the relevant assumptions and adjustments, if any, made in respect of information also included in the financial report”. Analogously, the Canadian Institute of Chartered Accountants (CICA) devotes a section of the Guidance on preparation and disclosure of Management Discussion and Analysis to illustrating the recommended practices for key performance indicators disclosure (CICA 2002, 2009). Similarly, the ASB’s Reporting Statement: Operating and Financial Review explicitly identifies the necessary presentation requirement for each indicator disclosed in the Operating and Financial Review (ASB 2006).Footnote 3 Such requirements were affirmed once again in the Guidance on Strategic Report issued in 2014 by the Financial and Reporting Council (FRC 2014).Footnote 4

Regulators and standard setters largely agree about the “firm specific” nature of indicator disclosure. The Financial Accounting Standards Committee (FASC) states: “The Committee believes mandating a standard set of disclosures related to, for example, customer satisfaction, quality, and the like would not best serve investors. Rather, we believe that companies should be encouraged to provide such disclosures voluntarily” (AAA 2002, p. 360). A similar approach is adopted by the CICA (2002, 2009) and the ASX (2003). At the same time, the idea that performance indicators should be clearly defined and explained by preparers is largely supported (EFRAG 2014). In its Practice Statement Management Commentary, the IASB explicitly states that, to be useful for external users, indicators should be accompanied by sufficient information to help users understand the message that the measures are intended to convey (IASB 2010). In other words, performance indicator disclosure regulation should focus more on the modalities of presentation than on the type of information disclosed, because performance measures can be misleading if they are inconsistently calculated or presented (ESMA 2014).

4 Research design

4.1 Sample selection

To verify the effects of the regulatory intervention introduced by the Directive, we focus on a context where the national legislator and the standard setter have implemented limited actions to integrate the original text. As mentioned earlier, the majority of European countries have not implemented rules or guidelines concerning performance indicator disclosure before or after the adoption of the Directive. This is the case in Italy, where the Directive was adopted in 2007 with an amendment of the Article 2428 of the Civil Code, devoted to management commentary disclosure. In 2008, the Italian National Council of Chartered Accountants [NCCA, Consiglio Nazionale dei Dottori Commercialisti e degli Esperti Contabili (CNDCEC)] issued a consulting document, Considerations about Management Report (CNDCEC 2008), that provides operative proposals to enable listed and non-listed companies to comply with the new Article 2428. This document suggests the inclusion of certain financial indicators that are considered important for assessing company performance. However, it does not provide a framework for performance indicator disclosure. Consequently, the contents and modalities of performance indicator disclosures in the annual report remain widely discretionary, as in many other European countries.

To assess the effects of EU mandatory requirement on disclosure practices, we have compared the indicator performance disclosure in Italian annual reports before and after the implementation of the Directive, which came into effect for fiscal years beginning after December 31, 2007. We have investigated disclosure practices over a 6-year period to allow us to better interpret any changes observed due to the introduction of the new requirement. Companies listed on the Italian Stock Exchange in December 2010 were eligible for inclusion in this study, excluding banks, insurance companies, holding companies, and real estate companies, as these industries have different reporting requirements, legal requirements and disclosure practices (Hossain et al. 1994). The resulting list consisted of 206 companies. To guarantee data homogeneity, companies listed after January 2005 were eliminated, reducing the number to 126. From this list, we extracted a sample of 75 companies, through a random sampling stratified by industry. Table 1 reports the breakdown of our sample by industry.

Table 1 Breakdown of sample by industry (following the Italian stock exchange classification system)
Full size table

4.2 Performance indicator disclosure measurement

The problematic nature of quality measurements is widely recognised and discussed in financial accounting literature (Healy and Palepu 2001; Core 2001; Beattie et al. 2004).

To assess performance indicator disclosure practices, we have considered two measures of disclosure quality: the number of performance indicators published in management commentary statement (QUANT) and their modalities of presentation (PRES).

In line with the previous disclosure literature, the rationale for the first measure is simple: the greater the number of indicators, the more disclosure is likely to satisfy information needs. The amount of disclosure is the most employed proxy in disclosure literature to assess disclosure quality (Wallace et al. 1994; Botosan 1997; Lang and Lundholm 2000; Lim et al. 2007, to name just a few).

The QUANT variable for each company was obtained as the sum of the number of financial and non-financial indicators collected for each report. By construction, the QUANT variable is a discrete measure, with a range from zero to infinity.

Over time, several authors have highlighted the inadequacy of the disclosure indexes that assume the amount of disclosure on specified topics as a proxy for a complex, multi-faceted concept such as disclosure quality (Beattie et al. 2004). Several authors highlight that disclosure practices may differ not only with regard to the quantity of information released, but also to the modalities of presentation (Beattie et al. 2004; Beretta and Bozzolan 2008; Chatterjee et al. 2008; Aripin et al. 2010). This is particularly true for performance indicator disclosure: paradoxically, an increase in the number of indicators could lead to a decline in disclosure quality, if certain modalities of presentations are not fulfilled.

Despite the fact that presentation modalities are essential for the usefulness of performance indicator disclosure (CICA 2002; G100 2003; ASB 2006; IASB 2010), to the knowledge of the authors, only Aripin et al. (2010) have developed a disclosure index based on these aspects. Elaborating on Aripin’s study, we have referred to the recommendations provided by several standard setters and professional bodies to develop our presentation modality index (SEC 2003; CICA 2002; G100 2003; IASB 2005; ASB 2006). According to these recommendations, our index consists of the following six items: Data source; Previous amount; Peer or sector data; Graphs and tables; Purpose and Future target. Table 2 reports a brief definition of each item.

Table 2 Items included in the PRES variable
Full size table

To apply this index, as a first step, we have assessed the six items for each indicator published by each company, following a dichotomous scoring: 0 in case of absence of the item; 1 in case of presence. To calculate the score of each indicator, we have divided the number of recorded items by the total number of items.Footnote 5 As a consequence, the score for each indicator ranges from 0 (none of the disclosure items is present) to 1 (all the disclosure items are present). As this study is not focused on any particular group of users, to minimise the subjectivity of our analysis we have used an un-weighted index, which assumes that each item has the same relevance for users (Beattie et al. 2004).

As a second step, to obtain the score of the variable PRES for a company i, we have determined the arithmetic mean of the scores reported for all the indicators published by the same company. By construction, PRES falls in the range [0, 1]: 0 indicates that none of the indicators published by the company i presents any disclosure item; 1 indicates that each indicator published by the company i shows all the required items.

The following formula synthesises the calculation of variable PRES for the firm i:

$$PRES_{i} = \frac{1}{n} \mathop \sum \limits_{t = 1}^{n} \frac{{\mathop \sum \nolimits_{j = 1}^{6} item_{j} }}{6}$$

where n = the number of indicators disclosed by the firm i; itemj = the item measuring the jth presentation aspect, where j ranges from 1 to 6.

4.3 Data collection

We analysed the performance indicator disclosure included in management reports (Relazione sulla Gestione). This is the section of the annual report affected by the Directive in Italy. We used the manual content analysis technique to investigate all six management reports (2005–2010) of each company. We chose manual content analysis following Linderman (2001) who underlines the limitations of computerised analysis tools when the categorisation procedures are highly complex. This is the case for performance indicator disclosure, as different labels can be used to identify the same indicator, and the items concerning presentation can be treated and displayed in very different ways.

The performance indicators were identified referring to the definition proposed by the ASB: “Key Performance Indicators are quantified measurements by reference to which the development, performance or position of the business of the entity can be measured effectively. They are quantified measurements that reflect the critical success factors of an entity and disclose progress towards achieving a particular objective or objectives” (ASB 2006, p. 8). According to this definition, we identified a performance indicator when:

  • It was explicitly stated as a measure of performance through the use of labels such as: “performance indicator”; “economic measure”; “financial measure”; “financial ratio”; “alternative performance measure”; “non-GAAP measure”.

  • It was presented in tables or graphical representations, often referred to as “Main performance results”; “Highlights”; “Key performance indicators”.

  • It was located in the management report narratives, i.e., in the analysis of the business.

We created two different categories of performance indicators: financial and non-financial indicators, as the Directive explicitly requires the presence of both these categories to be disclosed.

The performance indicators were labelled as financial when they captured financial performance, financial position, or cash-flows, either presented as an absolute value or as an index. According to the classification proposed by Matsumoto et al. (1995), four categories of financial performance were used to classify the financial indicators:

  • growth measures (i.e., Sales growth rate; Total asset growth rate);

  • profitability measures (i.e., Gross margin; EBIT margin; EBITDA margin; ROI);

  • leverage measures (i.e., Debt/Equity; Fixed Assets/Equity; Equity/Total Assets);

  • liquidity and cash flow measures (i.e., Current ratio; Net Working Capital/Sales; Cash Flow/Sales).

The performance indicators were labelled as non-financial when they were not based on conventional accounting figures. The non-financial indicators were classified into four categories, referring to the main multidimensional performance models (Lynch and Cross 1991; Kaplan and Norton 1992; Wright and Keegan 1997):

  • market measures (i.e., Market share, Retention rate, Satisfaction level, Number of new clients, Loyal customers);

  • internal processes measures (i.e., Production capacity, Order backlog, Volumes traded; Percentage of directly operating stores);

  • innovation measures (i.e., New products in the pipeline, Number of new patents registered, Hours of training per employee);

  • environmental and social measures (i.e., Emission reductions rate; Energy savings rate; Employee injury frequency rate).

The collection procedure included multiple phases. First, two coders selected a pre-sample of 10 documents from the original sample and accurately analysed the contents. Each coder reported the indicators collected and the presentation items for each company on a list; then, the two lists were compared. Inconsistencies were resolved through discussion, and the coding instructions were documented in a coding manual. Then, the same coders applied the coding rules to the remainder of the sample. Finally, the reliability was assessed by calculating the agreement ratio. The overall rate of agreement was considered satisfactory, determined as greater than 95 % (Clatworthy and Jones 2003; García-Osma and Guillamón-Saorín 2011). According to our main coding rules, concerning the collection of indicators:

  • Stock market measures were not collected (i.e., the Price to book value or Dividend yield) even if they included financial measures (i.e., the Price to earnings).

  • Mandatory information was not collected. This applies to the Net financial position, the disclosure of which is mandatorily required by the Italian Securities Exchange Commission (Commissione Nazionale per le Società e la Borsa, CONSOB).

  • Measures that simply report figures from the financial statements were excluded (i.e., the labour cost figure or depreciation figure), since they were not regarded as performance indicators.

  • Adjusted or alternative performance financial measures were gathered. According to the ESMA Recommendation on Alternative Performance Measures (ESMA 2005), we defined these as financial measurements that indicate significant adjustments to the line items of the income statement, balance sheet, or cash flow statement. Several formulations indicating alternative measures were taken into consideration, such as “adjusted,” “normalised,” or “recurring” (i.e., “Adjusted EBIT”, “Recurring EBITDA”, and “Normalised earnings”).

  • Measures related to both the overall and segmental performance were included in the analysis. When the same performance measure was repeated for different strategic business areas or geographical areas, it was collected once.

  • When the same measure was presented both as an absolute value and in ratio form (i.e., EBIT and EBIT margin), it was collected once.

5 Results

5.1 Univariate analysis

Table 3 reports the main descriptive statistics for the number of disclosed indicators (QUANT) in the pre-directive period (2005–2007) and the post-directive period (2008–2010) the introduction of the Directive. On average, the number of indicators disclosed has increased, from 5.76 indicators in the pre-directive period to 9.58 indicators in the post-directive period. This increase is statistically significant, as confirmed by the t test. In the period 2008–2010, unlike the previous period, each firm published at least one performance indicator. Furthermore, the limited increase in standard deviation suggests that the growth of indicators in management reports is generalised among firms. This result is in agreement with the previous evidence from UK companies (Black Sun Plc 2007; PwC 2007; Tauringana and Mangena 2009).

Table 3 Descriptive statistics for the QUANT dependent variable
Full size table

To better understand the main types of contents conveyed by firms, Table 4 illustrates the published indicators according to their nature. Table 4, Panel a, reports the descriptive statistics for financial indicators. On average, financial indicators disclosure increased in the post-directive period the adoption of the Directive compared to the pre-directive period, from 5.11 to 8.26. This increase is statistically significant. Table 4, Panel b, shows that firms seem primarily interested in publishing financial indicators that illustrate their profitability performance. The number of profitability indicators rose from 2.52 in the pre-directive period, to 4.15 in the post-directive period. This increase is likely related to the fact that profitability measures are the most widespread financial indicators and many of them can be easily calculated from financial statement figures. Furthermore, the number of both Liquidity/Cash Flow and Leverage indicators shows a statistically significant increase in the post-directive period. However, the extent of this increase is limited compared to the major step exhibited by profitability measures. On the contrary, companies do not improve their indicator disclosure about growth performance after the Directive’s introduction.

Table 4 Variable QUANT by type
Full size table

Table 4, Panel c exhibits the number of non-financial indicators in the two examined periods. On average, companies communicated 0.65 non-financial indicators in the pre-directive period and 1.32 non-financial indicators in the post-directive period. Despite this statistically significant increase, our results show that the presence of non-financial measures in annual reports continue to be limited: on average, companies publish just over one non-financial indicator after the introduction of a mandatory requirement. Table 4, Panel d, documents that all four non-financial categories show an increase that is statistically significant. However, this result indicates that non-financial information is especially devoted to illustrating external aspects, through both Market and Environmental & Social performance indicators, where the latter are explicitly mentioned by the EU Directive. On the other hand, the few indicators concerning Innovation and Internal process document that an accurate explanation of a company’s internal performance is still lacking.

Looking at the presentation issues (PRES), Table 5 exhibits little alignment of disclosure practices with international recommendations. Despite the fact that the average score shows an increase (from 0.36 in the pre-directive period to 0.48 in the post-directive period), overall the level of attention paid by firms to presentation modalities remains poor, as it is far from the optimal level, which is one. The decreasing standard deviation suggests that the differences among firms have diminished over time.

Table 5 Descriptive statistics for the PRES dependent variable
Full size table

Table 6 shows the breakdown of the items included in the variable PRES. These results, which are based on all the indicators published, point out the frequency of each item included in our presentation modality index. Two items show a substantial and statistically significant increase after the introduction of the Directive: Data source and Previous amount. On the other hand, Peer data and Future targets are very poorly disclosed, before and after the introduction of the Directive. This result might be influenced by the fact that this information has higher costs (especially proprietary costs).

Table 6 Breakdown of presentation modalities items
Full size table

For the rest, the increasing presence of Graphs and tables illustrating performance indicators is in line with other empirical evidence showing a growing presence of pictures and graphical representations in annual reports, which are expected to make information easier to find and understand (Nejati 2013). The item Purpose shows an increase that, albeit statistically significant, is not consistent: only 14 out of 100 indicators report an explanation that relates the indicator to the strategy adopted by the company. This result is not surprising, considering that performance indicator disclosure mainly consists of financial indicators that are only indirectly related to a company’s strategy.

5.2 Fixed-effects regression

We estimated a multivariate regression model that takes into account the influence of other factors that previous studies have proved to be related to disclosure practices. A similar analysis was conducted by Brown et al. (1999) to investigate the effects of the Australian Corporations Law Reform Act on both the disclosure quantity and the timeliness of the listed Australian companies and by Owusu-Ansah and Yeoh (2005) to investigate the effects of the Financial Reporting Act on the mandatory disclosure practices of the listed New Zealand companies. To our knowledge, no previous study has used this method to investigate the effects of an intervention that regulates management commentary information.

The panel data consists of company observations over the same six-year period, 3 years before (2005, 2006, 2007) and three years after (2008, 2009, 2010) the introduction of the Directive in Italy. The final sample consists of 450 observations.

While firm disclosure practices can be influenced by several factors, we only control for some important company-specific characteristics that are recognised as being among the most influential in the disclosure literature: size, leverage, industry, profitability and corporate governance. A positive relationship between the company size and the extent of its disclosure is well supported by both the agency theory and the proprietary costs theory (Lang and Lundholm 1993; Wallace et al. 1994; Hossain et al. 1995; Watson et al. 2002; Abdullah and Ku-Ismail 2008). Based on agency theory, a positive relationship between firm leverage and the extent of disclosure can also be predicted (Bradbury 1992; Mitchell et al. 1995; Patton and Zelenka 1997; Giner 1997; Prencipe 2004). Industry is also considered a significant factor in influencing the disclosure practices according to the signalling theory (Watson et al. 2002). As regards company performance, a positive relationship between a company’s results and its disclosure is supported by both signalling theory and agency theory (Watson et al. 2002; Giner 1997), and this relationship has been confirmed by several studies (Penman 1980; Lang and Lundholm 1993; Clarkson et al. 1992, 1994; Kasznik and Lev 1995; Miller 2002). Lastly, in the agency framework, corporate governance mechanisms, such as board composition and incentive plans, are predicted to reduce information asymmetries, thus improving disclosure quality (Eng and Mak 2003; Lim et al. 2007). Empirical findings show that board structure, measured by the proportion of independent directors on the board, positively affects disclosure quality (Leung and Horwitz 2004; Cheng and Courtenay 2006).

A fixed-effects regression was estimated to control for mis-specification problems due to omitted variables, which can bias estimated coefficients (Wooldridge 2010). A dummy variable was included to capture the effect of the change in the regulatory regime introduced by the adoption of the Directive. The fixed-effects model takes into account other firm specific unobservable variables such as management skills and firm culture (Owusu-Ansah and Yeoh 2005).

Two equation models were tested in the analysis:

$${\text{QUANT}}_{\text{it}} = \,\upbeta_{0} + \,\upbeta_{1} {\text{DIREC }} + \,\upbeta_{2} {\text{SIZE}}_{it} + \,\upbeta_{3} {\text{LEV}}_{\text{it}} + \,\upbeta_{4} {\text{PROF}}_{\text{it}} + \,\upbeta_{5} {\text{INDDIR}}_{\text{it}} + \varSigma_{{{\text{s}} = 1\,{\text{to}}\,5}}\upbeta_{\text{s}} {\text{IND}}_{\text{i}} + \,\upalpha_{\text{i}} + \,\upvarepsilon_{\text{it}}$$
(1)
$${\text{PRES}}_{\text{it}} = \,\upbeta_{0} + \,\upbeta_{1} {\text{DIREC }} + \,\upbeta_{2} {\text{SIZE}}_{\text{it}} + \,\upbeta_{3} {\text{LEV}}_{\text{it}} + \,\upbeta_{4} {\text{PROF}}_{\text{it}} + \,\upbeta_{5} {\text{INDDIR}}_{\text{it}} + \varSigma_{{{\text{s}} = 1\,{\text{to}}\,5}}\upbeta_{\text{s}} {\text{IND}}_{\text{i}} + \,\upalpha_{\text{i}} + \,\upvarepsilon_{\text{it}}$$
(2)

where i denotes firms and t represents years. The error term is composed of fixed firm and time components, and a residual error that we assume is independently distributed across i and t.

The variables included in the model are defined as follows:

  • QUANT is the number of indicators collected from the company’s management report.

  • PRES is the quality score of the presentation modalities.

  • DIREC is a dummy variable that takes the value 0 if the observation is drawn from the pre-directive period (2005, 2006, 2007) and 1 for the post-directive period (2008, 2009, 2010).

  • SIZE is the natural log of the firm’s total sales at the end of the fiscal year.

  • LEV is the firm’s debt-equity ratio at the end of the fiscal year.

  • PROF is the firm’s return on total assets at the end of the fiscal year.

  • INDDIR is the percentage of independent directors on the board at the end of the fiscal year.

  • IND is a vector of dummies representing the industry grouping (Consumer goods; Consumer services; Energy, Chemicals and Mining; Utilities; ICT; Industrial goods and services).

The variables SIZE, LEV and PROF are extracted from the AIDA Database (Bureau van Dijk information provider). INDDIR data is collected from companies’ annual Report on Corporate Governance.

The descriptive statistics for the control variables are reported in Table 7. The variability in the descriptive statistics is not considered to be high enough to represent a serious concern for the interpretation of the results.

Table 7 Descriptive statistics for independent measures
Full size table

The regression is estimated after controlling the pair-wise Pearson correlation coefficients of the variables reported in Table 8. The results show that the two dependent variables (QUANT and PRES) are positively correlated; this might indicate that the level of attention a firm pays to the modalities of disclosure increases with the number of indicators published. As the correlation coefficients among the control variables are relatively low, they are unlikely to pose a serious problem to the interpretation of our results (Hossain et al. 1995).

Table 8 Pearson correlation matrix
Full size table

Both the fixed-effects model and the random-effects model have been run in our regression analyses. The main difference between the two models is their assumptions about individual effects. The fixed-effects model arises from the assumption that the omitted effects in the model are correlated with the variables included in the model. Thus, the fixed-effects formulation implies that differences across groups can be captured in differences in the constant term. In contrast, if the individual effects are strictly uncorrelated with the other variables in the model, then it might be appropriate to model the individual specific constant terms as randomly distributed across cross-sectional units; this is the formulation proposed by the random-effects model (Wooldridge 2010).

Table 9 reports the estimates of Eq. (1), with QUANT as the dependent variable.

Table 9 Regression results for Eq. (1 ) with the disclosure measured as QUANT
Full size table

The estimation for the DIRECT coefficient is positive and significant. This result confirms our univariate analysis. After the introduction of the Directive, the number of indicators published by a firm is higher than in the period before its introduction. No major difference is observed between the estimations calculated with the random-effects model compared to those estimated with the fixed-effects model. The Hausman test rejects the null hypothesis that the coefficients estimated by the random-effects model are the same as the ones estimated by the fixed-effects model (Wooldridge 2010). Thus, the estimations produced by the fixed-effects model are preferred.

Among the control variables, SIZE is statistically significant, and its positive sign is aligned with previous evidence (Watson et al. 2002). In contrast, neither firm profitability (PROF) nor leverage (LEV) exhibit any relationship with the number of indicators published. These results seem to suggest that firm disclosure policy is not managed on the basis of short-term performance. Considering that the indicators communicated are mainly financial measures derived from the accounting figures, the absence of any difference among industries is not surprising. Moreover, the lack of any relationship between the amount of indicator disclosure and the percentage of independent directors on the board is aligned with Allegrini and Greco’s (2013) findings on voluntary disclosure provided by listed Italian companies. As highlighted by the authors, this result is likely to be influenced by the peculiarity of the Italian agency setting, which is characterised by large controlling shareholders and an overlap between ownership and management.

Table 10 reports the results of the regression analysis for Eq. (2), where PRES is used as a dependent variable.

Table 10 Regression results for Eq. (2 ) with the disclosure measured as PRES
Full size table

The coefficient for the dummy variable DIRECT is positive and statistically significant. Thus, the regression analysis confirms that presentation modalities have improved in the post-Directive period, in line with the results obtained in the univariate analysis. The Hausman test indicates that the fixed-effects model is also preferred in this analysis.

6 Conclusions and limitations

This study makes a contribution to the debate about the regulation of management commentary. Specifically, we aim to understand whether a regulation with a low level of specification is effective in regulating management commentary information. Focusing on a specific relevant piece of information, that is performance indicators, we investigate whether the requirement included in the Modernisation Directive (2003/51/EC) concerning the communication of performance indicators in the annual report has increased the quality of disclosure practices.

Our results for the Italian context document that the implementation of the Directive is associated with an increase in the number of indicators and an improvement in the modalities of presentation. We cannot interpret this association in terms of causality. However, the absence of any influence of the event under investigation seems unlikely, since our analysis involves the period immediately before and after the introduction of the Directive. Moreover, we can reasonably exclude that our results are only driven by a contingent event, like the financial crisis that, since 2008, has effected the world economy. Even though our sample companies show decreasing profitability and higher leverage in the post-Directive period, no any statistically significant association is documented between these variables and disclosure practices.

Our results are aligned with previous evidence based on the UK (Tauringana and Mangena 2009). Moreover, we found that larger companies communicate more indicators than other firms, in line with Watson et al. (2002), and they pay greater attention to the modalities of presentation. For the rest, neither industry nor board-independence show any significant influence on disclosure practices.

At first sight, these results seem to document that a mandatory intervention, although with a loose level of specification, such as that introduced by the Directive, is associated with an increase in disclosure practices. However, it is questionable whether such an improvement has substantially lead to high quality disclosure practices. Indeed, looking at companies’ disclosure after the introduction of the Directive, it seems that companies are oriented to minimise their efforts to comply with the new requirements, focusing on common financial indicators and paying low attention to disclosure presentation modalities. The increase in the number of indicators is predominantly related to financial measures and among these, to measures illustrating profitability. On the other hand, non-financial indicators continue to be quite substantially ignored, especially those related to company strategy. Looking at presentation modalities, the improvement documented by our presentation modality index does not testify to the overall quality of disclosure practices. In fact, this improvement basically deals with two aspects: the reconciliation with financial statement and the amount for the previous year. Other pivotal aspects, such as the disclosure of peer data and future targets continue to be overlooked. In conclusion, our results confirm Beattie and McInness’ (2006) concern about the risk associated with a regulation approach for management commentary which leaves managers with high discretion.

Bearing in mind the need to guarantee a sufficient level of discretion over performance indicator disclosure, these results offer several points for regulators to consider. First, the Directive regulates performance indicator disclosure on a stand-alone basis, which fails to clearly specify its purpose and objectives. Additionally, the Directive addresses both financial and non-financial indicators, although they show different characteristics. While financial indicators can be autonomously calculated by users, the non-financial measures are typically associated with higher proprietary costs. The absence of a broad legislative disclosure framework on performance indicator allows companies to be compliant by disclosing only common financial indicators and non-financial indicators with low proprietary costs. This fact could explain why, in recent years, a few frameworks have been developed to address management commentary disclosure, for instance the IFRS Practice Statement Management Commentary (IASB 2010), the IIRC Integrated Reporting project (IIRC 2013), and the OECD report on Intellectual Assets and Value Creation: Implications for Corporate Reporting (OECD 2006). All these reports ascribe a central role to performance indicator disclosure.

Finally, any regulation promoting the inclusion of performance indicator disclosure should be complemented by a list of requirements that guarantee a high and uniform standard in presentation modalities among firms. The absence of adequate presentation increases the risk of low comparability or even misleading interpretations by users, especially when firms disclose measures that are non-GAAP indicators (IOSCO 2014; ESMA 2014).

This study is limited in a number of ways. The first limitation is the sample. We have chosen to study Italian companies to better isolate the effect of the Directive, but other countries could be studied to demonstrate the broader applicability of our results. Another limitation is that an un-weighted index has been used to measure the presentation modalities, as it is not possible to rank the items included in the index based on their importance to users. The same point can be made regarding the information quantity measurement; it is not possible to list the indicators based on their relative informative value for users. For this reason, we assigned the same weight to each indicator. Thus, further research could be conducted on user preferences for the informative content of each indicator and the importance of the various aspects of disclosure quality.