Assessing and Improving the Quality of Sustainability Reports: The Auditors’ Perspective

Abstract

This article presents, an analysis of the opinions of assurance providers regarding the quality and the limitations of sustainability reports and their recommendations to improve them using the Global Reporting Initiative (GRI) as a framework. The qualitative content analysis of 301 assurance statements for sustainability reports from mining and energy companies provides a comprehensive view of the main outcomes of the assurance process, including its limitations, the application of the GRI principles and suggestions for improving sustainability reports. Taking into account the perceptions of practitioners a priori well informed on the quality of sustainability reports—namely assurance providers—this paper complements the current literature on sustainability reporting and its assurance, including critical approaches that question the reliability of sustainability reports, stakeholder engagement and the accountability of reporting practices. This study contributes to the debates surrounding the quality of sustainability reports, the added value of assurance statements and the ethical issues underlying the assurance process. It also contains important practical implications for auditors, standardization organizations and stakeholders.

Introduction

Sustainability reporting has become a mainstream practice in the communication of corporate commitment to and performance on sustainability issues (Fonseca et al. 2014; Hahn and Kühnen 2013; Junior et al. 2014; Perego and Kolk 2012). In 2015, more than 90% of the top 250 largest companies worldwide published a sustainability report, most of them using the Global Reporting Initiative (GRI) framework, which has become the reference model in this area (King and Bartels 2015). Nevertheless, the credibility and reliability of sustainability reporting have been widely criticized in the literature (Cho et al. 2015; Gray 2010; Milne et al. 2006; Moneva et al. 2006). To address these criticisms and instill confidence in corporate reporting, an increasing number of reports are being verified by assurance providers, which can be either accounting or consulting firms.

It is assumed that the assurance process demonstrates that sustainability reports and their underlying reporting practices have been verified by independent auditors—also called assurance providers—who share their conclusions on the quality and reliability of the information disclosed (Dando and Swift 2003; King and Bartels 2015; Rasche and Esser 2006). Assurance statements also frequently comment on the limitations of the report and make recommendations as to how the company might improve its reporting practices (Ball et al. 2000; Deegan et al. 2006; Gürtürk and Hahn 2016; Perego 2009). Although the independence of assurance providers and the quality of assurance statements have been criticized in the literature (Ball et al. 2000; Fonseca 2010; Manetti and Toccafondi 2012; O’Dwyer and Owen 2005, 2007; Park and Brorson 2005), most studies have highlighted the relevance and importance of the assurance process in improving the credibility and reliability of sustainability reporting (Manetti and Toccafondi 2012; Moroney et al. 2012; Simnett et al. 2009). This literature has contributed to a greater understanding of the role of assurance providers in the legitimation of corporate sustainability reporting. Nevertheless, the opinions of assurance providers on the quality, limitations and improvement of sustainability reports have been largely overlooked.

This article presents an analysis, based on the content analysis of a large sample of assurance statements, of the opinions of assurance providers regarding the quality, the limitations and the recommendations to improve GRI-based sustainability reports.

This study makes three main contributions to the sustainability reporting literature. First, it contributes to the debates on the quality of sustainability reporting and stakeholder involvement. The quality of sustainability reports can be defined as the transparency of information and compliance with basic reporting principles such as materiality, stakeholder inclusiveness, completeness, comparability, balance, accuracy and reliability (GRI 2013a). The literature on this issue has mainly focused on the analysis of sustainability reports by researchers themselves rather than the analysis of other experts, practitioners and interested parties. As a result, the perceptions of stakeholders, including assurance providers, have not received sufficient attention (Adams and Evans 2004; Ball et al. 2000; Manetti and Toccafondi 2012; O’Dwyer and Owen 2005, 2007). Although the independence of assurance providers is debatable, they are assumed to be relatively well informed about corporate reporting practices and to publish statements that are as rigorous and reliable as possible (Gilbert and Rasche 2008; GRI 2013a; Iansen-Rogers and Oelschlaegel 2005; King and Bartels 2015). It can therefore be assumed that the analysis of a large number of assurance statements will provide a comprehensive overview on the quality of reports that could complement the current literature on this issue. Second, although the recommendations of auditors are focused on specific reports, the analysis of a large array of statements provides a more extensive view on the avenues for improvement of sustainability reporting in general. These recommendations also indirectly reflect the main limitations of sustainability reporting, and their analysis could contribute to the critical literature in this area. Likewise, the critical content analysis of statements can raise ethical issues concerning the managerial capture and lack of independence of the assurance process, which have been debated in the literature (Adams and Evans 2004; Hummel et al. 2017; Michelon et al. 2015). Finally, by investigating the outcomes of assurance statements through the lens of the GRI principles for the content and quality of reports, it is possible to shed further light on reporting organizations’ compliance with these principles and the extent to which they are seriously taken into account by assurance providers themselves. How the GRI principles for the content and quality of report are integrated by assurance providers can be indicative of the quality of the verification process, which is assumed to be based on relevant criteria and standards (AccountAbility 2008; GRI 2013a).

The remainder of the paper is organized as follows. First, the literature on the assurance of sustainability reports and its contribution to the reliability of reporting practices is described. Second, the method for the qualitative content analysis of assurance statements is explained. Third, the main findings of the study are detailed in terms of outcomes of the assurance process, statements on the content and quality of information, report limitations and suggestions for improvement. Finally, the main contributions of the paper, its practical implications and avenues for future research are examined.

The Assurance of Sustainability Reports

Instilling Confidence in Sustainability Reporting

Sustainability reporting has been defined as “the practice of measuring, disclosing, and being accountable to internal and external stakeholders for organizational performance towards the goal of sustainable development” (GRI 2006, p. 3). Nevertheless, to be credible and useful for stakeholders, the information disclosed must be reliable, appropriately presented and clearly follow defined rules (Adams and Evans 2004; Boiral 2013; Fonseca 2010; Manetti and Becatti 2009). To achieve this goal, the GRI proposes to follow two sets of principles: one set defining the content of reports and the other related to their quality (GRI 2006). The principles for report content focus on the consistency between the information disclosed and the organizational context, particularly in terms of the organization’s activity and its stakeholders’ expectations. They include stakeholder inclusiveness (identification of stakeholders and response to their expectations), the sustainability context (presentation of information in the broader context of sustainability), materiality (relevance of topics covered by the report with regard to the organization’s impacts and stakeholders decisions) and completeness (reports should release sufficient information to assess the organization’s performance). The principles for report quality focus on the presentation and transparency of information: balance (information should not be focused on positive aspects only), comparability (information should be comparable over time and between organizations), accuracy (information should be sufficiently detailed and valid to assess performance), timeliness (regularity and recentness of information), clarity (readability and understandability of reports) and reliability (information can be depended on).

Although the development of the GRI framework in sustainability reporting is generally considered to have improved the quality of information (Deegan 2002; Gilbert and Rasche 2008; King and Bartels 2015), the lack of reliability and transparency of sustainability reports have been increasingly criticized in the literature (e.g., Cho et al. 2015; Milne et al. 2006; Unerman et al. 2007). Some studies have highlighted the managerial capture of information and the lack of involvement of stakeholders in the reporting process (Ball et al. 2000; Owen et al. 2000; Smith et al. 2011). The successful and insubstantial rhetoric of many reports has also been criticized (Cho et al. 2015; Fonseca 2010; Livesey and Kearins 2002). From this perspective, sustainability reports appear to be marketing tool intended to positively influence the perceptions of stakeholders rather than be reliable source of information (Cho et al. 2012; Talbot and Boiral 2015). The critical literature in this area has also shown the disconnection between the information disclosed and the significant sustainability challenges faced by companies (Boiral 2013; Moneva et al. 2006). Moreover, the information disclosed in GRI reports rarely seems to comply fully with the requirements and technical protocols of this reporting framework (Boiral and Henri 2017; Talbot and Boiral 2015).

These criticisms tend to undermine the credibility of sustainability reports and their usefulness for stakeholders. A third-party assurance process is generally considered to be the main response to restore or enhance public confidence in these reports (De Beelde and Tuybens 2015; Iansen-Rogers and Oelschlaegel 2005; Kolk and Perego 2010; Manetti and Becatti 2009; Park and Brorson 2005). Engagement with assurance can be defined as “an engagement in which an assurance provider evaluates and expresses a conclusion on an organisation’s public disclosure about its performance as well as underlying systems, data and processes against suitable criteria and standards in order to increase the credibility of the information for the intended audience” (AccountAbility 2008, p. 23). The rapid expansion of the assurance process reflects the need to increase the credibility of the disclosed information in the eyes of stakeholders. In 2015, nearly two-thirds of the reports from the 250 largest companies worldwide were verified by external auditors as compared to 30% in 2005 (King and Bartels 2015). This expansion reflects the development of an “audit society” (Power 1997a, b, 2003) in which the auditing and verification practices prevalent in the field of accounting have gradually permeated other areas, including sustainability reporting. The predominance of accountancy organizations, which represent nearly two-thirds of sustainability report assurance providers (De Beelde and Tuybens 2015; King and Bartels 2015; Moroney et al. 2012), also reflects the development of this “audit society” in that it is dominated by the practices and institutional arrangements that were first established and are now widespread in the financial area (Boiral and Gendron 2011; O’Dwyer et al. 2011; Wong and Millington 2014). Although the assurance of sustainability reports is based on specific standards—in particular AA1000 and ISAE 3000—these standards are themselves based, to a large extent, on general auditing principles (i.e., independence and impartiality of auditors, definition of the scope and different levels of assurance engagement and the organization of assurance statements). While these principles are applied to various areas, they predominate in accounting and financial auditing.

Overall, the application of these auditing principles is assumed to instill confidence in sustainability reports and to improve their reliability (Dando and Swift 2003; Gürtürk and Hahn 2016; Hodge et al. 2009; Kolk and Perego 2010; O’Dwyer and Owen 2005; Perego and Kolk 2012). Nevertheless, this optimistic perspective is debated in the literature.

The Benefits and Controversies of the Assurance Process

Although the benefits and implications of third-party assurance remain controversial, most studies agree that the verification of sustainability reports by independent external auditors is desirable or even necessary (Dando and Swift 2003; Manetti and Becatti 2009; Moroney et al. 2012; Park and Brorson 2005). First, by providing an assessment of corporate disclosure on complex issues where reliable information is difficult to obtain, the assurance process is assumed to reduce uncertainty and information asymmetry between managers and stakeholders (Gürtürk and Hahn 2016; Moroney et al. 2012; O’Dwyer et al. 2011). Second, the audit process can have a disciplinary effect and encourage companies to improve their sustainability practices, including information disclosure (GRI 2013b; Park and Brorson 2005). From this perspective, it can be assumed that third-party verification fosters the integration of the GRI principles defining report content and quality. This integration can explain the positive effect of assurance on the quality of reports and the promotion of a self-regulatory approach, particularly in geographic regions characterized by weak institutional pressures (Kolk and Perego 2010; Perego 2009). Third, the assurance process has been found to indirectly enhance stakeholder consultation (Manetti and Toccafondi 2012; Perego and Kolk 2012). Audits may entail interviews with concerned parties, in particular employees and, to a lesser extent, external stakeholders. Moreover, the internalization of GRI reporting principles should encourage stakeholder responsiveness. As a result, external assurance can contribute to stakeholder accountability. Although the beneficial impact of this process on internal practices remains understudied, most studies agree that the assurance process tends to improve confidence in sustainability reports and, more generally, increase an organization’s social legitimacy (Hodge et al. 2009; Manetti and Toccafondi 2012; Moroney et al. 2012; Simnett et al. 2009).

Nevertheless, according to the critical literature on sustainability reporting assurance, such confidence is questionable. First, the independence of assurance providers is debatable given the underlying commercial relationships between auditors and companies (Ball et al. 2000; Owen et al. 2000; Perego and Kolk 2012). These commercial relationships tend to encourage auditors to be rather uncritical and to not seriously question the reliability of information released by companies. Overall, a commercial relationship can compromise the professional skepticism and impartiality that should characterize third-party assurance (Boiral and Gendron 2011). Second, the managerial capture of information and the lack of involvement of stakeholders in the assurance process have been criticized (Adams and Evans 2004; Ball et al. 2000; Hummel et al. 2017; Michelon et al. 2015; Owen et al. 2000). This managerial capture is related to the control of managers over the information disclosed in reports and the dependence of auditors on the information released by companies. Third, the professionalization of assurance providers and rigor of practices in this area seem questionable. Unlike financial auditing, the assurance of sustainability reports is not necessarily based on well-recognized standards and well-established professional bodies with clear requirements in terms of training and experience. Some audits may be conducted quite superficially with the intention of providing a kind of commercial certificate mostly intended to increase the social legitimacy of reporting companies (Ball et al. 2000; Park and Brorson 2005; Smith et al. 2011). Such behavior is in line with the legitimacy theory, which claims that many companies subjected to strong institutional pressures tend to superficially adopt new practices in order to improve their social legitimacy (Meyer and Rowan 1977; Michelon et al. 2015; Smith et al. 2011). By adopting similar assurance processes based on recognized institutional arrangements arising from the area of accounting and reflecting the “audit society” (Power 1997a, b, 2003), reporting organizations tend to become isomorphic and better able to respond to social expectations for more accountability in sustainability reporting (Boiral and Gendron 2011; Gürtürk and Hahn 2016; Perego and Kolk 2012).

Despite controversies over the assurance process, most critical studies do not directly question the importance and legitimacy of third-party verification of sustainability reports. Some of these critical studies are even optimistic about the trend toward more robust verification practices, increased dialogue with stakeholders and improvements in the quality of reports arising from more rigorous assurance processes (Ball et al. 2000; Manetti and Toccafondi 2012; O’Dwyer et al. 2011). Such improvements could be partly related to auditors’ assessment of reports and their recommendations to improve their quality.

Assessing the Quality of Sustainability Reports Through Assurance Statements

The main outcomes of the assurance process are presented in a publicly available report or assurance statement generally structured around similar themes (i.e., scope of the verification process, level of assurance, methods and criteria used by auditors, limitations, conclusions and recommendations). Although the main objective of the assurance process is not to highlight the limitations of the reports or to propose avenues for improvement, these aspects are frequently addressed in the statements which, according to the assurance standard AA1000, are based on “a set of findings, conclusions and recommendations” (AccountAbility 2008, p. 21). Likewise, according to the GRI, assurance statements should indicate “whether the assured information is fairly presented, free of material misstatements and reported in accordance with reporting criteria” (GRI 2013b, p. 10). Moreover, the statements can include a “comment on any noteworthy limitations” (op.cit.p.9) and a “summary of recommendations for further action or attention” (op.cit.p.10). As highlighted by Gürtürk and Hahn (2016), “recommendations can also be delivered to the management directly without including them in the public assurance statement.” However, the information communicated directly to managers outside assurance statements is generally confidential and not publicly available. Since nearly half of assurance statements seems to contain specific recommendations for the reporting company (Gürtürk and Hahn 2016), the analysis of this type of information through a large sample of statements seems relevant to shed further light on the quality of sustainability reporting in the eyes of auditors.

Surprisingly, such analysis has been largely overlooked in the literature on sustainability reporting, which relies on the observations of researchers rather than assurance providers (Cho et al. 2015; Fernandez-Feijoo et al. 2014; Fonseca 2010; Gray 2006; Michelon et al. 2015). Similarly, the literature on assurance statements has not focused on the limitations of reports and recommendations for improvement, although the existence of these elements has been briefly mentioned in a few studies (Ball et al. 2000; Deegan et al. 2006; Gürtürk and Hahn 2016; Junior et al. 2014; Kolk and Perego 2010; Manetti and Becatti 2009). For example, according to Ball et al. (2000), recommendations are an integral part of assurance statements and should be taken into consideration, although they tend to reflect a “managerial turn” in the verification practice. This perspective is in line with the GRI and AA1000 standards, which encourage assurance providers to formulate recommendations. Conversely, according to Manetti and Becatti (2009, p. 296), “the aim of the assurance services should be only to express a professional opinion on the reliability of the information given in the social report, refraining from giving advice to the management.” However categorical, this position is more in line with the accounting perspective conveyed by the ISAE3000 standard, which encourages assurance providers to remain neutral and to clearly separate commentaries and recommendations from the rest of the statement (IAASB 2011; Iansen-Rogers and Oelschlaegel 2005). Although this issue remains understudied, a few studies have confirmed that accounting firms involved in the assurance process are less inclined to formulate recommendations than consulting firms (Deegan et al. 2006; Perego 2009).

Whatever the reasons explaining the differences in the content of statements, the analysis of recommendations formulated by assurance providers and their opinions on the limitations of sustainability reports are relevant for at least two reasons. First, such an analysis could contribute to the debates on the quality and reliability of sustainability reports from a different perspective—and one that has been clearly overlooked in the literature. The current literature remains essentially focused on the content analysis of sustainability reports and rarely involves interviews inside organizations or the perceptions of stakeholders (Manetti and Toccafondi 2012; O’Dwyer et al. 2011; O’Dwyer and Owen 2007). Interestingly, although they largely depend on the information released by organizations, assurance providers can collect data from various sources—including interviews and on-site visits—to verify the quality of sustainability reports (Gürtürk and Hahn 2016; Iansen-Rogers and Oelschlaegel 2005; Manetti and Toccafondi 2012; Park and Brorson 2005). As a result, one can assume that some assurance statements provide relevant information on the quality of reports and avenues for improvement. Second, this type of study could indirectly contribute to the critical literature on the managerial capture of sustainability reports and the assurance process (Boiral 2013; Hummel et al. 2017; Jones and Solomon 2010; Michelon et al. 2015; Smith et al. 2011). If managerial capture is predominant, it should be reflected by an absence of, or lack of substance in, comments from assurance providers on the limitations of reports and related recommendations for improvement. Otherwise, one can assume that the statements tend to reflect, to some extent, the professional skepticism that should, in principle, underlie the assurance process (Fernandez-Feijoo et al. 2014; GRI 2013b; King and Bartels 2015; Manetti and Becatti 2009). Obviously, the conclusions and the language adopted by assurance providers are expected to be shaped by a political correctness and optimism that characterizes auditor–client relationships (Dogui et al. 2013; O’Dwyer and Owen 2007; Park and Brorson 2005). But to presume that all professionals are necessarily overwhelmed by this relationship and therefore cannot express some degree of reflexivity or critical sense seems overly simplistic. It seems more reasonable to assume that such reflexivity, skepticism and critical sense do permeate, to some extent, some assurance statements and that this information can be relevant in the analysis of the quality of sustainability reports.

Methods

The objective of this study is to analyze the opinions of assurance providers regarding the quality, the limitations and the recommendations to improve GRI-based sustainability reports in the mining and energy sectors. The focus on the opinions expressed in assurance statements with respect to the reliability and transparency of sustainability reports requires a qualitative-exploratory research design. More specifically, regarding the data analysis methodology, qualitative content analysis of statements was used. Qualitative content analysis can be defined as “a research method for the subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns” (Hsieh and Shannon 2005, p. 1278).

Data Collection

The study focused on sustainability reports from the mining and energy sectors published between 2006 and 2013 with an A + application level of the G3 GRI framework. The focus on mining and energy sectors is justified by the sustainability impacts of companies in this area and the intensity of institutional pressures on such industries (Boiral 2013; Fonseca 2010; Fonseca et al. 2014; Hilson and Murck 2000). These impacts and pressures reinforce the need for third-party certification of sustainability reports to enhance their credibility in the eyes of stakeholders (Fonseca et al. 2014; Gürtürk and Hahn 2016; Manetti and Toccafondi 2012; O’Dwyer et al. 2011). To improve the homogeneity of the sample, the study focused on G3 GRI reports with the highest application level (A+). The GRI G3 version was launched in 2006 and was used until 2013–2014.¹ As a result, the sample of this study included most GRI G3 reports with the A+ application level published in the mining and energy sectors before the introduction of the G4 version. The selection of reports was based on the GRI search engine,² which offers a wide range of possibilities for searching, selecting and sorting sustainability reports, in terms of sector of activity, year of publication and GRI version. To facilitate data analysis, all reports selected were in English and included a statement from assurance providers. At the end of the selection process, 138 reports from the mining sector and 163 from the energy sector were obtained (see Table 1).

Table 1 Sustainability report sample distribution by year and sector

Sustainability reports were then clustered geographically based on the country where the assurance engagement took place, which usually corresponds to the country where the head office of the company producing the report is located. With regards to the distribution by continent of assured sustainability reports (see Table 2), the presence of companies from Asia and Europe has to be emphasized, as they account for almost 60% of the total sample. The higher relevance of these geographical areas is consistent with the worldwide dissemination of GRI reporting reported in the literature (e.g., Alonso-Almeida et al. 2014). Regarding the distribution of the different assurance providers for the analyzed sample, 62% of the statements were provided by accounting firms while 33% were provided by consulting firms, with no significant statistical differences between the two sectors of activity (see Table 3).

Table 2 Sustainability report sample distribution by continent and sector

Table 3 Distribution of assurance providers

Data Analysis

Qualitative content analysis techniques seek to interpret the content of text data through the systematic classification process of coding and identifying themes or patterns (Hsieh and Shannon 2005). Regarding the codification process, the established protocol closely followed the qualitative content analysis method suggested by Mayring (2014), who proposes procedures of inductive category development where categories are seen as tentative and revised step by step. The categorization process was conducted through the qualitative analysis software QDA Miner. First, the assurance statements were extracted from sustainability reports and transferred into the QDA Miner software. Second, a categorization grid reflecting the objectives and outcomes of the study was developed and used for data analysis. In line with the qualitative content analysis approach (Cho and Lee 2014; Mayring 2014; Hsieh and Shannon 2005), this categorization grid was reorganized during the data analysis process to better take into account issues relevant to the study that emerged in the analysis process. The QDA Miner software facilitated the creation, merging or subdivision of categories reflecting the main findings. The categorization process was independently conducted by two coders. To ensure the reliability of this process, each category was clearly defined and discussed with the two coders (Miles and Huberman 1994). The double blind coding of all transcripts made it possible to reduce possible bias related to different interpretations of the data collected and to improve the standardization of the categorization process (Thomas 2006). Regular meetings between the coders and the researchers involved in this study helped to refine the categorization grid and to assess the relevance of the creation of new codes. Although qualitative approaches are not suited to quantification (Gephart 2004), certain tendencies or frequencies were measured when relevant. In most cases, the quantification of data was conducted through the creation of subcategories and the measurement of the proportion of statements covered by those categories. For example, a specific category on the improvements observed over time by assurance providers in the quality of reports was created and was used to categorize 24% of statements. The QDA Miner software facilitated the measurement of frequencies associated with different categories and subcategories. At the end of the categorization process, the results of the two coders were analyzed, compared and summarized in separate files. There were no significant differences in the results of the categorizations process from the two coders. Third, the most relevant categories were structured around five meta-categories in line with the objectives of this study. Table 4 summarizes the categorization tree used in the study. All in all, 31 main categories were used in this study. In line with the categorization process of qualitative content analysis (Cho and Lee 2014; Mayring 2014; Hsieh and Shannon 2005), some of those categories were subdivided into subcategories for deeper analysis of specific issues, including the measurement of certain tendencies.

Table 4 Categorization tree: main meta-categories and categories identified

Four, relevant and representative passages were selected from the five meta-categories to illustrate the main findings.

Findings

The Main Outcomes of the Assurance Process

Assurance statements may cover various themes, including the criteria for report presentation, the objectives of the assurance process, the scope of verification and information on assurance providers. All statements investigated also contained a conclusion describing the main outcomes of the assurance process and the opinions of assurance providers on the quality of the sustainability report. Although the conclusions are formulated in measured and cautious terms, they are essentially intended to reassure stakeholders about the reliability of the sustainability report. This reassuring rhetoric takes two main approaches. The most common approach is to highlight the absence of major or material misstatements, errors or inaccuracies in the sustainability report. This negative phrasing is reflected in 56% of all statements and is essentially used by accounting firms. By highlighting the absence of problems rather than the quality or reliability of reports, negative phrasing is the most cautious way to reassure stakeholders:

Nothing has come to our attention to cause us to believe that the Freeport-McMoRan Copper & Gold Inc. self-declared application level of A + , in relation to its reporting against the GRI G3 Sustainability Reporting Guidelines, is materially misstated. (Freeport-MacMoRan Copper 2011, p. 40)

Nothing has come to our attention that causes us to believe that the sustainability data has not been properly collated from the information reported by sites. (MOL Group 2012, p. 225)

In 24% of statements—essentially those from consulting firms—assurance providers are more positive and highlight the quality of reports in terms of accuracy, balance, clarity, reliability or fairness. Although this phrasing seems less cautious, it generally remains rather elusive and does not compromise assurance providers. Moreover, the negative and positive phrasings of conclusions are not necessarily mutually exclusive and are used jointly in 20% of statements to describe different aspects of sustainability reports:

According to the audit scope, the information and data submitted in the report were evaluated as accurate and free from significant errors or misrepresentations, accessible and understandable to the stakeholders. (Grupo CPFL 2012, p. 255)

To support their conclusions, 69% of statements refer to the application of the GRI principles. Nevertheless, the extent to which the reports comply with those principles is rarely detailed and most statements essentially confirm very briefly that the reports meet the GRI requirements. Overall, the rhetoric used by assurance providers remains optimistic, although certain limitations are also mentioned. This optimism is reflected in the description of improvements observed over time in the quality of reports. Those improvements are mentioned in 24% of reports and imply that assurance providers have analyzed several consecutive reports. More than half of the comments on improvements concern stakeholder engagement and responsiveness. Although most of these comments remain general and unspecific, some are quite informative and show the efforts of companies to improve stakeholder relationships:

Banarra identified strong support within LGL for stakeholder engagement and inclusivity. Site-based engagement appears particularly strong, with significant examples of site-based engagement mechanisms including Lihir Island’s weekly meeting with the Landowners Association and Bonikro’s bi-monthly meeting with chief’s [sic] from the local villages through the Community Liaison Committee. (Lihir Gold 2010, p. 119)

In 2009, Newmont Mining revised their corporate standard on stakeholder engagement, and created new corporate standards with explicit and prescriptive guidance on stakeholders mapping and managing expectations and commitments. The implementation of these new standards at each site will help the company improve the uniformity of the company’s social responsibility processes. (Newmont Mining 2010, p. 63)

Assessing the Content of Reports

According to the statements analyzed, the assessment of the content of reports is quite heterogeneous (see Table 5).

Table 5 Assessment of the content of sustainability reports (% of statements)

The most frequently assessed principle is the materiality of reports, covered in roughly two-thirds of statements (see Table 5). The emphasis on this principle seems justified by the importance of materiality in evaluating whether the indicators and information contained in the reports reflect the organization’s main impacts and, more generally, stakeholders’ concerns. In certain statements, the verification of materiality seems to shape the whole verification process and be the main focus of the audit:

Did the performance indicators, statements and claims reported reflect BHP Billiton’s significant economic, environmental and social impacts? Were internal and external factors considered in determining the performance indicators, statements and claims included in the report? Does reporting include information on performance? (BHP Billiton 2010, p. 76)

How the materiality was verified in practical terms, however, remains unclear in most statements. Surprisingly, with the exception of two reports that briefly mention the GRI application-level check (OMV 2013; Verbund 2013), no statement refers to the tests or checklists proposed by the GRI to verify the compliance of reports with the principles detailed in this guideline, including the materiality of reports. The same remark applies to the other GRI principles, the verification methods for which are rarely explained.

Stakeholder inclusiveness and responsiveness is explicitly covered in roughly 42% of all reports (see Table 5). This proportion seems relatively low considering that the raison d’être of assurance statements is to improve the credibility of sustainability reports in the eyes of stakeholders and to better respond to their concerns (GRI 2006; Gürtürk and Hahn 2016; Manetti and Toccafondi 2012; O’Dwyer et al. 2011). Although most statements indicate that interviews were conducted and sites were visited during the verification process, the scope of this data collection and whether it concerns stakeholder issues is unclear. Overall, the assessment process of stakeholder responsiveness is not substantiated. It seems to mostly rely on the information disclosed by the reporting company and focus on internal procedures rather than the analysis of the concerns actually expressed by the interested parties:

The Company is engaged in dialogue with five stakeholders through different channels. The material issues emerging from the dialogue were collected and prioritised based on inputs from stakeholders, and the results are reflected in the Report. (Sesa Goa 2011, p. 70)

As part of the yearly stakeholder consultation process, during 2012 Enagas carried out an online survey and a Focus Group with its key stakeholders. In both processes they were asked to evaluate various of the Company’s CSR issues. (Enagas S.A. 2013, p. 408)

The principle of completeness is covered in 38% of statements. This relatively weak coverage can be partly explained by the difficulty of assessing whether the information disclosed by companies is sufficient, reasonable and does not omit material information. Although certain statements suggest that the information disclosed on specific issues could be more complete, the degree of completeness that can be expected from sustainability reports is unclear. As a result, the principle of completeness is essentially described through negative phrasing reflecting the absence of observed misstatements:

Based on RSK’s review and within the reporting boundary defined by MASDAR, RSK is not aware that the Report omits relevant information that would significantly influence stakeholder assessments or decisions or that reflect significant economic, environmental and social impacts (MASDAR 2012, p. 132).

Finally, the sustainability context is covered in 10% of all statements and in only 6% of those from the energy sector. Although this principle seems essential to place the information disclosed in a wider context (e.g., geographical specificities, capacity of local ecosystems to absorb pollution, living standards of surrounding communities), it is virtually ignored in most statements. One possible explanation is that the assurance process focuses on documents released by the organization, whereas the verification of the sustainability context would require enlarging the scope of auditors’ analysis through the introduction of contextual, non-standardized and complex information from various sources uncontrolled by reporting organizations. Some statements mentioned having examined the sustainability context, but only in relation to internal documents and interviews:

Our assurance process also included […] discussion on sustainability with senior executives at the different plant locations and at the corporate office to understand the risk and opportunities from sustainability context and the strategy RMML is following (RMML 2014, p. 2)

Assessing the Quality of Information

The assessment of the quality of information is also heterogeneous and is essentially focused on a few GRI principles (see Table 6).

Table 6 Assessment of the quality of information (% of statements)

The assessment of the accuracy of information is the most widely covered principle and is highlighted in 52% of all statements (see Table 6). This proportion seems relatively high considering the diversity of GRI indicators and the difficulty of precisely measuring sustainability performance on issues as various as biodiversity impacts, human rights and anti-corruption practices. Nevertheless, as indicated in about 35% of all statements, the assurance engagement does not cover all indicators and is focused on specific sections of sustainability reports. As such, some statements, mainly the ones from accounting firms, only focus on quantitative indicators that can be verified using recalculation (Sumitomo Metal Mining 2013). Moreover, the methods used for assessing the accuracy of information and the indicators concerned are rarely explained in the statements analyzed. In most cases, the verification of accuracy is simply mentioned along with other GRI principles, particularly reliability, completeness and materiality of information. Overall, the importance of accuracy, particularly in statements from accounting firms, seems to mostly reflect the emphasis on this principle in auditing practices in general. Nevertheless, as acknowledged in some statements, the transfer of this principle from accounting to sustainability is not unequivocal:

Non-financial data is subject to more inherent limitations than financial data, given both the nature and the methods used for determining, calculating, sampling or estimating such data. Qualitative interpretations of relevance, materiality and the accuracy of data are subject to individual assumptions and judgements. (Gold Fields 2010, p. 142)

The same type of remark applies to the principle of reliability, which is covered in 44% of all statements. The verification of this principle assumes an in-depth analysis of the internal procedures, evidence and sources of information used in sustainability reports. Such analysis appears to exceed the limited level of assurance provided by most auditors, who also tend to give the benefit of the doubt to reporting companies:

Examination of the reliability of the supplied data was not included in the auditing, because the Auditor was confident that the report could be verified to a Moderate level as having a minimal likelihood of containing errors, based on available evidence and from selective interviews conducted with people in charge of each performance area. (Korea Midland Power 2011, p. 72)

Although the balance of information is considered to be one of the main challenges of sustainability reports (Cho and Patten 2007; Cho et al. 2015; Hahn and Lülfs 2014; Talbot and Boiral 2015), it is covered by only a quarter of statements. Moreover, with very few exceptions, the balance of information is not explicitly interpreted, in line with the GRI framework, as a clear representation of both negative and positive material facts (Czech Coal 2012, p.131). Rather, this principle is associated with the neutrality of reports and the importance of presenting information in a “balanced manner” (e.g., Korea National Oil Corporation 2012, p. 83; Xstrata 2006, p. 91), but without defining this concept or how it is verified in practical terms:

Based on the information reviewed, IRAS is confident that this report provides a comprehensive and balanced account of the environmental, safety and social performance of ARM during the period under review. (ARM 2013, p. 123)

The report covers the significant issues and challenges that the company has faced during 2006 in a balanced manner. (Xstrata 2006, p. 91)

Likewise, the interpretation of the comparability principle, which is mentioned in 13% of reports, appears quite partial. The GRI defines comparability as the possibility of comparing performance both over time and relative to other organizations (GRI 2006, p. 14). With the exception of one statement, which observes the improvement of the reporting quality and comparability both between different reporting periods and with other companies’ reports (Norilsk Nickel 2012), assurance providers focused only on the comparability over time. Such a focus may be explained by the ease of identifying information—tables, figures—that compares performance longitudinally in the sustainability report verified. Conversely, comparisons between organizations would require more complex information from various sources to analyze whether the performance of the reporting company is actually comparable. Moreover, such analysis may appear to be ambiguous and subjective.

The same remark applies to the principle of clarity, which is covered in only 10% of statements. Although this principle is essential to improve the readability of reports, the criteria for its evaluation seem, at best, ambiguous, which can explain why it tends to be overlooked by assurance providers.

Finally, the principle of timeliness is virtually ignored in assurance statements. When mentioned, timeliness is neither explained nor analyzed in relation to specific issues such as the lack of recent information on an important issue or the need to clarify the time period of certain data.

The Limitations Observed by Assurance Providers

Although the language used by assurance providers is optimistic and rarely critical, 23% of statements from the mining sector and 20% from the energy sector explicitly mention some limitations and deficiencies related to the reporting process or auditability of information. The limitations observed by assurance providers are focused on four main issues: internal practices and processes, accuracy and reliability issues, absence of information and auditability (see Table 7). These issues are complementary and not mutually exclusive. Interestingly, compliance with the GRI framework and its specific reporting principles is very rarely mentioned. As a result, the problems reported in statements are not formulated in relation to specific requirements, guidelines or standards but rather presented as a general observation that does not put the assurance provided into question.

Table 7 Limitations and deficiencies observed by assurance providers (% of statements)

Firstly, 10% of statements highlight limitations related to organizational practices, including the reporting process. For example, the lack of internal guidance (Newmont Mining 2010) or clear methods to calculate certain environmental issues (Total 2013) were mentioned by assurance providers. The lack of stakeholder dialogue and failure to adapt the sustainability report to stakeholders’ specific concerns were also mentioned (e.g., Czech Coal 2010; Oil and Natural gas Corporation 2012; Vedanta Resources 2013). Finally, some statements criticize the internal practices for the identification of material issues or the monitoring of information:

Specific projects do not always appear to be selected in a strategic manner—with the maximum developmental return on investment considered—and systems to monitor and evaluate projects for socioeconomic impact do not appear to exist in an adequate form. (Xstrata South Africa 2011, p. 157)

The existing materiality determination process needs to bring out all material aspects related to individual operational sites and aggregated at corporate level. (Vedanta Resources 2013, p. 111)

Secondly, 10% of statements question the accuracy and reliability of information on certain issues. Data inaccuracy and errors are associated with methodological issues, data aggregation or lack of material indicators. Yet, most statements indicate that these issues have been corrected by reporting companies, were observed only a few times, concern specific activities only, or are not significant enough to question the quality of the whole reporting process:

Certain site-reported data was found to be inaccurate and/or unreliable on a few occasions, although none of the identified errors were deemed significant enough to warrant a statement of qualification, and all errors were adequately addressed prior to the conclusion of this engagement. (African Rainbow Minerals 2010, p. 73)

Occasional technical inaccuracies in the environmental data were identified due to data transfer and calculation methodologies among some non-material indicators that were corrected in the final draft of the report. (Czech Coal 2010, p. 124)

Thirdly, comments on the absence or lack of information were observed in 9% of statements. Although the insufficiency of information is related to the principle of completeness, the lack of compliance with this GRI principle is not clearly evidenced. Rather, the statements mention indicators and issues that should have been more thoroughly covered in sustainability reports:

Basis for Qualified conclusion[:] the Report does not provide sufficient representation of Rosneft’s performance regarding greenhouse gas emissions. (Rosneft 2010, p. 129)

We consider that BP could have covered the following subject areas in more depth in the Report: influencing the performance of business partners in relation to sustainability issues, disclosure of future environmental performance targets. (BP 2010, p. 34)

Finally, 3% of reports highlight the poor auditability of reports and the difficulty of accessing or verifying certain information. This difficulty can be related to various issues (e.g., lack of clarity in reports, unavailable or inaccessible information, data entry errors or absence of documentation on methodological aspects). Nevertheless, just like most other limitations observed, these issues are presented as minor problems or as a consequence of the difficulty of data collection:

For HR11, nothing has come to our attention that causes us to believe that grievance mechanisms do not exist, however we were unable to obtain sufficient appropriate evidence that the existing mechanisms could accurately track the number of grievances related to human rights due to inconsistencies in definitions used, tracking methods, and availability of documentation. (Goldcorp 2013, p. 2)

Suggestions for Improvement

The lack of explicit references to the limitations of reports is partly compensated for by frequent suggestions for improvement, which are proposed in half of all statements. Although they remain positively framed, recommendations tend implicitly to respond to some weaknesses observed in sustainability reports that are rarely clearly mentioned in the statements. These recommendations revolve around seven complementary and not mutually exclusive areas for improvement in sustainability reporting: stakeholder engagement; control and internal verification; data collection; scope of reports; identification of material issues; clarification of objectives and strategy; and standard compliance (see Table 8).

Table 8 Suggestions for improvement of sustainability reporting (% of statements)

Firstly, a quarter of statements suggest that reporting companies improve their stakeholder engagement. This proportion seems relatively high considering that the principle of stakeholder inclusiveness is covered in only 42% of statements; thus, most verifications of stakeholder inclusiveness result in recommendations to reporting companies. These recommendations cover various aspects such as dialogue with stakeholders (Midland Power 2011), procedures to identify key stakeholders (Bharat Petroleum 2008; Lihir Gold 2010; Sesa Goa 2011), clear descriptions of stakeholder engagement (Novatek 2011) or enhancement of stakeholder responsiveness (Korea Midland Power 2011). Overall, the suggestions proposed by assurance providers seem relevant while remaining quite uncritical and elusive:

A systematic and documented process for identifying and engaging key stakeholders on issues of concern should be implemented. (Bharat Petroleum 2008, p. 69)

The Auditor recommends that KOMIPO establish more diverse communication channels with all stakeholders and report information not just from stakeholder interviews, but also regarding performance status and plans, in order to enhance responsiveness. (Korea Midland Power 2011, p. 73)

Secondly, 21% of statements suggest improvements in the control and internal verification process of reporting companies. These suggestions may be related to internal practices for improving the reliability of information and the auditability of reports. Suggestions in this area cover the evaluation of the information disclosed in sustainability reports (PT Kaltim Prima Coal 2010), revision of internal control procedures (Rio Tinto 2007), implementation of efficiency evaluation criteria (Tatneft 2010), development of measurable quantitative targets (BG Group 2013) or monitoring of the socioeconomic impact of projects (African Rainbow Mineral 2011). The implementation of internal audits is also recommended in certain statements:

Implementation of systematic monitoring and auditing of environmental data will help ensure more accurate and reliable data. (Czech Coal 2010, p. 126)

PTT GC should consider […] introducing internal verification processes such as audits and top management level review to increase the accuracy of the datasets. (PTT Global Chemical Public Company 2013, p. 135)

Thirdly, 17% of statements include suggestions focused on the data collection process. These suggestions generally concern internal practices for improving the accuracy of reports and preventing errors in the management of information. Improvements to the frequency and rigor of this process (De Beers 2007; Vedanta Resources 2013), involvement of each reporting unit (Hess Corporation 2012) and the management of the sustainability database (Xstrata 2007) have been highlighted. The prevention of mistakes related to data aggregation is also mentioned, although the nature and scope of the errors identified by assurance providers remain unclear:

We encourage Codelco to further strengthen its information systems in order to prevent errors in the aggregation and collection of data. (Codelco 2011, p. 184)

Reinforce the key indicators information recollection systems and processes for environmental and social data in order to prevent compilation mistakes. (Penoles 2011, p. 70)

Fourthly, improvements in the scope of reports are suggested in 16% of statements. Suggestions in this area may be related to the incompleteness of reports or the absence or insufficiency of information. The inclusion of subsidiaries, branches or facilities not covered by sustainability reports is frequently recommended, particularly for large and diversified companies that do not necessarily disclose information on all their activities (Korea Gas Corporation 2012; ONGC 2013; Standard Oil 2010). Likewise, certain important issues such as GHG emissions (BHP Billiton 2007) and the life stage of mining operations (Barrick Gold 2009) need to be better reported. Interestingly, a few statements refer to the GRI requirements, although those references are rarely explicit and clear:

Hindalco should ensure capturing of data and information pertaining to “partially” and “not reported” core indicators of GRI G3.1 guidelines and report on the same in accordance with the commitments made in the report. (Hindalco 2012, p. 94)

While the Company has attempted to report on all core indicators, further strengthening of data collation and review systems may be considered, to ensure completeness of each core indicator. (MSPL 2009, p. 70)

Fifthly, 16% of statements suggest improvements in the materiality of reports. This proportion seems relatively low considering that the materiality principle is the most frequently verified by assurance providers. Suggestions in this area essentially concern the formalization of criteria to determine issues for reporting (Abeinsa 2013; Abengoa Solar 2013; PT Timah 2010), the materiality determination for different sites (PTT Public Company 2013; Vedanta Resources 2013), the disclosure of information on controversial issues (EVN 2010; PT Kaltim Prima Coal 2010; SK Innovations 2012), the inclusion of the value chain in this process (Abeinsa 2013; Abengoa Solar 2013) and the revision of material issues on a regular basis (PT Kaltim Prima Coal 2010; Santos 2012). Some statements also highlight the importance of taking stakeholder concerns into account in the determination of materiality:

Ensure that the materiality process systematically incorporates the views of stakeholders through an engagement that can serve both corporate strategy and future reporting. (Czech Coal 2010, p. 126)

Materiality test is increasingly used to better understand stakeholders’ specific interests. More comprehensive and organized processes would help S-OIL to find issues most material to its business and stakeholders. (S-Oil 2009, p. 85)

Sixthly, 11% of statements suggest that the reporting companies clarify their sustainability objectives and action plans for the future. This type of information is supposed to improve the comparability of sustainability performance over time. It should also improve organizational accountability by facilitating performance monitoring and verification of the achievement of objectives. Suggestions in this area are also expected to improve the transparency of the organizational strategy on critical issues:

Consider the development of objectives and strategies for carbon reduction and energy conservation programs. (Barrick Gold 2012, p. 96)

Rosneft published its goals and objectives related to sustainability performance in the Report. We suggest that Rosneft report its progress against these goals and objectives. (Rosneft 2010, p. 129)

Finally, 8% of statements include suggestions for the improvement of compliance with various standards on reporting, accountability and stakeholder relationships. Although it is not related to a specific GRI principle, compliance with standards may cover various requirements, depending on the standard considered. Nevertheless, the statements rarely explain the issues that need to be addressed and the benefits that can be expected through a better alignment with different standards:

Future management of stakeholder engagement would benefit from alignment with AA1000APS and reporting should include more detail on the engagement processes and results and the inclusion of responses to the results of engagement. (Novatek 2011, p. 63)

We recommend that Codelco continue to adapt its collection systems to the GRI G3 version requirements. (Codelco 2011, p. 184)

Discussion

The objective of this study was to analyze the quality of sustainability reports, their limitations and avenues for improvement from the perspective of the assurance providers in charge of the verification of those reports. At first glance, the discourse of assurance providers seems shaped by an optimistic and cautious rhetoric in which the problems and weaknesses of sustainability reports are rarely directly and explicitly addressed. This cautious attitude can be partly explained by the limited or moderate level of assurance provided in two-thirds of assurance statements. This level of assurance sends a signal that the verification process has not necessarily been extensive and that its conclusion needs to be taken with caution. The current rhetoric of statements can also be explained by pressures from reporting companies, who tend to use the assurance process as a tool to improve their social legitimacy (Manetti and Toccafondi 2012; Michelon et al. 2015; O’Dwyer et al. 2011). As highlighted in the literature, the managerial capture and commercial aspects of the assurance process limit the independence, transparency and critical distance of assurance providers (Boiral 2013; Jones and Solomon 2010; Michelon et al. 2015; Owen et al. 2000). From this perspective, the statements provided in sustainability reports tend to reflect the companies’ expectations and to legitimize the quality of the information disclosed. This tendency is reflected in the small proportion—15%—of statements containing comments on the limitations or weaknesses observed in the reporting process. Conversely, 25% of statements highlight the progress achieved by reporting companies, particularly in terms of stakeholder engagement. These findings lend credence to the critical literature on the optimism, predictability and managerial capture of assurance statements (Ball et al. 2000; Boiral 2013; Milne et al. 2006; Moneva et al. 2006; Smith et al. 2011). As highlighted by Hummel et al. (2017), such capture of assurance statements raises important ethical questions in terms of independence, professionalism and objectivity of assurance providers. The application of those principles, which underlie assurance standards, such as AA1000 and ISAE 3000, seems quite theoretical and is not reflected in the uncritical and optimistic nature of most statements. Nevertheless, a further examination of a large number of assurance statements shows that, beyond their politically correct appearance, these documents are not necessarily devoid of substance and professional skepticism. The suggestions for improvement contained in half of the statements indirectly respond to the limitations observed by assurance providers and contain some relevant information on the need to correct certain weaknesses. The suggestions most frequently made are related to stakeholder engagement and echo the literature on the need to enhance responsiveness and dialogue with interested parties (Kolk and Perego 2010; Manetti and Toccafondi 2012; O’Dwyer et al. 2011). Not surprisingly, no suggestions for the possible involvement of stakeholders in the verification process of sustainability reports were found. Although the relevance of such involvement has been highlighted in the literature (Junior et al. 2014; Perego and Kolk 2012), encouraging such involvement could question the legitimacy of assurance providers themselves and the control they, with the reporting companies, exert over the verification process. Overall, the statements remain an exercise of legitimation for both reporting companies and assurance providers. Yet, they can suggest interesting avenues of improvement that are not necessarily well covered by the literature on sustainability reporting, such as the clarification of objectives and strategy, standard compliance or the scope of reports. Conversely, important criticisms related to the GRI principles that have been stressed in the literature, such as the lack of balance and comparability of reports (Boiral and Henri 2017; Cho and Patten 2007; Cho et al. 2015; Hahn and Lülfs 2014; Talbot and Boiral 2015), are very rarely mentioned in the assurance statements.

Generally speaking, this study shows that the GRI principles are not systematically verified by assurance providers and that some may even be ignored. Certain GRI principles such as the sustainability context, clarity of information and timeliness of reports are almost never reviewed or mentioned. Most statements focus on a few principles also applied in the verification of financial reports, such as the accuracy, reliability and completeness of information. This tendency can be explained by the paradigmatic position of financial audits, which have shaped auditing practices in sustainability reporting (Boiral and Gendron 2011; O’Dwyer et al. 2011; Power 1997a, b). It can also be explained by the predominance of accounting firms, which are marked by institutional arrangements similar to those observed in the finance and accounting areas (Kolk and Perego 2010; Moroney et al. 2012; Perego 2009).

According to the legitimacy theory (Cho and Patten 2007; Gürtürk and Hahn 2016; Perego and Kolk 2012), such predominance strengthens the mimetic and normative isomorphism (DiMaggio and Powell 1983) of assurance statements, i.e., their tendency to imitate the statement practices legitimized by accounting firms and to reproduce the norms shared by auditors with similar educational backgrounds. The assurance of financial and sustainability information by the same accounting firms and the development of integrated reporting (i.e., the integration of information statements covering financial and sustainability issues into a single document) tend to strengthen this isomorphism. It can also provide economies of scale and improve the credibility of information (Huggins et al. 2011). In line with the legitimacy theory (Meyer and Rowan 1977; DiMaggio and Powell 1983), the isomorphism of assurance statements is shaped by the search for legitimacy of organizations. This isomorphism may explain the formal appearance of statements, which appear to be both rational and quite disconnected from the substantial sustainability issues faced by companies in the mining and energy sectors. It may also explain why principles more specifically related to sustainability issues, such as the sustainability context, tend to be overlooked by assurance providers. From this critical perspective, assurance statements tend to appear as a rational myth (Meyer and Rowan 1977), that is a formal practice loosely connected with real issues and adopted quite symbolically to improve the legitimate and rational image of the organization. This rational myth tends to camouflage, through reassuring accounting rhetoric, the lack of transparency and reliability of sustainability reports which has been highlighted in the literature (e.g., Ball et al. 2000; Cho et al. 2015; Milne et al. 2006; Smith et al. 2011). Such camouflage has ethical implications because it conveys a misleading picture of confidence and rationality to stakeholders, including the socially responsible analysts and investors who use sustainability reporting to assess the corporate performance in this area. The negative relationship between the assurance process and sustainability performance of reporting organizations found by Hummel et al. (2017) seems to lend credence to this camouflaging effect potentially played by assurance statements. Overall, whatever their rigorous appearance, accounting principles underlying the assurance process seem to be ill adapted to the qualitative, complex and multifaceted nature of the information contained in sustainability reports.

Contributions

First, this study sheds more light on the conclusions of assurance statements through a content analysis of a large sample, and it is focused on issues that have been overlooked in the literature. Most studies in this area are based on a limited sample or mainly describe the general features of statements (e.g., level of assurance, type of assurance provider, scope of verification) rather than their conclusions on report quality and its limitations or their suggestions for improvements. Some authors consider the personal judgment and recommendations of assurance providers to not be relevant or to fall outside the normal scope of the assurance process (Manetti and Becatti 2009). By focusing on these judgmental aspects through a study based on 301 statements, this paper sheds new light on the quality, limitations and avenues to improve sustainability reports from a perspective that complements the current literature. Although some observations and recommendations of assurance providers (e.g., the need to improve stakeholder engagement and reliability of information) are clearly in line with the literature (Adams 2004; Ball et al. 2000; Cho et al. 2015; Dando and Swift 2003; Manetti and Toccafondi 2012; Talbot and Boiral 2015), others (e.g., auditability and information access, data collection practices, clarification of objectives) remain understudied.

Second, the paper contributes to the literature on stakeholders’ perceptions on the quality of sustainability reports (Belal and Roberts 2010; Solomon and Solomon 2006; Solomon et al. 2013). These perceptions have been overlooked in the literature (Hahn and Kühnen 2013; Solomon and Solomon 2006; Unerman et al. 2007), which remains essentially focused on the analysis of sustainability reports by researchers rather than stakeholders, including auditors. Nevertheless, all interested parties are not necessarily well informed about sustainability reporting, standards and practices. By focusing on the opinions of a specific category of practitioners, namely assurance providers, this study contributes to a widening of perceptions on the complexity of sustainability reports. Although assurance providers tend to be biased by the managerial capture of the reporting and verification processes (Ball et al. 2000; Jones and Solomon 2010; Owen et al. 2000; Smith et al. 2011), they are also assumed to read sustainability reports thoroughly, to have an expertise in this area and to exercise, as far as possible, their professional skepticism (AccountAbility 2008; Iansen-Rogers and Oelschlaegel 2005; Junior et al. 2014; Manetti and Becatti 2009; Perego and Kolk 2012). Our findings show that assurance providers tend to express this skepticism indirectly, by highlighting possible avenues for improvement rather than stressing limitations or issues of non-compliance. The positive wording of recommendations allows auditors to add more substance and critical distance to their discourse without questioning the predominant optimistic and legitimizing rhetoric of statements. This finding contributes to reconciling the critical approaches—mostly based on the legitimacy theory in which assurance statements are essentially a public relations exercise largely controlled by managers (Ball et al. 2000; Deegan and Blomquist 2006; Fonseca 2010; O’Dwyer and Owen 2005, 2007)—with more functionalist approaches that defend the relevance and usefulness of this practice (Adams and Evans 2004; Dando and Swift 2003; Fernandez-Feijoo et al. 2014; Hodge et al. 2009; Manetti and Toccafondi 2012). From this perspective, assurance providers strive to find a balance between, on the one hand, the pressures from reporting companies in search of more legitimacy and, on the other hand, the basic requirements of audits in terms of independence, skepticism and professionalism.

Third, this study explores the criteria used by assurance providers to assess the quality of sustainability reports. Although all reports analyzed are based on the GRI framework, the requirements of this framework tend to be overlooked in the majority of assurance statements. This finding is all the more surprising given that the GRI—which is considered to be the most widely used and detailed reporting framework (King and Bartels 2015)—clearly defines principles for the content and quality of reports. One could assume that assurance providers would use those established principles to guide and legitimize their verification process. Instead, the integration of those principles appears to be very heterogeneous and uncertain. With the exception of the materiality of reports and, to a lesser extent, the accuracy of information, GRI principles are rarely explicitly taken into account in assurance statements. Moreover, when those principles are mentioned, little information is released on how they have been used, in practical terms, in the verification process. Overall, this paper provides confirmation of the verification process and shows deficiencies in the criteria used by assurance providers.

Practical Implications

The paper has practical implications for assurance providers, standardization organizations and stakeholders.

First, to increase the credibility of their verification and the readability of assurance statements, auditors should clarify the criteria used and systematically refer to established standards, particularly the GRI principles. Although those principles are not a panacea, their substantial application would improve the quality of sustainability reporting. By more systematically and explicitly verifying those principles, assurance providers would encourage reporting companies that use the GRI to internalize the reporting requirements of this framework. Such internalization would enhance the quality of sustainability reports and could contribute to address certain criticisms on the greenwashing tendencies, managerial capture and lack of ethics of the reporting process in general (Cho et al. 2012; Hummel et al. 2017; Moneva et al. 2006; Talbot and Boiral 2015). Moreover, the systematic verification of the GRI principles would also increase the legitimacy, credibility and clarity of assurance statements by addressing critical issues debated in the literature—such as the lack of balance of reports—that remain obscured by the optimistic rhetoric of auditors. To reduce the managerial capture of information and appearance of collusion with companies, assurance providers could also more substantially rely on information not controlled by reporting organizations (e.g., interviews with stakeholders, complaints investigations, reports from governmental agencies or incidents described in newspapers). Comparisons between this information and the content of sustainability reports would certainly reveal significant gaps, as many authors have suggested (Adams 2004; Boiral 2013; Gallhofer et al. 2006; Sikka 2006; Talbot and Boiral 2015). Such a counter-accounting approach is also in line with the need to further involve stakeholders in the verification process and to improve the added value of this practice (Jones and Solomon 2010; Junior et al. 2014; Perego and Kolk 2012).

Second, standardization organizations should clarify the criteria that need to be prioritized by assurance providers and how those criteria should be applied. Although the GRI provides a description of each reporting principle and a short checklist to verify its application (GRI 2006), their substantial integration into the assurance process can be tedious and would require more guidance to support the verification process. The implementation manual launched with the G4 version of the GRI (GRI 2013a) is certainly a step in the right direction, and it provides more detailed information on the implementation and verification of reporting principles. Nevertheless, given the lack of consideration for the GRI requirements observed in the statements analyzed, the use of this type of manual by assurance providers seems uncertain at best. More importantly, the GRI and its principles for the quality of reporting are virtually ignored by the main assurance standards used in this area, namely the AA1000 and ISAE 3000 guidelines. As a result, assurance providers are not inclined to apply these principles seriously but tend rather to adopt a procedural approach modeled after accounting practices that have not been sufficiently adapted to the verification of information on sustainability issues. Generally speaking, the compatibility between reporting and verification standards should be improved to encourage more substantial verification, particularly on issues that tend to be overlooked by assurance providers, such as the sustainability context, the balance of reports and the comparability of information.

Third, stakeholders should put pressure on companies and assurance providers to release more detailed information on the verification process. Although this study shows that assurance statements are not necessarily devoid of substance, it is based on an analysis of a large quantity of statements which, taken individually, often appear short and perfunctory. More information on the limitations observed and avenues for improvement would be useful to stakeholders interested in the quality of sustainability reports and to monitoring improvements in this area. For example, extra-financial rating agencies involved in the evaluation of sustainability performance and proxy voting services for responsible investors could require companies and assurance providers to share a more detailed verification report or to make it available upon request. Similarly, assurance providers could solicit information from rating agencies and other stakeholders involved in the evaluation of sustainability performance. This type of collaboration and exchange of information would be mutually beneficial for the stakeholders involved in the measurement of sustainability performance and assurance providers alike.

Limitations and Avenues for Future Research

The limitations of this paper and its empirical findings suggest various avenues for future research.

First, although the analysis of a large number of assurance statements is relevant to shed more light on auditors’ perceptions of the quality of sustainability reports, the outcomes of the verification process are clearly shaped by managerial pressures and companies’ quest for legitimacy (Ball et al. 2000; Fonseca 2010; Jones and Solomon 2010; Michelon et al. 2015; Owen et al. 2000; Smith et al. 2011). As a result, the findings of this study reflect what assurance providers and companies agree to share publicly and tend to obscure negative aspects that could damage corporate image. Likewise, due to the qualitative approach used in the fieldwork, it could not be asserted to what extent the lack of substance and the differences observed in assurance statements are due to the auditors themselves, the standards or guidelines used or to other factors, which could raise concerns of confounding effects. Future studies could interview assurance providers and managers to investigate the gap between the information disclosed in assurance statements and their actual opinions on the quality of sustainability reports. Similarly, investigating the perceptions of various stakeholders—including employees, NGOs and practitioners in the area of responsible investment—on this gap and, more generally, on the reliability of assurance statements, would provide a more comprehensive picture of the relevance and usefulness of current verification practices.

Second, although the suggestions for improvement mentioned in statements are certainly indicative of the changes that reporting companies should implement, the impact of those suggestions remains uncertain. Future studies could investigate the extent to which the outcomes of the verification process are seriously taken into consideration by managers and how they may influence organizational reporting practices. Such studies could take a longitudinal approach to analyze the possible changes and improvements in reporting practices over time, whether they are related to previous recommendations from assurance providers or not. Interviews with managers and assurance providers could also help to deepen our understanding of auditor–auditee relationships, and the tendencies observed in sustainability reports (e.g., most difficult principles to apply, improvements observed and changes in the nature and scope of the verification process).

Third, this study is based on the reports that use the G3 version of the GRI, which was used until the end of 2015. Although the reporting principles have not changed from the G3 version, it is assumed that the new GRI G4 will improve the quality of sustainability reports (Boiral and Henri 2017; GRI 2013a; Jones et al. 2015). Future studies could investigate to what extent this assumption is justified and how it is reflected in assurance statements. Assurance providers’ knowledge of this new version and their familiarity with the GRI principles in general could also be further investigated. The lack of integration of these principles observed in this study could be partly explained by the training, experience and educational background of assurance providers, who may believe that the GRI—whatever version is considered—is not in line with the norms of their practice or is simply not relevant to the conduct of the type of audit companies expect them to deliver. The influence of the assurance standard used (e.g., AA1000, ISAE3000) on the verification process could also be further investigated. This type of investigation would cast more light on the determinants of the normative and mimetic isomorphism that permeates the rhetoric of assurance statements (e.g., influence of the educational background of assurance providers, role of assurance standards, guidelines and templates used by different auditors, transfer of accounting procedures to the assurance of sustainability reports).