Keywords

FormalPara Prelude

One hour after sending out the survey questionnaire (that forms the basis for the current study) to approximately 2000 potential respondents, the author received a phone call from a deeply annoyed manager of a well-known cultural organisation. He had one and only concern—why does the author waste his and her own time on such nonsense (referring to the evaluation of organisational performance)? This was not just a call; it was a wake-up call for the author that something needs to be done to wake up the managers of Estonian Cultural and Creative Industries (CCI) in order to help them realize their full potential. Soon, another phone call followed with a calm voice (from a small CCI organisation) specifying one question in the questionnaire “What is meant by a “Written Mission Statement”? Is it really expected to write down something that is well recorded in the minds of our team members?”. After an intense, but not very successful effort to explain the essence of strategic management, the author was convinced that the chosen research direction was the right one and that there was a serious practical need for the current study.

1 Introduction

The current chapter aims to find out how the heterogeneous background of CCI organisations relates to the evaluation of organisational performance, challenges and skills gap in CCI organisations and develop a framework explaining their relations. A systems management approach is used and CCI organisations are targeted as systems that exist in a dynamic environment (Jensen and Sage 2000). The following factors are expected to shape the organisational performance and its evaluation of CCI organisations—challenges and skills gaps (Jensen and Sage 2000). Both are considered vital for effective organisational performance (Almatrooshi et al. 2016).

It is important to highlight that the current study is based on the expectation that an understanding of what it is that CCI organisations are struggling with the most (which challenges and skills gaps in particular), helps not only to understand their (potential) resistance to evaluation, but in addition, it also contributes to forming some practical recommendations for future research agenda.

In the current chapter, a wide definition of CCI is used—CCIs are those industries which have their origin in individual creativity, skill and talent and which have the potential for wealth and job creation through the generation and exploitation of intellectual property (Department of Culture 2001, p. 5). CCI is considered to be heterogeneous (Eikhof and Haunschild 2006), but not cohesive (White et al. 2014). The current study analyses the CCI organisations not only from a general perspective, but also considers the heterogeneity of the respondent organisations—form, size, sub-sector and age. This is an important consideration as sub-sector and size, among other variables, have already been proven to affect the management of organisations (Turbide and Laurin 2014). However, as creative organisations are often seen as hybrid organisations (commercial firms that further some aspect of the public good) (Rushton 2014), the borderlines between not-for-profit and for-profit organisations in CCI are not as straight-forward as in the more traditional business environment. Therefore, it is important not just to focus on the organisational form, but also other factors that differentiate CCI organisations—size, sub-sector and age.

In the current chapter “organisational performance” is understood as a mixture of goal attainment, relations between the organisation and its environment, and behaviour of organisation participants (Ford and Schellenberg 1982). While “evaluation of organisational performance” and “performance evaluation” and “performance measurement” are currently used as synonyms and refer to the ongoing monitoring and reporting of accomplishments, particularly progress toward preestablished goals (United States Government Accountability Office 2005, p. 3). The monitoring activities should not be seen just as one of organisational performance evaluation activities, but this function seeks to ensure that all the three levels of decision-making—rational, process and transactional—fit well together (Freeman 2010).

The evaluation of organisational performance in general is a rather new research domain in CCI. For instance, in Germany, the Evaluation Society has already been dealing with the topic of evaluation since 1997; however, its Working Group for Culture was only created in 2007 (Birnkraut 2011). So far, no Working Group for Culture or Creative Industries exists in the Evaluation Society of Estonia. Hence, the assumption of the current study is that it is not common for Estonian CCI organisations to evaluate their performance regularly and systematically.

However, there are countries (Australia, the UK etc.) where on a state level specific tools for CCI organisations have been developed to measure their performance (Birnkraut 2011). The aim of these planning and evaluation tools is to raise the quality of management and performance in CCI organisations and to develop the CCI sector, as well as the individual organisations within it. The current chapter presents examples of planning and evaluation tools that have been used in CCI organisations and are proven to be successful.

As there is hardly any academic literature available on CCI evaluation practices in Estonia, the theoretical part of current chapter can only rely on international literature on the evaluation of organisational performance and on previously published general reports by CCIs (that might be classified as grey literature), while only the data for the empirical part was collected from Estonia. Thus, the central research question of the current chapter is the following—How does the heterogeneous background of CCI organisations relate to the evaluation of organisational performance, challenges and skills gap in CCI organisations? The following sub-questions were formulated in order to find an answer to the research question:

  • How is the data on performance collected and/or analysed in CCI organisations in Estonia?

  • Is there any significant correlation between the evaluation of organisational performance and the challenges or skills gaps in CCI organisations in Estonia?

  • How do the challenges related to the evaluation of organisational performance in CCI organisations differ based on the form, size, age of the organisation and sub-sector in CCI organisations in Estonia?

  • How do the skills gaps related to the evaluation of organisational performance in CCI organisations differ based on the form, size, age of the organisation and sub-sector in CCI organisations in Estonia?

2 Characteristics of CCI Organisations

Management in the CCI organisations is usually considered to be complex for many reasons. The organisational phenomena in CCIs are unique (Pick et al. 2015), full of controversies (Banks and O’Connor 2009) and unpredictability (Faulkner and Anderson 1987). Thus, CCI organisations are facing a great number of managerial challenges—complex relationship between management, art and technology (Hodgson and Briand 2013). Based on the existing theoretical literature and reports, the current sub-chapter will attempt to map the challenges and the skills gaps in CCI which have direct impact on the management of the sector.

Potts and Cunningham (2008) describe CCI as a dynamic sector with substantial sub-sectoral variety over time and in terms of their business models. No doubt the heterogeneity within the CCI subsectors results in each having its own managerial specifics (Jeffcutt and Pratt 2002). Therefore, it is necessary to consider a wide variety of different conditions, and internal and external challenges (not only managerial challenges), when trying to understand how CCI organisations work, because they all may influence the outcome—the organisational performance. Nevertheless, it is important to keep in mind that what unites these diverse CCI organisations, is the “creative product” as the outcome (Jones et al. 2015) and the specific “art-commerce relation”, which makes the CCI a “special case” (Ryan 1992, p. 44). This is the reason why the current chapter includes the entire Cultural and Creative Industries looking at the full range of CCI organisations whose outcomes may all be defined as creative.

2.1 Challenges Affecting CCI Organisations

Perhaps the most widespread challenges concern the general management of CCI organisations. Berziņš (2012) found in his study in Latvia, that the strategic management process is more complicated in CCI organisations than in more traditional industries, mainly due to additional factors and parallel functions. The study by Küttim et al. (2011), which compared CCI organisations in Estonia, Latvia, Finland, and Sweden, revealed that CCI managers characterize their own management style as lacking a market orientation, overlooking managerial mistakes, weakness in planning time, organisation, and financial matters; which may all be called management related challenges.

The resources obtainable in the external environment shape the survival of players in the CCI (Noyes et al. 2012). However, not just the limited financial means, but also the lack of understanding of the principles of financial management, are considered quite common in CCI organisations (Jones et al. 2004). A study by Tscherning and Boxenbaum (2011) revealed that CCI organisations in Denmark lack competencies in the areas of finance, while another comparative Baltic-Nordic study pointed out entrepreneurial competencies, among others financial planning, as important challenges in CCI organisations (Küttim et al. 2011). Thus, both studies refer to financial challenges in the CCI.

The external environment of CCI organisations is quite challenging due to its production process that often requires long, high-risk development phases (European Creative Industries Alliance 2012, p. 22). This means that CCI organisations produce symbolic content (Bilton and Leary 2002) that has a symbolic value for the end-users, which is more or less intangible in nature (Towse 2010). Nevertheless, creative products and services in an open market compete with mass production. Therefore, the constantly changing competitive environment is challenging for CCI organisations. Following table (Table 1) summarizes the main challenges and skills gaps in CCI organisations.

Table 1 Overview of the main challenges and skills gaps in CCI organizations (compiled by the authors)
Full size table

The challenges described above are expected to limit the ability of CCI organisations to work as effectively and sustainably as they could. When looking at the current challenges as a missed opportunity, the Cultural and Creative Industries could change the way how challenges are faced.

2.2 Skills Gaps in CCI Organisations

As leadership competencies are considered among the key factors that contribute to organisational performance (Almatrooshi et al. 2016), the existing skillset of managerial and entrepreneurial competencies need to be targeted when describing the essence of CCI organisations. It is quite typical to small organisations (thus to many CCI organisations) to have skills gap in management in general (Jeffcutt and Pratt 2002). Also, the results of the study by Küttim et al. (2011) showed the lack of a wide range of competencies in CCI organistions: the lack of knowledge of the business environment, financial and accounting skills, weakness in management in general, but also in financial management, business law, strategic thinking and planning, etc. One of the conclusions of the study was that a creative entrepreneur should understand a little of everything—be a generalist—in order to run a successful business (Küttim et al. 2011, p. 372).

CCI organisations require more know-how in coaching for business strategy, finance for project development and strategic planning and business skills in general (Tscherning and Boxenbaum 2011). Quite often, CCI organisations tend not to have a horizon for long-term commercial planning and face major challenges concerning strategy and business development (Tscherning and Boxenbaum 2011). Furthermore, Jeffcutt and Pratt (2002, p. 10) have highlighted that in practice most managers of CCI organisations do not have either a core task or a core competency in management. The empirical part of the current chapter analyses whether these challenges and skills gaps affect the evaluation of organisational performance in CCI organisations and how.

3 Evaluation of Organisational Performance as a Solution

Creativity can be neither planned nor controlled when managing creative workers (Manning and Sydow 2007). However, the research object for the current chapter is not creativity as such, but the CCI organisations whose outcome is usually creative. Therefore, the current chapter is based on the belief that the organisational performance of CCI organisations should not only be planned and controlled per se, but on a consideration that a regular evaluation of organisational performance will contribute to a more effective management of CCI organisations. First, the essence and benefits of evaluation of organisational performance will be introduced and then some methods and tools will be discussed.

The initiative to practice evaluation of organisational performance may come from the organisation itself or be externally driven. In some countries, the financing of arts institutions depends on the results of their performance measurement (Birnkraut 2011)—thus, existing business plans, marketing plans, performance contracts, objectives, and key performance indicators are needed to receive (government) support (Caust 2003). However, it is not just financial stability that is a possible positive outcome of the evaluation of organisational performance. For example, the quality of performance in general or the ability to avoid risks may improve as a result of performance evaluation (Radbourne et al. 2009). Furthermore, Gstraunthaler and Piber (2007) point out additional benefits of practicing organisational performance evaluation in CCI organisations: identifying a “wrong” direction and together with the stakeholders concerned, it is the basis for sustainable development (Gstraunthaler and Piber 2007, p. 366). Also empirical evidence from the performing arts sector shows that arts organisations can maximize the following quality indicators based on the feedback of their customers: knowledge-transfer or learning, risk management, authenticity, and collective engagement (Radbourne et al. 2009). The key to the success in any industry is linked to involvement, joint planning and shared responsibilities in daily evaluation practices—the organisations corresponding to the needs of their target groups can be competitive in the long-run. Therefore, it is always important to understand and be aware of the needs of target groups.

Thus, the evaluation of organisational performance provides CCI organisations the opportunity to understand the best developmental direction and the needs of the organisation’s target groups. Birnkraut and Heller (2008) reasoned that evaluation of organisational performance could be a suitable way to help the organisation to ensure the maximum efficiency of their operations. In case the managers of CCI organisations have acknowledged the need for additional managerial and entrepreneurial skillset, they could use the evaluation of organisational performance to support decision-making and contribute to higher quality organisational performance. Thus, there is reason to believe that practicing evaluation of organisational performance in CCI organisations could be a solution to some of the challenges faced by CCI organisations. This raises the question of how this should be done.

At the beginning of the evaluation process, it is necessary to decide, which performance measurement indicators and tools to use. Performance in museums is often measured on the basis of a set of key indicators which are derived from accounting systems developed for management purposes in enterprises (Gstraunthaler and Piber 2007, p. 363). It is advised to begin with some generic measures like quality, customer satisfaction, product/service cost structure, and some financial criteria (Neely et al. 2000). However, often organisational efficiency is measured by meeting budget and time constraints (Miron et al. 2004) and usually via the question: how do organisations know if they are successful or at least efficient? First, the goals and indicators need to be set, and the data collected and analysed. When those preconditions have been fulfilled, corrective measures come into play in order to learn from mistakes and choose the right path based on identified gaps—this forms the essence of evaluation of organisational performance.

3.1 Evaluation Tools Used in CCI Organisations

Most scholars seem to agree that a one-model-fits-all organisational performance evaluation method does not exist (Birnkraut 2011) and Caust (2003, p. 60) has emphasized that industry models that have no direct relevance to arts should not be used. Based on the literature the following organisational performance evaluation tools are used in CCI organisations (Table 1): Balanced Scorecard, Gap-analysis, Benchmarking, Social Return on Investment (SROI), different Quality Frameworks, Framework Model for Evaluating the Performance of Arts Organisations, Data-envelopment analysis (DEA) and Artistic Vibrancy Framework. All the above listed tools are described briefly in Table 2 below.

Table 2 Selection of evaluation tools used in CCI organizations worldwide (compiled by the authors)
Full size table

Most of these evaluation tools take financial aspects into consideration, but do not focus on them and pay more attention to the uniqueness of organisations (among others that most CCI organisations provide mission-based performance) through their general performance as such. These evaluation tools have been used, for instance, in the UK, Japan, and Spain, mainly in museums and libraries (Vitaliano 1998; Hammond 2002; Chen et al. 2005; Chou et al. 2012; del Barrio and Herrero 2014; Ma et al. 2014) and are also considered to be more or less suitable for the evaluation of CCI organisations. The tools described in the table above are briefly compared below.

The Balanced Scorecard is based on monitoring indicators to improve organisational outcomes (Boorsma and Chiaravalloti 2010). It was essentially used to align the management and strategy of a business but has also been successfully implemented in arts organisations in the form of some modified versions (Birnkraut and Heller 2008). It mainly targets the internal aspects of the organisation, while the Gap Analysis consists of a listing of characteristic factors of the present situation which takes into account the external and internal aspects. It is all about listing the factors needed to achieve future objectives and highlighting gaps that exist and need to be filled. The method has been used in art galleries, museums and when assessing the quality of festivals or performances (Kilbride and Norris 2014).

Data-envelopment analysis (DEA) was initially developed to evaluate non-profit organisations. Today, this fairly standardized technique has been widely applied to assess cultural institutions, particularly museums and libraries (Chen et al. 2005; del Barrio and Herrero 2014).

The methods described so far are applicable to single organisations to measure their organisational performance or some of its aspects. However, it is only possible to use Benchmarking when there exists another organisation as a benchmark, since this method is about comparing the data of one organisation with metrics from similar organisations in the same field of activity. It has also often been used in libraries and museums, most probably because the systems are similar enough to make the comparison possible (Selwood 2002; Reichmann and Sommersguter-Reichmann 2010). There has also been at least one attempt to create a general benchmarking raster for CCI policies at the local level (Montalto et al. 2012).

Social Return on Investment (SROI) is a method that allows organisations to generate and measure value of their outcome in monetary terms (Krlev et al. 2013). In spite of its focus on financial issues it has also been effectively used to assess arts programs and the impact of investments in culture. However, SROI is limited in its capacity to compare across museums and galleries (Zappalà and Lyons 2009).

There are also a few tools that have been specially designed to suit the needs of CCI organisations. One of them is the Quality Framework that was launched by Creative Scotland and is a continuous improvement tool targeted especially at creative organisations (Scottish Arts Council 2009) and gathering outcome-based evidence (Turnbull 2011). Sorjonen and Uusitalo (2005) have also suggested a framework model for evaluating the performance of arts organisations, and this has been successfully used in Finnish arts organisations. The same authors suggest that each organisation should design its own measurement system of outcome indicators, process indicators, and structural indicators measuring efficiency and effectiveness (Sorjonen and Uusitalo 2005).

In conclusion, even though important, the chosen evaluation tools play a smaller role than the fact that there is an evaluation orientation and evaluation results are implemented in reality, meaning that evaluation of organisational performance is used as a learning tool (Russ-Eft and Hallie 2009).

4 Research Methodology

The research philosophy of the current chapter is based on the assumption by Gioia et al. (2012) stating that the organisational world is socially constructed and that the people constructing their organisational realities are “knowledgeable agents”. Namely, CCI managers know what they are aiming at and are able to express this.

4.1 The Population and Sample

In 2011, there were 7066 CCI organisations in Estonia (Eesti Konjunktuuriinstituut 2013). The sample used for the current analysis dates from 2016 and includes 460 employees of CCI organisations. For some stages of analyses also 42 organizations which do not belong to CCI are included (the control group). Table 3 describes the sample of the participant organisations with a focus on their organisational performance evaluation practices and mind-set.

Table 3 Description of the respondent organizations based on their organizational performance evaluation practices and mind-set (n = 460) (compiled by the authors)
Full size table

The table above illustrates the level of agreement with 6 aspects of evaluation of organisational performance in CCI organisations (regular performance analysis on a daily basis; efficient system for performance analysing; methodology for analysing performance; planning depends on the analysis of previous results; positive attitude of managers towards evaluation; achieved results are compared against the set goals) based on the number of employees, organisational form, age of the organisation and sub-sector. Table 3 illustrates some contradictions of the evaluation of organisational performance in respondent organisations. First, in majority of the CCI organisations planning depends on the analyses of previous results, but methodology or organisational performance evaluation systems are not widespread. And there is one more interesting characteristic of respondents—it is less in common to compare achieved results against set goals then using analyses of previous results for planning.

4.2 Data Collection

The questionnaire was composed using the core elements of a similar Danish study (Tscherning and Boxenbaum 2011) targeting creative enterprises, and a self-assessment tool introduced in the USA (BTW Consultants 2010) and for cultural organisations (Birnkraut 2011). The online survey environment in Google Forms was accessible to participants for 2.5 months (from mid-January until the end of March 2016).

The questions covered, besides the formal characteristics (age, size, type and sub-sector), also the skills and challenges of CCI organisations. The respondents had 5 options to choose from (ranging from strongly agree to strongly disagree), and therefore it should not have been difficult for the respondents to position themselves using such a set of options. As the questionnaire was anonymous, there is no reason to believe that CCI managers would intentionally conceal their challenges or (gaps of) skills.

4.3 Data Analysis

The current chapter covers the first stage of a longer research project, and therefore mainly presents descriptive statistics (cross-tables) and correlations. A Pierson Correlation Analysis was used to identify significant relationships between the evaluation of organisational performance variables and variables of challenges and skills gaps.

The data analysis consisted of the following steps:

  • First, the answers to the open-ended question concerning actual examples of data collection and analysis tools used in the responding organisations were analysed. In order to describe the reality in the most natural way, the respondents were asked to name any existing data collection and/or analysis method or tool used in the current organisation. Coding was used to categories the answers.

  • On the second stage, a Correlation Analysis was run in order to detect variables (challenges and skills gaps) that correlate with aspects of organisational performance evaluation (mind-set and practices).

  • On the following stage, special attention was paid to those challenges that were at least weakly correlated with the evaluation of organisational performance. Cross-tables were subsequently created according to the number of employees, organisational form, age and the sub-sector of CCI organisations. Only correlating variables were included in the cross-table.

  • Next, special attention was paid to those skills (gaps) that were at least weakly correlated with the evaluation of organisational performance. Cross-tables were then created according to the number of employees, organisational form, age and sub-sector of the CCI organisations. Once again, only correlating variables were included.

5 Results

First, the organisational performance evaluation mind-set and practices of CCI organisations in this study are briefly described and then the research questions answered in more detail.

The majority of the CCI organisations in Estonia tend not to have any methodology for analysing performance or efficient system for performance analysis. In the majority of the CCI organisations in this study, planning depends on the analysis of previous results regardless of the number of employees, organisational form, age of the organisation or sub-sector. While “organisational form” seems to cause the biggest difference—in the public sector the “methodology for analysing performance” is most widespread compared to other organisational forms and it is in municipal bodies where the “regular performance analysis on a daily basis” is most common. In both the public sector and municipal bodies, the “efficient system for performance analysis” is more common than in the remaining organisations.

CCI organisations with more than 10 employees are more active in most of the aspects of evaluation of organisational performance. The evaluation practices and mind-set also tend to differ based on the age of organisations as the organisations that are more than 25 years old are the most active in most aspects of the evaluation of organisational performance. Surprisingly, in the youngest organisations, the results achieved are compared against the set goals the most.

A quick look at the sub-sectors and the following “trends” seem to be most typical:

  • Museums practice more regular performance analysis than organisations of other sub-sectors, while regular analysis does not seem to be typical in the entertainment software sector.

  • The broadcast sector responded exceptionally positively to the “planning depends on the analysis of previous results” and “managers’ positive attitude to evaluation” (most probably due to the small number of respondents in the sample). Furthermore, the “efficient system for performance analysis” and “methodology for analysing performance’ is more in common in the broadcast sector than in other sub-sectors.

  • Surprisingly, it is the broadcast sector and the art field where the achieved results are compared against the set goals the most.

More detailed results based on the research questions are presented below and briefly commented upon in the following discussion.

5.1 How Is the Data on Performance Collected and/or Analysed in CCI Organisations in Estonia?

As the authors expected that evaluation of organisational performance is not very widespread among CCI organisations in Estonia and managers are not very familiar with the specific evaluation terminology, the respondents were asked rather general open-ended question at the end of the structured questionnaire—“How do you collect or analyse feedback from your stakeholders (please name the tools and/or methods used)”. Coding the responses revealed the results presented in the following table (Table 4).

Table 4 Most common ways of collecting and analyzing performance data from stakeholders by CCI organizations in Estonia (compiled by the authors)
Full size table

A total of 299 respondents (65% of sample) decided to answer this question. The responses reveal more information about how data is collected (mainly face-to-face, surveys, etc.) and only 2 specific data analysis tools were mentioned—Google Analytics (3) and Cost-Benefit Analysis (1). However, they cannot be considered CCI-specific tools.

A few rather passive and/or self-evident approaches were mentioned by the respondents as ways to collect data; therefore, the understanding that clients should come and give feedback on their own initiative (for instance, in social media or in a guest book) seems to prevail, instead of CCI organisations collecting it intentionally. However, a small number of respondents also referred to outsourcing data collection and analysis or contributing to a larger survey. 2.2% of respondents are regularly not collecting or analysing any data. One response is worth highlighting in particular—“Please forgive me, but this question does not make any sense to many creative units because “business model”, “quality system”, etc. have nothing to do with creativity, which is driven by internal and idea-based needs”—as it contributes to the general picture of how evaluation of organisational performance is understood among CCI organisations in Estonia.

5.2 Is There Any Significant Correlation Between the Evaluation of Organisational Performance and the Challenges or Skills Gaps in CCI Organisations in Estonia?

Table 5 below illustrates the correlations between six organisational performance evaluation variables and 14 challenges and skills gaps of CCI organisations studied. Significant correlations are highlighted.

Table 5 Correlations between the evaluation of organizational performance, challenges and skills gaps typical of CCI organizations (compiled by the authors)
Full size table

Based on the correlation table, it becomes evident that “no confidence in income” correlates significantly with all the evaluation variables; however, all the correlations are negative. Furthermore, more finance-related challenges correlate with some aspects of the evaluation of organisational performance—“making a profit” and “financial management”. There are two more variables—“challenging strategic planning” and “challenging analysing and reporting”—that are negatively related to most of the aspects of organisational performance evaluation. Surprisingly, only “protecting copyright is challenging” relates weakly, albeit positively, to some aspects of organisational performance evaluation. On the other hand, “compliance with laws” only correlates negatively with some aspects of organisational performance evaluation.

5.3 How Do the Challenges Related to the Evaluation of Organisational Performance in CCI Organisations Differ Based on the Form, Size, Sub-sector and Age of the Organisation?

Table 6 below presents the level of agreement with the following 3 challenges—“no confidence in income”, “making a profit” and “protecting copyright”—in CCI organisations based on the number of employees, organisational form, age of the organisation and sub-sector.

Table 6 Agreement with the statements about following challenges: no confidence in income, making a profit and protecting copyright based on the number of employees, organizational form, age of the organization and sub-sector (compiled by the authors)
Full size table

As seen from Table 6 above, CCI organisations with more employees seem to face less challenges. “Making a profit” and “no confidence in income” seem to be issues especially for the smaller organisations.

The biggest struggle for most of the organisational forms tends to be coping with “making a profit”, while only for NGOs “no confidence in income” is a mayor issue as well.

Age group tends to make a smaller difference than number of employees or organisational form in CCI organisations. Surprisingly, organisations that have existed 6–10 years, experience more financial challenges than younger or older organisations. One more surprise was detected—it is the oldest age group, which organisations consider “protecting copyright” more common challenge than younger organisations.

“Making a profit” tends to be a dominant challenge in all sub-sectors of CCI organisations. Music and Broadcasting industries also challenge with “no confidence in income”, especially compared to Entertainment Software where this seems to be a minor issue. “Protecting copyright” is especially relevant for organisations in Handicrafts, Architecture and Libraries, but is not an issue at all in the Entertainment Software sector.

5.4 How Do the Skills Gaps Related to the Evaluation of Organisational Performance in CCI Organisations Differ Based on the Form, Size, Sub-sector and Age of the Organisation?

Table 7 below illustrates the level of agreement with the following skills gaps of “financial management”; “strategic planning”; “being in compliance with laws” and “analyses and reporting”—in CCI organisations based on the number of employees, organisational form, age of the organisation and sub-sector.

Table 7 Agreement with statements about the following skills gaps: financial management, strategic planning, being in compliance with laws, and analyzing and reporting; based on the number of employees, organizational form, age of the organization and sub-sector (compiled by the authors)
Full size table

All in all, CCI organisations tend to agree less with having skills gaps than facing challenges, discussed in the previous sub-chapter.

Table 7 above reveals that CCI organisations that do not have any paid staff members, struggle the most with all skills gaps measured, while “strategic planning” is also quite challenging for organisations with 6–10 employees. Surprisingly, “analyses and reporting” does not seem to be an issue for organisations that have more than 10 employees.

When looking at different forms of organisation, the largest differences could be seen between NGOs and foundations. NGOs tend to struggle more with “financial management” and “strategic planning” then other types of organisations, while “strategic planning” is of less concern for foundations than any other organisation form.

Older organisations seem to struggle slightly more with “financial management” and “strategic planning” than younger organisations, while the youngest organisations seem to lack primarily the competence to “analyse and report”. Organisations that are older than 25 years, tend not to struggle with “compliance with laws” or “analyses and reporting”.

Compared to the other sectors, the Performing Arts and Entertainment Software sectors tend to lack “financial management” skills more. While “strategic planning” tends to be more complicated for the Performing Arts, Film and Video, Music and Handicrafts sectors. “Being in compliance with laws” tends to be less challenging for the Publishing sector than other sub-sectors. A lack of skills for “analysing and reporting” seems to be the biggest issue for the Entertainment Software industry compared to other sub-sectors and is non-existent in the Broadcast industry.

6 Discussion

Resources that are obtainable in the external environment are said to shape the survival of players in the creative industries (Noyes et al. 2012). The current study found some empirical evidence, that challenges and skills gaps may be associated with the evaluation of organisational performance. As Clarence W. Barron has said “everything can be improved” and based on the empirical evidence found, this is also true of the efficiency and efficacy of strategic management and planning in CCI organisations. This chapter explored the challenges and skills gaps that might limit an organisation’s ability to develop and expand. The findings do not yet provide a definitive answer to the question of whether evaluation of organisational performance is the right solution for improving the performance, competitiveness, effectiveness, service quality, resilience or sustainability of CCI organisations. They nevertheless highlight the need for more research aimed at improving the knowledge of strategic management in the CCI sector.

The results revealed differences between CCI organisations based on size, age, legal form and sub-sector, thus, CCI organisations can be considered heterogeneous. The current study also revealed that CCI organisations face different kind of challenges that correlate with their organisational performance evaluation practices. However, the findings did not indicate any usage of existing organisational performance evaluation methodologies referred to in the existing literature. This is rather surprising, as the benefits of evaluation of organisational performance should be known among the managers of CCI organisations in Estonia.

The study by Turnbull (2011) has shown that the successful evaluation of performance helps organisations become more conscious of the competencies they hold and of their artistic quality and as a result, make more informed decisions about their performance (Epstein and Mcfarlan 2011). For instance, the quality frameworks and organisational performance measurement tools make the organisation think about where they stand (Turnbull 2011) and where they would like to be. This might lead to better planning and analysis and in longer run to a higher quality of organisational performance.

7 Conclusion

As the study aimed to find out how heterogeneous background of CCI organisations relates to the evaluation of organisational performance, challenges and skills gap in CCI organisations, the simple answer would be—“directly”! Thus, CCI organisations in Estonia are diverse in form, size, sub-sector and age. However, the results also revealed that there is no one single dominant variable that affects all the aspects of evaluation of organisational performance. The challenges correlating with the practices and mind-sets of organisational performance evaluation differ based on the organisational form and sub-sectors the most, while the size and age of the organisation seem to play a slightly less important role. On the other hand, the results also revealed that the skills gaps correlating with organisational performance evaluation practices and mind-sets differ the most among different sub-sectors and number of employees.

To summarize the most important findings, the study also revealed that some elements of evaluation of organisational performance are more in common in Estonia than others—for instance, planning in most of the CCI organisations examined depends on the basis of the analysis of previous results. Most of the data on performance in Estonian CCI organisations is gathered from stakeholders through face-to-face contact and surveys. A few rather passive and/or potentially self-evident approaches were also mentioned by the respondents that might refer to a widespread approach that stakeholders should come and provide feedback on their own initiative, instead of the CCI organisations collecting it intentionally.

The results revealed that having “no confidence in income”, “challenging strategic planning” and “challenging analysing and reporting” do negatively influence most of the aspects of evaluation of organisational performance. The biggest challenge for different types of organisations tends to be coping with “making a profit”. CCI organisations with more employees seem to face less challenges in general. Organisational performance evaluation practices seem to be least different based on the age of the organisations.

CCI organisations that do not have any paid staff members, struggle the most with all skills gaps measured. Also, empirical evidence was found that, older organisations seem to struggle more with financial management and strategic planning than younger organisations, while the youngest organisations mainly lack the competence of analysing and reporting, but also financial management.

The following framework of factors related to the evaluation of organisational performance is suggested by the authors as a result of the study (Fig. 1).

Fig. 1
figure 1

Framework for the evaluation of organisational performance in CCI organisations (compiled by the authors)

Full size image

The contribution of the study is mainly practical. As it mapped the skills gaps of CCI organisations; policymakers and managers of CCI organisations could use the results as an input to contribute to skills development of CCI organisations by developing evaluation/assessment tools and providing trainings to improve the level of needed competences. Thus, there are reasons to assume that raising awareness of the benefits of the organisational performance evaluation and the existing evaluation tools, could improve the sector in a longer run.

Due to the limitations of the current research design, we call for more research on the factors that affect the evaluation of organisational performance by suggesting following questions for further research:

  • How could national and supranational institutions help to improve the competencies for practicing the evaluation of organisational performance?

  • What could be the ultimate benefits for CCI organisations to implement the evaluation of organisational performance?

  • What characterizes CCI organisations that do or do not perform evaluative practices?

Such a list of questions cannot be answered by a single study, and different angles and approaches should be addressed to tackle the issues of strategic management in CCI for the sake of improving the sector.