Self-Service Business Intelligence

1 Analytical Empowerment of Business Users

In recent years, Business Intelligence (BI) systems have undergone two distinctly different fundamental changes. On the one hand, social media systems, machine sensors, devices like smart phones, and other sources generate new data which often differ from traditional operational data regarding their structure, rate of growth, and volume (McAfee and Brynjolfsson 2012). On the other hand, the scope of BI has been extended from strategic questions to operational tasks so that more employees have a need to apply BI (Böhringer et al. 2010). These developments have created more demand for BI reporting and pattern search. In addition, the frequency of requests for changes by businesses has also increased (Yu et al. 2013). Consequently, BI specialists who are either IT professionals or experienced BI users in functional departments (so called power users) has become an even bigger bottleneck than before (Kobielus 2009). Unexperienced users (so called casual users) who need to make time-critical business decisions sometimes act, therefore, without exploiting all available data (Abelló et al. 2013). In response to these developments, the approach of Self-Service BI (SSBI) has been suggested (e.g., Imhoff and White 2011). SSBI should, on one hand, empower casual users to perform custom analytics and to derive actionable information from large amounts of multifaceted data without having to involve BI specialists. Power users, on the other hand, can accomplish their tasks with SSBI more easily and quickly than before. Or, in Eckerson’s words (2009): “users create exactly the reports they want, when they want them…”.

The description of this term does not reveal the difference to the goals of traditional BI systems, but SSBI systems offer new features to accomplish the goals. One new option that will be described below in more detail can serve as an example: in some SSBI applications, business users (casual or power users) are enabled to integrate external data (e.g., from social networks) with data in the data warehouse and analyze them “on the fly.” SSBI also takes advantage of new software architectures as explained below.

The desire to let business users access enterprise data has existed for a long time. SQL was partly developed with this goal in mind (Codd 1974). It indeed made the access easier, but mainly for programmers. The next attempt was undertaken with the introduction of data warehouses (Devlin 1996). This enabled power users to retrieve data and build reports or models by themselves. Meanwhile, more users experience the need to analyze data quickly and independently. While the goals of traditional BI have not changed, in SSBI the computing environment and new tools can fulfill promises and meet the users’ needs. The first step in this direction was taken when BI architecture changed from client/server applications to Web applications. In this architecture, client software installations for new users (and updates for old users) are no longer necessary, users can access the resources through an interface they are familiar with (the web browser), and pushing data to client machines has become obsolete. Today, the central resources can even reside in the cloud, adding flexibility while decreasing total cost of ownership of BI systems. Additionally, advances in hardware and distributed systems make a fast analysis of large amounts of data possible.

Business users are accustomed from other IS they use, privately or at work, to carry out tasks without the intervention of specialists (e.g., online-trading of stocks or direct handling of human resources information), so they are not only capable to accomplish SSBI with the right tools but they may even expect to have this option. Of course, they cannot be expected to write the program code. This “analysis democratization” does not mean that all business users are equal. They continue to have different demands and skills. However, if they have the appropriate skills and access rights, they are not doomed to wait until BI specialists have prepared the requested information for them. End-user analysis shifts from simple consumption of information to the opportunity of trying out different representations and analyses (Stodder 2015). The users are empowered to move from exploitation of data to their exploration. At the same time, the coordination effort between business users and BI specialists should be eliminated as far as possible.

2 Levels of Self-Service

The self-service concept can be implemented with respect to different tasks: access to prepared reports or data resources, direct access to data, access to functions, or creation of new resources. The system support necessary varies with these tasks. A corresponding classification is shown in Fig. 1. Software applications or systems exist for each of the shown levels. Some are add-ons to existing BI systems while others stand alone. Not all of the features shown are new but their implementation for a large group of users, the user friendliness, and the power exceed the possibilities of traditional BI systems.

Fig. 1

Levels of self-service

2.1 Usage of Information

At the lowest level, the users receive access to information that has already been created (existing reports) or where they only need to set parameters before processing them. This differs from traditional reporting systems by the fact that not only a few standard reports are provided. The user obtains access to all reports that are potentially relevant to her. The advantage of this approach is that it is well suited for casual users without special analytical or tool skills. Basic insights can be derived easily if the needed information is found and interpreted correctly. For deeper, individual insights, this solution is not flexible enough.

A further improvement at this level includes the provision of reports and dashboards with a “drill anywhere” possibility (Eckerson 2009). Users start their analysis on a highly aggregated level. If questions that are more specific need to be answered, users can drill down predefined paths step by step until they obtain data at the required detail level (or reach the bottom of the aggregation ladder). For a quick provision of data outside of the data warehouse, these do not need to be imported into it. A switch to another data source is either intuitive from a front-end perspective or the user does not notice it at all. This guidance simplifies the analysis but it remains restricted to what has been prepared by BI specialists.

2.2 Creation of Information

At the second level, users can be granted access to data at the lowest disaggregated level available in the system to create new information from it. The rationale is that others cannot foresee all the needs that users may have and prepare appropriate views for them. In principle, with SQL a non-procedural language for general access to data already exists. However, it has turned out to be too complex for the majority of business users. The new tools create virtual views almost on the fly, even from big data files if necessary. Depending on the product, casual users can choose whether they want to analyze existing data as flat, relational, or multidimensional files (Hänel and Schulz 2014). Thus, they are no longer dependent on BI specialists to select the data they need. This, however, also carries the risk that incorrect data excerpts or aggregates are selected because casual users may have less understanding for complex data relationships that still lie in the background.

In addition to the preparation of reports and charts, business users can also be given the opportunity to autonomously perform advanced analytics (e.g., predictive analytics and text mining) that go beyond the analyses of historical data. Due to the complexity of these functions, business users are often not able to correctly formulate their analysis requirements. It is necessary to make analytical functions available to them without the need to master a statistical package. Here, again, there is also a risk of faulty analysis. Especially mathematically and statistically inexperienced users have little chance to assess the correctness of their calculations.

2.3 Creation of Information Resources

Functions at this level clearly go beyond traditional BI systems. In traditional BI systems, the data from different sources are combined and offered to the user as a unified source. Due to the emerging variety of data sources and user requirements, this data preparation is becoming increasingly challenging. Therefore, business users can be given the opportunity to autonomously harness new data sources for analysis that are not preprocessed by IT. They create new information resources by (temporarily) combining these data with corporate data. A simple implementation can give business users the opportunity to load data into a personal workspace. Systems that are more complex enable autonomous integration of various sources with corporate data. This creates new pitfalls. Relationships need to be identified while usage of poor quality data or the sidestepping of existing access rules by sharing these data with other users must be avoided (Stodder 2012).

Another possibility that can be offered to business users at this level is to combine different functionalities using reusable components that have been prepared by IT professionals. Due to the preprocessing, these elements can be joined through simple drag-and-drop to form a so-called mashup, e.g., in the appearance of a dashboard (Kobielus 2009). The actual complexity of this approach remains hidden to the business user.

2.4 User Roles

The more flexibility is offered to the users, the more BI skills they must bring along (Spahn et al. 2008). Therefore, the idea of SSBI does not mean the same for all users. The fit between users and appropriate BI tools needs to be determined based on their specific tasks, their informational demands, their computer skills, and their analytic skills (Eckerson 2014). This allows adjusting the self-service concept to every business user giving her enough flexibility but without overtaxing her with functionalities. A strict classification of users into discrete groups according to their business function is not necessary as they may have alternating needs and different skills independently of their function; it is sufficient to control their individual access to data. As mentioned above, a rough division of user types into casual and power users can be made based on the level of their analytic and tool skills. Casual users consume information most of the time while power users also produce information, either for themselves or for others (Eckerson 2014).

3 Current System Support and Future Developments

Different levels of SSBI require different system support. Some support for self-service has been available for many years, often under a different name. However, a recent survey of 348 BI users shows that the majority of users can enjoy self-service in only one out of eleven data preparation steps (Stodder 2015): transferring data from a company data repository to an individual desktop (e.g., for further analysis in a spreadsheet). For all other steps, e.g., create new dimensions, enrich data, integrate structured and unstructured data, the help of BI specialists is usually needed.

Flexible retrieval of existing data, incl. creation of new views on data, is difficult for business users due to the complexity and variety of data structures. BI specialists, therefore, often create simpler presentations of data structures by “flattening” the data for end users even if they are actually stored in a complex relational or a multidimensional database (e.g., SAS LASR Analytic Server¹). Research is still going on to determine what is the best presentation of data (structures) for business users who want to conduct data exploration on their own (e.g., Hänel and Schulz 2014). Another approach to empower users is the provision of apps by BI specialists that casual users can combine to create their own interactive dashboards (e.g., InfoApps by Information Builders²). These go beyond parametrization or drill-down functionality.

In the past, mainly aggregated operational data (perhaps enriched by some structured data on competitors) were used in BI systems. Nowadays, data from different, often volatile sources and with different degrees of structure are also used for analysis. However, these data need not to be imported through a regular extract, transform and load (ETL) process. Data virtualization can be used to provide these data to the user for analysis only, rather than to store them physically. The modeling can be carried out at the time of analysis based on current analytical requirements (e.g., Cisco Information Server 2015³). Performance issues that have prohibited such an approach in the past can be partially ignored nowadays due to advances in hardware (Imhoff and White 2011). Moreover, because of the direct access to source data, it is possible to deliver real-time information to business users, which is often needed for operational tasks. The integration of these sources with enterprise data is another challenge.

Even though the majority of data integration tasks will still be performed by IT specialists, business users can also partly execute ETL and data integration by themselves. In a simple case, users may be given the possibility to load spreadsheets into personal BI software (e.g., IBM Cognos Express,⁴ Microsoft PowerPivot for Excel⁵). Specialized applications support the blending of external data with corporate data for an integrated view (e.g., Alteryx⁶ and Trifacta⁷). Abelló et al. (2013) suggest creating “fusion” cubes, which augment existing company cubes by dimensions and facts that may originate from the web or some other unstructured source. The new elements are described in the Resource Description Framework format. Such data integration is prone to quality problems and needs to be carefully controlled, which is also an organizational issue that will be discussed below. Varga et al. (2014) propose integration of heterogeneous data sources through automatic handling of metadata (relating to systems and users). In other words, systems that are designed to support SSBI at this level must be able to use metadata provided by external data resources or even extract them from the data automatically.

Once additional data are made available to business users (without intermediaries), they should also be enabled to analyze the data by themselves. To provide these capabilities, the most appropriate algorithm (e.g., for forecasting) or the most appropriate visualization is automatically chosen by the system based on the selected data (e.g., SAP Lumira⁸). However, one shot will usually not be sufficient; therefore, the user must be able to interactively improve the analysis or visualization (e.g., through appropriate filtering, aggregation, or clustering) until she gains the required insight. The goal is to reduce the time needed to understand data, to find patterns, and to derive insights (Stodder 2015).

SBBI can be applied in all industries and many functional areas. For example, a German pharmaceutical company has stopped delivering data cubes to regional sales people but is offering them self-service dashboards instead (Ziff Davis 2012). SSBI can also be rolled out to customers. A North American insurance company offers their policyholders an app-based dashboard so they can retrieve bills, eligibility information, and claim details (Information Builders 2015).

4 Organizational Issues

The idea of SSBI is not yet widespread within companies. According to recent statistics, only about 22 % of potential business users can use some form of self-service in practice (Logi Analytics 2015). There are also reports of failed deployments. A simple provision of data and analytical functionalities to a large number of users does not lead to success. In one example, a company compiled a portfolio of 26,000 reports after a few years of employing SSBI (Eckerson 2008). The company has subsequently abandoned this approach and created 300 standard reports that met most of the existing informational needs.

One way to support diffusion of BI use in companies is to assist users in helping each other (user collaboration). This can be done through enterprise social networks and wikis. Users can, for example, reuse and adapt reports developed by other users in other parts of the organization without knowing each other (e.g., Alpar et al. 2015). Social features such as a star rating or the number of accesses can also help business users to evaluate the quality of reports. In addition, information retrieval techniques and recommender systems can support business users in finding relevant information (Imhoff and White 2011).

The importance of data governance and security increases with the addition of (external) data. Ideally, central systems and directories for access rights can be used when temporary or permanent new data structures are created. It must be decided, for example, who is allowed to add data, which metadata must be provided for the added data, how long they should be kept if at all, should there be a minimum data quality requirement. Otherwise, SSBI could become a loophole for security breaches. However, access rights should not be so restrictive as to hinder the idea of SSBI. Business users must be encouraged to use their creativity and mine the potential treasures buried in the available data without technical restrictions.