Keywords

1 Introduction

Analytics and analysts are growing in all the work streams. Analytics is used in different ways for different information and hence we have different types of analytics used all over. Analytics is used to improve the performance of an organization to get better profits along with better outcomes of their applications. In order to improve the performance, the past data should be recorded and analysed and prediction should be done. All the fields use analytics in which each field does analytics in a different way and hence gets a different name. Henceforth, we have huge count of analytics. We have started our research by collecting various types of analytics that are available excluding the basic four types; viz., descriptive, diagnostic, predictive and prescriptive analytics. We collected all the analytics in the alphabetical order and we have noticed that, this whole data can be grouped together based on their similarities. In the later sections, we have researched about the companies in that field and implementation of big data analytics to achieve profits.

Before the invention of data processing tools like Hadoop and Spark, traditional file systems, DBMS, RDBMS were used to store and process the data. These could not handle huge amount of semi-structured and unstructured data. Hence, to handle different types of data, we make use of tools such as Hadoop and Spark, which makes the work easier and very efficient. This is still a developing stream where technologies are getting updated on a large scale.

Size in terms of Big Data is not a fixed quantity. The size of the data has increased to terabytes and petabytes. This will increase in the future, as there is no limit for the streaming data. In 2012, Gartner updated Big Data’s definition as, “Big Data is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.” [1] The characteristics viz., volume, velocity and variety are not sufficient to determine all the characteristics of the data anymore [2]. Hence the number V’s are extended to 10 namely, volume, variety, velocity, veracity, validity, value, variability, venue (public and private cloud) [3], vocabulary, vagueness.

2 Data Processing Techniques

Data storage systems play a very essential role. The efficiency of storage is important challenge and hence there are various techniques available. Traditional file systems were used initially and were of great use, but this had many disadvantages and hence to overcome them, Data Base Management Systems (DBMS) came into use [4]. Execution time was reduced, security was increased and memory storage was possible when DBMS was introduced. Moreover instead of checking each and every alphabet just to get the record of a single file, we can use the database system, which helps in an easy retrieval of the files with less efforts (except that the user should know the query language). This reduced the complexity in retrieving data and also the access time.

Though there was a significant improvement in the performance after using database management systems and relational database management systems, there were few disadvantages when it has to deal with huge amount of data [5]. Hence, Big data tools like Hadoop and Spark came into existence, which can handle vast data very efficiently. The disadvantages of DBMS when it has to handle huge amount of data are: Data dependency (it becomes complex when dependencies are high), storage (if the data is huge, space required will be high which is cost ineffective), causation (when filtering of the data is done, important information might be lost which effects decision making), etc.

Since data can be collected from various sources, there are different types of data available. These are majorly from digital sources. Some of the different types of digital data are: Structured data i.e., the data have a high degree of organization and will be in a proper order, which makes search very easy. Semi-structured data means that the data is partially in-order. Examples of semi-structured data are mails, various markup languages (HTML, XML, SGML) etc. Unstructured data is the kind of data that is being collected from various sources. Generally, this data does not have a pre-defined data model or it is not organized in a pre-defined manner. Even if the data has a structure it is considered to be unstructured if it does not follow a predefined structure.

One of the most essential advantages of Big data analytics is that it can handle unstructured data with ease which contains audio, video files along with all the kinds of data available (like text, sensor data etc.) [6].

Figure 1 explains the change in amount of usage of technologies over a period of time. Along with the time, technologies that are used to store and manage the data also got progressed which provides an easy and efficient management. In 1960, only file systems (F) were used whereas in 1965, DBMS (D) was introduced and it overtook the usage of file systems. In 1980, RDBMS (R) was found to be better than DBMS, which resulted in high fall in the usage of file systems. In 2006, Hadoop (H), a big data tool was invented, which was found to be very efficient in handling vast amount of data and then Spark (S) (which is also a big data tool) was introduced in 2012, which overcame the disadvantages of Hadoop.

Fig. 1.
figure 1

Extent of usage of different technologies

Full size image

3 Description of Various Types of Analytics

The four main types of analytics are:

  • Descriptive analytics: History is an essential factor in the case of analytics and hence, we need to maintain a log of things those occurred in the past which helps us to learn what has happened. As the name suggests, descriptive analytics means a description of an event, or summarization of raw data to make it human interpretable. Data mining algorithms are used to identify patterns from already available information of the past and help us in data visualization. The main application of this is in E-commerce websites, to keep track of the customer’s wish list and suggest good products accordingly. It is also used to predict a customer’s financial performance [8]. This is considered to be as the preliminary stage for analysis.

  • Diagnostic analytics: Troubleshooting a problem is a very important task. It helps us to find out the root cause of the problem and helps us to fix it so that the problem does not repeat. Diagnostic analytics is the next phase of Descriptive analytics and considered to be the second stage of analysis. In this phase we analyze and find out what happened and why did it happened. Like Descriptive analytics, Diagnostic analysis also makes use of data mining algorithms and takes a deeper insight of the data to analyze. This is a very essential step because the same mistakes would cost us a lot and we need to prevent committing same mistakes as done in past. This is applied mostly in Social media to analyze the number of posts and media shared.

  • Predictive analytics: Interest in future has lead to the introduction of predictive analytics. It uses the historical data and by applying machine learning techniques and statistical algorithms the future outcomes are predicted [9]. The result does not depict of what will happen in the future, instead it tells what might happen along with the reliability factor. Some of the applications are: increase in life expectancy is possible by predicting the disease by analyzing the symptoms [10] in medical field. In the same way, the profits of the business is increased exponentially just by using this predictive analytics on their products. Fraud detection is a very big application of predictive analytics.

  • Prescriptive analytics: Prescriptive analytics relates to both descriptive and predictive analytics, which means that the outcomes of both these analytics are used to give the best solution for a situation. In a form, it is an extension of predictive analytics in means that, when predictive analytics says what might certainly happen, prescriptive analytics talks about why will it happen and what can be done further along with what might happen (the best possible solutions particularly). Applications of prescriptive analytics are slowly growing and few of them are in healthcare, pricing, travel and transportation optimization, oil production through fracking [11].

The process of analysis should initiate with descriptive followed by diagnostic, predictive and prescriptive. Following the above order gives the best outcome.

4 Application

As analytics are used in various fields, we have done a study on how they are used in different organizations.

4.1 Healthcare/Medical

In the evolution of healthcare practices and research, big data analytics is playing a major role nowadays. The main reason for this is that big data can withstand with all structured, semi-structured and unstructured data. Moreover a large volume of data is being generated due to advances in the technology in medical and clinical research field [12]. By implementing big data analytics in the medical field, doctor’s work would be made easier to take decisions within seconds and improve patient’s treatment. One of the leading companies that is using big data analytics in the field of health care is ALTEN Calsoft Labs. Their solutions help organizations to get actionable insights to build competitive strategies based on the four fold path of capture, store, and process and analyze. Archimedes, one of the companies that use big data analytics in the healthcare sub-verticals, is best known for its robust disease models and simulations. Kyruus, another such organization using network analytics to identify and understand key social connections that drive commercial, organizational, and health results. Predilytics and Recombinant also belong to this list. Most of the organizations use predictive analytics as it helps us to predict based on the past information which also means it uses Descriptive analytics. OptumHealth is another leading organization in the Global Healthcare Data Analytics market. Its care solutions include health management solutions such as wellness, complex medical support, decision support systems, and physical health programs [13]. Quantzig healthcare analytics solutions help clients in the healthcare industry including pharma, medical devices, diagnostics, and healthcare delivery companies to address key challenges, reduce costs, increase margins, and gain a competitive market advantage. Healthcare analytics solutions include integration of electronic medical records, clinical trials, hospital records, physician notes and pharmacy data to create consolidated and actionable insights, and risk mitigation and resolution of issues for medical cost management, providers’ claim processing, and improved payment accuracy. Verisk Analytics is among the leading vendors in the Global Healthcare Data Analytics market. The company provides five types of health solution suites: Enterprise Analytics, Payment Accuracy, Quality & Compliances, Revenue Integrity, and Life and Legal [14]. The Enterprise Analytics suite helps organizations to make informed decisions as well as improve cost containment and quality of care provided to patients. The Payment Accuracy suite helps in fraud detection and real-time claims that uses Predictive analytics. MEDai’s health record management solution helps to manage patient records and provides an expert level of care to patients. Its healthcare compliance solution helps discover policies and procedures needed to be followed in healthcare enterprises. MedeAnalytics is also one of the prominent vendors in the market. The company provides a wide range of products for hospitals and individuals. MedeAnalytics’s solutions are delivered using the SaaS model. Its product, Clinical Performance Manager, provides a daily insight into clinical cost and the performance of the individual physician [15]. Health care uses mainly of Descriptive analytics by storing the past data of the patients which is useful for predicting the patient’s health status and while the patient has to get diagnosed, along with Descriptive analytics, Diagnostic analytics is also used get the insight.

4.2 Business

The concept of big data has been around for years. Applying analytics to the data that flows into the business can provide insane amount of value to the organization. Earlier only few companies understood this and applied basic analytics and later on when the term big data evolved, almost all the companies adopted it. Data analytics offers both speed and efficiency in decision-making based on the past data. Companies use both In-Memory analytics and predictive analytics. In-Memory analytics is by analyzing data from system memory (instead of from your hard disk drive), you can derive immediate insights from your data and act on them quickly [16]. This technology is able to remove data pre-processing and analytical processing latencies to test new scenarios and create models; it’s not only an easy way for organizations to stay agile and make better business decisions, it also enables them to run iterative and interactive analytics scenarios. Predictive analytics technology uses data, statistical algorithms and machine-learning techniques to identify the likelihood of future outcomes based on historical data. It’s all about providing a best assessment on what will happen in the future, so organizations can feel more confident that they’re making the best possible business decision. Some of the most common applications of predictive analytics include fraud detection, risk, operations and marketing. SAP’s best Big Data tool is its HANA in-memory database, by which the company says can run analytics on 80 terabytes of data, integrate with Hadoop, search text content, harness the power of real-time predictive analytics, and more. Oracle has its Big Data Appliance that combines an Intel server with a number of Oracle software products. They include Oracle NoSQL Database, Apache Hadoop, Oracle Data Integrator with Application Adapter for Hadoop, Oracle Loader for Hadoop, Oracle R Enterprise tool, which uses the R programming language and software environment for statistical computing and publication-quality graphics, Oracle Linux and Oracle Java Hotspot Virtual Machine. Google is more of a cloud services company but it is making a push into Big Data analytics by offering BigQuery, a cloud-based Big Data analytics platform for quickly analyzing very large datasets. Unlike most services, you send data up to BigQuery rather than store it in the cloud. Apart from these Tier 1 companies, there are other companies using big data analytics for marketing purpose. Starbucks manages to open new stores in very close proximity with their other stores. It uses big data to determine the potential success of every new location prior to expanding their operations. With location-based data, traffic data, demographic data, and customer data, they’re able to estimate the general success rate of each new store, so they can choose locations based on the propensity toward revenue growth, thus decreasing the financial risk of each new store. T-Mobile is using big data to help reduce their customer turnover rate. By analyzing big data, they can determine the core causes for turnover, allowing them to implement effective solutions that will keep more clients on board [17]. As a telecom company, they accrue boundless quantities of data every year, and without big data management, the ability to analyze the data would be greatly inhibited. In business analytics, all the four types of analytics are used and very essential unlike medical/healthcare.

4.3 Sports

Big data in sports industry has been gaining momentum since 2007 and now tracking has become normal. The sports world isn’t immune to the impact of big data Statistics-driven sports like Major League Baseball and the National Football League have long crunched numbers to make key decisions; so using big data is a natural progression. Analysts and trainers pored over data to predict performance and develop strategies [18]. Whether motivated by profit or the quest for a win, greater efficiency and increased accuracy, the sporting world is embracing big data to improve performance. Kitman Labs has collaborated with several Olympic teams to prevent future sports injuries by using Big Data and Analytics. The sports and data technology company Kitman Labs uses a unique Athlete Optimization System. This system allows team performance directors, coaches and trainers to understand how athletes are responding physically, as well as mentally to the stresses endured during training and exercise at levels of high competition. Any signs of negative response can trigger the staff to adjust an athlete’s training and recovery program to proactively avoid injury. Data analysis is used to identify undervalued players when constructing baseball teams [19]. Predictive analytics looks at patterns in historical data to determine future performance and trends. With algorithmic and biomedical advances, the sports industry has greater confidence in predicting and measuring the success of current and future players. In the NBA, data analysis has provided teams with better ways to measure player efficiency and defensive effectiveness. A player’s value can be measured by a number of metrics, including player efficiency rating, win shares and wins above replacement player. Coaches famously watch video to gauge opponents’ skills and improve their own player’s performance, using knowledge gleaned in the training room, but now they’re also using big data analytics to gain an edge. With RFIDs, sensors and GPS trackers, training and coaching staff can capture information, feed the data into analytical engines and use it to influence strategic decision-making. This approach can help them choose exactly the right player for any given play. Ultimately, the goal for athletes, trainers, coaches, broadcasters and others involved in sports decision-making is to leverage real-time data to improve live performance. Like smaller companies that must find an edge to compete successfully with larger enterprises, a team with a smaller budget or more limited pool of athletes can use big data and analytics to gain an advantage. In that way, big data can make sports smarter.

4.4 Education

Online courses and learning systems have been gaining tremendous popularity over the last few years. While their ease of access and availability makes them a very useful medium for knowledge sharing and learning, they do not keep the learners and their learning abilities in mind. Big data allow for very exciting changes in the educational field that will revolutionize the way students learn and teachers teach [20]. Big Data can help to create groups of students that prosper due to the selection of who is in a group. Students often work in groups where the students are not complementary to each other. By using algorithms it will be possible to determine the strengths and weaknesses of each individual student based on the way a student learned online. This will create stronger groups that will allow students to have a steeper learning curve and deliver better group results. It will give students the opportunity to develop their own personalized program, following those classes that they are interested in, working at their own pace, while having the possibility for (offline) guidance by professors. Big Data can give insights in how each student learns at an individualized level. Each student learns differently and the way a student learns affects the final grade of course. Some students learn very efficiently while other may be extremely inefficient. When the course materials are available online, it can be monitored how a student learns. This information can be used to provide a customized program to the student or provide real-time feedback to become more efficient in learning and thus improve their results. Predictive analysis can serve many segments of society as it can reveal hidden relationship that may not be apparent with descriptive modeling. Analytics advancement plays an important role in higher education planning [21]. The descriptive modeling can help to evaluate the teaching staff and their excellence in imparting the education. Using predictive analytics on all the data that is collected can give educational institute insights in future student outcomes. These predictions can be used to change a program if it predicts bad results on a particular program or even run scenario analysis on a program before it is start. Big Data can help provide insights to support student’s learning needs. For instance, learning analytics as a fundamental component of Big Data in higher education provide researchers with opportunities to carry out real-time analysis of learning activities. By performing retrospective analysis of student data, predictive models can be created to examine students at risk and provide appropriate intervention, hence enabling instructors to adapt their teaching or initiate tutoring, tailored assignments and continuous assessment. Big Data can afford to shape a modern and dynamic education system, which every individual student can have the maximum benefit from that. Furthermore teachers have valuable tools, were they do not have before, which can make their decisions more specific and are able to choose a big variety of new learning methods. Hence the Big Data are actually involved to change the way of industries including the education. In the new era of data, the traditional difficulties will no longer exists, keeping the good methods. The education system will be enriched with new learning ways, making more efficient and targeted. But the way of this new era, have just began and there are many difficulties such as the lack of experienced personnel on the science of Big Data and Data analytics which can be overcome in the course of time.

4.5 Stock Market

One of the most popular applications of analytics is found in Stock Market analysis. It shares changes every second. If you are able to get good amount of share through stock market investment, that does mean that you have the ability to accurately predict stock behaviour. However, luck factor involves in it but apart from that the knowledge developed through years of experience of dealing in stocks is the important factor [22]. The key point is to analyse data and decipher the relevant patterns. Enormous amount of data is fed into the systems and conventional data mining algorithms are used to process it. Followed by this, big data analytics is applied to the processed data and investments are made. Reliance securities have integrated online trading platform, which uses analytics to provide robotic insights to investors. It scans and captures vast amounts of gathered data, processes it using advanced algorithms, and presents real time analyses. It helps the firm get a macro view of stocks, and provide analytics and optimal risk strategies. Kotak Institutional Equities uses a web-based platform named Consumer Querimetrix that explains and predicts short-term behaviour of Indian investors by analysing vast amounts of data [23]. It uses machine-learning techniques and merges big data analytics to provide consumer insights and capture inflection points. Angel Broking has also incorporated big data analytics in its day-to-day operations to automate processes, speed up activities and enhance customer experience. The firm uses analytics to predict margin-limit multiplier, e-mails and calls classifier, and analysing customer sentiments, queries and complaints. Currently, about 30% of Angel Broking’s trades happen online and nearly 80% of its new clients demand online access. HDFC Securities is using a mobile app that helps its clients to trade stocks, track market movements, manage portfolios and analyse industry trends, using big data analytics. The company has also implemented Oracle SuperCluster that supports its increasing customer base and daily transaction load. The platform has increased online trading speed by up to 60% and enabled HDFC to produce reports 67% faster while reducing risk and cutting data center costs. Aditya Birla Financial Group has implemented an online solution, which provides data and insights for its entire business, and supports its customer centricity vision. It helps the firm generate reports and provide information to all sales, marketing, and customer service related units [24]. The solution provides informed insights for faster turnaround, better administration, and better understanding of customers. It has helped the company bring down processing time by 30–40%.

4.6 Web and Entertainment

Web Analytics helps us to keep track of Visitor’s behavior, performance of website and data flow. In other words it is the collection, reporting and analysis of website data. The primary concern is to use the website data to determine success or failure of these goals to drive strategy and improve the user’s experience. It is not a measuring web traffic but can be used as a tool for business and market research, and to assess and improve the effectiveness of a website. It is often used in customer relationship management analytics. Based on the purchases in the past, the customer is given suggestion, monitoring their likes and frequently searched items to give better recommendation, observe geographic regions from which most of the searches are made, and predicting which products, customers are most and least likely to buy in the future [25]. This can help to improve the ratio of revenue to marketing costs. In addition to these features, Web analytics may include tracking the click through and drill down behavior of customers within the Web site, determining the sites from which customers most often arrive, and communicating with browsers to track and analyze online behavior. Web Analytics follows four basic steps viz., Collection of data, processing the data into information, Developing KPI (Key Performance Indicators) and finally formulating the online strategy. There are two categories of web analytics viz., Off-site and On-site. Off-site web analytics refers to the measurement of a website’s audience, share of voice and the comments. It is regardless of whether you own or maintain a website. On-site web analytics measures the performance of your website in a commercial context. This data is typically compared against key performance indicators for performance and used to improve website. Most of the web analytics tools are free. Google offers a free web analytics tool to track number of users visiting your website and to measure traffic sources and goals [26]. It basically generates reports on Audience Analysis, Acquisition Analysis, Behavior Analysis and Conversation Analysis.

The future of this industry is centred on the convergence of digital and analytics solutions. Therefore, in order to gain customer insight, the enterprises are eager to transform their media delivery. Entertainment solely depends on the audience experience. Audience analytics can help organizations continuously capture audience response from multiple sources so that they can deliver the right content to the right person at the right time. Subsequently, each target audience has a different and unique experience, leading to increased revenues from consumers, advertisers, and overall viewer market share. Media and Entertainment industry demands that content creators and distributors develop a new way to leverage big data to understand and connect with audiences.

As mentioned previously, we have broadly classified the fields as Medical, Business, Sports, Education, Stock market and Web and Entertainment. Figure 2 is a plot that shows the proportion in which these fields are using different types of analytics. From this research we find that Stock Market makes the highest use of analytics whereas Medical and Business fields makes use of analytics almost in an equal proportion. Sports and Education use in moderate amount and Web & Entertainment uses the least. By this we can infer that there will be a great improvement in Education, Sports and Web & Entertainment fields if the use of analytics is significant.

Fig. 2.
figure 2

Usage of big data analytics in the six major fields

Full size image

5 Conclusion

As the technology is increasing, the size of the data that we get from this is also getting increased proportionally. We get data from many sources like sensors, websites, and electronic gadgets etc., which are contributing terabytes and petabytes of data every day. Every organization produces data in large amount irrespective of it being small or big. This huge amount of data needs efficient organization and management. Analyzing the data that has been well organized, stored and managed is equally important to do computations on that data. Since, there are different types of analytics available, usage of these in an efficient way will fetch better results. Each organization uses these analytics in their own way; depending in which field that organization is working. Therefore, we have given a brief description of the organization’s field and their techniques of using these different types of analytics in order to obtain high profits in an efficient manner. This will provide the new comers in this field of big data analytics to understand the concepts of what is big data analytics, types of analytics present and their use in different fields. It also provides an easy path for entrepreneurs in ways of how to achieve better profits by appropriately using analytics in their start-ups. Not only that, but also can be used by other organizations which are in the initial stages of using analytics for a better performance.