Keywords

1 Introduction

It does not raise doubt that marketing science and commercial aspects are the base of marketing activity in both online and offline environments. However, especially for online marketing, the technological background and supporting technologies play the key role. They are developed in several directions including campaign planning [12] or real time optimization towards better conversions [22]. New methods are implemented in the area of algorithms for computational advertising including adaptive approaches [20] or linear mathematical models [9] with their extensions [23]. Attempts to increase the effectiveness of online commercial activity often leads to negative side effects such as growing intrusiveness of online marketing content [33] and, as a result, physical or cognitive avoidance [14]. Searching for compromise between content intrusiveness and its influence on user experience within a web system is one of directions of research in this area [18, 34]. The approach proposed in this paper integrates data related to effectiveness of online marketing content together with the evaluation of its intensity and negative impact on user experience. Changes of online environments are taken into account and multistage decision support with the use of MCDA methods is introduced. Direct application of the MCDA approach in this class of problems is hampered as the MCDA methodology is based on assumptions of stability of parameters forming part of the decision-making support process, e.g. datasets, criteria, decision variants and evaluations. In online planning parameters may change dynamically and are conditioned on the changes in audience characteristics, variable efficiency of advertising message or competitors’ activity [5]. Employing the classic MCDA approach without considering time evolution is the way to oversimplify the problem [17]. In this context the paper presents an adaptation of the MCDA approach for the needs of dynamic decision-making support in the online environment in the process of multi-stage planning of marketing activity. The solutions were verified on the basis of data from real advertising campaigns. The paper is organized as follows: section two includes the review of literature, section three presents the conceptual framework and assumptions for the proposed approach. In the next section empirical results are presented followed by a summary in section five.

2 Literature Review

The development of electronic media and the growing role of online advertising create the area for searching for new solutions both in the practical and scientific dimension. The main purpose is usually to increase efficiency of marketing activity in multiple dimensions. On the level of advertising message, tasks which include the use of persuasion, colors, animation and call to action images [32] as well as identification of factors affecting efficiency [30] are realized. On the operational level, optimization in real time [10] and factorial methods [6] are used. Other areas include the use of available broadcasting resources [7], personification of message [20] or choice of message content on the basis of context [29]. The basis for implementing marketing activity is planning and scheduling ad expositions with the participation of available broadcasting resources. Plans are implemented using advertising servers which carry out the selection of marketing content as a response to a request coming from an internet browser [2]. The problem of optimization of the selection of advertisements was formulated as a task of linear programming with maximizing the number of interactions under given constraints which include the number of times an ad was displayed in a given period [22]. The basic model of linear optimization was developed towards a compromise between searching for and exploration of decision-making solutions [23] and a balanced distribution of broadcasts [9]. Other methods on the operational level are based on the monitoring user activity and maximizing likelihood of interaction [16]. In other solutions the selection of advertising content in based on user profiles created during internet sites browsing [13]. Another model takes into account pricing strategies in the process of managing advertising space and maximization of revenue from the sale of advertising space [12]. The literature review shows that majority of the available optimization systems and models is oriented towards increasing the number of interactions within a webpage and automatic selection of advertisements so that it is maximized. Even though maximizing broadcasts of invasive forms of advertising may increase the financial result, in a short while it may also result in the decrease of user experience and negatively affect the brand perception. In a typical process of designing websites user experience should be taken into account for better functionality and creating solutions oriented on internet users’ needs. A question arises here about the level to which it is worth increasing the intensity of the marketing message in order to draw users’ attention and maintain profit at an acceptable level without disturbing user experience excessively. Excessive use of video, audio and animation within online content results in the problem of excessive burdening with commercial content and brings side effects negatively affecting the user [26]. The solution proposed in this paper integrates parameters related to effectiveness of message and its negative impact on the recipient resulting from the intensity of employed persuasive mechanisms. Taking into account several stages of decision-making support gives the opportunity to reflect the changeability of preferences and measurement data. The basis of earlier-proposed linear models are Pareto-optimal solutions where bringing tasks down to one function makes it more difficult to take into account decision-makers’ preferences and criteria. In MCDA approaches the basis are nondominated solutions and they feature the possibility of taking into account qualitative factors subject to subjective assessment as well. The next section presents assumptions for the proposed solutions.

3 Conceptual Framework

The conceptual framework presented in this stage is the continuation of earlier research based on searching for compromise between marketing effectiveness and user experience [18]. In the proposed approach data from multi-stage campaign is integrated with measurements of intensity of message and subsequently global and local objectives are employed in the evaluation of results from various perspectives. The proposed model allows obtaining compromise solutions on the basis of measurements from a real environment and a decision-maker’s preferences. Methodologically, framework is based in on the Dynamic Multi-Criteria Decision Analysis (DMDCA) which is an extension of static approach [5] along with an indication including the taking into account the dynamics of domains in the MCDA subject matter [1, 21]. At present majority of work done in this field is oriented towards the expansion of the classic MCDA model allowing its areas oriented application for various aspects of dynamic decision making. They are focused on changes in the MCDA domain such as variable sets of decision variants or criteria for their assessment and expand the classical MCDA paradigm (see [15]) with additional components of the decision making process such as changeable spaces or domain analyses [31]. The synthesis of approaches can be found, i.e., in the following works: [5, 19]. Due to the specific nature of online marketing-related issues and great dynamics of environment the research assumes a constant form of the set of decision variants as well as the family of criteria for their evaluation. It is at the same time in line with the specific nature of the discussed decision-related problems. The aspects of dynamics in multicriteria decision making process which were highlighted includes changeability of partial evaluation of decision variants in time (performance tables and global performance of variants – see [15]). The analysis of impact of this changeability on the final result of the decision making process is taken into an account. The presented procedure is based on the classic MCDA framework (see [15, 27]). Additionally, assumptions allowing of the dynamic decision situation modelling were introduced. The research procedure itself was based on a five-stage course of the process of multiple-criteria decision-making proposed by Guitouni [15]. It is composed of: (I) the structuring of the decision situation, (II) the preferences articulation and modelling, (III) the aggregation of these preferences, (IV) the exploitation of this aggregation, (V) the recommendation. The multiple-criteria procedure itself covers stages (III) and (IV), whereas the decision problem can be characterized by stages (I), (II) and (V), where stages (I) and (II) address the input data of the decision process and stage (V) defines the output data. The introduction here of the dynamics of modelled decision situations requires expanding the classical MCDA model with consequences of implementation of the time factor. Let t k denote k-th time period for which the multiple-criteria decision model is built, and, let T denote a set of time periods T = {t 1 ,…,t k ,…,t p }. The intention of the decision maker in k-th multiple-criteria decision problem is to select the alternative that best meets their preference for a specific set of criteria. Further consideration was adopted as a solution to the problem of decision-making to maximize the outcome of the transformation F designating the degree of fulfillment of the criteria selected by the successive decision variants as shown in the equation:

$$ G\left( {a_{k}^{b} } \right) = \hbox{max} F\left( {C\left( A \right)} \right) $$
(1)

where \( a_{k}^{b} \) is the most preferred alternative selected from a set of decision-making variants A in period t k and \( G\left( {a_{k}^{b} } \right) \) is a performance variant \( a_{k}^{b} \) denoted also as an assessment of the fulfillment of criteria C. The course of a DMCDA process formulated in this way is presented in Fig. 1.

Fig. 1.
figure 1

Integration of MCDA methods with multi-stage measurement of performance in online environment

Full size image

The presented procedure expands classical paradigms presented in works [15, 27] with the aspect of modelling the dynamics of modeled domain. In the presented model, particularly, it has the form of a cyclical process of generation of structure of individual performance tables and their aggregation. The research presented in the paper assumes a balanced impact of each performance table on the final evaluation support result. It’s worth to notice, that he choice of an aggregation strategy itself can be realized in accordance with the specifics of a given decision problem and the following can be examples of such [8]: Time Appreciated Aggregation, Time Depreciated Aggregation, Time Period Mostly Appreciated Aggregation. As shown above, the framework was formulated using classical assumptions (set of alternatives, set of criteria, outcome of each choice, preference structures of decision makers and stakeholders are fixed and steady). However, it may be expanded also where the assumptions are not fulfilled. For instance, the changeable spaces listed above can be effectively modelled using the theory of Dynamic Multiple-Criteria Decision Analysis, habitual domains, and competence set analysis presented by Po-Lung Yu and Yen-Chu Chen [31].

4 The Empirical Study

Structuring of the decision problem was carried out in stage I. For this purpose the set of decision variants (A), the set of criteria (B) and performance tables of individual variants were defined. The set of decision variants (A) consisted of advertisements located on a web site. Five advertisers were taken into account here and for each of them ten ad units differing in the level of intensity of impact on the recipient. Therefore, the total of 50 ad units formed the discussed set of variants. Each variant was examined in terms of 3 criteria: C1 - conversion rate, C2 - intensity of advertising message, which may negatively influence user experience and C3 - profit of an portal operator which can be treated also as advertising costs covered by the advertiser. Conversion rate (CR) is a basic measure of effectiveness of advertising expressed by the ratio of the number of desired user interactions to the number of contacts with the advertising content in which they can be potentially realized [18]. In the case of online advertising a desired interaction may be e.g. user’s “clicking” on an ad unit, e.g. a banner, and the number of contacts is equal to the number of times a given ad is displayed. The CR coefficient was designated a posteriori in the performed analyses on the basis of real data. In turn, intensity of an ad was specified in a subjective study. The broadcaster’s profit was calculated on the basis of the number of interactions and costs covered by the advertiser. Due to the fact the research focused on the changeability of preferences in time, it looked separately at the efficiency of variants in three equal time periods for which different conversion rates were obtained due to audience characteristics and interest in promoted services. Moreover, aggregation of three efficiency rankings into one was carried out using a group procedure and the efficiency of variants averaged from three time periods was examined. The obtained efficiencies for projected variants represented by conversion rate (CR) for each from three periods for first advertiser are presented in Table 1.

Table 1. Examples of criteria efficiencies of variants for selected advertisers.
Full size table

Stages II and III, i.e. modelling and preference aggregation, require in particular selecting a calculating procedure (MCDA methods) [27]. The research applied the Promethee method based on the outranking relation [35]. The method allows the application of six preference functions: usual criterion, U-shape criterion, V-shape criterion, level criterion, V-shape with indifference criterion, gaussian criterion [3]. Promethee allows obtaining a total preorder of decision variants (Promethee II) and carrying out the aggregation of individual rankings into a group ranking (Promethee GDSS - Group Decision Support System) [4]. Therefore, it is suitable for the above discussed structure of a decision problem. In stage II, for the Promethee method, the following needed to be done: defining the weight of the criteria and directions of preferences, selection of criteria preference functions and defining values of thresholds for the criteria. The selection of preference functions and values of thresholds greatly affect the order of the variants in a ranking [24, 25]. Moreover, the type of preference functions applied depends of the type of criteria. For quantitative criteria it is recommended that functions using the following are applied: V-shape criterion, V-shape with indifference criterion or gaussian criterion [11]. The developed decision model applied the V-shape criterion. This function uses preference threshold p, whose value should fall within reliable min and max values taken by a given criterion [28]. For threshold p, the developed model adopted the value of two times standard deviation. When it comes to the weights of criteria, for the purpose of our analysis it was assumed that all criteria are equally significant. Full preference model with assigned weights and direction for each from three periods is presented in Table 2.

Table 2. Preference model in the discussed decision problem.
Full size table

Five rankings of preference variants were obtained in stage II: 3 individual rankings from subsequent time periods, a ranking based on averaged values and a ranking obtained using the Promethee GDSS group procedure based on three individual rankings. These ranking are presented in Table 3.

Table 3. Obtained variant rankings for top twenty ad units.
Full size table

Stage IV of the research procedure is based on the exploitation of the obtained solution. For exploitation of individual rankings the analysis of their changeability in time was carried out. This analysis shows that in a dynamic environment such as the Internet, and in particular an internet ad, user preferences may be subject to constant, significant changes. It may be proven by the fact that out of 10 best variants of the first individual ranking, only 6 variants featured in the top of the third ranking (A5.10, A3.3, A1.10, A1.3, A5.3, A1.7). The position of variant A1.6 will serve as another example, which fell from position 10 in the first ranking and position 8 in the second ranking to position 32 in ranking 3. Great changeability of preferences obtained for individual variants in subsequent time periods are shown by rankings’ scatter graphs presented in Fig. 2. This is why the analysis of individual rankings shows the need for permanent preference research in DSS systems operating on dynamic data.

Fig. 2.
figure 2

Scatter graphs of rankings of variants in individual periods with pairwise period comparison and showed localization of each design variant for each advertiser.

Full size image

Exploitation of rankings: based on averaged values and aggregated to group evaluation, was performed through the application of the analysis of rankings’ sensitivity to changes of criteria weights. The purpose behind performing it was to find guidelines for optimal decision variants depending of decision makers’ preferences. Findings of the sensitivity analysis for an averaged ranking and for a group ranking were presented respectively in Figs. 3(a) and (b). The comparison of results of the sensitivity analysis for both rankings shows that aggregation of preferences from three individual rankings into a group ranking gives more transparent results that a ranking based on averaged criteria values. In a group ranking, across the entire field of criteria values, there is a smaller number of dominant variants, which allows obtaining more transparent recommendations. It can be observed above all in the case of the Conversion Rate criterion for which along with the increase in its weight, for the averaged ranking, variants A1.9, A1.3, A5.10 and A5.4. dominate subsequently. In turn, in the group ranking, only two variants are dominant: A1.9 and A1.3.

Fig. 3.
figure 3

Results of the analysis of sensitivity to changes in criteria weights in (a) averaged and (b) group ranking

Full size image

Stage V, i.e. drawing up the recommendation, is based in the results of stages III and IV. The sensitivity analysis carried out for the group ranking obtained using the Promethee GDSS methods indicates high stability of the obtained solution for dominant variants. Variant A1.3 remains the best in terms of weights: between 15 % and 100 % for the Conversion Rate criterion, between 27 % and 72 % for the invasiveness criterion and between 0 % and 40 % for the profit criterion. If the weight of the profit criterion is greater than 40 %, then the best variant may be A1.10. In turn, when the weight of the invasiveness criterion is lesser that 27 % then variant A1.10 may be assumed as optimal. The following variant dominations resulting for the sensitivity analysis may be assumed as doubtful: A1.9 for the weight of the Conversion Rate lesser that 15 % and A1.1 and A3.1 for the weights of the invasiveness criterion greater than approx. 75 %. This doubt results from the fact that in individual rankings the position of these variants is characterized by great dynamics and in the most current ranking (Period 3) they take remote positions.

5 Summary

The presented results confirm the ability of application of the proposed procedure for obtaining decision solutions given changeability of measurement data and the presence of multiple criteria. The obtained results show discrepancies in the application of the averaged and group approaches. Making decisions based on averaged values may lead to simplifications which lower the quality of decision. A significant element introduced in the model was the taking into account of both characteristics relating to the effectiveness of message represented by the Conversion Rate as well as parameters relating to the impact intensity. Research results indicate the dynamics of obtained solutions in the field of internet advertising and its effectiveness. This is why there is a need of constant evaluation of the effectiveness of advertising in relation to its other aspects, such as invasiveness and profits for the service owner or costs borne by the advertiser. A certain way to capture this changeability allowing the drawing up of the recommendations valid for a slightly longer period of time is averaging individual solutions obtained for subsequent time periods. In the research case, aggregation of individual rankings into a group ranking proved a more functional way of averaging which allowed preparation of clearer recommendations. The conducted research opens further research directions which should cover, among others, differentiation of weights for individual rankings in such a way so that most up-to-date rankings have highest weights (time appreciated or depreciated aggregation [8]). Thus, they would have greatest impact on the aggregated ranking by means of a group procedure and they would allow obtaining more up-to-date user preferences. Another direction can be application of data from subsequent time periods in the construction of fuzzy values for criteria preferences and building a ranking with the application of a selected fuzzy MCDA method, e.g. Fuzzy Promethee or Fuzzy TOPSIS.