Keywords

1 Introduction

The furniture manufacturing sector of the Baltics is facing serious challenges common in all European countries, namely, the growing global competition for customized solutions [1]. New standards to be followed in the industry tend to increase production costs, extend manufacturing time and cause frequent errors in the product quality. To maintain sustainability, companies need decision support instruments, allowing an immediate reaction to customized orders and proper evaluation of manufacturing procedures, costs and deadlines. The task to estimate costs as precise as possible and as early as possible has become critically important in customized manufacturing. However, cost estimation at an early design stage makes a serious challenge for other sectors as well, e.g. nanotechnology, aircraft manufacturing, etc. [2]. New opportunities to deal with the problem may be suggested by newly emerging trends in extensive data research [3] and by operational research into decision support systems supplemented by machine learning techniques [4]. Complex decision making problems, as well as cost estimation problems, are often attributable to absence, insufficiency or inaccuracy of available data. Additional reliability could be acquired complementing an intelligent system with a human knowledge intervention and applying outcomes of behavioural operational research. In such cases, a structured expert judgement (SEJ) can be used. Therefore, it is worth discussing a meaningful use of expert knowledge to compensate the complexity of price calculation and foresee possible errors. Apart from scientific and methodological issues of how to integrate the output of structured expert judgement into an intelligent cost estimation system, there are additional questions to be tackled, e.g. expert recognition and building trust between the user of the decision support system and the expert. Thus, the complexity of cultural processes additionally enhances the complexity of customization.

The goal of the present research is to look into the cultural pattern of competence recognition within furniture industry with the purpose to adjust the structured expert judgement strategy as an instrument to validate expert input into the decision support tool for cost estimation.

The main question about a methodological recommendation for expert selection for SEJ is analysed in the frame of broader sectorial picture, e.g. (a) the pricing process, ICT usability and experts; (b) the accuracy of pricing forecasts by experts. The broader context of the research context has led to the choice of methods.

The paper is organised as follows. First, we are giving an overview of what is known about the widespread pricing methods then we are going to analyze how expert knowledge is applied in cost estimation and finally, we are going to discuss how to achieve a better integration of a particular strategy of structured expert judgement with a behavioural pattern adopted in customized manufacturing for the sake of price estimation. The research also reveals how to identify appropriate experts among the company’s employees for further cost estimation.

2 Literature Review

Cost Estimation.

Having screened the most common cost estimation techniques, one can find a simple and reliable method to estimate the cost by dividing a product into its constituent elements and estimating the cost on the basis of prices of individual components and time necessary to produce the element [5]. However, the simple technique based on adding up prices of individual components eventually turns into a complex task. The first moment for uncertainties to show up is when the choice between possible design solutions has to be made. Thus, the early design stage needs additional attention and reliable cost estimation procedures and models to be used [6]. A more comprehensive approach to the task of cost calculation is based on unbundling elements according to different levels of uncertainties. Three categories can be distinguished here: facts, estimated values and application parameters [5]. Facts are what is measured exactly at the end of the production process. Estimated values are the result of statistical observations or the data collected through experiments (e.g. working hours). Application parameters cover the part of the cost that comes up from different scenarios.

Customized manufacturing typically relies upon traditional cost estimation methods, most of which are based on the reaction to complexity. So far, attempts to estimate costs manually, especially at the early design stage, have failed to produce positive results due to the lack of accuracy, incompleteness of data and extensive consumption of time [7].

More and more sectors, including manufacturing, software engineering, process engineering, construction industry and scientific research, refer to instruments of artificial intelligence. Therefore, newly developed methods based on artificial intelligence and massive amounts of data have become a good opportunity for customized manufacturers. At the moment, the most popular analytical instruments include: analogous cost estimation, bottom-up estimation techniques, and computing technology combined with artificial intelligence.

Expert-Based Knowledge.

Even with an application of various methods and multiple methodologies, the accuracy of cost estimation is still insufficient and is under question every time a new customized order is received. Also, because of tight deadlines, additional sources of knowledge must be sought in order to either correct or justify the preliminary numbers. A possible solution of the situation might be expert advice. Expert contribution is widely used in different areas “where an explicit conceptual framework does not exist or where data are very impoverished” [8]. Before addressing the question of how to deal with expert knowledge, we need to tackle the issue of expert recognition. There have been permanent debates about who may be referred to as an expert and how to select the best expert in the field. Recently, expert judgement has been increasingly recognized as another type of scientific data. Recognition of expert-based knowledge is almost inseparable from some cultural issues lying behind understanding the value added of professional knowledge. There is some tension between professional knowledge and any other knowledge, which could be represented by a complementary area or use of knowledge. In the meantime, there are more and more studies claiming that professional knowledge is more valuable than “practical wisdom of clients” [9]. It seems that these questions are still not answered completely, and there is some gap in the literature when product customization is discussed in terms of price estimation.

Although it has been recognized as an adequate source of additional knowledge for decision-making, expert knowledge is still difficult to integrate into a universal knowledge management system. Also, from the perspective of knowledge retention, different strategies to ensure knowledge accumulation are recognized. Knowledge retention can be implemented by maintaining three different approaches: technology-based, interaction-based (capturing the process and practices) and culture-based (best practices with interactions of professionals) [10]. The technology-based approach relies on collection and storage of information and factual data [11]. Interaction based knowledge accumulation means learning from previous mistakes and the culture-based approach means specialist expertise in a particular field [12]. Accuracy of pricing strongly depends on a chosen culture-based strategy that has direct links with practice and expert interaction. Culture based strategies differ between sectors depending on their size, nature, and other criteria. The choice is typically limited by formal skills of an expert and inclusion of other experts is understood as a cultural process, which is an integral part of price setting. The expert-based approach shall mostly consider the following: the complexity, dimensions and specifications of the furniture, the current market variables, the size of the market for specific furniture, the existence of risks in the market and the possible sales margin. However, the systematic process of selecting and using experts in price estimation is often ignored by businesses [12].

Structured Expert Judgement.

Expert judgement includes a wide range of methods - from one opinion to think-tanks with external validation. Very often, parameters necessary for decision making or modeling physical or biological behaviour are not precisely known. Experts in different fields may have the necessary valuable knowledge of models and parameters applicable in their specific field of interest. Quantitative assessment and aggregation of expert opinions can provide an essential contribution to decision making and can lead to an optimal choice of model parameters. Expert opinions can be combined either by identifying them individually and explicitly and then applying some mathematical rule or by giving the group an opportunity to discuss problems and negotiate to agree on a consensus, what is also known as behavioural aggregation. Behavioural research has shown that a person’s ability to encode judgements in probabilistic terms varies according to his experience [13]. There is some evidence that mathematical aggregation can outperform behavioural approaches [14].

Various mathematical methods can be practically used in combining expert judgement. Mathematical methods usually ensure that there is a common understanding of the questions among experts and the use of certain weights to combine expert assessments of uncertainties. The simplest method is to deem all the weights equal. Equal weight combinations have obvious advantages, however they have drawbacks as well. An expert whose distributions are very different from those given by other experts may have a significant impact on the resulting decision. This is a significant drawback if the expert assessments cannot be justified based on performance. As more and more experts are brought into the study, the equal weight decision maker can tend to become quiet diffuse [12]. Cooke’s Classical approach [12, 15] which is known as structure expert judgement, is most frequently applied among all expert judgement methods.

Expert judgement is used in a wide range of areas, including nuclear safety [16, 17]; aircraft engineering [18]; air traffic control [19] and software production [20]. Expert judgement is also used extensively for cost estimation [21], where experts have to make assumptions and judgements about the cost of a new product. Expert judgement based estimation approaches are most commonly used in the software industry [20]. For example, the main procedural steps set in EXCALIBUR software are following [12, 22]: formation of an expert group, expert assessment of variables, identification of the true values, rating of experts in terms of calibration and informativeness, conversion of ratings into weights, questioning of experts about uncertainty of the results of possible measurements within their domain of expertise.

3 Methodology

To capture the cultural approach of the behaviour pattern towards expert recognition and to measure the accuracy of expert knowledge in the process of price estimation, a mix method research methodology has been chosen. The mix method strategy comprises three studies: a qualitative study, a quantitative study and an SEJ experiment. The qualitative study was conducted to find out what pricing stages are used by companies and how companies use expert knowledge to justify the accuracy of the price. This research has to provide the knowledge we need to invite the experts to the SEJ experiment and what kind of questions about prices are worth asking and what accuracy of price prediction we could expect. The quantitative survey was conducted with the aim of obtaining quantitative data on the use of expert knowledge for price calculation understanding the cultural pattern of competence recognition within furniture industry. The third study was an SEJ experiment during which expert elicitation and expert ability to predict the price were studied. The combination of the methods is expected to determine whether there are specific behavioural trends that could limit the application of SEJ in a furniture company. Another objective to be achieved is to investigate the possibility to adjust the structured expert judgement strategy as an instrument to validate expert input into the decision support tool for cost estimation.

Quality Study.

The research was based on the semi-structural interview method. Two companies working in customized manufacturing were selected for the research. Data for the research were collected in the course of 26 interviews with a wide range of specialists within the company, including CEOs, managers, product developers, constructors. All interviews were transcribed and coded. The responses were grouped into categories and subcategories by means of the qualitative data analysis software NVivo. A total of 174 pages of text and 905 coded notions were grouped into four generalized categories: price estimation, the organizational structure, employee engagement and production processes. Having made the qualitative classification of statements into categories, a comparative analysis of the collected interview data was conducted to reveal fundamental similarities and differences between approaches of individual respondents. As the scale of the study was very large, only 1/5 of the data was used for the expert database. The present article analyses only the category of price estimation.

Quantitative Survey.

The survey was conducted by an international market research company in the period from January to February 2019. 146 Lithuanian companies were interviewed as a sample representing furniture manufacturers. A set of 35 questions compiled specifically for the survey included five major factors of the conceptual model: the organizational structure, employee engagement, price estimation, production processes, and IT implementation. Although the overall research comprises a wide range of issues associated with customized furniture manufacturing, this particular survey focuses only on the part regarding price estimation and expert recognition. Here, questions about the structure of the organization, employee involvement in product pricing and employee assessment most typically taken into account are dealt with. The used 35-item questionnaire is reliable as the value of Cronbach’s alpha coefficient is 0.809. The internal consistency of the 7-item list is also satisfactory for the Cronbach’s alpha equal to 0.818.

Descriptive statistics and analysis of non-parametric statistics are used in the quantitative study. The Cochran’s Q test is used to check existence of statistical differences between experts (company representative) opinions using their agreements as dichotomous variables. A null hypothesis that there would be no difference in percentages of agreements between the experts has been made and tested with the significance level of 0.01. In order to run Cochran’s Q test, the study was designed so as to meet assumptions suggested by Sheskin [23]. A correlation analysis has been carried out to identify the strength of relationships between pairs of survey questions related to SEJ using Spearmen’s Rank correlation coefficients with the significance level of 0.01. The relative strength of relationship between two question-items is based on interpretation of the relative strength of the significant Spearmen’s correlation coefficients [24].

Structured Expert Judgement.

The third part of the research is based on a structured approach to expert judgement [12, 15] in order to identify appropriate experts among the company’s employees for further cost estimation. The composition of this group is an important issue. The survey was carried out in one of the Lithuanian furniture manufacturing companies. Ten employees (nominated as experts) took part in the research: a CEO, a constructor, a chief accountant, a chief financial officer, the head of the finance department, an IT manager, a designer, a senior project administrator, a project administrator and a chief product manager. The structured expert judgement involves two generic quantitative dimensions of expert performance: calibration and information. The calibration questionnaire included the following elements: general questions about the expert and seed questions. Each expert had to answer twenty seed questions about their experience, compiled for expert elicitation. Then, a list of several products was given for price estimation. Experts had to suggest possible prices on the basis of available data about the product, such as the materials, the quantity and the operational procedures. An EXCALIBUR software package [25] was used for structured expert judgement elicitation on the basis of Cooke’s Classical Model [12], where two separate scores (calibration and information scores) are estimated and multiplied together to get the overall weight for each expert [26]. In order to estimate the scores, experts are given a set of seed questions. Experts were asked to submit their estimates with a certain tolerance, for example, minimum and maximum probable values, in other words, to specify quantiles of the distribution of interest, such as the 5th, 50th or 95th. The calibration and information scores combined together lead to a weight estimate for each expert [12, 15].

4 Findings

Qualitative research focused on the pricing process and its interfaces with engineering processes. After a detailed disclosure of the management stages of the price calculation, it becomes clearer what the issues of calibration and forecasting of SEJs are relevant to the furniture sector and their experts. The qualitative research has revealed an interesting approach to pricing and its importance in customized production. Pricing is a complex critical process crucial in customized production. The study has shown that it is extremely difficult to determine the exact price of a piece of furniture because it depends on a great deal of various factors. Each company uses different methods and methodologies of pricing. However, there are some general trends in modeling furniture prices. For example, some companies set prices after preparatory design works, while others rely upon immediately calculated average prices. Such differences affect the exact final price. Also, companies account for different factors when calculating their prices. Some pay more attention to the product design and the materials used, others focus primarily on the production time and the hourly cost of necessary operations. Bigger companies have departments responsible for price calculation. For example, a project department creates order and sends it to the pricing department. The pricing department calculates the costs and sends the numbers back to the project department where a certain mark-up is added. Smaller companies cannot afford having pricing department and therefore delegate the function to individual experts.

In terms of general trends in modeling furniture prices, analysis of price calculations comes up with two aspects of trends. The first aspect entails tangible and measurable product elements, such as product dimensions, production time, the number of produced units, the used materials and the market segment.

The other aspect is about intangible parameters that affect price calculation. Words like guesswork, collaboration, collective, meetings, intuition, client are typical here. The second trend is primarily predetermined by the influence of experts or experienced employees. It is the experts who are responsible for setting the product price. The study shows that most companies are looking for expert opinion, proposals or assistance in determining the final price of individualized furniture while experts admit that the process is based more on predictions and intuition rather than rationality. The main participants in pricing include senior managers, finance managers, engineer sand and even company directors.

The notion of “expert/experts” entails such concepts as self-determination, meetings, intuition, director’s influence, independence, competence or qualification. The concepts are not only invoked to describe an expert but also are referred to by experts in price estimations.

The study clearly shows a certain conflict between the trends (aspects). Pricing is mostly determined by measurable criteria whereas expert judgement is largely about intangible elements difficult to measure. However, all the elements are equally important in pricing and no factor can be neglected as insignificant as each of the factors can have a significant impact on the final price of the product. All elements must be logically combined into a reasonable price expression.

Quantitative Survey – Cost/Price Estimation with a Limited Set of Professionals.

In Lithuanian furniture sector is dominated by small and medium-sized enterprises with only a few large companies employing more than 800 people. This determines their specific behaviour in dealing with pricing issues. 53% of the surveyed companies stated that the director is involved in the price calculation (14% among those claimed that the director is the only person responsible for pricing), 37% named the owner (10% – only the owner), 33% – the head of production department, 25% – the constructor, 24% – the designer and 17% – the project manager. In 37% of the companies, only one employee is involved in the calculation of prices: 14% – the director, 10% – the owner, 12% – other. 25% of the companies involve two employees, 17% – three employees, 13% – four, 5% – five, 3% – six employees. An interesting fact is that the majority of the companies have more than one employee responsible for pricing. Most companies recognize this stage of manufacturing as important and sensitive to the quality. That is why companies see the need to have an “advisor” to assist in the calculations.

Not surprisingly, opinions of only few delegated specialists are taken into consideration during the price estimation stage. In 20% of the surveyed companies, the director’s proposals are the most important when pricing is discussed, in 16% it is the head of the production department and in 10% – the owner. 17% of the companies give major importance to proposals by the head’s of the production department or another specialist (a project manager, a designer, a constructor or a technologist). Thus, a total of 33% of the companies tend to rely upon proposals by the head of the production department.

76% of the surveyed companies noted that only the director or the owner make the final decision on the price offered to the client. A further 11% of the respondents named the deputy director, the product manager, the project manager, an engineer designer or constructor as a co-maker of the final decision along with the director/owner. Thus, the owner or director has a say on the final decision on pricing in 87% of the companies. In 4% of the companies, the final decision on pricing is made by the product manager and in 2% – by the project manager. Practically in all surveyed furniture companies, prices are decided mostly by the director or other highly ranked manager delegated the power of decision.

To have a general idea about experiences and opinions about specialists (1) whose opinions are taken into account in the process of pricing, (2) who make final decisions on pricing, and (3) who have the greatest influence on the competitiveness of the product, a null hypothesis that there would be no difference in percentages of agreements between the experts has been made. The conducted Cochran’s Q test maintained that there is no statistically significant difference in the proportion of expert opinions on the three question-points under consideration: (1) χ2(145) = 168,458, p = 0,089; (2) χ2(145) = 62,969, p = 1,000; (3) χ2(145) = 145,087, p = 0,482.

Companies mostly trust employees who contribute to competitiveness – designers, constructors and product managers. On the basis of calculations of the significant Spearmen rank-order correlation coefficients between pairs of questions about the level of the organizational structure, the professional status of the representative in the company; the rank of the specialist responsible for pricing, the rank of the specialist whose opinion is valued most and the rank of the specialist who makes the final decision on pricing, several assumptions can be made:

  1. (1)

    The simpler the structure of the company is (e.g. all employees directly accountable to the head of the company), the higher professional status is required for pricing (e.g. the director or the owner) (rs(144) = −0.214, p = 0.01).

  2. (2)

    The higher position a company representative is holding, the more likely they are to take into account proposals of a senior employee (rs(144) = 0.250, p = 0.02).

  3. (3)

    The higher the qualification of a specialist in charge of pricing is, the more likely their price estimates are to be taken into account (rs(144) = 0.323, p = 0.07).

SEJ.

The previous research has shown that during the cost estimation process companies mostly trust such employees as designers, constructors and product managers. These findings, as well as obtained assumptions, were partially confirmed during the research based on a structured approach to expert judgement in order to identify appropriate experts among company’s employees for further cost estimation. While recognition of the ability of top managers to predict prices has not been corroborated by SEJ results. It was noticed that a well-composed group of experts could be a possible resource in assessing uncertain aspects of cost estimation, as it has been observed in the quantitative survey.

Two sets of questions were prepared and used in the expert ranking experiment: (1) questions about the furniture sector, furniture economics and company demography and economics and (2) questions exceptionally about company economics and demography. The results of the expert judgement based on Cooke’s classical model obtained by means of EXCALIBUR software (see Fig. 1) show that only two experts out of ten were found suitable as experts for further cost estimation, namely, the chief financial officer and the senior project administrator. An additional study was carried out having left only questions about internal processes in the company (11 seed questions) to rank the other experts. The results (see Fig. 2) show that the highest weights were given to the following experts: the IT manager and the chief product manager.

Fig. 1.
figure 1

Results of scoring experts (20 seed questions, global weights).

Full size image
Fig. 2.
figure 2

Results of scoring experts (11 seed questions, equal weights).

Full size image

To achieve the goal of the research another experimental step was carried out to find out the possibility of using the structured expert judgement strategy for cost estimation in customized furniture manufacturing. Authentic data provided by a Lithuanian furniture manufacturing company were used in the survey. The data included the actual costs of the products. During the experiment, methods based on machine learning techniques were also used for cost estimation [27]. Six experts among company’s employees took part in the experiment: the chief accountant, an IT manager, two designers, a project administrator and a chief product manager. Whereas the results of the quality study showed that pricing is mostly determined by measurable criteria, the costs were evaluated on the basis of the following data: the cost of materials, the list and duration of the manufacturing procedures. The obtained results are presented in Fig. 3. Expert No. 1 (the chief accountant), Expert No. 2 (the IT manager) and Expert No. 5 (the chief product manager) were identified as the most appropriate in the previous study. The results show that the cost estimation given by Expert No. 1 (the chief accountant) is very close to the actual price of the product and the cost obtained by machine learning-based techniques. However, in order to use machine learning techniques, it is crucial to have a proper set of historical data for the training process and to identify essential data features to obtain accurate results. Meanwhile, experts can make cost estimates in the absence of historical data.

Fig. 3.
figure 3

Results of cost estimation for six products (straight line – the real cost of the product, dotted line – machine learning-based cost estimation)

Full size image

To sum up, a well-composed group of experts can be a possible solution in assessing uncertain aspects of cost estimation. The composition of such a group is an important issue. The general requirement for the experts is that the group needs to have some specific knowledge necessary to understand the technical, organizational and financial side of the cost estimation. Apart from hiring expert team members, who have the necessary knowledge, is also advisable to refer to the necessary external experts for alternative opinions. Also, it has to be noted that essential insights and valuable opinions may come from individuals who cannot be immediately identified as experts [28].

5 Conclusions

The main intention of this study was to investigate whether there are behavioural patterns within the particular sector that limits smooth integration of different methods (machine learning and expert judgement) during operational decision (cost estimation) task.

Price estimation at the early design stage not only gives an opportunity for optimization of costs but also positively contributes to the development of customized production processes. The survey in the principles that companies would expect from a Custom Order Price Estimation System has come up with strong arguments in favour of material-based pricing systems, whereas systems based on ex-peer evaluations are subject to criticism. Analysis of how companies rely on experts has revealed several trends. Pricing in furniture industry is the prerogative of several professionals who are responsible for setting the right price. This was confirmed by both the quantitative survey and a qualitative interview. The trend has also been confirmed by means of an SEJ experiment. Selected experts among company’s employees demonstrate better results both in expert rating and price forecasting. Meanwhile, recognition of the ability of top managers to predict prices has not been corroborated by SEJ results. The fact might reflect both a limited distribution of responsibilities and a clear division of management responsibilities.

It was noticed in the research that a well-composed group of experts could be a possible solution in assessing uncertain aspects of cost estimation. The general requirement for the experts is that the group needs to have some specific knowledge necessary to understand the technical, organizational and financial side of the cost estimation. Although the cultural model of the furniture sector would recommend a slightly different approach: the top executives and the best engineers in this sector are seen as experts. This should be taken into account when developing methodological recommendations for the implementation of the structured expert judgement.

Finally, a conclusion can be made that a cultural pattern could express itself in corporate behaviour depending on the employee experience, education and skills. Knowing the cultural boundaries of expert recognition and the fundamentals that make an impact on the rationality of decision making, one can adjust an operational research tool (for instance SEJ) according to sector specificity.