Keywords

1 Introduction

Artificial intelligence (AI) technologies are increasingly utilized across platforms and in different service contexts. However, understanding the degree to which they are accepted, and under which circumstances, requires further investigation. Traditionally, within the information systems research, technology acceptance (TA) has been extensively studied through different models. These models include a plethora of variables that precede individual responses, such as usage intention, behavioral intention, attitude, and satisfaction towards the technology. Two of the most frequently utilized models are the Unified Theory of Acceptance and Use of Technology (UTAUT) [13], and the Technology Acceptance Model (TAM) [2]. Moreover, users’ TA level is also affected by the presence of anthropomorphism, perceived trust, risk, privacy, enjoyment, and satisfaction.

First of these set of factors is anthropomorphism, which is the indication of having the human-like characteristics present in technology, such as a human appearance, emotions, or intentions. Previous research has shown that AI technologies that primarily interact with humans like chatbots [11] and recommendation agents [9] have better adoption and intention to use rates when the agent has anthropomorphic features. The second factor, trust, and its antecedents in artificial intelligence technologies are highly investigated [8]. These investigations include performance and attribute-related factor such as system competencies or personality and communication. Aside from these performance and attribute-based factors of AI agents or systems, enjoyment and satisfaction are also factors that have been shown to increase an individual’s intentions to adopt or use the technology [5, 15]. Other factors related to the features of an artificial intelligence system are the risk and privacy levels perceived by humans. A growing stream of research has shown that these features that can negatively affect individuals’ trust in the system lead to lower continued use [4, 7]. Apart from these factors, AI systems have been documented to be affected by other attributes as explainability and emotional conversationality, which in turn can reduce an individual’s cognitive load [6], help build better interactions with the technology, and lead to a more natural type of communication [1].

Although these attributes, which include the conversational, emotional, and explainable factors of AI, have been studied across many disciplines. It is not yet clear which one of these factors leads to the strongest levels of acceptance. Thus, the goal of this research is to investigate the factors driving TA in AI systems, through a meta-analysis approach.

2 Methods

2.1 Finding Sample of Articles

To complete the literature search, we follow the guidelines from past meta-analyses done on both AI and acceptance models [8]. We conducted our first search on the Web of Science platform, which covers the majority of the publishers and extends to multiple fields of research. We developed an initial search term that includes various chatbot systems and acceptance terms that will correlate to our interest in this meta-analysis. We have started with key terms such as “chatbot”, “UTAUT”, “TAM”, and “usage intention” or “adoption intention” and revisited our search term in each iteration of title and abstract screening.

Our keyword search term(s) were based on acceptance and AI keywords, which comprised AI factors within or without technology acceptance models, adoption or usage intention constructs, and AI keywords used in business contexts, respectively. We also developed some inclusion criteria regarding the publication year and language of the articles. AI research has transformed quite in recent years; consequently, the decision was made to include the research done between 2021–2022 in English. Furthermore, for specific research fields of interest, the search terms were narrowed include only “Information systems”, “Business”, and “Management” fields. Inclusion criteria included empirical research that has been done within our scope of research questions.

For our following round of literature search, we have selected ScienceDirect as a platform since it included research from consumer research and the social sciences as business or management compared to other platforms. For this platform, we have narrowed our search term(s) to keywords such as “chatbot”, “adoption intention”, “usage intention”, and included more general terms such as “usage” or “adoption”. These additional inclusion criteria were inserted to eliminate any additional noise that may emerge from searching for other AI terms. Per our previous search using the Web of Science platform, this search also included research articles published in English within the last decade. We did not include the specification for business-related fields for this search to find articles that fit our research questions from other industries such as health, tourism, etc. From both searches, there were 1633 and 644 returns from Web of Science and ScienceDirect respectively, resulting in 2277 articles to screen. Due to time limitations at hand, we started the title and abstract screening process with extraction of the data. After the screening process for title and abstracts of the results returned from these search terms, we have extracted 18 articles for our preliminary search, following the PRISMA guidelines (Fig. 1).

Fig. 1.
figure 1

PRISMA flowchart for identifying studies via databases for meta-analysis.

Full size image

2.2 Selection of Articles

For the next stage of our review, we have created an inclusion criteria list. After screening for every criterion within our search returns, 18 articles and total of 87 effect sizes were identified and extracted. Although there were more records returned after our search, only 18 were selected. The code-book and article information coded are available upon contact with corresponding author. The reason we selected these articles was to be able to show a preliminary set of results. Articles were coded following the Coding Scheme in Appendix A (Fig. 2). The criteria followed for this process to include articles may also be found in Appendix B.

2.3 Analysis Protocol of the Effect Sizes

The pre-processing stage consisted of converting the results from the statistical tests and regression models to effect sizes. Due to the fact that all 18 articles included in the analysis did not report any effect sizes, we converted them manually. The majority of the articles reported t-values of the relationships between AI acceptance characteristics and their precedent usage, adoption intention, attitude, or satisfaction. Only nine articles reported p-values for their tests. While the one remaining study did not report t-values, it included \(\beta \) and standard errors from the regression model, which can be converted to t-values. The transformations for all the values are presented below:

$$\begin{aligned} \textit{d}=\frac{(2*\textit{r})}{\sqrt{(1-\textit{r}^2)}}, \end{aligned}$$
(1)
$$\begin{aligned} \textit{d} = \frac{\textit{t}}{\sqrt{(\textit{t}^2+n - df)}}, \end{aligned}$$
(2)
$$\begin{aligned} \textit{t} = \frac{\beta }{SE}, \end{aligned}$$
(3)

where d and r are the effect size Cohen’s d and r, t is the t-value, n sample size, and \(\beta \) and SE are the coefficient and the standard error of the structural equation model results.

The analysis for the relationship between characteristics of acceptance and the precedent factors (i.e. usage, adoption, attitude, and satisfaction) is the main relationship of interest. Others include the role of conversational and emotional attributes, the type of industry, and finally the personal versus professional uses of AI on effect size strength. In order to have a reasonable amount of variation in the analysis, there were at least five effect sizes for each moderator. Furthermore, we only extracted the total or direct effects of interest from articles that included structural equation modelling and were relevant to our primary research question. Finally, since this is a presentation of preliminary results, p- values less than 0.1 will be taken into consideration as marginally significant. To test the effects of the interaction of interest with moderators we will use R package metaphor [14], which will help execute these tests with a random-effect multi-level meta-regression model.

3 Results

3.1 Homogeneity Analysis and Publication Bias

As is the standard procedure for a meta-analysis article, we completed several analyses. Firstly, we performed a homogeneity test of the sample, given that we are investigating the presence of any moderators or other variables that might produce the variability within the effect sizes. The homogeneity test revealed there were indeed moderating factors associated with the effect of factors on acceptance (Q(86) = 1565.81, p < .0001) within our sample. Finally, Egger’s test [3], trim and fill [12], and Rosenthal [10] methods revealed no asymmetry within our sample, suggesting no publication bias.

3.2 Factors Driving Acceptance

A meta-regression model including all our moderators showed that the AI factor that driving a strong effect on acceptance is conversational (r = 1.79, p = 0.0177, CI = [0.188, 1.979]) and explainable (r = 0.81, p = 0.0726, CI = [−0.045, 1.033]) attributes of artificial intelligence, compared to the traditional measures of acceptance.

We further investigated whether explainable, conversational, and emotional attributes are more useful in some industries than others. Based upon our findings, having explainable features in AI interacts positively with acceptance in both e-commerce (p < .0001, CI = [−1.1023, −0.5764]) and tourism (p < .0001, CI = [0.6395, 1.235]). Although we did not find any significant interaction between conversational attributes and industry types, having this attribute in an artificial intelligence system produces larger effect sizes than not having it.

Similarly, an interaction effect between the emotionality of the agent and acceptance within each industry was not significant except e-commerce, which produced stronger effects when there were emotional cues made by the AI (p = 0.0172, CI = [0.0921, 0.9492]). When it comes to personal and professional use of these AI agents in service context, our initial results also showed interesting but non-significant results. Based on our results, in professional settings, all three attributes (conversational, emotional, and explainable) positively affected acceptance compared to when these attributes were absent. Moreover, emotionality was the only factor that increased acceptance in personal use compared to the other factors.

3.3 Robustness Check

To be able to do a robustness check, we have disattenuated the reliability scores that are used in each study to measure independent and dependent variables. These reliability scores are collected from these variables ranged from 0.63 to 0.98, and 0.76 to 0.98, respectively. After the calculation of disattenuated r and variance of disattenuated rs, these values are used to do an additional robustness check. This model also showed us that there may be heterogeneity in our sample (Q(86) = 3470.88, p < .0001).

4 Discussion

Through our initial analysis of the literature, we have identified which factors play an important role in the TA literature. We found that different industries that utilize AI technologies have different needs when it comes to certain factors of AI. Furthermore, emotional, conversational, and explainable factors do not affect TA in AI artifacts unanimously. In other words, employing different factors to different sectors are crucial to individuals’ acceptance rates.

Surprisingly, when the effect sizes produced within these articles were compared, factors such as satisfaction and privacy had a stronger effect on TA than the traditional acceptance models. After investigating these relationships across industries, emotional aspects of AI were found to negatively affect acceptance in sectors that could increase vulnerability or include more vulnerable populations. The health sector is one such example. On the other hand, the presence of conversational factors of AI do not appear to affect acceptance in other sectors such as e-commerce and tourism.

4.1 Limitations

The findings discussed here are preliminary and are based on the limited sample of the literature reviewed to date. Consequently, we are unable to report definitive results relating to every interaction effect within our moderators, at this stage. There is another methodological limitation caused by our limited sampling of the extant literature which is the range of confidence intervals in our results. We hope to alleviate this limitation, once we gather more effect sizes from our search.

4.2 Implications and Conclusion

Nonetheless, after initial findings, the AI technologies that exist in companies can be repurposed to put more emphasis on primary factors such as conversational, emotional, and explainable factors, which may increase individuals’ acceptance and usage of these technologies. More importantly, prioritizing the perceptions and feeling relating to privacy protection while interacting with AI technologies is more crucial than other traditional antecedents of acceptance.