1 Introduction

Several studies in information systems (IS) and human-computer interaction (HCI) have attested that website visual design is a key factor in the success of e-retailers (Cyr 2008; Flavián and Guinalíu 2006; Hassenzahl 2004) by influencing user responses, such as emotions (e.g., Éthier et al. 2006, 2008), perceived quality of the vendor (e.g., Fogg et al. 2003; Loiacono et al. 2007), and trust and behavioral intentions (e.g., Holsapple and Wu 2008). Over the past decade, customers have increased their expectations of the visual design aspect of products, which then have driven businesses to come up with innovative product design strategies to meet those expectations. For example, Apple™’s products and websites have established a benchmark for design that many technology companies now try to emulate in efforts to reach consumer markets.

In the marketing context, superior visual design helps vendors distinguish their products from those of competitors and gain recognition in a crowded marketplace (Bloch 1995; Schmitt and Simonson 1997). A successfully implemented visual design strategy can create identity for the organization and its brands, as well as provide value by satisfying customers’ aesthetic needs (Schmitt and Simonson 1997). Consequently, visual design has become a critical strategy in many business areas (Hoegg et al. 2010). In this study, we adopt concepts of visual appeal established in the marketing domain to shed light on how online users react to the visual appeal of websites.

Although website visual design features have been broadly explored in the IS and HCI literature, most studies have focused on the designers’ perspectives of the design features, such as color (Cyr et al. 2010), layout (Deng and Poole 2010; Geissler et al. 2001), simplicity (Karvonen 2000), complexity (Pengnate et al. 2018), and the use of images (Cyr 2008; Cyr et al. 2006; Cyr et al. 2009; Hassanein and Milena 2007). Only a few studies have focused on users’ perspectives, such as gender (Cyr and Bonanni 2005), culture (Reinecke and Bernstein 2013), and familiarity (Flavián et al. 2006). Therefore, while the importance of website visual design is widely recognized, individual differences in how online users perceive website visual designs, as a factor in the users’ perspective, are not well understood and have not been sufficiently investigated in the literature, especially in the online context. Thus, the primary goal of this study is to address this gap in the literature by providing a better understanding of how individual differences in aesthetics influence online user perceptions and responses. This is an important area that should be recognized since it can enhance performance of the online strategy by preferably shaping users’ perceptions toward websites, such as engagement and intention to use the websites.

The importance of individual differences in visual appeal has been recognized in numerous studies. In the marketing literature, Bloch et al. (2003) reported that individual differences in the centrality of visual product aesthetics (CVPA) is an antecedent of customers’ perceptions of products. Studies in the HCI and IS domains have also provided evidence of individual differences in website design aesthetics (e.g., Lindgaard et al. 2006; Lindgaard et al. 2011; Tractinsky et al. 2006). However, only a few studies have investigated the impact of such individual differences and mixed results were reported (e.g., Golander et al. 2012; Phillips 2007). Moreover, while prior studies have examined individual differences in the centrality of visual website aesthetics (CVWA, as a variation of CVPA), they applied the CVPA measurement items directly in the website design context (e.g., Yoo and Kim 2014). Consequently, the study of the CVWA is still in its infancy, and the results from previous studies must be interpreted with caution since they have been limited by the lack of a valid and reliable measure to capture individual differences in website aesthetics. Therefore, it is necessary to develop a measure of the CVWA as well as to explore its effects on user responses to provide deeper insight into how people may differ in their ability to distinguish nuances in website visual design. An understanding of the CVWA is especially important for website designers because website visual appeal is considered the most effective strategy for immediately attracting and engaging online users and potential customers (Aztek 2016; Corcione 2017; Lindgaard et al. 2006; McLeod 2016).

Thus, the overarching objectives of this study are to develop and validate an instrument for measuring the CVWA and investigate the effects of the CVWA on users’ responses. Specifically, we focus on examining how the CVWA moderates online users’ perceptions toward websites, including the site’s visual appeal, trust, and intention to use the website. For example, for some users, the visual aesthetics of a website may play a major role in determining their attitude toward the website and may guide their decision as to whether to interact with it or move past it to another website; however, for other users, this may not be the case. This concept reflects the meaning of website visual design, which is carried in the eyes of online users and which has rarely been studied. Note that, in our context, the terms “visual appeal” and “visual aesthetics” are used interchangeably, with the understanding that both terms are associated with the appreciation of beauty (Hassenzahl 2004).

A series of three experiments was carried out in this study to develop and validate the CVWA measure and to examine the effects of CVWA on user responses. The first experiment aimed to examine the effects of the CVWA on user responses using the scale derived from the CVPA measure proposed by Bloch et al. (2003). The second experiment was conducted to develop and validate the CVWA measure. Finally, the third experiment was designed to replicate the first experiment using the CVWA measure.

In the next section, we present the literature concerning website visual design framework and the connections to CVWA. This is followed by a report of the three experiments. Then, we discuss the implications of the results for future research.

2 Background and Theoretical Framework

2.1 Individual Differences in the Centrality of Visual Product Aesthetics in the Website Context

Several studies in the information systems (IS) discipline have highlighted the importance of individual differences as a crucial factor in influencing online user’s behavior and responses. For example, in the online commerce context, Lin et al. (2019) suggested that the effects of online users’ perceptions of interactivity and risk on attitude toward websites were moderated by gender. Chen et al. (2015) conducted a study in a cross-cultural setting and reported that cultural differences affected the relationship between trust and e-loyalty. Karumur et al. (2018) found that online users’ personality traits such as personality, motivation, and interests were highly correlated with the users’ behaviors and preferences in online recommender systems.

In addition, the effects of individual differences have been reported in the mobile commerce area. Kim and Hwang (2012) found that personal factors such as maturity and socio-economic status were strongly related to users’ perceptions of mobile service quality. Furthermore, in the virtual world setting, Zhang et al. (2019) examined individual differences of avatars and found that the interaction networks in the virtual environment were determined by the avatars’ age and gender. Therefore, the findings from these studies provide empirical support for the notion that individual differences are key determinants of users’ subsequent behavior and responses.

In terms of visual aesthetics, in the marketing literature, Bloch et al. (2003) proposed the concept of the centrality of visual product aesthetics (CVPA) to account for individual differences in consumers. They defined CVPA as “the level of significance that visual aesthetics holds for a particular consumer in his/her relationship with products” (p. 552). The CVPA encompasses three related dimensions: (1) acumen, or the ability to recognize, categorize, and evaluate product designs, (2) the value a consumer assigns to product appearance in enhancing personal and even societal well-being, and (3) the level of response to visual design aspects of products. Table 1 contains the CVPA measurement items from Bloch et al.’s (2003) study.

Table 1 CVPA dimensions and measurement items
Full size table

In the website design context, the evidence that visual aesthetics are perceived differently among individuals is supported by some empirical studies. For example, Tractinsky et al. (2006) reported differences among participants’ average ratings of web page visual aesthetics. Interestingly, their results also revealed that some participants rated the web pages as fairly unattractive, whereas others rated the same set of web pages as highly attractive, giving an early indication of individual differences in aesthetic perceptions of website designs.

The findings of a more recent study by Papachristos and Avouris (2011) were consistent with those of Tractinsky et al. (2006). They reported that participants’ ability to rate website visual appeal were considerably inconsistent. In addition, in Jacobsen’s (2010) fMRI study, it was reported that individual differences in aesthetics played a substantial role in influencing individuals’ aesthetic judgment and response. Yoo and Kim (2014) reported that the CVPA is not significant in website design perception; however, the CVPA items were applied directly in the website context, and as a result, the website aesthetic aspect was missing in their study. These studies support the notion that website visual appeal can be perceived differently by individuals. Nevertheless, the concept of website visual appeal remains to be investigated, especially from the user’s perspective, since it is still not well understood and has not been explained by the existing studies.

2.2 Hypothesis Development

Several studies have suggested that website visual design is a critical factor in determining users’ perceptions, judgments, and behavioral responses toward websites (e.g., Cyr 2008; Norman 2004). It is important to note that, in this study, we focus on users’ holistic aesthetic impression, which considers all the visual design elements as a comprehensive dimension of website visual design features. Several studies in the IS and human-computer interaction (HCI) literature suggest that such design features are used as executional cues that trigger users’ emotional responses (Éthier et al. 2006, 2008; Norman 2004), which, in turn, influence the users’ perceptions of websites, such as trust (McKnight et al. 2002), satisfaction (Flavián et al. 2006), and quality of e-retailers (Loiacono et al. 2007; Schlosser et al. 2006). Some examples of website visual design features investigated in the literature include color (Cyr et al. 2010; Kim and Moon 1998), design clarity and perceived visual appeal (e.g., Cyr 2008; Cyr and Bonanni 2005; Gefen et al. 2003; Hampton-Sosa and Koufaris 2005; Karimov et al. 2011; Robins and Holmes 2008; Vance et al. 2008; Wang and Emurian 2005; Zhang et al. 2009), design quality of website aesthetics (Jiang et al. 2016), and images (e.g., Cyr et al. 2009; Karimov et al. 2011; Wang and Emurian 2005).

Regarding the theoretical framework, the connection between website visual appeal and users’ behavioral responses can be explained by the affect-as-information model (Schwarz 1986). The affect-as-information model posits that emotional states evoked by users’ aesthetic impressions influence perceptions of and responses to the stimuli they are experiencing. A person’s emotional state alters the assessment of new stimuli by pushing them in the direction of the valence of the emotion (positive or negative) that is already being experienced. As a result, emotional states influence individuals’ cognitive processes in judging new stimuli. With respect to website visual appeal, visually-pleasing websites can invoke positive emotional states (Éthier et al. 2006), which then positively affect the users’ evaluation of the websites. On the other hand, poorly-designed websites can invoke negative emotional states, which adversely shape the users’ perceptions of the websites. These perceptions subsequently shape behavioral intentions. Positive emotional states induce approach actions, such as physical movement toward, engaging in, and exploring the environment, whereas negative emotional states cause avoidance behavior, such as physical movement away from the stimuli (Deng and Poole 2010; Schwarz 1986).

According to the literature, website visual appeal is considered an overarching concept with multiple aspects. Previous studies have attempted to understand the mechanism of website visual appeal on users’ responses. For example, Lavie and Tractinsky (2004) distinguish between two types of aesthetics — classical and expressive. While classical aesthetics are primarily characterized by pleasant, clear, clean, and symmetric designs, expressive aesthetics consider creative, fascinating, and sophisticated designs. These two dimensions have been adopted by studies that investigated website aesthetics (e.g., Coursaris and van Osch 2016; Lindgaard et al. 2011; Tractinsky and Lowengart 2007). In addition, visual design feature is considered an executional cue that fulfills viewers’ intrinsic hedonic needs (Holbrook and Hirschman 1982) by evoking emotional states (Éthier et al. 2006; Roseman et al. 1996). Consistent with this notion, in the website context, visually-pleasing websites have been found to influence users’ positive emotional states (Éthier et al. 2006, 2008; Hwang and Kim 2007; Lin et al. 2012), which further shape the users’ subsequent judgment of the websites (Loken 2006). Such an effect can be considered a halo effect (Nisbett and Wilson 1977) that carries over to the evaluation of other attributes of the websites.

Previous studies have investigated website visual design in multiple levels to understand its effects on users. Some studies focus on specific, low-level visual design components, such as color (Cyr et al. 2010; Kim and Moon 1998), layout (Geissler et al. 2006), and images (e.g., Cyr et al. 2009; Hassanein and Milena 2004; Karimov et al. 2011; Wang and Emurian 2005). Other studies address website visual design at a more comprehensive and higher level of abstraction, such as hedonic and pragmatic dimensions (Hassenzahl 2004) and classical and expressive aesthetics (Lavie and Tractinsky 2004). This study, in contrast, considers users’ response to the overall visual design of the website; that is the visual appeal influences users’ holistic aesthetic impression of the website.

According to Tractinsky and Lowengart (2007), user perceptions of website visual aesthetics are considered a function of website design characteristics and user characteristics. The website design characteristics are the objective properties of the visual design (e.g., shape, color, or layout) and may be used to intentionally effect the desired user perceptions. However, users may not have the interpretation or aesthetic perception that the designer intended, since users may be influenced by individual dispositions, such as individual differences, nationality, or prior experience (e.g., Papachristos and Avouris 2011; Tractinsky et al. 2006). These findings from the previous studies have raised questions regarding differences in the ability of individuals to recognize website visual appeal. Therefore, we propose the following hypothesis:

  • Hypothesis 1: CVWA moderates the influence of website visual appeal on users’ perceptions of visual appeal; that is, website visual appeal has a greater effect on the perception of visual appeal for users with a higher CVWA compared to users with a lower CVWA.

In addition to perceived visual appeal, we examine online trust, which is another key factor in the success of a website especially in the e-commerce setting. Online trust has been found to affect several critical determinants of website success, such as satisfaction (Flavián et al. 2006), perceived website usefulness (e.g., Gefen et al. 2003), intention to use (e.g. Holsapple and Wu 2008), intention to buy (e.g., Lim et al. 2006), and e-loyalty (e.g., Cyr 2008; Flavián et al. 2006). According to McAllister (1995), trust consists of cognitive and affective dimensions. Cognition-based trust relies on rational evaluation, available knowledge, and good reasons (Jeffries and Reed 2000). Website designers can enhance cognitive-based trust by incorporating certain attributes into the websites, such as reliability, guarantees, and professional credentials (Pennington et al. 2003). On the other hand, affect-based trust relies on emotional bonds evoked by the user’s first impressions of the website. Website design features such as color, images, and appearance are generally considered important components that develop first impressions (e.g., Hampton-Sosa and Koufaris 2005). Such first impressions form the basis for the user to decide whether the website vendor will deliver the product/service as promised. In addition, McKnight et al. (1998) suggest that affective-based trust can be affected by a personal tendency to trust. This disposition to trust reflects the extent to which an individual demonstrates a centrality to be willing to trust. CVWA appears to be in line with the concept of disposition to trust in establishing affective-based trust; therefore, we propose the following hypothesis.

  • Hypothesis 2: CVWA moderates the influence of website visual appeal on trust; that is, website visual appeal has a greater effect on trust in website providers for users with a higher CVWA compared to users with a lower CVWA.

Reflecting previous studies in the area of website adoption, we also examine intention to use websites, as it has been suggested as a major factor that determines the success of online commerce websites (e.g., Pennington et al. 2003; Reinecke et al. 2013). According to Schwarz (1986), emotional states inform individuals’ approach or avoidance behaviors. For example, positive emotions induce approach actions such as using a website, and negative emotions may lead to avoidance behaviors such as leaving the website. There has been consistent evidence that the extent to which emotions motivate users to approach or avoid a website is moderated by user characteristics, for example, a metamotivational state (Deng and Poole 2010), attitudes (Fishbein and Ajzen 1975), and subjective norms (Schepers and Wetzels 2007). In this light, we also propose that the CVWA differs among individuals, as website visual appeal may stimulate individuals to use or avoid the website at different levels of arousal. Therefore, we propose the following hypothesis.

  • Hypothesis 3: CVWA moderates the influence of website visual appeal on intention to use the website; that is, website visual appeal has a greater effect on the intention to use the website for users with a higher CVWA compared to users with a lower CVWA.

3 Overview of the Experiments

A series of three experiments was conducted to examine the impact of CVWA on user responses. Experiment 1 replicated Bloch et al.’s (2003) study by adopting the measure from the CVPA scale developed in the marketing literature to test the effects of the CVWA in the website environment. A confirmatory factor analysis (CFA) was conducted to examine the CVPA construct validity. The effects of CVPA on perceived visual appeal, trust, and intention to use the website were tested on a hypothetical website, which was created specifically for this study to avoid the effects of content and brand familiarity. The realism of the experimental website was retained by mimicking a legitimate charity website. The charity website was selected as a stimulus to avoid the confounding effects of website attributes such as product aesthetics (we were interested only in the role of website design aesthetics), price, or features. Three versions of the charity website were created. They varied only in terms of visual appeal: all three websites provided the same information, content, and features to avoid the effects from other variables that may impact the observed variables. The websites differed only with respect to various visual design components suggested in the visual aesthetics literature (e.g., Kimball 2013; Nielsen 2005), specifically, different levels of color, contrast, and the use of images, as shown in Fig. 1.

Fig. 1
figure 1

Conditions of the three experimental websites

Full size image

In general, the results from Experiment 1 revealed that the CVWA potentially influenced perceived visual appeal, trust, and intention to use the website in the expected direction. However, the effects of the CVWA were not strong enough to elicit a statistically significant impact. Consequently, Experiment 2 was designed specifically to develop an alternative measure of the CVWA, the CVWA measure. Based on the CVPA measure, the CVWA measure was created as a semantic differential scale and then tested for validity and reliability. In addition, Experiment 3 was conducted to investigate the effects of the CVWA using both measures: the CVPA and the CVWA measures. The effects of the CVWA on perceived visual appeal, trust, and intention to use the website were examined on the hypothetical charity website conditions 1 and 3 in Experiment 1.

4 Experiment 1: Assessing CVWA Using the CVPA Measure

4.1 Methodology

Our experimental study was conducted in a computer lab to avoid environmental confounding factors. As mentioned in the previous section, this experiment aimed to evaluate participants’ CVWA using the CVPA measure and to observe its effects on perceived visual appeal, trust, and intention to use websites.

4.1.1 The CVPA Measure

The CVPA measure we used was adapted from the CVPA items developed by Bloch et al. (2003). All items in the three dimensions of the CVPA measure were reworded to reflect the website visual design and online business settings. Table 2 presents the CVPA measurement items used in this experiment.

Table 2 CVPA items and loadings from the CFA results (Cronbach’s alpha = 0.811) (* item dropped due to high cross-loading values)
Full size table

4.1.2 Participants and Procedure

A total of 99 undergraduate students enrolled in a major Midwestern university participated in the experiment (25 females and 74 males). One unit of extra course credit was offered as an incentive for participation in the experiment. Participants were briefed about the study’s general objectives and were given written instructions regarding the experimental task. They answered the CVPA questions on a seven-point Likert scale and were then randomly assigned to one of the three conditions of the charity website (Fig. 2). Each condition contained an approximately equal number of participants. Participants were asked to perform a hypothetical task—consider making an online donation to the victims of a natural disaster. After viewing the website, participants were directed to an online questionnaire to rate their perception of visual appeal, trust, and intention to use the website on seven-point Likert scales. There were no time limits to viewing the website and completing the questionnaire. The experimental session lasted approximately 30 min.

Fig. 2
figure 2

Means and standard deviations of the CVPA groups and website conditions

Full size image

4.2 Results

Task durations were investigated to identify participants who might not have paid attention to the experiment and might have participated just for the reward. The task durations were sorted and the 5% of participants with the longest and the shortest durations were excluded from the analysis. Consequently, the data set contains 91 usable responses, from 24 females (26.4%) and 67 males (73.6%). The subjects were between 18 and 55 years old. Of the participants, 84 (92.3%) checked or sent email messages every day and 26 (28.6%) made 1–3 online purchases per month.

We then performed manipulation checks of website visual appeal using ANOVA. Participants exposed to website conditions with a higher level of visual appeal were more likely to agree that the website is more visually attractive than were those assigned to website conditions with lower visual appeal. The average visual appeal of the high-visual-appeal website, the moderate-visual-appeal website, and the low-visual-appeal website is 6.00 (Std. Dev. = 1.03), 3.83 (Std. Dev. = 2.05), and 1.41 (Std. Dev. = .66), respectively. Tukey’s HSD results indicate that the aggregate difference among the means of visual appeal across the three groups is statistically significant at the 0.05 level. Therefore, the manipulations of website visual appeal were deemed successful.

4.2.1 CVPA Measure Assessment

The CVPA measure was assessed for reliability and convergent validity. An exploratory factor analysis (EFA) was conducted to explore the underlying dimensions of the CVPA measure as a one-factor, second-order model. Our results demonstrate that the CVPA measure comprises three conceptual sub-dimensions, a finding that is in concert with the results from previous work on the CVPA (Bloch et al. 2003). We then conducted a confirmatory factor analysis (CFA) to examine the convergent validity of the CVWA construct. The results demonstrate relatively high correlations between items of the same construct. As a rule, items in their corresponding construct load highly if the loading coefficient is above 0.6 and do not load highly if the coefficient is below 0.4 (Hair et al. 1995). We found two items (VAL2 and RES1) that did not load well on their corresponding dimension of the CVPA and they were removed from the analysis. The complete list of CVPA items and loadings from the CFA results are shown in Table 2. The construct reliability of the CVPA items assessed by Cronbach’s Alpha is 0.811, suggesting that the internal consistency of the items is acceptable.

4.2.2 The Influence of Individual Website Aesthetics on User Responses

A further analysis was conducted to examine whether the levels of individual website aesthetics influence user responses toward websites that differ in aesthetic quality. While we expected a main effect of superior design generating more positive evaluations, our overall research hypothesis is that users with high individual website aesthetics are more discriminating in their responses (perceived visual appeal, trust, and intention) than users with low individual website aesthetics.

A 3 × 3 factorial design was analyzed to test the proposed hypotheses. An overall CVPA score was computed for each subject from the items that loaded highly on the three sub-dimensions of the CVPA measure. The mean CVPA score for this sample is 6.42 (Std. Dev. = 1.21). The sample was divided into three equal groups by CVPA scores, and a series of two-way ANOVAs was conducted using the 3 × 3 design (three levels of website visual appeal vs. three levels of CVPA). The mean scores for the high CVPA subject group (n = 30), moderate CVPA subject group (n = 30), and low CVPA subject group (n = 31) are 7.69 (Std. Dev. = 0.48), 6.57 (Std. Dev. = 0.23), and 5.06 (Std. Dev. = 0.78), respectively.

The results of the series of ANOVAs on perceived visual appeal, trust, and intention indicate that the three conditions of website visual appeal are statistically significantly different for perceived visual appeal (F = 80.04, p < 0.001), trust (F = 16.50, p < 0.001), and intention to use (F = 28.06, p < 0.001). With regard to the interaction effects of the CVPA, even though some of the effects are in the predicted direction, the coefficients did not reach a conventional level of statistical significance. Figure 2 presents means and standard deviations of the dependent variables on the three website conditions and CVPA groups.

4.3 Discussion

While the findings suggest that the CVWA can potentially shape user responses, the effects are not significant at the 0.05 level. Possible explanations for these findings include the weak effects of the CVWA and CVPA measurement error. Regarding the weak effects of the CVWA, the results contradict evidence from previous studies that individual differences in aesthetics tailor users’ aesthetic impressions (e.g., Jacobsen 2010; Papachristos and Avouris 2011; Tractinsky et al. 2006). Therefore, further study is needed to confirm these findings.

With regard to CVPA measurement error, the current research instrument and methodology may not be effective as an assessment of participants’ CVWA. As presented in Table 2, two of the CVPA items (VAL2 and RES1) had high cross-loadings across the CVPA dimensions and were removed from the analysis. In addition, item VAL 3 has a relatively low factor loading on the value dimension, 0.703, which is only slightly over the acceptable threshold for the factor-loading value (e.g., Bagozzi et al. 1991; Chin et al. 1997; Straub 1989). Furthermore, given the nature of the CVPA items (e.g., “Being able to see subtle differences in website designs is one skill that I have developed over time” and “I have a pretty good idea of what makes one website look better than its competitors”), it would be quite difficult for a participant to score at the low-prejudiced end of the rating scale. This problem could confound measurement bias such that it did not permit an individual to give an unprejudiced response (Arnold and Feldman 1981; John and Robins 1994; Lewicki 1983; McConahay 1986). Such measurement bias has been reported in prior studies, especially when factors related to individual differences and centrality were investigated (e.g., Balzer 1986; Varma et al. 1996). In this experiment, the CVPA items were embedded in a self-rating scale; thus, it seems very possible that the CVPA scores indicate prejudice. Consequently, it is reasonable to argue that the CVPA measurement items and the self-rating scale may not be an effective method to assess one’s CVWA. For this reason, in Experiment 2, we propose the development of an alternative measure of the CVWA, one that can assess individual website aesthetics implicitly, to minimize measurement bias.

5 Experiment 2: Developing an Alternative Measure of CVWA

In this experiment, we propose an alternative strategy to implicitly assess CVWA by applying the concepts of indirect attitude measurement. In general, this methodology provides an estimate of participants’ CVWA without directly asking them to consider their attitudes regarding website aesthetics. Overall, this approach requires participants to perform a set of tasks involving rating a series of 18 website screenshots in the three major dimensions of the original CVPA measure—acumen, value, and response. The measure was developed by using the semantic differential technique of Osgood et al. (1957).

5.1 Methodology

5.1.1 CVWA Measure Development

The centrality of visual website aesthetics (CVWA) measure was developed in this experiment to implicitly assess participants’ individual website aesthetics by following the scale development approach suggested by previous studies (e.g., DeVellis 2016). Specifically, as discussed in the previous experiment, certain items in the original self-rating CVPA measure may not be suitable to assess individual differences due to bias and prejudice. Therefore, rather than participants’ having to submit explicit responses, we designed an alternative set of measurement items to implicitly capture the CVWA. In general, such measures require participants to perform a task by responding to a certain set of pretested stimuli. Such implicit measures have been widely used in the psychology literature, especially when assessing individual differences and centrality, such as implicit attitude (e.g., Fazio et al. 1995; Greenwald et al. 1998) and variance and arousal (e.g., Langer et al. 2008).

We also designed the CVWA measure as a semantic differential scale to minimize the problem of acquiescence, which happens when individuals tend to overly rate their favorable attributes and ignore their unfavorable attributes (Furnham 1986). The acquiescence problem has been commonly found in Likert-based scales but can be reduced by converting the scales into semantic differentials (e.g., see Friborg et al. 2006). Therefore, a semantic differential scale is more appropriate than the self-rating scale used in Experiment 1 to assess the CVWA.

While Bertamini et al. (2013) developed an implicit semantic differential measure to evaluate the relationship between symmetrical visual patterns and emotional response, rather than specific visual design elements, we sought to develop an implicit measure of individual centrality of holistic aesthetic impressions in the website design domain. To the best of our knowledge, a CVWA measure has not been reported in the literature, possibly because individuals’ centrality of aesthetics and its effects have rarely been studied, especially in the online context. Therefore, it is necessary to develop an efficient and robust measure for the CVWA.

Consistent with the CVPA measure, the semantic differential pairs for the CVWA measure were developed based on the three CVPA dimensions (acumen, value, and response). The semantic differential terms were derived from the definitions of the CVPA dimensions proposed by Bloch et al. (2003). Specifically, for the acumen dimension, the semantic differential terms were “Very Attractive – Very Unattractive” to represent individuals’ ability to recognize, categorize, or evaluate website visual aesthetics (Giese et al. 2014). For the value dimension, we used the terms “Very Satisfied – Very Dissatisfied” to present the value of the website aesthetics that enhanced individuals’ emotional state, especially satisfaction, which has been reported as a major factor influencing the value of business (e.g., Cyr et al. 2010; Flavián et al. 2006; Yang and Peterson 2004). Last, the terms “Will Definitely Use – Will Never Use” were used for the response dimension since it represents individuals’ approach-avoidance behavioral response toward a website (e.g., Deng and Poole 2010). The semantic differential terms were then examined by graduate students majoring in English to assure that the terms represented the corresponding dimensions of the CVPA.

Originally, a set of 30 website screenshots with various degrees of visual appeal was used as the stimuli to elicit participants’ individual website aesthetics. Overall, these websites were carefully selected by website design experts based on three criteria: (1) they do not belong to well-known vendors, (2) they represent a wide range of corporate and institutional websites, including corporate, e-commerce, entertainment, and information sites, and (3) the websites cover a wide range of visual appeal, including very poor to very professional visual appeal. Regarding visual appeal, the website stimuli were selected by website design experts based on key visual design principles suggested by previous studies, such as visual symmetry (Jacobsen and Höfel 2003), color (Moshagen and Thielsch 2010), and contrast (Lim et al. 2007).

A pretest was conducted on a separate sample of 35 undergraduate students to assess the visual appeal of the 30 websites. In this pretest, participants rated the visual appeal of each website in random order on a seven-point Likert scale. The results show average ratings of visual appeal for the high-visual-appeal websites and low-visual-appeal websites of 6.37 (Std. Dev. = 1.667) and 1.23 (Std. Dev. = 1.660), respectively. However, in order to shorten the experimental session, only 18 websites (nine websites with the highest visual appeal ratings and nine websites with the lowest visual appeal ratings) were used in the main experiment. From the 18 websites, six websites (three high visual appeal and three low visual appeal) were assigned to each of the CVPA dimensions. Table 3 presents the CVPA and CVWA measures. Note that a few items in the CVPA measure have been slightly reworded from those used in Experiment 1 to better describe the CVPA dimensions in the website design context.

Table 3 Items in the CVPA and CVWA measures
Full size table

5.1.2 Participants, Stimulus and Procedure

Eighty-three undergraduate students (35 females and 48 males) who had not participated in Experiment 1 completed this experiment. One unit of extra course credit was offered as an inducement for participation. Participants were asked to rate the CVPA measure and then performed a set of tasks to assess the individual website aesthetics by using the CVWA measure. In addition, before the CVWA measure tasks had been carried out, participants were directed to complete a set of 10 trial questions that mock the actual semantic differential items as warm-ups and to acquaint them with the CVWA rating tasks. The experimental session lasted approximately 20 min.

Regarding the CVWA tasks, each of the 18 websites was randomly displayed to avoid systematic errors (Liu and Salvendy 2009) with the corresponding semantic differential pair shown in Table 3. One website was presented at a time as shown in Fig. 3. Participants were asked to rate the semantic differential scale item ranging from 1 to 10 for each of the website.

Fig. 3
figure 3

An example of semantic differentials in the CVWA measure

Full size image

5.2 Results and Discussion

After screening for participants who might have not paid attention to the experiment, the dataset contained 77 usable responses (32 females and 45 males). Demographic information from participants was compared to the information from those who took part in Experiment 1 using a series of t-tests. The results do not reveal any statistically significant differences between the two groups.

Confirmatory factor analysis (CFA) was conducted on the CVPA items to assess the measurement validity. The results are consistent with those of Experiment 1—VAL2 and RES1 items appeared to have high cross-loading values and were removed from the analysis. The remaining items from the CVPA measure were averaged to formulate the overall individual website aesthetic scores.

Regarding the CVWA measure, the measurement items were assessed for instrument quality by testing the construct validity and reliability. The results of an exploratory factor analysis (EFA) suggest that, in line with the CVPA measure, the CVWA appears to be a higher-order factor model with three lower-order factors (or constructs)—acumen, value, and response.

Consequently, CFA was conducted to assess the construct validity, which is demonstrated when there are relatively high correlations between measures of the same construct (convergent validity) and low correlations between measures of different constructs (discriminant validity). Construct reliability was assessed using composite reliability (CR) and Cronbach’s alpha. Factor loadings of the CVWA constructs, CR values, Cronbach’s alpha, and average variance extracted (AVE) are presented in Table 4. CR values range from 0.961 to 0.982 and the Cronbach’s alpha values range from 0.939 to 0.972. Both the CR and Cronbach’s alpha values are above the acceptable level suggested by Hair et al. (2006). Thus, the CVWA measure proposed in this experiment demonstrates convergent validity.

Table 4 CVWA measurement items, CR, Cronbach’s alpha, AVE, and loadings from the CFA results. Bold text indicates the items in the CVWA dimension
Full size table

Discriminant validity was determined to ensure that constructs differed from each other such that the correlation between items in any two constructs should be lower than the square root of the average variance shared by items (e.g., AVE) within a construct (Hair et al. 1995). As shown in Table 5, the square root of the variance shared between a CVWA dimension and its measurement items is greater than the correlations between the dimension and other dimensions. Therefore, the CVWA measure satisfies the criteria for discriminant validity.

Table 5 Discriminant validity of the CVWA measure. Diagonal elements in boldface represent the square root of AVE.
Full size table

In addition, the results of a correlation analysis between the CVPA and CVWA measures indicate that the overall correlation between the two measures is 0.107 (p = 0.291), which is not significant at the 0.05 level. The correlations between the three dimensions of the CVPA across the measures were then examined. The results suggest that only the response dimension of the CVPA and CVWA measures is significantly correlated at the level of 0.05, as presented in Table 6. Overall, the results from Experiment 2 provide evidence that the CVWA measure can potentially outperform the CVPA measure, according to the construct validity testing. Therefore, further investigation using the CVWA measure to assess the effects of CVWA on user responses should be carried out.

Table 6 Correlations between the dimensions of the CVWA assessed by the CVPA and CVWA measures (* significant at the 0.05 level)
Full size table

6 Experiment 3: Investigating the Effects of the CVWA on Perceived Visual Appeal, Trust, and Intention to Use the Website

The findings from Experiment 1 suggest that the effects of the CVWA may be stronger when participants evaluate website conditions with high and low levels of visual appeal—not with moderate visual appeal. These findings are consistent with the results from prior studies suggesting that, as an inverted U-shaped function, stimuli with extreme levels of visual appeal are more correlated with visual appeal ratings than those with moderate level (e.g., Bassili 1996; Pham et al. 2001). Therefore, only the charity website conditions with high and low visual appeal (Conditions 1 and 3) are used as stimuli in Experiment 3.

Regarding the CVWA measure, the results from Experiment 2 provide evidence that the CVWA measure may be more effective than the CVPA measure for capturing the CVWA. Therefore, the performances of these measures were evaluated in Experiment 3. Note that because of high cross-loading values in Experiments 1 and 2 for items VAL2 and RES1, these two items were not included in the CVPA measure. The user responses (perceived visual appeal, trust, and intention to use the website) and the three hypotheses tested in Experiment 1 were examined in Experiment 3.

6.1 Methodology: Participants, Stimulus, and Procedure

A total of 84 undergraduate students (33 females and 51 males) who did not take part in Experiments 1 and 2 participated in this experiment. One unit of extra course credit was offered as an inducement. The experiment was conducted in a computer lab in multiple sessions, each session lasting approximately 30 min. Participants were asked to perform the same hypothetical task as in Experiments 1 and 2—answering the questionnaire using the CVPA measure and the CVWA measure. Subsequently, participants were asked to consider making an online donation to the hypothetical charity using the website. Before the data were collected, as in Experiment 2, a block of 10 trial questions was administered to acquaint participants with the CVWA measure procedures. These data were not analyzed further. Participants were asked to evaluate the charity website conditions (high and low visual appeal) presented in random order. Then participants were asked to rate the website on visual appeal, trust, and their intention to use the website as well as answer demographic information questions.

6.2 Results

The dataset contains 80 usable responses (32 females and 48 males). Demographic information and the online experience of the participants in this experiment were compared to those of the participants who completed Experiments 1 and 2. The results do not reveal any statistically significant differences between the groups of participants.

As in the previous experiments, confirmatory factor analysis (CFA) was conducted on the items in the CVPA and CVWA measures. The results resemble those of Experiments 1 and 2, suggesting that the CVWA measure has satisfactory validity.

CVWA scores for both the CVPA measure and CVWA measure were calculated based on the procedures used in Experiment 2. For each measure, the sample was divided into two groups by the CVWA scores (high vs. low). A series of t-tests was conducted to test the proposed hypotheses regarding the effects of CVWA on perceived visual appeal, trust, and intention to use the website. The results are presented in Fig. 4.

Fig. 4
figure 4

Results from the hypothesis testing (* significant at the 0.05 level)

Full size image

Overall, the results from both measures are consistent and in the expected directions, especially for the website condition with high visual appeal. According to the CVPA measure, high CVWA participants discriminate more strongly than low CVWA participants across all three dependent variables (perceived visual appeal, trust, and intention to use the website). These results are similar to those using the CVWA measure. However, CVWA effects were not found in the website condition with low visual appeal. Therefore, regarding the hypothesis testing, Hypotheses 1 are 3 are partially supported when using the CVWA measure for the website condition with high visual appeal. Regarding Hypothesis 2, the direction of the CVWA is consistent with those of Hypotheses 1 and 2 but the effect is weaker (p = 0.064) such that it is not statistically significant at the 0.05 level.

Furthermore, we conducted a correlation analysis to further explore the relationships among the three dimensions of the CVWA assessed by the CVWA measure and perceived visual appeal, trust, and intention. The results, presented in Table 7, reveal that only the high-visual-appeal website condition is significantly correlated with the CVWA at the sub-dimension levels. In addition, the response dimension of the CVWA is statistically significantly correlated with all three user response variables when website visual appeal is relatively high.

Table 7 Correlations between the CVWA dimensions and user responses (* significant at the 0.05 level)
Full size table

6.3 Discussion

The results of Experiment 3 confirm our reasoning for the effects of the CVWA on user responses as well as further attest to the validity of the CVWA measure. As suggested earlier, the effects of website visual appeal are stronger for participants with a high CVWA than those with a low CVWA, especially when the visual appeal of the website is high. The directions of the CVWA effects are as predicted and consistent with those of Experiment 1.

The results of this experiment indicate that the effects of the CVWA are elicited when participants interact with websites that have a high level of visual appeal. These results are in line with the findings of prior research investigating the moderating relationships between personal traits and website design quality (e.g., Turkyilmaz et al. 2015). In this respect, website visual appeal, which is an aspect of design quality, can evoke emotional states which subsequently elicit higher levels of arousal for users with a high CVWA than for those with a low CVWA.

The results also suggest that one’s CVWA influences perceived visual appeal and intention to use the website, but not trust. As previously discussed, trust can be formed based on cognition or affect (McKnight et al. 1998); therefore, these results lead us to infer that the CVWA plays little role in establishing online trust, which could indicate the importance of some non-emotional process underlying online trust development. These findings could be concerned with the cognitive evaluation of website features that exist in addition to the affective aspects. Prior online trust research has provided evidence of a strong impact of cognitive website design features on trust formation, such as user reviews (Ye et al. 2009), third party assurance seals (McKnight et al. 2004), and familiarity (Gefen 2000), which may have a direct influence on trust and are not moderated by the CVWA.

Regarding the CVWA measurement, the findings suggest that the CVWA measure outperforms the CVPA measure in assessing CVWA. Given the various factors that confound individuals’ responses to the CVPA measurement items, it would be difficult to draw clear inferences from findings based on the CVPA measure’s scores. This is not to imply that the CVPA measure lacks measurement validity since the results were in the expected directions; however, it is less effective in capturing the CVWA than the CVWA measure. In addition, according to the analysis of the correlations between the sub-dimensions of the CVWA and user responses, the response dimension appears to be significantly correlated with all three user response variables. These results provide evidence that the more cognitive-related dimension of the CVWA may produce a stronger effect on user response; however, more detailed aspects of these findings await further investigation.

7 General Discussion, Limitations and Conclusion

The major aims of the study concerned the development of the CVWA measure as well as the investigation of the CVWA’s effects on user responses. The results of the research clearly attest to the importance of the variability that exists among online users with respect to their centrality of website visual aesthetics. Summarizing the earlier findings, the results of Experiment 1 provide evidence that the CVWA potentially affects, albeit rather weakly, perceived visual appeal, trust, and intention to use websites. That is, users with a high CVWA may be more inclined to use aesthetics in their evaluations of websites than users with a low CVWA. In Experiment 2, in addition to the direct measure, we proposed the development of the CVWA measure to implicitly capture participants’ CVWA. The results suggest that the CVWA measure potentially outperforms the CVPA measure. As a result, Experiment 3 was conducted to test the hypotheses of the effects of the CVWA in the low-visual-appeal and high-visual-appeal website conditions. The findings of Experiment 3 provide evidence that the effects of the CVWA are more salient when users interact with websites with high levels of visual appeal.

The findings of this research suggest that the CVWA is both a theoretically and managerially relevant factor. While the effects of the CVWA on online users’ responses are mixed in previous studies (Bovee 2004; Phillips 2007), our investigation provides evidence of significant effects of the CVWA, especially when using the CVWA measure. Thus, the major theoretical contribution of this study is establishing the importance of the CVWA construct in the related literature. The current study raises an interesting area for further research—whether individual differences moderate the effects of website visual appeal on user responses. Therefore, the CVWA should be considered in the future investigations that seek to test relationships between website visual appeal and user responses.

In addition, aligning website design features and the CVWA dimensions generally represents another research direction of promise. Our results suggest that the dimensions of the CVWA, especially the response dimension, reveal various effects on user responses. Future research can explore the impact of specific website visual design features on users’ CVWA at certain levels of mental processing (see Jacobsen and Höfel 2003).

As for practical contributions, the outcome of this research will be of interest to managers and web designers. The present study has direct application to the managerial question of how to effectively design websites targeted to a specific user segment, information that can help managers create websites that improve user experience and, consequently, elicit the desired behaviors. The results of this study can also assist managers in their decision to customize their website designs based on the CVWA of their users. The influence of website visual design is stronger on users with a high CVWA. Therefore, managers can customize their website designs based on their target customers’ CVWA. For example, a website that offers products or services for design professionals (e.g., a design-related magazine) can assign higher priority to visual-related design strategies. Furthermore, managers must be careful when customizing the designs of their websites, since the design may unintentionally reduce the visual appeal and, in turn, affect their users, especially those with a high CVWA.

As is often the case when new research instruments are developed, certain limitations are inherent in this research. First, the CVWA measure needs to be further validated in future studies, especially for measurement and method errors. Second, while this study focuses on a holistic view of website aesthetics, it would be worthwhile to see whether the effects are produced by specific visual design factors such as lower-level design features (e.g., color, layout, or graphics, see Cyr et al. 2010), by aspects of website visual design (e.g., simplicity and diversity, see Moshagen and Thielsch 2010; Wani et al. 2017), and by dimensions of user experience (e.g., hedonic and pragmatic, see Hassenzahl 2004). Testing the CVWA measure in such contexts may provide additional checks for validity as well as useful information regarding design features that determine one’s CVWA. Finally, it might also be profitable to explore how other individual differences that may affect users’ aesthetic impressions in different ways, such as gender (e.g., Cyr and Bonanni 2005; Lin et al. 2019) and cultures (e.g., Chen et al. 2015; Cyr et al. 2010), interact with users’ CVWA.

Online users have differing abilities for evaluating website visual design aesthetics. The findings of this study suggest that their CVWA influences their responses and that the CVWA measure outperforms the CVPA measure in assessing the CVWA. We look forward to future research investigating such possibilities in a systematic and more detailed fashion. In summary, in order to develop truly effective and efficient websites, it is essential for online vendors to understand their users’ characteristics and preferences, since users rely on different website attributes as signals in making decisions or evaluations of website vendors and because their decisions are critical factors in the success of online commerce.