1 Introduction

Electronic commerce over the Internet has the potential to benefit both vendors and buyers and has received much attention from researchers and practitioners in order to establish good practices in this area. From vendors’ perspective, thousands of organizations have set up web systems in order to take advantage of the opportunities Internet has to offer [1–3]. From buyers’ perspective, these readily available web systems can be used to support, in whole or in part the electronic procurement (e-procurement) process. Fully fledged web systems typically include electronic catalogs, shopping carts, payment systems and order tracking systems. Despites the strategic importance of e-procurement [4, 5] and the magnitude of the business-to-business transactions over the Internet [6], little or no research explored which requirements should be considered the most important when using web systems from different vendors. Little is known for example, on which functional requirements (FR) and non-functional requirements (NFR) are essential to a productive environment for e-procurement and how users perceive them to be.

The objective of the paper is to assess the major FR and NFR of web systems currently used to support business-to-business transactions and to examine their relationship to users’ performance on the job. Data collected from 133 corporate buyers from about 130 Canadian organizations doing business in a wide range of industries helped identify the FR and NFR requirements engineers should be more concerned about when developing or evaluating such systems. All respondents had access to web-based procurement systems and most of them used such systems to support their daily tasks. Although based on feedback from corporate buyers working for Canadian companies, our findings are likely to apply to the broader group of corporate buyers using web systems for procurement purposes since respondents use the same procurement process described in relevant international supply chain management literature and access web systems of vendors around the world.

The remainder of this paper is organized as follows: Sect. 2 reviews relevant literature and presents the research hypotheses, Sect. 3 describes the methodology, Sect. 4 presents findings from this study, Sect. 5 includes a discussion of the findings and their possible implications on requirements determination of successful web systems, and Sect. 6 concludes this paper and outlines further work.

2 Assessing existing web-based procurement systems

2.1 Theoretical background

In the system development lifecycle, requirements determination defines the expected functional and NFR for the to-be system. These requirements can also be used as the basis for the evaluation of a deployed system [7]. In the case of web-based procurement systems, FR describe the expected services provided by such systems and specify the expected input and output and the related processing. NFR on the other hand deal with the quality attributes of web systems and are often defined as constraints placed on such systems. Reliability, security, and usability have been considered to be key quality attributes for e-procurement systems [7, 8]. Such NFR specify how well e-procurement systems must perform the FR identified.

With regards to FR, previous work in the fields of information systems (IS) and supply chain management [9–11] has shown that e-procurement systems should support the identification, the selection and the execution phases of the procurement process. The identification functionality of such systems is defined in terms of ability to allow users to spot potential vendors of a given input. The selection functionality is defined in terms of ability to give buyers an online access to pricing and other relevant information that let them compare different offerings and select a supplier for the pondered input. Finally, the execution functionality of a procurement system is defined in terms of ability to facilitate the online exchange of information needed to complete a transaction and the related post-sale activities, allowing users to check the availability of a given input, place orders, make payments and track the status of their orders and of their shipments online.

Non-functional requirements typically focus on the usability, the security, the reliability, and the performance of e-procurement systems [7, 8, 12]. Usability refers to the degree to which a particular system is easy to understand, to learn and to use under specified conditions [13]. Security means that the use of the e-procurement system is hazard-free and does not create a potential for a loss [14, 15]. Reliability is defined in terms of proper functioning and uptime that allow an e-procurement systems to maintain a specified level of performance [16, 17].

It is reasonable to assume that usage of readily available web-based systems is willful behavior since corporate buyers can carry out their procurement tasks through other means such as on-site visits, phone calls, catalogs, etc. IS researchers have relied on many models based on applications of the Theory of Reasoned Action [18, 19] and on the quality of IS [3, 20] to explain user acceptance of a new system. Such models have consistently shown that systems’ characteristics as perceived by potential users play a critical role in their adoption and can improve users’ performance on the job.

2.2 Research hypotheses

The objective of this research is to assess the most critical FR and NFR of existing web systems used for e-procurement and to link them to users’ performance on the job. First, the assessment of NFR for users and non-users is examined. If, as suggested by existing models on systems’ adoption, corporate buyer’s decision to use readily available web systems is based on her assessment of their quality, then users are likely to rate significantly higher such procurement systems on each requirement than non-users. Also, because the FR cannot be adequately assessed by non-users, this part of the analysis focuses only on the NFR as perceived by respondents. In this context:

  1. H1:

    Users rate significantly higher the extent to which web systems meet the usability requirement than non-users.

  2. H2:

    Users rate significantly higher the extent to which web systems meet the security requirement than non-users.

  3. H3:

    Users rate significantly higher the extent to which web systems meet the reliability requirement than non-users.

The next phase of the analysis focuses on users’ assessment of the FR and the NFR of the web systems supporting the procurement process. Based on findings from the factor analysis discussed in Sect. 3, the execution functionality needed to be refined, hence distinguishing execution services—which refer to placing orders and making payments- from post-sale services—which support tracking the status of orders and of shipments.

First, possible interactions between FR and NFR are examined. Because NFR specify how well (i.e., how user-friendly, how reliable, and how secure) a system must perform its FR, usability and reliability should normally apply to all functions of a web system. Security however applies mostly to the execution services and to the post-sale services because these two functionalities require users’ actions that may cause some loss [7, 14, 15]. Therefore:

  1. H4:

    Usability is positively correlated to the identification function.

  2. H5:

    Usability is positively correlated to the selection function.

  3. H6:

    Usability is positively correlated to the execution function.

  4. H7:

    Usability is positively correlated to the post-sale function.

  5. H8:

    Reliability is positively correlated to the identification function.

  6. H9:

    Reliability is positively correlated to the selection function.

  7. H10:

    Reliability is positively correlated to the execution function.

  8. H11:

    Reliability is positively correlated to the post-sale function.

  9. H12:

    Security is positively correlated to the execution function.

  10. H13:

    Security is positively correlated to the post-sale function.

Second, web systems that meet the FR and NFR associated with the procurement process can improve the performance of users. When used to support the identification and the selection phases of the procurement process, web systems with all the desirable functions and attributes help reduce buyers’ search costs as it becomes easier and faster for users to contact more potential suppliers and decide on the “best” one [21, 22]. When used to support the execution and the post-sale phases, such systems help buyers reduce the processing costs for their transactions by eliminating paperwork, reducing data re-entry, improving information accuracy and reducing lead times [5, 23]. In this context:

  1. H14:

    The identification function is positively correlated with a reduction of users’ search costs.

  2. H15:

    The selection function is positively correlated with a reduction of users’ search costs.

  3. H16:

    The execution function is positively correlated with a reduction of users’ processing costs.

  4. H17:

    The post-sale function is positively correlated with a reduction of users’ processing costs.

3 Methodology

Subjective measures are very common in IS research. To collect data used in this research, a questionnaire was sent to each of the 988 certified professional purchasers members of the “Corporation des Approvisionneurs,” an organization affiliated to the Purchasing Management Association of Canada. Respondents were surveyed about the desirable functions and attributes of the web systems they generally use for procurement purposes and about the benefits from such usage. A total of 133 corporate buyers returned a usable questionnaire (for a response rate of ∼13.5%). The sample spanned corporate buyers from over 130 organizations doing business in a wide range of industries.

Following Churchill’s [24] guidelines, multi-item Likert scales were developed in order to assess the FR and NFR of web systems and their relative benefits as perceived by respondents. In accordance with Nunnally [25], seven-point (1 = Strongly disagree and 7 = Strongly agree) scales were used to ensure statistical variability among survey responses. Whenever possible, pre-existing scales were used because they have been tested for reliability and validity properties; new ones were based on prior work in the field. With regards to the FR, new scales based on the list of tasks involved in the procurement process were used to measure the identification, the selection, the execution, and the post-sale functionalities of web systems [9]. With regards to the NFR, the scales for usability and for reliability were adapted from Segars and Grover [26] and Goodhue [27], respectively, whereas the measurement for security was based on Jarvenpaa and Tractinsky [28], and Palvia [29]. Finally, the assessment of the reduced search costs was based on Bakos [21] and Barua et al. [22], while the measurement of the reduced processing costs was based on Jones and Beatty [5]. The questionnaire was pilot-tested by five corporate buyers, four academics, and one IS professional for completeness and readability. Their suggestions helped develop the final questionnaire.

Following Hair et al. [30] guidelines, the questionnaire was later validated prior to data analysis using factor and reliability analyses. As mentioned previously, findings from the factor analysis revealed the need for a refinement of web systems’ FR which, in this study, include identification, selection, execution, and post-sale support. Findings from the factor analysis revealed also the need for the deletion of one item (SECUR1) related to security as it loaded on the incorrect factor. For the rest, the results of the factor analysis and the large values obtained for all Cronbach’s alphas confirmed the validity and the reliability of the scales. The score for each factors equaled the mean score of its retained items. Table 1 summarizes the results of the validity and the reliability analyses performed. The items used to measure each construct are presented in Table 4.

Table 1 Validity and reliability analyses
Full size table

4 Results

4.1 Respondents’ profile

All 133 participants had Internet access at work and 110 (82.7%) reported using web systems to support supply chain operations. The most common age group of participants was 30–39 (36.8%), followed by 40–49 (32.2%), 50–59 (27.6%), and 20–29 (3.4%). Most of them worked in the manufacturing sector (50%), the rest worked in the retail industry (43%) and in the service sector (7%). In terms of annual volume of transactions, respondents had purchased on average over $25.5 millions worth of input. All the participants were computer savvy: on average, they have been using computers at work for more than 12 years.

4.2 Preliminary data analysis

A first step in examining our data involved testing the normality of variables under study. The results of the Kolmogorov–Smirnov tests performed on the refined constructs and presented in Table 2 have shown that the assumption of normality is rejected for 6 of the 9 constructs investigated. Therefore, nonparametric (Mann–Whitney and Spearman correlation) tests were used for all subsequent analyses (α = 0.05) because these tests assume very little about the underlying distribution. Next, to assess the possible non-response bias, Mann–Whitney tests were used to compare data obtained from early respondents (i.e., those who had sent a completed questionnaire within the first 3 weeks) to that of late respondents (i.e., those who had sent a completed questionnaire after the third week). The rational for this screening is that late respondents are likely to have similar characteristics to non-respondents [31]. The chi-square test (α = 0.05) was used to examine the possible relationship between the response time and the decision to use readily available web systems to carry out procurement tasks. Findings summarized in Table 2 show no significant differences between early and late participants, indicating that respondents were representative of the larger population of corporate buyers targeted in this study. Finally, an analysis of possible links between respondents’ profile and the variables under study was conducted to identify potential confounding factors that may affect the conclusion of this research. Findings presented in Table 2 show a significant and positive correlation between computer literacy and ratings related to the selection functionality of web systems (Rho = 0.239 and sig. = 0.017). No other significant relationship was found.

Table 2 Descriptive statistics and respondents profile analysis
Full size table

4.3 Assessment of FR and NFR of web systems and their relative importance

The first phase of the analysis assesses the NFR as perceived by all respondents and examines their relationships to respondents’ decision to use or not to use readily available web systems to support the procurement process. On average, respondents have rated web systems high on usability (mean 5.41) but low on reliability (mean 2.72) and security (mean 2.73). Mann–Whitney tests were used to examine the hypothesized difference in NFR assessment between users and non-users of web systems. Findings show support for H1, indicating a significant difference in usability assessment between users and non-users (Z = −2.051 and sig. = 0.040) with a higher mean rank (69.10) for users than for non-users (51.46). Findings show however no support for H2 and H3, indicating no significant differences for the assessments related to web systems’ reliability (Z = −0.367 and sig. = 0.713) and security (Z = −0.392 and sig. = 0.695). Based on these findings, usability seems to have more effect than reliability or security in users’ decision to adopt readily available web systems to carry out their procurement tasks.

The subsequent phase of the analysis focuses on users’ assessment of the FR and the NFR of the web systems supporting the procurement process. With regards to the FR, web systems rate higher on identification (mean 6.11) and selection (mean 4.23) services than on execution (mean 2.47) and post-sale (mean 3.47) services. With regards to the NFR, web systems rate higher on usability (mean 5.52) than on reliability (mean 2.74) and security (mean 2.71). Next, possible interactions between FR and NFR are examined. Spearman correlation tests were used to investigate the relationships between FR and NFR. Findings presented in Table 3 do not support any of the hypothesized relationships: the correlation coefficients were all low (varying from −0.132 to 0.128) and all non-significant.

Table 3 Hypothesis testing
Full size table

The last phase of the analysis examines how FR as perceived by web system users correlate with the reported performance on the job. As mentioned above, users indicate that web systems support to a greater extent their identification and their selection tasks and to a lesser extent their execution and their post-sale tasks. They also report on average a greater reduction in search costs (mean 5.29) than in processing costs (mean 3.33). Results from Spearman correlations presented in Table 3 show a meaningful and significant relationship between web systems identification functionality and the reduction in search costs (Rho = 0.494 and sig. = 0.000), hence supporting H11. Findings show however no significant correlations between the other functionalities and the corresponding performance measurements.

5 Discussion and implications

The objective of this study was to assess desirable FR and NFR of web systems in the context of e-procurement and examine the relationship between such assessments and the performance on the job reported by users. In doing so, this study provides requirements engineers with informed feedback on functionalities and attributes of web systems currently used for procurement purposes. Requirements engineers can use these findings to determine which requirements are more important to focus on when developing an e-procurement system or simply when evaluating possible commercial off-the-shelf products. Findings show that on average, web systems rate high on usability and on the identification function. Furthermore, users report important reduction in search costs. Results also indicate that usability is a critical factor as it seems to have a positive effect on individuals’ decision to use such web systems. This is consistent with IS acceptance research which suggests a positive relationship between ease of use and adoption behavior [19]. Also, the fact that web systems support the identification of potential suppliers and that their usage is associated with buyers’ reduction in search costs is consistent with research on the task-technology fit which posits that a better fit between tasks requirements and systems’ FR can improve users’ performance [32].

Findings show as well that web systems rate average on the selection function, which indicates that such systems somewhat support the selection process but not to the extent of allowing a reduction in search costs. Moreover, the analysis of confounding factors indicates that computer savvy users tend to rate higher the selection function of web systems. Possible explanations for the overall lower assessment of the selection function include the facts that buyers may prefer more personal contacts at this phase of the procurement process [23], that web systems do not necessarily (or easily) provide pricing, product and other relevant information needed to select a supplier [1]. Also, the selection of business partners can require a formal process (such as writing, sending and evaluating Requests for Proposal or Quotation) that is not easily handled through web systems [9]. Results finally show that web systems rate quite low on reliability, security and on the execution and the post-sale functions, and that users report a limited reduction in processing costs. After a follow-up with some of the surveyed buyers we understood that possible explanations include the fact that not all vendors deploy fully fledged transactional web systems [1] and that buyers may prefer to use more traditional means (such as paper-based orders, fax, phone, etc.) to execute transactions and to follow up on them. As a result, buyers may not benefit from web system-based reductions in processing costs. Previous work has shown that security is an issue that can affect the execution of electronic transactions over the Internet [7, 14, 28]. This study shows that security is of concern to buyers and that web systems provide limited support for the execution of transactions but these two findings are not linearly related. Results from additional questions on the survey form reveal that 57% of the respondents use web systems deployed by well-known suppliers only to execute business transactions, which indicates that security is an important requirement for e-procurement systems.

Likewise, reliability is a concern for users of web systems but our data analysis shows no support for linear relationships between the reliability and the functionalities of web systems. These findings do not mean however that reliability is not an important requirement for web systems. A possible explanation for this equivocal findings is that this reliability issue may be more due to the unpredictable traffic on the Internet causing delays in transmission or loss of data packets [16, 17] than to the faulty functioning of web systems that, most probably, are professionally developed and maintained. Future research should further investigate the origin of web systems’ lack of reliability.

Requirements engineers should note that the findings related to the security and the reliability of web systems for e-procurement do not mean that these two NFR are not important because, as shown in previous work in this field, the level of adherence to these two attributes does affect the quality of such web systems.

Finally, findings show no interaction between FR and NFR. This is somewhat surprising since, by definition, NFR constrain or limit the functionality defined in the FR. A possible explanation for this result is that the interaction between FR and NFR exists but not in a linear fashion and therefore cannot be captured through correlation analyses.

6 Conclusions, limitations, and future work

This research assesses typical FR and NFR of existing web systems used for e-procurement and how they can affect the performance of buyers. As such, it gives insights about desirable features for e-procurement systems thus providing the requirements engineering community with an informed feedback on the most important FR and NFR for e-procurement systems and how they can affect users’ performance on the job. However, similar studies in different settings are necessary to verify the generalizability of our results. Also, because our findings are based on correlations, careful experimental research can be greatly beneficial to verify the internal validity of this study. This research has some limitations. First, it considers all web systems accessed for procurement purposes to be somewhat homogeneous: all of them typically include electronic catalogs, shopping carts, payment systems and order tracking systems and all of them have the same level of adherence to each NFR identified. In reality, the functionalities and the quality of such systems can differ from one vendor to another. Second, this research assumes that procurement tasks to be very similar for all corporate buyers. Although the procurement process typically involves the identification, the selection, the execution and the post-sale activities, professional buyers may specialize in selected tasks of that process, which may affect their assessment of web systems’ FR and NFR. Although we do not consider that these two factors can significantly affect our findings, further research should take into account these two limitations. It should also take into account additional factors that explain the decision to use readily available web systems and factors that contribute to the deployment of successful web systems. Relevant additional factors include integration requirements, efficiency requirements and the identification of application classes as they can affect the desirable requirements of successful web systems.