Keywords

1 Introduction

Web service is a software system that is unified by URI (unified resource identification). As a special kind of service, Web Service not only realizes the characteristics of remote access through network, but also inherits the characteristics of autonomy, openness and self-description of general services. Different organizations have different understandings and definitions of Web service. However, there was no fixed definition of Web service so far. There are descriptions and understandings of Web service by several large enterprises and institutions in the following.

The definition of the W3C organization: Web Service was a software application that used URI to unify the identification, and used XML to defined, described interfaces and binding. Web service is found and used by other users by network, and finished interacts through XML messages at last. The definition of SUN Company: Web service should include the following five characteristics. First, it provides an external interface, which exchange data in XML format. Second, the out Web service can be access by Web. Third, the services among the systems support relationship are loosely coupling. Fourth, if Web services completed registered and the services would be located. Fifth, it supports the specification of Web service protocol and implemented message communication used XML. The definition of IBM Company: Web service is the smallest application module that has the characteristics of self-description, self-contained and support matched with other Web service. Web service can implement description, search, publish and be called anywhere in the network environment. Whether service users asked for simple application requests or complex composite business processes, it can accomplish tasks by calling the Web Service. When a Web service deployed successfully, any other application can be discover and invoke deployed service through a UDDI service registry to accomplish the task. The definition of HP Company: Web service is a service that solves user’s problems through Internet, and transacted and processed tasks on behalf of applications and users.

Web services are platform independent, low coupled, self-contained and programmable. Web applications that describe, publish, discover, coordinate, and configure them using an open XML (a subset of the standard General Markup Language) for developed distributed interoperable application. The rise of Web service has been accompanied by the introduction of service-oriented software architecture (SOA), which provided a new paradigm of standards-based, loosely coupled, cross-platform distributed computing on the Internet. Individual Web service provides specific capabilities, and in order to meet the needs of users, more and more real projects need to integrate and combine multiple Web services to provide comprehensive and complex value-added services composite Web services. Members of the service can communicate with each other and handle user operations and requests in a logical manner. With the further development of Internet application, Web service composition is bound to get concerned and applied widely. In order to ensure the quality of composite Web services, model checking is used to verify the conformance and the related properties of the model of composited Web services with its implementation [1, 2]. Comprehensive and adequate testing for the implementation of Web services composition is required. However, because of the dynamic and distributed characteristics of the Web service composition itself, many traditional software-testing technologies have lost their original effectiveness to the Web service composition. Therefore, we need to study the new testing techniques and methods for the Web service composition, to provide a powerful support for the performance, function and reliability of composite service.

At present, Web service portfolio testing has been studied and some research results have been obtained. The purpose of this paper is to systematically summarize and analyze the existing methods and techniques for the testing of Web services composition. Although some researchers have made a definite analysis and discussion about this problem, we think that this problem is still needed further investigating. Web services testing analyzed and summarized by Hong, Bozkurt and Ebrahim [2,3,4,5], however, the research status of Web services composition testing had not been emphasized. Web services composition testing has discussed by categorized the test methods completed by Rusli et al. [6] in long before. After that, there were other study results have been published. Therefore, it is necessary to make a new and comprehensive survey summary.

The structure of this paper is as follows. The first section, introduces the definition of web services in detail; The second section, analyzes and discusses Web service architecture; The third section summarizes and describes several web service composition testing methods; The fourth section, introduces and summarizes testing techniques about Web service composition; The last section summarizes and prospects about the research on web service composition testing.

2 Analyzed and Studied the Web Service Architecture

In general, Web Service used to invoke remotely. The traditional software testing technology could not be simply apply to measure tested work of Web service application system; In view of this, Zhang and Zhang [7] proposed a criteria contain J attribute indicators such as accuracy, fault tolerance and testability, which can be used to evaluate the reliability of Web services application systems. According to this reliability criterion, we can effectively eliminate inappropriate Web services.

Testing framework [8] based on JUnit used in unit testing of application system widely. Therefore, Zhang et al. [9] had attempted to propose a suitable unit test framework for BPEL, which included Composition Model, test architecture, life cycle management and so on. In this framework, the test function divided into several test process (TP) and control TP process (CP), with the life cycle of TP be controlled by TP provided beginTest and endTest.

Dong et al. [10] put forward an automated testing framework based on WSDL. Given that the message contained in WSDL didn’t fully assist the test work, so the WSDL extension specification was referenced in this framework, it included four other extensions such as into/output (I/O) dependency. According to these extensibility, this test framework could deduce test data and operation flow, formed a complete test case at last.

Akehurst et al. [11] defined constraint for each object in BPEL based on the Object Constraint Language (OCL), and implement the Java classes of verification based on these restrictions. At the same time, Akehurst established a Meta-model based on specification of BPEL, and the associations in objects defined by BPEL were represented as UML diagrams.

Looker et al. [12] put forwarded a test method based on Fault-Injection. Because of the SOAP packet format used by Web services to exchange messages was based on XML, Looker was able to add an injector server between the service provider and the service requester by modified the container of the Web service, to monitor all messages exchange between the service provider and the service requester, and according to the setting of test cases insert a message that might cause an error into a normal message, and observed whether the Web service under tested could correctly correspond to messages with exceptions, such as error content, missing content, and so on.

Offutt et al. [13] proposed to use data disruption to generated different SOAP parameter data, and analyzed the messages in response to verified correctness of the peer-to-peer Web service. Offutt et al. [13] proposed three methods of disruption message: Data Value Perturbation (DVP), RPC Communication Perturbation (RCP), and Data Communication Perturbation (DCP). DVP was mainly based on the parameter message format defined by WSDL, such as string or numeric value, through the method of boundary value analysis to generate different parameter messages. RCP used mutation operators to calculated parameters to generate different new parameter messages. DCP is a Web services testing that used XML messages to send messages. Offutt et al. [11] modified the contents of XML messages in SOAP by some rules, These XML messages are used to test the access to a Web service’s database whether correct or not.

Chen, Li and Zhang [14] proposed a development and test environment that could flexibly define the process – WSCE, which enables the combination of Web services to carry out in a very convenient way. In this architecture, Yu proposed two mechanisms such as virtual partner and inspector service, to help developers verify the correctness of the process or not.

Tsai and Paul et al. [15] put forwarded a test framework of WSTF (Web Services Testing Framework). This framework was based on agent technology and could be applied to SOA architecture system.

3 Web Service Composition Testing Methods

Web services testing methods had many similarities to traditional software testing and there are differences of them. Web service testing required service requesters, service providers, and UDDI accomplished together. The comparative results between Web service testing and traditional software testing were shown in Table 1.

Table 1. Traditional software testing and web service composition testing
Full size table

Compared with the traditional software testing and the characteristics of the Web service composition itself, the tester could not have all the test information, because of the component service is black box test, the tester could not have the source code of the component service, so it was unable to get all the features of a component service and build a rich test model. Therefore, in the test of Web service composition, some scholars have studied how to expand document parsing or build model technology to obtain sufficient test information. Furthermore, the dynamic binding of Web service composition makes it difficult to predict the operating environment and behavior of the combined service, the generation of test prediction was difficult, however, traditional software testing techniques target and software behavior were predictable, static, and non-distributed, so it could not be applied to Web service composition testing. As with traditional software, the compositional Web service also has a software evolution process, but the evolution process was dynamic, the changes didn’t limited to internal structure or variable of the program. Instead, component services are upgraded or replaced, business processes are replaced, and interface information for component services is changed, and these evolutionary processes exist throughout the operation of the system, so additional information is needed to support the regression test. As we could see from the comparison above, the difference between Web service comparison testing and traditional software testing exists in the whole process of testing. According to the testing process, this paper summarized and analyzed the technologies and methods of Web composite service testing in different stages.

4 Web Service Composition Testing Techniques

One of the major features of Web services testing is that most testers do not gain the source code. Therefore, all white-box testing relevant techniques are not used. How to test Web services effectively is becoming a hot issue in Web services research.

4.1 Web Service Composition Testing Based on EH-CPN

The Web composite service testing technology based on EH-CPN mainly have the following steps: First of all, we analyzed the data flow in the extended colored Petri net through OWL-S document transformation, and find the used pairs for all of the variable definitions and the used chains for the definitions of corresponding input and output. Next, the definition that all input and output used chain extended to get an executed test sequence. The above test sequences meet the full definition of use coverage criteria. The test data was generated by the test case generated method that combined the equivalence class partition and condition constraint, then combined test data and test sequences to generated test cases. Input the OWL-S document in the developed test tool of TWCS, and TWCS would generated a colored Petri net that extended levels corresponding to OWL-S, then find out used chains that all the input and output of all variables and extend it to an executable test sequence. Input the number of test cases corresponding to each test sequence, and completed the generated the number of test cases was same. Using the proxy occupancy program in .NET Framework SDK to generated proxy for each Web service; finally, the proxy service be used to completed the call of the corresponding Web service, executed the test case and completed the test of the Web service.

4.2 Web Service Composition Testing Based on Mutation

The idea of mutation testing was to detect the effectiveness of test cases by embedded errors in the program and guided the generation, selection and reduction of test cases, and to achieve the purpose of tested at last. It was a test method based error. The idea of mutation testing was proposed based on white box testing first, and the object of mutation testing was program code. With the development of mutation testing, the idea of mutation testing could be using for Web services testing, and guided test cases generation, selection and reduction of test cases [16].

There were a lot of research in Web service mutation test [17]; this section introduced the mutation-based Web service composition testing, which takes following steps to test the Web composite service workflow: Firstly, parsed OWL-S document and extracted information such as the type, format, and etc. input format accepted by the composite Web service to be tested and workflow. Generated the initial test data set based on the type, format, and other information of the input format, and test data could be generated randomly or by the usual methods of boundary value analysis or equivalence class division. At the same time, the workflow information of Web composite service is analyzed and found nodes that meet the variation conditions in the workflow. According to the corresponding change rules, changed the OWL-S document information to generate new mutant that injected the wrong to the original composite service to be wrong version of service composition.

After completed the above work, entered the same test data should be returned different test results executed the original service composition and the variant Web service composition, because of the workflow of the service composition was changed. If the output was different, the variant was killed and the test data could identify the wrong Web service combination and it was an effective test data. The test data should be retained and added an effective test data set for later test data selection. It indicated that the test data is invalid if the output results were same, the new test data should be redesigned or repeated above tests and expanded the effective test data set [18].

4.3 Web Services Testing Based on Interactive Behavior Specifications

This section introduces test problem of service interaction from the service requester’s point of view. A Web service tested method that leverages interaction behavior specification. The main steps were as follows:

Firstly, the behavior specification of Web service should be described correctly. UML is widely used in system modeling in industry and academia because its simplicity and standardization, using sequence diagrams of UML and the behavior rules of Web services described by OCL. It is provided to the service requestor in the form of an XML file (XML metadata interchange) together with WSDL document, and XML documents could be generated automatically by existed UML modeling tools.

Secondly, used an extended state machine model of ELTS described the behavior rules of the service. Added semantic information based on traditional LTS generated extended ELTS to strengthen the function of LTS described data flows. ELTS was based on the implementation relationship in a certain formal defined, and introduced a new implementation relationship through generated algorithm of traditional LTS. Through this new relationship given corresponding test data generated algorithm and test cases with test coverage. It used to test whether the interaction behavior specification was consistent with service implementation or not [19].

4.4 Testing Techniques Based on Formal Methods

Test case generated based on formal method divided into model detector and formal analysis technology. The test case generation method based on model detector was an input model, which converts the service combination described of BPEL into a certain detector, and used formal method to describe the demand model that the composite service should satisfy, used them as input of the model detector to produces test cases. Most of the detectors used in the study included Nu SMV, SPIN and BLAST [20,21,22,23]. Test case generated method based on formalized analysis technology, which described by BPEL through a formal method or other formalization methods, such as Petri nets, automata or process algebra, then use existed analytical techniques of the formalized method to generated test cases, such as references [24,25,26].

Petri net was a modeling and analysis tool for distributed systems. It was a directed graph composed of repository, change, and directed arc. It was easy to described sequence, concurrency, conflict and synchronization of the processes and components in the system. Compared with other system models, true concurrency was a unique advantage of Petri network. The modeling method based on Petri net could described all kinds of control flow in the combinatorial process, but it could not reflect overall state of composited service directly.

Automaton was a mathematical model with clear semantics. It was suitable for describing discrete input and output systems. The system has a limited state, different states represent different meanings. In actual needs, the system could complete prescribed tasks in different states and transfer to another state. The automaton modeling method could described internal state of Web service composition directly, but it could not described interaction behavior between two services. The ability to described concurrent activities in composite process was limited, and there was a space explosion problem. Process algebra was a formal modeling language based on algebraic methods. A group of operators was used as a process component in grammar. The semantics of the operator was defined by a structured operating semantic method. In this way, a process could be regarded as label transition system (LTS). A significant feature of process algebra was attributed concurrency to non-deterministic, that is, considered the behavior of concurrent processes as all possible interlace behavior of each process. The behavior of concurrent execution was suitable for described concurrent interactive systems. The modeling method of process algebra had strong description ability and rigorous computational reasoning ability, but its expression was more complicated, not intuitive and difficult to understand.

Miao and Chen et al. [27] proposed a testing approach to model-based testing for Web applications, which designed and implemented a web application testing system based on this model. Taking the UML state diagram of Web application as system test model, used UML sequence diagram described test target, and the FSM test model is constructed by transformation and combination, automate generate test case; test model visualization and automation of test execution were come true. It mainly focused on functional testing in article: Model-Based Testing for Web Applications. The performance tests, load testing, usability testing, compatibility testing and security testing not verified.

Qian and Miao et al. [28] proposed a test path generate approach, which illustrated by SWLS (Simple Web Login System) as an example and presented an effective Web testing model for Web software testing. One of the main advantages of this approach is that you did not need to access back-end source code. In order to get PTT from PFD, they proposed a transformation algorithm by this method. They obtained test path from PTT by constructing path expression, and gave a possible way to describe test path in XML. Qian and Miao, which were full link coverage and full-page coverage, also proposed two important concepts. It is possible that a particular link will appear on a page only if provided a specific input in tested. But this web test method is not necessarily adaptable in new case, so it needs to further improve test path generation method, and develop new prototype to re-validate this web test model proposed.

Above several kinds of formal model could described the behavior of the Web service combination well and have relevant technical and tools support, there were some differences only in computational complexity. However, these methods required staff with relevant professional background knowledge and ignored data flow information modeling in combination process. Therefore, the non-formal test case generated method was discussed below.

4.5 Testing Techniques Based on Informal Methods

The informal method [29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51] that it converted control flow, data flow, message flow, behavior, etc. in a composite service described by BPEL and others into a graph model and used search technology and constraint analysis technology generated test cases.

The steps of test case generated method as follows: Firstly, build a model based on some features of Web service composition. Secondly, generated test paths based on model traversal. Thirdly, generated test cases based on constraint condition in above path. The following research work falls into this category.

On studied of test case generated problem in Web service combination, some researchers focus on control flow characteristics of Web service composition. Yuan et al. [29] proposed a BPEL test case generation method based on graph search, which used to deal with concurrent semantics of BPEL. This method described the WS-BPEL program by defined control flow graph (BPEL flow graph, BFG). BFG contained control flow and data stream of BPEL program. Generated test paths by traversed the BFG model, the constraints in test path as the input of constraint parser generated abstract test data and converted it into executable test cases automatically. However, the process of converted BPEL documents to BFG and test paths search process were done manually.

The method proposed by Yan et al. [30] was similar to Yuan’s method [29]. They converted the WS-BPEL program into an extended control flow graph (XCFG) and generated test paths based on XCFG, and then used a constraint parser generated test data from test path. However, it is different from the Yuan et al. method that Yan and others used symbolic execution methods to obtain a series of constraints from the test path by invoked component services, but this method produced abstract test case that it needed to be converted to an executable test case manually.

Mei and others proposed a test method based on XPath Rewrite Graph (XRG) [31, 32] that it combined the control flow graph (CFG) and XRG to solved possible integration problem caused by XPath in BPEL process. With the gradual deepening research, some scholars believed that the model based test cases produce techniques same as the method based on path generated test case. This method represents the test data by generated message parameters, but the generated test cases were high redundancy and low error rate. Hou [33] and Ni et al. [34] applied test technology based on message flow in Object Oriented Program (OOP) to Web service combination test first time. Wu and Huang [35] thought that binding internal state of single service, execution sequence among services and behavior of service closely related in runtime. Therefore, references [36,37,38] proposed an EDSM sequence test model (EFSM-SeTM) for Web service composition. They studied runtime test from the point of workflow view and proposed a scenario-based testing framework for Web service composition.

We summarized and analyzed informal testing method based on model. As shown in Table 2. It is shown that most of above test case generation techniques are semi-automated, even include the technology proposed in running test. Therefore, firstly, How to achieve full automation is a problem that can be further studied. Secondly, no test case generated technology involved Web services and the quality of Web services determined the correctness of entire Web service composition, so it is necessary to test the Web service.

Table 2. Classification of informal methods
Full size table

5 Summary and Outlook

It can be known from above analysis that some problem of Web service combination need to be research, although some results have been achieved in this area, which were mainly reflected in the following aspects.

Formal modeling technology of Web services combination need to be developed and researched deeply, such as research on the correlation, fairness and applicability of formal model, further research on the property analysis technology of Web services formal model. It is necessary to research Web services technologies made formal technology provide services and support better.

The quality of test cases was fundamental condition for effective Web service combination testing, which shown that high error rate, low redundancy and high coverage. How to obtain more constraints that can enrich test information generating test cases is a problem needed to be solved in academia and industry today.

Some researchers have been concerned about the runtime binding problem of Web service composition, Ni [34], Wu [35] and Sun et al. [38] designed test automation prototype tools, but they did not elaborated automation level of test technology, or did they verify the relationship between automation and runtime binding issues.