Introduction

Service Oriented Architecture (SOA) is an IT architectural approach to business applications that links loosely coupled business-centric tasks. These services can be accessed over a diverse number of devices and networks, from the Internet to corporate VPNs. SOA’s strengths are optimal reuse of corporate assets, speed to market, and the ability for businesses to quickly change and adapt to new market conditions.

However, because SOA is dynamic in nature, testing can become a particular challenge. Reuse resulting in speed to market can be highly advantageous, but testing requirements become more complex as a result. Qualitest offers a different approach than other testing organizations, with developed and proven assets that can be put into place immediately and which are extensible as your SOA development expands and changes.

This white paper will explore the components of SOA, then focuses in particular on the services and service bus aspects of the architecture. These two areas are where Qualitest’s approach to SOA testing is differentiated, offering unique advantages to your organization.

Understanding SOA

Below we define and describe each component of the above diagram.

Application Front End

The application front end links people, business processes, and information anywhere, anytime, using any channel, via destination-specific views delivered on the wide spectrum of available devices.

Service

A service combines information and business processes. While each service is a fundamental building block in the SOA, each one is completely autonomous from the others. Each service is typically limited in scope to a particular business function or group of functions.

Often SOA services are delivered via the Web. Web services allow businesses to communicate with each other and with clients via an Internet connection, communicating with one another without having to know how firewall-protected backend IT systems are configured. Web services are always on, promoting availability wherever the user happens to be. They have become a standardized way of integrating Web-based applications using the XML, SOAP, WSDL and UDDI open standards over an Internet protocol backbone, where:

  • XML is used to tag the data
  • SOAP is used to transfer the data
  • WSDL is used to describe the available services
  • UDDI is used to list the available services

Service Repository

An SOA repository is a database which holds the software and metadata that comprise an SOA registry. The registry is an evolving, interactive, controlled-access catalog to help manage SOA projects, allowing organizations to easily discover and communicate with one other via Web services.

As a metadata repository, the service repository helps validate content and support SOA workflow. The repository is where policies, processes, attributes and schemata related to SOA governance are recorded.

Service Bus

The service bus underpins a fully integrated and flexible end-to-end service oriented architectures. This infrastructure, often called an Enterprise Service Bus (ESB), connects services that employ different data, message formats, network protocols and programming languages. Service requesters and service providers communicate through service bus middleware.

Contract

Service contracts ensure reuse through the consistency of their design principles. Within service-oriented solutions, a service contract becomes the only method through which services interact with each other or with other potential services.

Generally, service contracts are standardized in the following three areas:

  • Functional expression
  • Data model
  • Service policies

Implementation

Interface

Because the SOA is a collection of applications accessed through a wide variety of devices and interfaces, a consistent user interface can be challenging. This set of services is most frequently presented to users through multiple views. Productivity is improved substantially when these views are implemented reliably and when basic UI principles taken into account over the entire SOA.

Business Logic

The underlying purpose of SOA is to transform business models into reusable and flexible services, thereby increasing productivity. Service-oriented applications convert business behavior into a functional service, all the while reusing standardized business logic within these services.

Data

Within an SOA, services use defined protocols that describe how services pass and parse messages using description metadata, which must be sufficiently detailed to describe not only service characteristics, but also the data they present to the end user. XML is often used in SOA to structure data that developers enclose in description containers.

Challenges of SOA Testing

Because the SOA is never static and is always online, changes can be introduced into the application at any time. Testing must ensure quality, compensating for the dynamic nature of SOA and avoiding the likelihood of unanticipated failure.

As with all testing, SOA testing should begin in the design stage with an assessment of the overall architecture. Regression testing should be an integral part of any SOA test suite.

Unit Testing

Unit testing should take place before integrating the service into the SOA. Some challenges include:

  • Services often don’t have UI. The QA team must create appropriate test harnesses to provide test data that can be analyzed.
  • Testing each service individually is the best way to fully test it, particularly because testing services together can cause daisy-chained issues

Integration Testing

Problems that might be encountered during integration testing of object-oriented systems may be amplified with SOA. These architectures include several considerations and challenges:

  • Dependency on third party services that can change without notice, creating testing delays
  • Late binding where it might not be possible to know which services is invoked when a site is called
  • Services that may be absent when testing or under load or maintenance
  • A variety of programming languages that may be used to create the loosely coupled services
  • Consistent compatible versioning of service across the SOA

Functional System Testing

The functional testing of Web services can be problematic. Functional testing’s underlying purpose is to provide requests to the service and analyze the responses’ accuracy. But with SOA’s loose federation of multiple (and growing) services, it is impossible to test every input and output. In addition to testing the business logic, the asynchronous nature of the services can add additional testing challenges, such as testing for unintended race conditions.

In addition, SOA application growth can become more than any one team can understand or test. Strategies that can arm testers, providing them with the ability to test the system without understanding every application in it, becomes critical to the testing effort, while at the same time often becomes the root cause of unexpected behaviors.

Non-Functional System Testing

In addition to functional testing, the following non-functional characteristics need to be tested:

  • Reliability
  • Efficiency
  • Usability
  • Maintainability
  • Portability
  • Performance

Because an SOA is most often deployed on the Internet, it can be impossible to predict the workload. Requests per second can suddenly increase beyond predicted levels. Therefore, any testing plan needs to incorporate tests that saturate the system to see what will happen in the case of system overload.

Security is another challenge for SOA, with authentication and authorization clear vulnerabilities. In this wide-open environment, it is difficult and often impossible to track unauthorized use of Web services.

Qualitest SOA Test Approaches

Testing the Service Bus

Testing the service bus means testing the connections between the systems. This can be done using:

  • FTP files for processing
  • Web service testing using SOAP or REST
  • JMS messaging

Generally a request is received and has to be translated into a request sent to another system. Testing must be done to check that:

  • The input is validated.
  • The correct initial response is returned.
  • The data is translated and supplemented as needed.
  • That correct request is built and sent to the next system in the workflow process.
  • That system in turn sends an initial acknowledgement response before typically processing and translating the data into a new request that is then returned to the bus.

Since the bus acts as a gateway, allowing many disparate systems to communicate with one another, data may have to pass through the bus several times to complete one business workflow.

Recently, Qualitest tested a client system that registered a new customer. The request passed through the bus six times in a variety of formats, picking up additional data from each of the different systems as it continued through the business process flow – and yet, the request was still completed in less than ten seconds.

The number of scenarios is virtually limitless when testing similarly complex workflows. This means that the testing effort can easily become equal to if not greater than the initial development effort.

Testing the Web Service and APIs

SOAP Web services are usually defined by Web Service Definition Language (WSDL), while REST services utilize a different mechanism. APIs are often given to the test team as a spec documentation.

Testing of the Web Service and API requires the following steps:

  • Testing the format of the request. XML is case- and often order-sensitive, which means that a field defined as UserID is not the same as UserID. In this way, XML tags are similar to variables that are assigned values.
  • Requests can include two fields with the same name, so they must be distinguished from one another. SOAP requests will accommodate this by utilizing name spaces.
  • Testing negative data is as important as testing valid data.
  • It is critical to test Web services individually. It’s insufficient to simply verify the response returned from the request. Additional background processes can often save data into a database, performing additional processing before the response comes back. This must be incorporated into the test case.

Specific Qualitest Reusable Assets

Qualitest has created a suite of reusable assets to expedite efficient testing of the Service Bus, Web services and the SOA API. These include:

  • Test case templates
  • Scripts that parse WSDLs, creating both the script to test them, as well as the data sheet that drives the test. This sheet is filled with default data, allowing us to comprehensively test a variety of WDLS. For instance, if a field is defined as a string 30 characters long, Qualitest’s script generates a test for 29, 30, and 31 characters.
  • Scripts that compare the Web service definitions between releases, a critical aspect of ever-changing SOAs. Since these change so frequently, particularly in the initial stages of an SOA, a script that auto-generates revised scripts is a great deal quicker, easier to manage, and less error-prone than manually editing changes every time a change is introduced.
  • Data mapping diagrams for ESB testing. Qualitest maps and follows each piece of data as it makes a path through the system, often changing variable names (and possibly formatting) as it moves from process to process. This is another critical test component and is the only way to fully test the system.
  • Scripts to compare the ESB’s configuration setting between every release. As the number of services increase, so do changes on configuration settings. An ESB with a 100 or more services, for instance, are likely to have multiple config settings changes with every release – and because these settings don’t require ‘code changes,’ they are often made without the test team’s knowledge.
  • Scripts that compare data sizes and data types for all of the connected services.

Conclusion

SOAs provide business organizations several key advantages. These include optimal reuse of corporate assets, speed to market, and the ability for businesses to quickly change and adapt to new market conditions. These loosely coupled services do not require intimate knowledge of all of the backend systems in the architecture, yet are able to communicate with each other, sharing business data and logic to a wide-spread corporation and its clients.

But the dynamic nature of SOAs creates specific testing challenges. In fact, the testing effort can easily become equal to if not greater than the initial development effort.

Qualitest’s unique suite of SOA test assets provides their clients with peace of mind. These assets can be deployed quickly, easily identifying bugs that might go undetected without them. Qualitest’s understanding of the challenges and its unique approach in solving them make them an industry leader in the testing of SOA