Selenium is a powerful tool for test automation that is rapidly gaining popularity and large scale adoption. When it comes to functional test automation of applications in a web browser, few tools are as cost effective or light weight. Since Selenium-based automation can be developed over a wide variety of technology stacks, it is also extremely configurable and capable of integration with many other tools. It offers the freedom to deploy test automation in a way that makes sense for your organization without binding the QA process to particular implementation models.
While this sounds great on paper, it can leave many organizations at a loss for how to properly implement certain strategies, especially distributed testing. The general objective of distributed testing is to run multiple tests in parallel in multiple environments. The motivation for such an activity can range from reducing test execution time to achieving cross-platform coverage.
Distributed testing with Selenium is implemented using Selenium Grid, a specific mode of Selenium server in which it acts as a hub for distributing Selenium tests to remote nodes. The details for establishing a hub and configuring nodes is well documented, but actually getting tests to execute with Selenium grid requires additional considerations. Here are some of the more important ones.
Infrastructure
There are more ways than one to go about acquiring the environments for distributed testing with Selenium. Many organizations would immediately start thinking about creating a lab managed on premises or in the cloud composed of virtual machines, device emulators and mobile devices. While this approach is one that works well in many places, for other organizations it may be cost prohibitive, slow to spin up, or difficult to maintain. Luckily, there are services available which tackle this problem head on. One such solution can be found in the form of SauceLabs, a cloud based service which provides environments for testing with Selenium and Appium, as well as a mode for manual test execution. With SauceLabs, you get all the combinations of environments, versions, and devices you need at reasonable price points. For those that have specific requirements requiring on premises testing of certain applications, an additional feature allows you to securely connect your Sauce account to internal servers.
Appropriate Coverage
One of the biggest advantages of a distributed testing solution is the ability to achieve greater test coverage in a manageable amount of time by utilizing resources intelligently. But should I test my new mobile app on 30 different devices? Do I really need to test 20 different versions of 4 browsers on 3 operating systems? Before diving into a distributed testing deployment, do the research and make a list of what coverage is both desirable and reasonable to accomplish. While this is a topic for an entirely different discussion, I would argue that it is an activity which is a strict prerequisite to any distributed testing deployment.
Parallel Execution
When asked about the challenges in implementing Selenium Grid, most that have only read and not implemented respond with something along the lines of “Oh it’s easy! Once you have the hub and nodes set up you just change one line of code!” While in some rare cases this may be true, this ignores the challenge of parallel execution, by which I mean the ability to have your tests executed at the same time without having to individually start them all as separate processes. If you’re working with a custom framework, developing a feature for parallel execution can be challenging, but it’s quite achievable. If you’re designing a framework from the ground up, be sure to include this in the initial feature list, as it’s easier to implement earlier in the process. And if you’re using or selecting a tool from a vendor, investigate to see if this feature is supported. If not, you’ll want to start considering solutions to the problem as early as possible in the process.
