The client is a corporate compliance and risk management consulting company. They support banking and tax audit efforts regarding companies worldwide, that use workflow management web forms which must be properly tracked for record keeping and auditing purposes through time stamps. The synchronization between users requires signoff dates to match. The workflow aspect of their software would break if users could not signoff the forms with proper date/time information. Keeping accurate records of timestamps is crucial too.
The client provides their consultants with pre-packaged and custom-tailored software packages. The developers had made their own GRC (Governance, Risk, Compliance) solution. The software is mostly web-based but Excel Add-Ins are also developed. The web-based software architecture follows a 3-tiered .NET pattern, with SQL on the back-end. The out-of-the-box components behave as workflow management software with reporting capabilities, dashboards, bulk editing of records, and customization of web-forms.
Prior to Qualitest’s engagement, many issues were being reported in production. Testing was being performed by financial, corporate compliance (non-software) consultants with little testing knowledge. This also took time away from their core business interactions with clients (big banks, media corporations, various large institutions). The development team (5 developers) loved programming but had little interest in testing.
Much of the testing required proper usage of multiple time zone APIs for logged events, records, batch record editing and workflow signoffs, requiring testing of all global time zones. This kind of localization testing includes considering different times of the year, leap years, holidays and daylight savings times (which may vary between countries even in the same time zone). Workflow management web forms were being used worldwide and required proper tracking of their timestamps for record keeping and auditing purposes.
We were largely on our own to self-manage and determine what was needed. The developers and the compliance consultants, while very good at what they do, lacked interest and background in testing. Much of the need involved the writing, design and execution of functional tests including use of a bulk Excel editing tool. But first, we had to learn how things work (which we documented, noting the lack), and formalize a standardized test suite based on IEEE test design templates (so that we’re not reinventing the wheel every time). Once we greatly expanded the scope of coverage, more bugs started pouring in. After these got fixed, the bug count fell back down to a more manageable level.
Due to the minimal test documentation at project launch, face to face requirements gathering was necessary. Our method of system breakdown, provided a firm foundation that had been lacking, which the client loved. System-wide test coverage was documented and shared with the development teams. The coverage was effective enough that it was used by developers to coordinate their development tasks as well for two releases.
Testing processes improved over time (improvements were proposed and implemented incrementally). One of the largest contributions was the increased test coverage, leading to more exposure into the client’s software and the uncovering of defects. Testing coverage continuously grew as a result of confidence in testing methods. The test approach was well documented and issues found during testing were properly submitted. Qualitest was introduced to more and more features to test over time, even the most difficult ones, such as the GRC solution and testing an in-house “custom fields” language with the ANTLR grammar as a test reference. Testing was started earlier and earlier in the development cycle. Eventually tests were being written before any development was available for testing.
Let’s look at some of the test variables used for time zone testing:
“When you design better tests, learn the software better and increase test coverage, you find a lot of issues that you wouldn’t even think are there. And if you don’t test for those, the users will find these types of things. We’re providing value.”
Lewis Sall, Project Lead, Qualitest Group
