Client Overview

Our Client is a large credit union headquartered in Pennsylvania.  Since its founding in 1934, the group has grown to over 450,000 members and roughly $7 billion in assets, placing it among the top one percent of all US credit unions. Their mission is to provide financial opportunities throughout members’ lives, which means continually updating their line of products to meet members’ evolving needs.

It’s Complicated: Third Parties, Many Moving Parts

To ensure members were getting the most out of their digital banking experience, our Client had converted from a legacy system developed inhouse to a new online and mobile banking platform hosted by Kony. The conversion was a major undertaking that involved many moving pieces and different components.

One crucial aspect of the project was performance and stress testing of the newly implemented Kony platform, which had been customized by a 3rd-party developer. Performance testing would include verification that the Kony cloud environment could handle average and peak member usage of the mobile- and online-based digital banking solutions simultaneously.

“Qualitest planned and executed comprehensive performance and stress testing of the online and mobile apps.”

Stressed for Success 

Qualitest planned and executed comprehensive performance and stress testing of the online and mobile apps to ensure that the new digital banking solution was working properly and had not suffered any degradation to end user response times. We tested both the mobile and online systems individually and then simultaneously, under average expected conditions and peak level usage.

  • For the mobile solution, we converted Postman collections that had been created by the client to JMeter test scripts.
  • For the online solution, we developed Workflow documentation based on the Client’s information and used it as the starting point to create JMeter test scripts.
  • For the combined mobile and online banking applications, we developed performance testing scripts that simulated the member activities with the highest known frequency.
  • Our plan called for leads to be generated from three different geographic regions, simulating how members use the production system.

The individual mobile and online tests simulated a load of 2,000 users for the average expected usage and 4,000 for peak level usage conditions. The combined mobile and online tests simulated a load of 4,000 users for the average expected usage and 8,000 for the peak level usage the system might need to withstand.

“The failure rate was under 1%, which is better than the industry standard for acceptability.”

Key Benefits

Qualitest’s comprehensive knowledge of digital engineering tools and platforms and our expertise with performance and stress testing saved the Client costs and time.

We provided the Client with special value by uncovering code and architecture errors potentially damaging to the end user experience so they could be resolved.

  • During the high volumes of simulated usage, we verified that the system could handle the peak load with no outage. The failure rate was under 1%, which is better than the industry standard for acceptability.
  • We used the hosted SAS platform BlazeMeter to run the performance tests, which meant the Client did not have to provide servers or hardware to execute the performance tests, minimizing total project time.
  • Through our rigorous testing practices and digital engineering expertise, we identified and alerted the Client to potentially damaging issues with the 3rd-party developer code and the architecture–issues that could have led to dramatic degradation of performance and the user experience.
Download the PDF