The client is a telecommunications and internet service provider, providing 3G and 4G services through its own network infrastructure.
To help optimize their website, the client sought tools to enhance insight analysis into consumer behaviors, likes and dislikes of the site’s functionality and usability. The customer experience analytics tools (crucial for building good retail CRM) chosen were ForeSee and Decibel, which track, record and measure user behavior and satisfaction. Both tools had been previously utilized on the site, collecting session data and customer input. This project involved deploying a new ‘Replay’ function that both tools had but that the client had never used, training the client on proper usage, and ensuring expected behavior and privacy.
The ‘Replay’ function allows analysts to watch customer’s online sessions after the fact. In addition to ensuring that the replay function was collecting and showing data correctly across browser and mobile device fragmentation, the client’s top priority was ensuring that this function complied with legal privacy requirements specifying that all customer personal identifiable information (PII) must be obscured and hidden during replay, including PII entered in forms by online users and PII saved previously on the site in personal accounts.
We integrated a higher level of quality QA into the website development scrum teams. Qualitest consultants filled the gap with experienced, fast thinking, fast learning and fast adapting testers in those teams. A Qualitest Test Consultant was integrated into the scrum team in charge of the ongoing site wide optimization efforts of the client’s online channel. The 3-month effort was driven by the team’s DDM (digital development manager) and myself as the team’s QA. Instructions came straight from the stake holders while the two of us worked on it with the insight team. We concentrated on:
1. Studying the ForeSee and Decibel tools:
2. Reviewing all PII on the client site:
3. Working with 3rd party developers to fix any defects found (PII’s that weren’t obscured correctly)
4. Documenting procedures to follow by all development teams when introducing future new pages or functions that include PII
5. Updating the site’s cookie policy to include information about the new functions
Separate test script templates were constructed for Decibel and ForeSee, based on the specific build and functionality of each one. Two sets of scripts were compiled and run using each template on every single page and function that showed PII on the client website.
All scripts were run on the top browser and top mobile phone used by client customers at the time. In addition, only Decibel scripts were run on a tablet since ForeSee was not configured to record tablet sessions. After testing these main browsers, a smaller sample of scripts was run across additional browsers and mobile devices according to a test matrix agreed upon with the development manager and the insight team.
Defects and fixes were managed in two avenues. While changes to the recoding procedure in Decibel could be managed by a client analyst, ForeSee changes needed to be sent to the 3rd party developers in another country, making the process take longer and become harder to complete.
All test templates were later entered to the test scripts repository for future use by the QA team.
Key benefits resulting from the testing solution for this project included:
