A multinational technology corporation that produces computer software, consumer electronics, personal computers, and related services needed to improve its user identification algorithms within very tight timeframes. The firm contracted with Qualitest to help solve the problem.
The specific scope of the project was to capture eye-tracking data of 600 participants from varying demographics on client-provided devices over eight weeks. Recruiting participants was a significant challenge of this project. Historically, high-velocity projects with short duration captures are more successful in high foot-traffic locations, such as malls and events where participants engage more organically. Due to the COVID-19 pandemic, the client had struggled to collect real human data at scale. Yet, the client needed to obtain a variety of datasets to modify its algorithms to work as intended. The client needed us to complete the capture in full without sacrificing quality or the health and safety of our staff or participants. Another challenge arose during the initial design of the project. The devices sent for the collection were delayed in delivery. When they arrived, they were many generations older than the client had expected. This required additional testing and modeling to ensure reliable and accurate data collection.
Given the short timeframe of this project, the execution team worked with the older devices to attempt to produce useable data. This included testing a variety of distance, lighting, and capture angle scenarios to ensure high-quality data capture. Throughout the pilot phase, the execution team sent data results to the client for continuous feedback. After a few days of testing, the client approved the scenarios, and the capture process began. At the same time, we developed a solution to address the low volume of organic traffic available during the study period. We recreated “the mall model” at our Q TestLab in Kirkland, WA. The result was a hybrid between walk-in and recruited participants. Due to our marketing tactics – such as our referral program – and aggressive outreach to our participant database, we successfully recruited the right mix of participants.
Initially, the client had requested 600 participant datasets in eight weeks. Delays in device procurement, struggles with the device cameras, the necessary calibration to validate the data, and the tight deadlines left only four weeks to capture data — half the time originally planned. We strategically re-organized our team to address the urgency of recruiting participants. Resources were moved from the capture side to participant management to accelerate our efforts and maximize the velocity of data capture. To improve the volume and value of the data collected, we deployed multiple capture stations. This approach provided additional variations in the data and countered the lower data quality from previous-generation devices. The participant recruiting team also took on execution tasks to keep the participant data streamlined and in order.
The client had been struggling to perform the data collections they needed to train their algorithms during the COVID-19 pandemic. This challenge impacted product development and put their deadlines at risk. We delivered the full number of captures by the intended due date and met the client’s quality standards in half of the expected time. Our ability to pivot quickly and develop new models for data collection, along with the flexibility of our staff to adjust roles, empowered us to meet the clients’ tight deadlines and gather the data needed. Even faced with technology challenges (old devices that required calibration and additional testing) and human challenges (recruiting and controlling the environment), we delivered new data that will continue to improve the algorithms and move the client’s products forward. Qualitest’s adaptability, combined with our goal of delivering high-quality data at velocity, shows that we can rise to meet any challenge.
