The Client needed help with collection of ground truth data prior to launch of consumer AR/VR headsets.
They required real-world, algorithm-ready quality data to be drawn from a wide range of diverse real-world home environments.
Working from detailed Client requirements, Qualitest set up staging sites across the globe in locations where Client’s products would launch.
We collected ground truth data using motion capture systems from 1,250 rooms in 335 homes and used RAID drives to deliver captured data.
The data collection to date has contributed to the launch of multiple VR headsets and dozens of software products.
The data we continue to collect helps shape product development and enables our Client to generate performance objectives for their product teams.
Our Client, a prominent social network leader, is also an innovator of AR/VR/MR (Augmented Reality/Virtual Reality/Mixed Reality) hardware and software, including headsets and smart glasses. Of paramount importance to the Client is ensuring their products and services are inclusive to all people and all environments, delivering a top performance anywhere they may be used.
For AI/ML-enabled products such as headsets, performance depends on the quality of the data used to build the algorithms. The Client needed algorithm-ready data collected from a diverse range of home environments to evaluate new AR/VR products’ performance for launch readiness and for corner-case training (training data for hard-to-track environments). They wanted collection sites to be established in places where they were launching the products, which included a variety of locations in North America, Europe and Asia Pacific.
“The Client needed algorithm-ready data collected from a diverse range of home environments.”
The Client hired Qualitest to help collect the ground truth data they needed. With a global network extending to four continents, we can scale our delivery capabilities to meet demanding data collection needs anywhere in the world. We were well-positioned to take this project head-on, knowing we would help evolve our Client’s groundbreaking products.
We worked closely with the Client’s team to develop data requirements, to refine procedures for collecting high-quality data using motion capture systems, to build tooling, to manage logistics of data and equipment, to secure IP and to handle privacy and regulatory issues, such as trade and customs.
We started with a proof-of-concept for data collection using homes in the San Francisco Bay area. After fleshing out the data requirements, we scaled up operations and proved the value of the data collected to Client stakeholders. Three traveling teams were dispatched to countries where the Client would launch, where they staged previously identified Airbnb locations and implemented data collection.
Our data collection process focused on using the Client’s devices the way end-users would be expected to use them: for example, playing games or watching media in different parts of the home. The Client wanted assurance that the headsets would deliver a smooth user experience regardless of type of home and the rooms and places they would be used—bedrooms, living room, kitchen, garage, hallways, in soft and bright lighting, at different times of day and even under different weather conditions.
We collected ground truth data using motion capture systems that could easily be set up in homes, later shuttling captured data from the locations to our Client’s campus for processing. To protect against data loss, we used large RAID drives. For secure transport and safeguarding of IP and equipment, we coordinated with our Client’s internal security teams to book local security.
“The project’s global scale grew to over 15 countries in 100 cities, utilizing 1250 rooms in 336 homes over the course of about a year.”
In our efforts to achieve the Client’s data requirements, the project’s global scale grew to over 15 countries in 100 cities, utilizing 1250 rooms in 336 homes over the course of about a year.
This project should prove valuable to the Client’s ability to innovate and push AR/VR boundaries further into the real world.
Download the PDF
