Client Overview
Our Client was founded in the 1970s and today it stands as a global technology giant renowned for its significant contributions to the computing industry. They play a pivotal role in shaping the digital landscape and their diverse product portfolio encompasses a wide array of software, hardware, and services.
Recently they have ventured into artificial intelligence, mixed reality, and other cutting-edge technologies. Beyond their technological prowess, our Client is recognized for philanthropic initiatives that emphasize education and accessibility. With a commitment to empowering individuals and organizations, they continue to be a driving force in the evolution of technology worldwide.
Overcoming internal constraints to deliver success with image labelling
Our Client’s challenges started even before their project kick-off date. Due to internal constraints the project start date was pushed out by 2 weeks, but the end date was not changed. The original delivery timelines were already very aggressive, as the expectation was to process thousands of mages within a very short time span.
The accuracy of the labelling was set at pixel level, and this slowed the process down considerably since the masking around image borders had to be done pixel by pixel to mitigate rejections. In addition, more than 95% of the images in the dataset were complex with multiple subjects, flowing hair, noisy backgrounds and extra objects like drum sets & instruments that also needed to be masked.
In the validation phase of the project the requirements were changed constantly, which resulted in multiple rounds of rework on the sample set and delaying the phase for 3 weeks.
Our Client approached us for image labelling services. The requirement was to create new datasets of segmentation/contour masks of facial images for training and validation of custom machine learning models.
Based on our Client’s requirements, the engagement was structured into 2 stages:
- Stage 1 – a onetime effort to process a large dataset of images, within short timeframe to meet the baseline dataset objectives.
- Stage 2 – ongoing support for inflight dataset requirements throughout the program lifecycle.
The program needed the team to be scaled to maximum capacity within 2 weeks, which included a training and infrastructure setup for a very large team. The project ramp up was seamless with a cohesive and well-functioning team steadily reducing the backlog which put the project on the right footing.
Assessing workflows and validating projects for greater success and ensuring quicker timeframes
The project plan implemented comprised of 3 phases:
1. The validation phase – this phase included guidelines, workflows and a sample dataset.
2. The scale phase – this phase included the team ramp up, benchmarks and improvements.
3. The steady state phase – this phase included velocity and quality.
The approach used was collaborative and seamless to produce optimal output at high levels of quality. During the validation phase systems, workflows, and project guidelines for annotation were validated and consisted of a small dataset of images to be annotated and delivered for evaluation and approval by client stakeholders.
Our Client provided approval for annotated validation dataset before the start of scaled annotation and during scaled data annotation, we continued to benchmark both data quality and velocity, with continuous improvements.
In addition, there were two clear strategies that were deployed:
Delivery Strategy
The data was uploaded to a designated cloud storage repository. Our data annotators then processed the images based on strict guidelines, and our onsite Quality Control team audited the Qualitest QC team audited images for acceptance criteria after annotation. Approved images were provided to our Client on a regular basis, and a factory model approach increased the project’s velocity. Our Client received daily progress reports with quality feedback, velocity, threats and blockers.
Quality Control Strategy
The internal offshore quality controllers were responsible for the quality of the data annotated for our Client and conducted internal quality checks along with feedback on the entire dataset. Rework and stringent controls were provided to receive zero rejections, and optimization based on Client recommendations, external audits, approvals and recommendations was conducted.
Establishing a clear onsite/nearshore AI data project management office to maximize success
In order to maximize success for our Client , we established an onsite/nearshore ground truth project management office created a seamless interface with the client project teams. This helped to reduce the high risk of adverse impact on the project due to ineffective communication between the client and offshore team due to time zone differences.
The onsite project lead liaised with our Client as a single point of contact for the project, participating in planning and status discussions to get a health check of how it was progressing. In addition, alignment was provided with the offshore team lead to ensure that delivery was aligned to our Client’s expectations and recommended course corrections based on feedback from our Client.
Additionally, the onshore project lead held weekly project sync meetings with our Client’s point of contact to convey weekly status updates, overall progress, potential risks and blockers. The onsite project lead was supported by the project administrator for all onsite project administrative paperwork and infrastructure support as needed for the project.
The offshore project lead was responsible for the overall management of service delivery by the execution team. The onsite project lead and offshore team lead held daily sync meetings, which were proven to be very effective in conveying our Client’s expectations and concerns, adjustments to the offshore delivery team and the ability to change priorities almost in real-time. Taking advantage of the different time zones, these issues and requests were often resolved by the time the offshore team began their day with minimal project downtime.
The execution process was templated with standard execution and quality control steps that helped in scaling the delivery and quality teams quickly, efficiently and minimized the learning curve. This resulted in productivity being at par with the project objective and productivity outcomes from day one of onboarding the data annotators
Key benefits
- A cost saving of 40% was achieved through the decision to have a dedicated offshore quality control team dedicated to the project.
- The percentage of first-time acceptance of images after the validation phase soared to 95%, which was a 50% improvement.
- The project was completed 8 days ahead of schedule with no unanticipated delays.
- The creation of a factory model enabled the delivery of tens of thousands of processed images earlier than the estimated project end date.
Download the PDF
