In July 2014, Qualitest Group conducted an in-depth survey on the factors associated with utilizing crowdsourcing and freelancing platforms for software testing. The survey focused on IT professional’s experience using crowdsourced testing and freelance testing platforms as compared to perceptions of those who were familiar but never used them. The survey included over 500 IT professionals of which only 33% qualified for the survey by holding a relevant management position and indicating familiarity with at least one of these models.
| Freelanced Testing | Crowd Testing | |
| Familiarity | 57% | 36% |
| Experience | 16% | 11% |
| Topic | User’s Experience | None User’s Perceptions |
| Cost savings | Experienced significant cost savings | Did not expect cost savings |
| Test Coverage | Experienced increase in test coverage | Did not expect increase to test coverage |
| Functional testing | Experienced improvements in functional testing under both models | Expect improvement in functional under a crowd testing model but not under a freelancer model |
| Usability Testing | The highest value in using these models is in Usability testing. These result are similar across both models and are consistent for both user’s experience and none user’s perceptions | |
| Flexibility | Potential operational flexibility is not a value experienced or perceived from both models by both groups | |
| Compatibility Testing | These models offer relatively low value in the area of compatibility testing. These result are similar across both models and are consistent for both user’s experience and none user’s perceptions | |
| Managed Vs. Direct Engagement | Managed engagement models are preferred by both users and none users. In the case of crowd testing, a managed engagement model is preferred more significantly than with a freelanced testing model | |
