Insights Blog The Future of Test Process Improvement

White Paper

The Future of Test Process Improvement

Testing can sometimes become a troublesome and uncontrollable process. It can take more time and money than originally planned, and sometimes still offers insufficient insight into the quality of the test process - possibly putting the quality of the software being tested and business process itself at risk. So what can we do?

Kelvin Kam

Improve your test process

Many organizations realize that improving the test process is essential for ensuring the quality of the software and overall business processes. However, in practice, it’s often challenging to define the steps required to improve and control the process. Nowadays, developments occur at a very fast pace. Productivity is continuously increasing and clients are demanding even higher levels of quality.  Even if your current test process is satisfactory, this process will need improvement in the future. Organizations should review their testing process and be proactive and forward thinking. They might prefer to have a defined standard for testing or a continuous improvement program that is constantly evolving to meet their customers’ needs as well as their own. Of course there’s very little point in reviewing your testing process if the basics are not in place to begin with.  This is how you could start:

Plan Ahead – Ensure at the very least that the organization has a basic test process in place as this helps to clarify testing responsibilities and overall approach, resulting in less confusion.

Involve Early– Getting testers involved at the earliest stage is proven to be beneficial, as it’s cheaper to find and fix defects at this stage of the project.

Risky Business – Consider running a Risk Workshop so that the high-risk areas are identified to inform the testing effort.

Focus the Mind – A test strategy will identify what’s to be tested and focuses the mind on what the testers are trying to achieve.

Buy-In – Ensure that your Project Managers buy into your testing process.

Communication is Key – Don’t forget to have regular meetings with all of the stakeholders to ensure all items are on track.

The TPI® Next model

TPI® Next is the Sogeti model for business-driven improvement of testing. It offers insight into the maturity of the current test process and identifies improvement actions to accomplish the desired test maturity level and business goals. Through extensive global adoption since 1998, TPI® Next has become the industry standard for test process improvement.

Recent updates to TPI® Next

Since the original release of TPI®, new topics have had an increased impact on test best practice. Examples include outsourcing, Agile testing, multiple test processes, etc. These topics are covered by the updated model, TPI® Next. Additionally, there are descriptions of how to deal with specific test types and/or test levels, like evaluation and integration testing to provide you with practical tips and tricks for your specific situation.  Enablers are a new addition to TPI® Next and clarify the relationship between testing and adjacent processes and how they can benefit from each other. This supports organizations that are already using software process maturity models such as CMMI® or ISO/IEC 15504.

The TPI® model has been developed based on the knowledge and the experience of Sogeti and its clients. The model offers insight into the “maturity” of the test processes within your organization. Based on this understanding, the model helps you to define gradual and controllable improvement steps.

The TPI® Next model offers business-driven test process improvement through the concept of clustering, which aligns the order of improvement efforts with your business drivers. It considers the different aspects of the test process, such as the use of test tools, design techniques or reporting. By evaluating various aspects, the strengths and weaknesses of the test process become clear.

These aspects are called the key areas. The TPI® Next model has 16 key areas. These key areas are as follows:

  1. Stakeholder commitment
  2. Degree of involvement
  3. Test strategy
  4. Test organization
  5. Communication
  6. Reporting
  7. Test process management
  8. Estimating and planning
  9. Metrics
  10. Defect management
  11. Testware management
  12. Methodology practice
  13. Tester professionalism
  14. Test case design
  15. Test tools
  16. Test environment

Test Maturity Matrix

The TPI® Next model uses a test maturity matrix to provide a visual overview of the key test areas and their respective maturities. The maturity matrix lists the 16 key TPI® Next test process areas. It then shows the overall maturity level based on the 16 key areas, with regard to four maturity levels – Initial, Controlled, Efficient and Optimizing.  The combination of key areas, maturity levels and checkpoints are used to identify the strengths and weaknesses of the current test process, and to help define actions for improvement.

Filling in the matrix makes it easier to evaluate the improvement proposals. The requirements for each level are defined in the form of checkpoints which are questions that need to be answered positively in order to qualify for that level. These checkpoints make an objective classification by maturity level possible. Improvement actions can be defined in terms of desired higher levels of test process maturity. Checkpoints help determine the actions required to achieve a higher level. Additionally, the model offers improvement suggestions to support test process improvement, which include a list of tips and ideas that can help in the effort to achieve the desired level.

TPI® Next Assessment Approach

The following outlines the generic approach for a typical TPI Assessment:

  • Obtain awareness for improvement – Agree on the need for improvement and get commitment from all involved that improvement is necessary
  • Determine the scope for improvement – Establish the scope for the assessment, identifying the areas to be targeted for improvement
  • Interview all your personnel – Interview all your key personnel that have direct or indirect influence over your test processes
  • Gather documentation – Gather and collate the key documents which feature in your test processes.
  • Establish your current test maturity – Using the TPI® Next model and the Test Maturity Matrix, chart your organization’s current test maturity, highlighting areas of strength and weakness
  • Define improvement actions – Using the TPI® Next model and the Test Maturity Matrix, define the improvement actions that will get your organization’s test maturity to the desired level
  • Present assessment findings – The key output of this assessment will be a detailed TPI Assessment report which will include:
    • Your organization’s current test maturity
    • Your organization’s desired test maturity
    • Action plan outlining how to get from your current to your desired level of test maturity

TPI® helps organizations to realize the desired test maturity levels and achieve more streamlined and efficient processes. It enables organizations to optimize their testing performance and identifies the strong and weak points of the current situation in an organization, while suggesting the necessary improvements. The improvements are then implemented according to a plan and are monitored to meet the business goals. With the ever-changing business ecosystem and dynamic needs of customers, business goals of an organization might change, and TPI addresses these by suggesting improvements to the test processes.

Benefits of Test Process Improvement

  • Increase effectiveness and efficiency of test activities
  • Align testing with organizational priorities and with other project processes
  • Improve value of testing to the organization
  • Improve overall software quality
  • Reduce down time
  • Reduce errors
  • Enhance compliance
  • More efficient and effective business operations
  • Long-term reduction in the cost of testing
  • Long-term reduction in the length of the test process
  • Effective base-lining and evaluation tool to determine if the test improvements and benefits envisaged are being achieved.