Test Completion Criteria for Test Levels

Usually, there is a misconception that test completion occurs when the testing phase is complete. In addition to this, it’s considered that it’s a final report. Which, in turn, needs to get testing sign-off before we go to production. However, test completion can occur at different project/test milestones. Let’s have a look at test completion criteria at different Test Levels.

• Completion of Sprint/Agile Iteration:Once the sprint finishes, the testing of the stories planned in the sprint also needs to complete within the sprint timelines. Often, there are outstanding defects or dependencies that remain unresolved within the sprint timelines. Therefore, it leads to testing spill over to the next sprint. A test completion report at the sprint level is a concise report. This report calls out the open defects/dependencies and failed/blocked test cases that can’t execute in the sprint. Based on this data, the Scrum Master decides whether to close the story (by taking exception from the product owner), or the story will carry over to the next sprint. This discussion can happen on the last day of Sprint, or as part of the Sprint retrospective meeting.
• Test Level Completion: You would already know by now that we have different test levels that are used to certify software from a testing perspective. These are Component Testing, Integration Testing, System Testing, and User Acceptance Testing. You can read our article on test levels if these are new terms for you. There is a defined process and criteria that govern whether we can move from one test level to another. Its captured as part of the test closure report for that test level.
  • Component Testing:Test closure at component testing usually has a limitation of getting the unit test case coverage. Additionally, it ensures there are no critical defects that will impact component integration testing. Test strategy governs the percentage of unit test coverage, and usually, kept at greater than 80%.
  • Component Integration Testing: Test closure at this level calls out the integrated components whose testing finishes (E.g., Cart with Address validation, Checkout with payment gateway, etc.). If the testing of all the integrated components completes, and there are no critical defects, then the testing moves to the next test level. This level is referred to as System testing.
  • System Testing: As system testing is the last level of testing before we give it to customers for user acceptance, the closure report is detailed. Usually, the test strategy defines the KPI (Key Performance Indicators) that needs to meet before we can exit System testing successfully. A Sample KPI looks like below.
    • 100% System Test Execution
    • 95% Pass Rate
    • 0 P1/P2 Defects and less than 50 P3/P4
    • Cross-Browser / Cross-Device testing is complete with >90% pass rate
    • Accessibility Testing is complete
    • Analytics Testing is complete
    • Performance Testing is complete with acceptable and agreed issues
    • Security testing is complete, and no major defect pending

As you can see, there are several KPI that require tracking for completion of System testing. Therefore, the test closure report is pretty comprehensive. Presenting this report to business stakeholders happens, and based on the results; they decide whether User Acceptance testing can start or not.

• • User Acceptance Testing: This is the last test level before the software goes to production. The test completion report usually contains the execution status and open defects. This report determines whether the software release can happen to production.
• Maintenance Release Completion: For maintenance releases, we usually don’t have comprehensive testing done, and a test completion report could call the new features added, and the corresponding testing happens for that along with open defects.
• Test Project Cancellation: In rare circumstances, a test project could get canceled or deferred, usually due to software no longer required or strategic decision by the business. In such cases, the closure report calls out the completed testing so far, and defects, and open dependencies. The completion report helps to ensure that when the project restarts, we don’t have to start from scratch.

As we have seen, the test completion is an important activity. It additionally determines the readiness of software at each test level. The completion report ensures transparency with stakeholders so they can make informed decisions about the software.

https://www.toolsqa.com/software-testing/istqb/test-completion/