Software Testing - Test Case Review

When a test engineer prepares a test case, he or she may skip some scenarios like entering incorrect data and writing incorrect navigation steps, all of which might have an impact on the overall test execution process. To avoid this, one round of evaluation and approval will be conducted before beginning the test. If some test cases are missed out and the review procedure is not done, then the accuracy of the test case document will be less. Only after the test case has been written must all cases be sent for review to another test engineer, known as a reviewer, for review.

In software testing, the examination of test cases is a critical step. Every functionality listed in the Software Requirement Specification is addressed by the test case. The test case should be effective as well as adhere to the test case writing guidelines. Each test case should be checked to ensure the success and thoroughness of the test.

Why Review Test Cases?

Peer review should be done on any work product that is deemed a deliverable. Test cases, which are important deliverables for the testing team, are included in this category. It is critical to write effective test cases that successfully uncover as many faults as possible during the testing process. As a result, a check is required to determine whether:

• Test cases are developed with the intention of detecting faults, and the requirement is correctly understood.
• Areas of potential impact are identified and put to the test.
• The test data is accurate and covers every possible domain class. There are scenarios that are both favorable and negative.
• The expected behavior is accurately documented.
• The test coverage is sufficient.

Why is Test Case review important?

Test Case Review is important because it helps in identifying areas that need further and more comprehensive testing, in generation of the actual test cases which are useful in documentation and as a ready-reference, in facilitating the selection of the most effective test cases and in determination of the time required by the selected test cases to be executed.

Test case review can also be described as a highly significant phase in the software testing life cycle that focuses on the quality of test cases. It is beneficial to carry out a comprehensive review to note all the gaps, uncertainties, and contradiction understand the defects before conclusion in that it becomes hard to notice the defects. It also leads to the establishment of standards, boosts on test coverage, and compliance to the requirements. In addition, completion of a sound review can drastically limit the amount of rework and maintenance lost in the future, in turn, saving time and money.

Who review the test cases

Most of test case reviews are done in a team with people drawn from different areas within the development team to get different views and a thorough review. The primary reviewers include:

• Test Leads and Managers: They make sure that the tested cases correspond to a test strategy and the goals of the project.
• Peers and Senior Testers: They quote on detail concerning the technicalities of the test cases and the functionality of the system in testing.
• Developers: Their contribution is useful in the confirmation of the technical likelihood and accuracy of the test cases in relation to the code.
• Business Analysts and Product Owners: It proves that test cases include all the business needs and those scenarios that users can encounter on the application.

What is Test Case Repository?

Keeping test cases organized pays off handsomely, particularly in medium to large projects. Furthermore, testing is a procedure that can be repeated. Everyone benefits from reusing test cases since it saves time. Large elements of projects can be repeated for testing. Maintaining a test case repository allows one to reuse past test resources as needed, which helps to save time. The good news is that keeping a well-organized test case repository isn't difficult at all. The quantity and variety of test cases that constitute the basis of testing cycles are often linked to the success of a software testing team. The assimilation of test cases may take a significant amount of time and effort, with the main focus being on creating a complete test case repository for each application. Test cases in the repository cover all essential permutations and combinations in workflow execution and transaction, ensuring that all system and user interactions are covered.

• A test case repository is a centralized storage area for all baseline test cases (authored, reviewed, and authorized).
• When the client provides the requirements, the developer begins constructing the modules, and the test engineer begins writing the test cases.
• The authorized test cases are stored in a test case repository.
• If a test engineer wishes to test the application, he or she must use the test case repository to get the test case.
• We can remove test cases from the test case repository if we don't need them.
• We have a separate test case repository for each version.
• Without the authorization of the test lead, test cases cannot be modified or changed once they have been baselined or saved in the test case repository.
• If there is a crash that affects the software, the testing team always has a complete backup of the test case repository.

Furthermore, because a test case repository increases over time, testers should keep it up to date with each new version of the business application or software product. If this is not done, it will lose synchronization with the software's actual function and behavior over time. As a result, subsequent QA cycles' findings will suffer as a result.

Benefits of Test Case Repository

The following are some of the benefits of a test case repository:

1. Reusability: Used test cases, especially when good test data is used, can be reused for regression test, hence affording time and effort in test case development.
2. Traceability: Indeed, it affords the best traceability especially for coverage of all the requirements from the test cases.
3. Consistency and Standardization: It makes ensure that test cases are written in a neat, readable and presentable format and structure for easy understanding and test execution.
4. Collaboration: It is for this reason, that through a shared repository there is better and easier sharing as well as updating of test cases among the members of the team.
5. Version Control: It enables one to verify the difference in the test cases so that the current or the most appropriate can be used.

Test Case Review Process

The following are the list of activities involved in the review process:. 

1. Planning

This is the first phase and begins with the author requesting a moderator for the review process. The moderator is responsible for scheduling the date, time, place, and invitation of the review. The entry check is done to make sure that the document is ready for review and does not have a large number of defects. Once the document clears the entry check the author and moderator decide which part of the document to be reviewed.

2. Kick-Off

This is an optional step in the review process. The goal is to give a short introduction on the objectives of the review and documents to everyone in the meeting. 

3. Preparation

The reviewers review the document using the related documents, procedures, rules, and checklists provided. Each participant while reviewing identifies the defects, questions, and comments according to their understanding of the document.

4. Review Meeting

The review meeting consists of three phases-

• Logging phase: The issues and defects identified during the preparation phase are logged page by page. The logging is done by an author or scribe, where a scribe is a person to do the logging. Every defect and its severity should be logged.
• Discussion phase: If any defects need discussion then they will be logged and handled in this phase. The outcome of the discussion is documented for future purposes.
• Decision phase: A decision on the document under review has to be made by the participants.

5. Rework

If the number of defects found per page exceeds a certain level then the document has to be reworked. 

6. Follow-Up

The moderator checks to make sure that the author has taken action on all known defects

Techniques for Test Case Review

There are three techniques to conduct a test case review:

1. Self-review: This is carried out by the tester who wrote the test cases. By looking at SRS/FRD, he can see if all of the requirements are met or not.
2. Peer review: This is done by another tester who isn't familiar with the system under test but hasn't developed the test cases. Maker and Checker are two other names for the same thing.
3. Supervisory review: This is done by a team lead or manager who is higher in rank than the tester who wrote the test cases and has an extensive understanding of the requirements and system under test.

Tips While Reviewing Test Cases

Below are some of the tips to be kept in mind while reviewing the test cases:

• In the review process, it's best to stick to version numbers. For example, if reviewing a test case plan for the first time, make it v.1. Once the tester has completed all of the changes, re-review it and make it v.1.1. One will be able to tell which one is the most recent this way, and there will be a complete record of the plan's changes.
• It is always preferable to meet with the tester face to face to ensure that he fully comprehends all of the review input.
• If at all possible, run test cases on the SUT (System Under Test) to gain a better understanding of the outcomes and actions involved in their execution.
• It is preferable to have a copy of SRS/FRD with you while reading the test case for reference.
• If you are unsure about a test case or expected outcome, consult with the client or your supervisor before making a decision.

Factors to Consider During Test Case Review

During the review, the reviewer looks for the following in the test cases:

1. Template: The reviewer determines if the template meets the product's requirements.

2. Header: The following aspects will be checked in the header:

• Whether or not all of the qualities are captured is debatable.
• Whether or not all of the traits are relevant is debatable.
• Whether all of the traits are filled or not is up to you.

3. Body: Look at the following components in the test case's body:

• The test case should be written in such a way that the execution procedure takes as little time as possible.
• Whether or not all feasible circumstances are covered is debatable.
• Look for a flow that has the maximum amount of test coverage.
• Whether or not to use the test case design technique.
• The test case should be easy to comprehend.

4. Text Execution Report:

• It is the last document produced by a test lead after all of the testings has been finished.
• The test execution report defined the application's stability and included data such as the number of cases written, ran, passed, and failed, as well as their percentages.
• The test execution report is a final summary report that defines the application's quality and aids in determining whether or not the program may be handed over to the customer.
• Every module has its own spreadsheet to track its progress.

This report was written by the test lead, and the test engineer submitted the particular features that he or she had tested and implemented.

This report is sent to the following addresses by the test lead:

• Development Team.
• Management.
• Test manager.
• Customer.

Where the list of failed test cases is required by a development team. There is a list of test case names, related status, and comments, as shown in the table below. The data from the Sales test case is shown in the table below-

Common Mistakes During Test Case Review

Below are some common mistakes checked during the test case review process:

1. Spelling errors: Spelling errors can sometimes cause a lot of confusion or make a statement difficult to grasp.
2. Replication of Test Cases: It relates to the reduction of redundant test cases. It's possible that two or more test cases are testing the same item and can be combined into one, saving time and space.
3. Standard/Guidelines: It's critical to examine whether all of the standards and guidelines are being followed correctly during the review process.
4. Redundancy: When a test case is rendered obsolete owing to a change in requirements or certain adjustments, it is referred to be redundancy. These types of test scenarios must be eliminated.
5. The manner used: Test cases should be written in a basic, easy-to-understand language.
6. Grammar: If the grammar is incorrect, the test case can be misinterpreted, leading to incorrect findings.
7. Format of Template: When a suitable template is followed, it is simple to add/modify test cases in the future, and the test case plan appears orderly.

Classifying Defects in Review of  the Test Cases

When these checklists are utilized consistently and problems are discovered, it is recommended that the defects be classified into one of the following categories:

• Incomplete test cases.
• Missing negative test cases.
• No test data.
• Inappropriate/Incorrect test data.
• Incorrect Expected behavior.
• Grammatical problems.
• Typos.
• Inconsistent tense/voice.
• Incomplete results/number of test runs.
• Defect information was not recorded in the test case.

Defects could sneak into production if test cases aren't thoroughly reviewed. As a result, production issues could be reported, thereby impacting the Software's quality. Resolving problems at this time would be much more expensive than fixing them if they had been discovered during the testing phase.

Conclusion

This is an essential process in testingsince it focuses on reviewing all test cases and developing efficient and effective test cases. Therefore, organizations can incorporate employees of various levels to participate in the review process, assisting in the identification of problems early and hence increasing the reliability of software products.

Also, when applied properly, a good test case repository leads to the optimization of efficiency, compliance, and sharing during the software testing process, thereby making a very significant contribution to the functionality of software testing.