Regression Testing 101:
Selective retesting to detect faults introduced during modification of a system or software component, to verify that modifications have not caused unintended adverse effects, or to verify that a modified system or software component still meets its specified requirements.
“…a testing process which is applied after a program is modified.”
How often do you perform regression testing on your code base? What is the recommended frequency of regression testing? What is the correct technique of regression testing?
This article will answer all these questions!
How do you decide right test case for Regression Suite:
The objective of regression testing is to uncover broken functionality that got induced with modification on existing code base, hence it is recommended to identify core functionality test cases from the existing Test Bank and based on the below factors:
- Test case covers critical functionality
- Test case is high priority test case compared to other testcases in that component
- This test case is part of the functionality that is directly impacted by the change performed
If answers for all the above are “Yes” then this test case is a candidate for Regression Test Suite
Should I automate regression test cases:
Test often, test more! This is possible only if you have automated your regression suite.
Having said that, it is all too common for regression testing to focus solely on the success rate of tests, whether automated or manual, without fully evaluating if this indicates the software is functioning as expected. In some instances, making changes to fixed bugs can uncover functional problems that were not addressed by the previous regression suite, making exploratory testing that much more beneficial.
How much of exploratory testing should I include:
It is important to weave an exploratory testing phase into the regression testing process; this step can greatly improve the quality of the software. In many cases, a human will catch an unknown issue, even where a test case indicates no such issue exists.
When developers make too many changes to the Software too frequently, it often leads to instability of the software, higher test failures, lot of bugs and unknown defects.
For this reason, it is beneficial to perform exploratory testing before a major release. This approach usually reveals failures that structured test many not discover and even points to improve test suite, that would significantly alter the the end-result of the software.
Qentelli brings to you an example:
Consider a simple, yet common application – an online shopping portal. During the purchase process, the application validates the address provided by the user. On the surface, this validation is independent of the payment type used (Credit/Debit/Bank Transfer etc). However, there is a bug in the code for payment through a gift card (address validation is not performed!).
Because gift cards are not commonly available in Test Environments and it was considered that payment methods don’t have any impact on address validation, the bug was deemed low priority. Only later when the bug surfaced in production, it was found that the address validation code was an older version causing another defect.
This type of “masked defect”, where one defect hides another can cause cascading issues due to dependencies in the software design. These types of issues can be problematic for the application development process. In cases where defects alter the behavior to such a degree that it is effectively hidden from the developers, it’s common for test cases to fail to notice these issues. Hence, it is recommended to perform a combination of Automated regression and Exploratory testing in every major release.
What should be the frequency of Regression testing:
Regression testing is usually performed after any changes are made to the code base. The frequency of regression testing may vary from project to project based on the complexity of the software and/or change made to the system. It is recommended to perform core regression testing, nightly or alternate days and complete regression suite execution, once a week. Generally speaking, the more often regression testing can occur, the more issues can be discovered and resolved, and the more stable and production-ready the application becomes
Qentelli’s Best Practices in Regression testing:
- Automate as many test cases as possible and eliminate effort for repetitive validation. Complement this with validation from a Business Expert to ensure the automation process did not miss any operational scenarios
- Prioritize and identify to include test cases that are absolutely required in final validation of the software.
- Test Data Management and ensure you have a way of generating and cleaning Test Data for the suite.
- Perform exploratory testing that empowers individual Tester, responsibility and freedom to examine and explore the software in various ways that are not documented.
To learn and explore more in detail, please write to us at firstname.lastname@example.org. Our experts will be delighted to engage with you. Also, you can visit Qentelli’s social links for more details- Facebook Twitter LinkedIn
Headquartered in Dallas, TX with global delivery teams in India, Qentelli is an Industry Thought Leader in Quality Engineering, Automated Testing and Continuous Delivery. With high performing engineering teams working in the dedicated Innovation Group, Qentelli brings design thinking to address complex business problems and enables Continuous Delivery across Enterprise IT through automation for its global customers.