From Wikipedia, the free encyclopedia

Regression testing is any type of software testing which seeks to uncover regression bugs. Regression bugs occur whenever software functionality that previously worked as desired, stops working or no longer works in the same way that was previously planned. Typically regression bugs occur as an unintended consequence of program changes.

Common methods of regression testing include re-running previously run tests and checking whether previously fixed faults have re-emerged.

Contents 1 Background 2 Regression test generation 3 Types of regression 4 Mitigating regression risk 5 Uses 6 Quotes 7 See also 8 Notes 9 References 10 External links

Experience has shown that as software is developed, this kind of reemergence of faults is quite common. Sometimes it occurs because a fix gets lost through poor revision control practices (or simple human error in revision control), but often a fix for a problem will be "fragile" in that it fixes the problem in the narrow case where it was first observed but not in more general cases which may arise over the lifetime of the software. Finally, it has often been the case that when some feature is redesigned, the same mistakes will be made in the redesign that were made in the original implementation of the feature.

Therefore, in most software development situations it is considered good practice that when a bug is located and fixed, a test that exposes the bug is recorded and regularly retested after subsequent changes to the program. Although this may be done through manual testing procedures using programming techniques, it is often done using automated testing tools. Such a 'test suite' contains software tools that allow the testing environment to execute all the regression test cases automatically; some projects even set up automated systems to automatically re-run all regression tests at specified intervals and report any regressions. Common strategies are to run such a system after every successful compile (for small projects), every night, or once a week. Those strategies can be automated by an external tool, such as BuildBot.

Regression testing is an integral part of the extreme programming software development method. In this method, design documents are replaced by extensive, repeatable, and automated testing of the entire software package at every stage in the software development cycle.

Effective regression tests generate sufficient code execution coverage to exercise all meaningful code branches. Therefore, software testing is a combinatorial problem. Every boolean decision statement requires at least two tests: one with an outcome of "true" and one with an outcome of "false". As a result, for every line of code written, programmers often need 3 to 5 lines of test code.^[1]

Traditionally, regression testing has been performed by a software quality assurance team after the development team has completed work. However, defects found at this stage are the most costly to fix. This problem is being addressed by the rise of developer testing. Although developers have always written test cases as part of the development cycle, these test cases have generally been either functional tests or unit tests that verify only intended outcomes. Developer testing compels a developer to focus on unit testing and to include both positive and negative test cases.^[2]

• Local - changes introduce new bugs.
• Unmasked - changes unmask previously existing bugs.
• Remote - Changing one part breaks another part of the program. For example, Module A writes to a database. Module B reads from the database. If changes to what Module A writes to the database break Module B, it is remote regression.

There's another way to classify regression.

• New feature regression - changes to code that is new to release 1.1 break other code that is new to release 1.1.

• Existing feature regression - changes to code that is new to release 1.1 break code that existed in release 1.0.

• Complete test suite repetition
• Regression test automation (GUI, API, CLI)
• Partial test repetition based on traceability and analysis of technical and business risks
• Customer or user testing
  • Beta - early release to both potential and current customers
  • Pilot - deploy to a subset of users
  • Parallel - users use both old and new systems simultaneously
• Use larger releases. Testing new functions often covers existing functions. The more new features in a release, the more "accidental" regression testing.
• Emergency patches - these patches are released immediately, and will be included in future maintenance releases.

Regression testing can be used not only for testing the correctness of a program, but it is also often used to track the quality of its output. For instance in the design of a compiler, regression testing should track the code size, simulation time and compilation time of the test suite cases.

• "Also as a consequence of the introduction of new bugs, program maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire batch of test cases previously run against the system, to ensure that it has not been damaged in an obscure way. In practice, such regression testing must indeed approximate this theoretical idea, and it is very costly." -- Fred Brooks, The Mythical Man Month (p 122)

• Quality control
• Software development process
• Software testing
• Unit testing
• Smoke testing

This article needs additional citations for verification. Please help improve this article by adding reliable references. Unsourced material may be challenged and removed. (September 2007)

• JUnit Factory: Automated Regression Test Generation for Java