1. Fundamentals of Testing

1.1.What is Testing?

Software systems are an integral part of our daily life. Most people have had experience with software that did not work as expected. Software that does not work correctly can lead to many problems, including loss of money, time or business reputation, and, in extreme cases, even injury or death. Software testing assesses software quality and helps reducing the risk of software failure in operation.

Software testing is a set of activities to discover defects and evaluate the quality of software work products. These work products, when being tested, are known as test objects. A common misconception about testing is that it only consists of executing tests (i.e., running the software and checking the test results). However, software testing also includes other activities and must be aligned with the software development lifecycle.

Another common misconception about testing is that testing focuses entirely on verifying the test object. While testing involves verification, i.e., checking whether the system meets specified requirements, it also involves validation, which means checking whether the system meets users’ and other stakeholders’ needs in its operational environment.

Testing may be dynamic or static. Dynamic testing involves the execution of software, while static testing does not. Static testing includes reviews and static analysis. Dynamic testing uses different types of test techniques and test approaches to derive test cases.

Testing is not only a technical activity. It also needs to be properly planned, managed, estimated, monitored and controlled.

Testers use tools, but it is important to remember that testing is largely an intellectual activity, requiring the testers to have specialised knowledge, use analytical skills and apply critical thinking and systems thinking (Myers 2011, Roman 2018). The ISO/IEC/IEEE 29119-1 standard provides further information about software testing concepts.

1.1.1. Test Objectives

The typical test objectives are:

• Evaluating work products such as requirements, user stories, designs, and code

• Causing failures and finding defects

• Ensuring required coverage of a test object

• Reducing the risk level of inadequate software quality

• Verifying whether specified requirements have been fulfilled

• Verifying that a test object complies with contractual, legal, and regulatory requirements

• Providing information to stakeholders to allow them to make informed decisions

• Building confidence in the quality of the test object

• Validating whether the test object is complete and works as expected by the stakeholders

Test objectives can vary, depending upon the context, which includes the work product being tested, the test level, risks, the software development lifecycle (SDLC) being followed, and factors related to the business context, e.g., corporate structure, competitive considerations, or time to market.

1.1.2. Testing and Debugging

Testing and debugging are separate activities. Testing can trigger failures that are caused by defects in the software (dynamic testing) or can directly find defects in the test object (static testing).

When dynamic testing triggers a failure, debugging is concerned with finding causes of this failure (defects), analysing these causes, and eliminating them. The typical debugging process in this case involves:

• Reproduction of a failure

• Diagnosis (finding the defect)

• Fixing the defect

Subsequent confirmation testing checks whether the fixes resolved the problem. Preferably, confirmation testing is done by the same person who performed the initial test. Subsequent regression testing can also be performed, to check whether the fixes are causing failures in other parts of the test object.

When static testing identifies a defect, debugging is concerned with removing it. There is no need for reproduction or diagnosis, since static testing directly finds defects, and cannot cause failures.