Notes on: The Fundamentals of Testing (Ch1)

• Software testing is costly in human effort

• The goal of testing is to be practical and cost effective

• Errors are where major system failures usually being. These initial failures can come in the form of an incorrect thought or a wrong assumption. When an error is written down it becomes a defect of a 'fault'

• A faulty component that is built into a system becomes a failure in the system

• Testing is to avoid and rectify

• Software failures can lead to

  • a. Loss of money

  • b. Loss of time

  • c. Loss of business reputation

  • d. Injury or death when there is high risk (i.e. aircraft carrying passengers)

• Exhaustive testing of complex systems is not possible

• The more risk, the more and better testing should be performed

• The software development process must handle competing demands for resources

• Resources triangle:

Testing is one component of quality assurance. Testing seeks to ensure that the system that enters service without defects that lead to serious failures

Notes continued:

• Prioritisation: achieving acceptable results from a finite and limited amount of testing.

  • a. Prioritize the most important tests first

  • b. Most important functions as defined by the users

  • c. This will also address the most significant risk

• Testing sets criteria of when its safe to stop testing, and how much level of defects can be tolerated in a delivered product

• Testing is an activity that reduces risk and improves quality

• Testing can be systematic (planned)

• Testing can be methodical (a process)

• Testing is an activity performed on the product (not just code)

• Testing has a variety of techniques

More on Testing:

• Testing explores the component or system with the main aim of finding and reporting defects

• Testing does not include the correction of defects, these are passed on to the developers

• Testers can also debug

Root cause - tries to answer why the problem came about in the first place

Quality assurance - ensures that the testing process is undertaken correctly

Quality control - checks the quality of something

Static testing - can involve reviews

Dynamic testing - exercises the program under test with some data

Testing Process:

• Test set up > test execution > record test results

• A good test finds a defect if there is one present

• Testing shows the presence of defects, not the absence of defects

• Exhausting testing is impossible

• The earlier a problem (defect) is found, the less it costs to fix

Defects Cluster together:

• System complexity

• The effects of change on change

• Development staff experience

• Development staff inexperience

Pesticide paradox - running the same tests will not continue to find new defects

Test planning:

• Answers what is going to be tested, how it will be achieved, and who will do them

• This activity produces plans and schedules

Test monitoring and control:

• Monitoring - what has been achieved and what is expected to be done at this point and time

• Control - taking necessary action to meet the original or revised objectives as given in the test plan

• Produces test reports which compares actual progress against the plan

Test analysis - what to test

• Identifies testable features

• Defines and identifies associated test conditions along with risk and risk levels

Test design - how to test

• create test cases

• prioritize test cases and sets of test cases

• identify test data

• designing test environment if necessary

Test Implementation

• Do we have everything in place to run the test?

• Creating and prioritizing test procedures

• Creating automated test script

Test Execution

• Running tests

• Recording the identification and version of what is being tested

• Comparing actual and expected results

• Reporting defects based on failures

• Recording results ( pass, fail, blocked)

• Produces the status of individual tests

Test completion - test summary report (i.e. Test Rail summary report)

Traceability between test basis and test work products

• Enables the evaluation of test coverage

• Makes tests auditable

• Relating testing to stakeholders

• Improving understandability of test progress reports

** Defect reports need to be raised against the software and not the individuals