Testing and Quality Control

Testing is the process of finding defects (bugs) by comparing the expected behavior of a program with the actual result. It does not prove the absence of errors: as Edsger Dijkstra pointed out, a test can show that defects exist, but not that they do not. Quality control (QC) focuses on finding defects in the finished product, while quality assurance (QA) focuses on improving processes so that defects do not appear.

Tests are run at several levels. Unit tests check a single function or class in isolation, integration tests check how modules work together, system tests check the whole application, and acceptance tests check that the product meets the customer’s requirements. According to the test pyramid idea, there should be the most fast and cheap unit tests, fewer integration tests and the fewest slow and complex interface tests.

Two main approaches are used to choose test cases: black box (based on inputs and outputs, without knowing the internal code) and white box (based on the code structure). Equivalence classes and boundary value analysis help choose input values: for example, if access is allowed from the age of 18, the values 17, 18 and 19 should be checked, because errors most often occur at the boundary. In Python a unit test looks like this: assert is_adult(18) is True and assert is_adult(17) is False.

Code coverage, the share of code executed by tests, is used to measure test quality. High coverage is not a guarantee of quality, because even a test without checks (assert) increases coverage. Other practices that keep quality high are code review, static analysis (a linter), running tests automatically in CI and regression tests, which verify that a new change has not broken previously working features. The earlier a defect is found, the cheaper it is to fix.

Key takeaways

Testing shows that defects exist; it cannot prove that they do not.
Levels: unit, integration, system and acceptance tests; the pyramid has the most unit tests.
Equivalence classes and boundary values reduce the number of tests and find errors efficiently.
Code coverage is useful, but it is not the only measure of quality.
CI, code review and regression tests keep quality stable.

Practice tasks

In Python, write a function is_adult(age) that returns True for ages 18 and above, and write at least five pytest tests for boundary values (17, 18, 19) and unusual values (0, -5).
Measure code coverage with the commands coverage run -m pytest and coverage report, find an uncovered line and add a test that checks it.
Test a web form (for example, a registration form) with the black box method: taking equivalence classes and boundary values into account, build a table of at least eight test cases (input and expected result).

Review questions

Test yourself: think first, then reveal the answer.

1How do unit tests differ from integration tests?Show answer

A unit test checks a single function or class separately from the others, while an integration test checks that several modules work correctly together.

2Why does 100% code coverage not guarantee the absence of errors?Show answer

Coverage shows that the code was executed, not that the result is correct; tests with no checks or weak checks also increase coverage.

3Why are regression tests needed?Show answer

To verify that a new change has not broken previously working features.