Black box testing is when test cases are derived solely from the specification of the object to be tested (product, component). White box testing, on the other hand, derives the test cases from the internal structure of the object, e.g., from its source code or software architecture.
Unfortunately, many medical device manufacturers neither specify the test cases nor derive them systematically with black box test methods. Instead, a tester simply clicks through the application. This is neither effective (errors are only found with an insufficient probability) nor efficient. This regularly becomes a problem in the audit.
Tests cannot prove correctness (of software in this case), i. e. the absence of errors. They can only confirm the existence of errors.
The objective of black box test methods is, therefore, to specify test cases that have a particularly high probability of finding errors. Which test cases would you use to test the following screen template?
Black box test methods help precisely in this respect to quickly derive the test cases that have a high probability of detecting faults.
There are several black box test methods, for example
In equivalence class-based testing, one considers value ranges in which the product or software behaves equivalently. For example, the following value ranges could be distinguished for the date of birth:
What valid and invalid birth dates are should be inferred from the specification. Unfortunately, manufacturers fail to provide sufficiently precise specifications.
Specify your tests while you are still developing the specification of the product. Thus, black box test methods help you achieve precise and indisputable specifications.
Once you have identified the equivalence classes, select one representative of each for the black box test. In the above example, this would be the following birth dates:
Competent software testers are capable of selecting and combining the appropriate black box test methods depending on, among other things
IEC 62304 does not require a specific test method. However, it does require a development plan (IEC 62304 Chapter 5.1), which must also specify how the software will be tested. I.e., medical device manufacturers must specify the test methods(s).
IEC 62304 requires manufacturers to specify and perform software integration tests (IEC 62304 Chapter 5.6) and software system tests (IEC 62304 Chapter 5.7) for safety class B and C software. Since the software system tests must verify that the software requirements (IEC 62304 Chapter 5.2) are met, and since software requirements should be specified as the software’s black box behavior, software should be tested by black box tests.
The FDA requirements do not go beyond those of IEC 62304 with regard to (black box) testing.
Requirements for black box testing also arise at least indirectly from ISO 13485: The standard requires adequate training and competence of the team. Dedicated and trained software testers are indispensable for professional software design and development. This cannot be another task of the developers.