Both IEC 62304 and the FDA require integration tests.
In software development, integration tests are a sequence of individual tests with the objective of finding errors in the interaction of different, interdependent (and already tested) software components of a software system.
Source: Johner Institute, based on Wikipedia
Integration tests verify whether software components that use interfaces of other components, i. e. call its methods, properly handle their returns (return values, exceptions).
Integration tests, like all software tests, are intended to improve the quality of the software by finding errors. To do this, they check whether the interfaces of two or more software components (also called software modules or software units) fit together.
These integration tests are necessary, although professional software developers have already tested the public methods of all components. Because component, module, or unit tests are not sufficient in order to find errors in the interaction of these components. This shows the following example very drastically:
One spacecraft crashed even though all modules were fully tested. However, one module (component) used SI units and another IU (Imperial Units). That the two interfaces did not fit could have found integration tests.
Many standards require integration testing and set requirements for it. For example, IEC 62304 requires integration testing and a verification whether the selected integration test method is suitable.
IEC 62304 sets the following requirements for integration testing:
Many manufacturers do not meet the requirement of the problem resolution process. Instead, they modify the software until it behaves according to specifications. This corresponds neither to the requirements nor to the idea of IEC 62304.
In addition, manufacturers must evaluate and ensure that the chosen integration test method is appropriate.
The manufacturer must evaluate the adequacy of the integration test procedure.
IEC 62304, Chapter 5.6.5
Since the 2015 version of IEC 62304, section 5.6.5 has been worded somewhat more comprehensibly, as the term "verify" has been replaced by "evaluate" and the term "correctness" by "adequacy".
It is, therefore, not a matter of reviewing the correct application of the test procedure but rather the suitability of the procedure.
What the authors of IEC 62304 mean by evaluation can also be derived from Annex B.5.6:
This evaluation of the test procedure is usually done by a review of the integration plan.
The first step is to define the objectives of the integration tests. The taxonomy of ISO 25010 provides a good checklist for the completeness of these objectives.
The selection and prioritization of these objectives should be risk-based. For example, if the timing of the software is critical, it should be evaluated during integration testing.
Determining the integration strategy is part of the test plan. Manufacturers should already determine the integration strategy as part of the software architecture. This determines the order in which the components are to be integrated. Typical representatives of integration strategies are:
In the bottom-up strategy, testers first test the hardware or operating system-related layers and then the higher layers (e.g., UI). This means that in the course of integration testing, they add the next higher layer each time, reviewing whether the test objectives are met.
In the Medical Device University, you will learn more integration strategies and how to meet the requirements of IEC 62304 (not only) for integration testing.
The specification of test cases is part of the test plan. In order to specify the test cases and, thus, the test data, manufacturers should use methods such as the black box methods. These help to systematically determine the test cases so that they find defects with a particularly high probability. This is what distinguishes good tests from bad ones.
The specification of the test cases also includes determining how the tests are to be carried out, documented, and their outputs evaluated.
Read more about blackbox testing techniques, such as equivalence class-based testing, and about software testing in general.
In order to perform the tests, the testers must develop mock objects or/and test drivers depending on the selected test strategy. With this, they perform the tests as intended in the test plan.
The documentation also includes an evaluation of the test result. In the event of an error, this documentation serves as input for the problem-resolution process.
In order to specify and perform integration tests, both programming experience and competence in software testing are required.
Software developers who have undergone further training to become software testers have this competence.
Manufacturers should not start software integration testing only when programming is complete. Instead, it is advisable to test the interaction of the available components following the integration plan from an early stage. In this way, errors can be found earlier and eliminated more easily.
During the maintenance phase, manufacturers should also perform the integration tests as regression tests. This way, they ensure that the changes do not reintroduce bugs that have already been fixed.
IEC 62304 requires manufacturers to review, as part of the software release, whether the software integration tests have been performed as specified in the design and development respectively integration test plan. This assumes that these plans exist.
Integration tests are an important instrument of software quality assurance.
The Johner Institute recommends that these tests be performed as a separate step and not together with the software system tests.
The integration tests help prove that the system can be assembled piece by piece from the components. This, in turn, confirms the desired weak coupling of the components and, thus, of a good architecture.
An architecture that fails to meet the "weak coupling and strong cohesion" paradigm becomes an architectural debt for manufacturers that becomes increasingly difficult to resolve. The expenditure for this "debt reduction" is a multiple higher than that for consistently accomplished integration tests.