Verification and Validation: Difference and Definitions
How do you differentiate verification and validation, and how are those terms defined? Even the IEC 60601-1 unclearly uses these terms.
Find out here how to safely go through authorisation and audit with a precise verification and validation.
Verification (definition and examples)
Verification of medical devices (in general)
Definition: The verification is an examination with objective means so that specific (product) properties are fulfilled. These (product) properties or characteristics can be found, for example, in a System Requirements Specification (SRS).
Examples of specified characteristics are the design of the user interface (see also verification of the suitability for use), the system's behavior to actions through its technical or data interface or the application part. So it could be specified that a certain tension must be present on a defibrillator in a specific pulse sequence. The verification would be examining whether this tension in the specified pulse sequence is actually present.
Verification of usability
Definition: The verification of the serviceability is the objective evidence that specified product characteristics are fulfilled with respect to the usability. So it is a subset of the verification and is required by the IEC 62366.
Examples of specified product characteristics are its font sizes, colours, contrast ratios, or general rules like flagging mandatory fields and such. This test (verification) is usually in the form of an inspection by an expert. One checks either against specified product features or against general rules, such as the ones described by the ISO 9241 family.
Want to learn more about the process for verifying and validating the suitability for use? Then we recommend the Usability & Requirements Seminar by Thomas Geis. And here you will find an overview of our technical medical and medical software seminars.
Validation (definition and examples)
The validation, however, is the objective evidence that a specific user in a specific context can reach their specific goals. This test has two aspects,
- "Classical Validation": The examination of whether one can ever reach the goal with the use of a medical device. The user objectives can be found described in the intended use.
- Validation of usability: The examination of whether the specified user in the specified context of use can reach the goals (purpose).
Read more here about the validation of medical devices.
Results of the verification and validation can be independent
It is quite conceivable that the verification of the medical device is successful and the validation is not - and vice versa, as shown in the following examples:
- The defibrillator was set to 3000 volts on the pads (verification successfully). The patient's heart was still not beating (purpose) after application of the product (validation failed).
- The defibrillator was set, instead of the specified 3000 volts, to only 2000 volts (verification failed). Nevertheless, the patient's heart beat (purpose) after the application of the product (validation successfully).
With more than 60 training videos auditgarant will show you how to achieve requirements for your medical products, specify and prove the verification and validation. For premium members, the checklists and templates are available e.g. for a validation plan and the documentation of validation results.
Verification and validation: Special features of standalone software
In the following you will find out more about
- regulatory requirements
- verification of software requirements in safety class A
- special characteristics of verification and validation of mobile medical apps
1. Regulatory requirements for the development of medical devices
The guidelines for medical products, especially the "Medical Device Directive" (93/42 / EEC) and the "Active Implantable Device Directive" agree: one of the basic requirements of a medical device is:
"For devices, which incorporate software or which are medical software in themselves, the software must be validated according to the latest technology , where the principles of development lifecycle, risk management, validation and verification should be considered."
Addition through the 2007/47 / EC
And how can you let your auditor know that you have validated your software according to the latest technology? Normally, you would follow a standard. For software verification, IEC 62304 is applicable, for the software validation, there is no specific standard.
By the way, the usual black-box testing is usually a verification and not a validation activity!
2. Verification of software requirements for security class A
According to IEC 62304 the software safety class decides on the need for software system testing, which corresponds with a verification of the software. For safety class A no tests are required, no Unit-, no integration, no system tests. Thus no verification.
According to ISO 14971 You must, however, verify all measures that contribute to risk control. So if you implement measures in software, you have to undergo verification - regardless of the security class!
3. Validation and verification of medical apps
To my contribution to the Mobile Medical Apps one asked the following important question:
For me the major question remains what separates the Apps from other software: How do I integrate the platform in the process of verification and validation? Say, it's enough that I'm testing it on a xy system with Android, and can I then put it on the market for all products with this version? Or do I have to test different hardware - for example, when used by Bluetooth does the app interface respond differently? What if there is a safety update on the Android version? What if other apps installed, who want to make use of the same hardware, interface? Do I have to test them all?
Strictly speaking, there are several questions. Let us address these individually:
a) To what extent do you have to verify Mobile Medical Apps?
The verification is to check that the specified product characteristics are met - in this case, the software requirements. One of these software requirements is a description of the run-time environment for which the previously mentioned platform is part of. Please look at the Institute Club, the video training for the Software Requirements Specifications if you want to learn more about it.
The platform includes aspects such as hardware as well as screen sizes, screen resolutions and the operating system, in this case certain Android versions, as well as the availability and correctness of data interfaces like Bluetooth.
Also an issue is the "Verification of the suitability for use", with which the manufacturer checks against the "specification of usability", which in turn contains its own claims or / and refers to standards such as the ISO 9241 family.
A full verification is excluded as are all tests generally. This means that all "tests" will only be random checks. Now the question arises, as to how extensive these samples need to be. The answer can only be found in the risk management. Only then can you estimate what risks might arise if that platform behaves differently than you think.
b) How are Mobile Medical Apps validated?
The response to the risk management also fits well for this question. But we can be even more specific: When validating it comes to the evidence that specified users in specific context reach their specified usage targets. How can this extent depend on the platform: An important factor is surely the size and resolution of the screens. The other factors should be constant or already be checked and controlled on the verification.
c) How do you deal with the fact that there are other apps?
At this stage you will be overwhelmed by the combinatorics: You will never be able to test the combinations of all possible apps. Again, you have to take risk management to help: If (unacceptable) risks arise that occupy other Apps’ interfaces (or interferes with the platform in any other way), you must choose the standard "triple jump" prescribed by the MDD :
- Strive for inherent safety (eg to not use functions, to not provide functionalities, to not use other platforms, etc.)
- Select precautions: For example, you could confirm that interfaces in the software work externally (eg via a "web service Ping" if data would be loaded or sent)
- Inform users about risks.
Again, testing is not a tool to prove the correctness of the Mobile Medical App.
We have prepared a presentation that provides a step-by-step overview on how to bring your Mobile Medical App to the CE marking.
The IEC 60601-1 also confuses Verification and Validation
If you get taught anything in a training for software development, it is the V-Model. Over and over again. It is also a wonderful model to explain a lot.
But how does IEC 60601-1 understand the this V-Model?
According to this norm (see figure) demands from user needs are derived on a programmable electrical medical system - for short PEMS. As well as system requirements.
The first thing you have to wonder, what does the norm consider user needs. It is an undefined term. The authors probably mean an unspecified mishmash of needs, desires, usage requirements and directly formulated system requirements. Anyone who handles terminology so imprecisely, will subsequently also achieve imprecise results.
Where this lack of precision leads, you can see for yourself: The norm believes that the fulfillment of PEMS requirements, or system requirements can be validated.
System requirements may be verified (in the test system) but not validated.
You can, however, by definition validate the usage requirements. But they don’t appear in the picture as previously discussed, but get lost in the midst of user needs.
More information on the verification and validation of software / medical products
Your next steps
- If you want to know how to formulate (Software) System Requirements, which are the subject of verification, then read the brief article about the software system requirements.
- If you want to know, however, what has to be considered in the validation, then read the article about validating software as a medical device.
- If these differences don’t make sense straight away, then I recommend the groundbreaking (at least for me) Seminar by Thomas Geis. It's called "usability, requirements and IEC 62366 ?. Thomas Geis is very strict with the terminology. And that makes his results so sensational.
- If you want to experience an innovation machine, visit the seminar - or book him right now. I sure will ...