Verification and validation: difference and definitions

What is the difference between verification and validation, and how are these terms defined? Even standards and laws use the terms incorrectly or in a misleading manner.

Find out how you can get your products such as medical devices safely through authorizations and audits thanks to precise verification and validation.

1. Verification

a) Definition

 Definition: Verification

“Confirmation by means of the provision of objective evidence [...] that the requirements set out [...] have been met”.

Source: ISO 9000

This definition does not explain what type of “requirements” need to be confirmed by the verification. To avoid any uncertainties and confusion with stakeholder requirements, limiting these requirements to product or component requirements is recommended.

 Definition: Verification

“Verification is a test using objective means that specified properties (e.g. of products, components) have been met”.

Source: Johner Institute, adapted from ISO 9000

These properties or features may be specified in a System Requirements Specification (SRS), for example, or in the component requirements

b) Examples

Examples of specified features are: 

  • Specification of the user interface (see next chapter)
  • Behavior of the system at the data interfaces (including interoperability)
  • Behavior of the system at “patient interfaces” It is possible for there to be a specification that a defibrillator must generate a certain voltage at a certain pulse sequence. The verification would then be a test of whether this tension is actually generated in the specified pulse sequence.

c) Special case: Verification in the context of usability

The verification of the usability is objective evidence that specified product features are met with reference to the usability. It is therefore a subset of the verification and is required by IEC 62366.

Examples of specified product features are font sizes, colors, contrast ratios, or general rules such as the marking of compulsory fields. This test (verification) is carried out for example in the form of inspections by usability experts. They check against 

  • specified product features (e.g. as described in a style guide or a UI mockup) and/or 
  • general rules such as for example those described in extensive detail by the ISO 9241 family.

Do you want to learn more about the process for verifying and validating suitability for use? We recommend the seminar on “Usability & Requirements” by Thomas Geis. 

2. Validation

a) Definition

Definition: Validation

“Confirmation by means of the provision of objective evidence […] that the requirements […] for a specific intended use or a specific intended application have been met”.

Source: ISO 9000:2015

Whether or not these requirements have been met for a specific use depends on the users and the usage context. For example, it is possible that the requirements of a defibrillator are met in an OR setting with professional users but not when it is used by laypersons on a wet street at night. 

The definition of the term “validation” should therefore also be expanded:

 Definition: Validation

“Confirmation by means of objective evidence that the specified usage objectives (purpose) can be achieved by the specified users in the specified usage context”

Source: Johner Institute

b) Examples

The validation is therefore the objective evidence that a specified user can achieve their specified usage objectives in the specified usage context. This test has two aspects:

  1. Classical validation: Testing whether the usage objective can even be achieved with the medical device. Clinical trials are an example of a validation of this type. A clinical performance assessment is carried out for in-vitro diagnostics.
    The usage objectives are described in the purpose. The clinical assessment and performance assessment is part of the validation and must show that the usage objectives have been achieved, in other words that the product is safe, efficient, and effective. 
  2. Validation of the usability: The test of whether the specified users can achieve the usage objectives (purpose) in the specified usage context effectively, efficiently and satisfactorily.

3. Software verification and validation

a) Regulatory requirements

The MDR and the IVDR both require:

In the case of products, the components of which include software, or products in the form of software, the software is developed and manufactured according to the latest technology standards. The principles of the software life cycle, risk management including information security, verification and validation must be taken into account.

MDR Annex I paragraph 17.2.

b) Fulfilment of the regulatory requirements

Manufacturers generally use IEC 62304 to prove that these requirements of verification and validation are met according to the latest technological developments. However, IEC 62304 only explicitly addresses the verification.

Software verification is usually carried out by means of:

The guidance document on MDCG 2020-1, which we summarized in our specialist article “clinical assessment of software”, provides a valuable addition.

4. Typical cases

a) Mixing up the various “validations”

The old Medical Device Directive (93/42/EEC) required:

“For devices which incorporate software or which are medical software in themselves, the software must be validated according to the state of the art taking into account the principles of development lifecycle, risk management, validation and verification.”

MDD supplemented to include 2007/47/EC

This requirement uses the term “validation” and “validated” in two different contexts, which should not be confused.

  1. Software validation in the narrow sense: this means the validation described above and should be understood as a delimitation from verification.
  2. Software validation in the broad sense: this validation corresponds to Computerized Systems Validation, or that which the FDA sets out in the guidance document “Software Validation”. Here, the term software validation is used as a synonym for software quality assurance. The latter comprises the life cycle phases: 
    1. Creation of the requirements
    2. Specification of the software architecture and the software design
    3. Implementation of the software
    4. Software testing (unit, integration, and system tests)
    5. In the case of standalone software, also validation in the narrow sense.

Data scientists know validation from machine learning models, which differ from the two validations just mentioned.

Further information

Learn more about the topic Computerized Systems Validation (CSV) here.

b) Lack of verification in safety class A

According to IEC 62304, the software safety class determines the need for software system tests that are necessary for a verification of the software. No tests are set out in IEC 62304:2006 for safety class A; no unit, no integration, and no system tests. This changed with amendment I, which sets out at least software system tests.

According to ISO 14971, all measures that contribute to risk control must be verified. If you implement measures in software, you must subject them to a verification regardless of the safety class.

c) Mix-up of verification and validation

In software development, the V model is still anchored in many people’s heads and is also referenced by IEC 62304 and IEC 60601-1.

But how does IEC 60601-1 see this V model?

According to this standard, requirements of a programmable electrical medical system, or PEMS for short, are derived from user needs; in other words system requirements (see figure).

First of all, we need to consider what the standard considers to be user needs. This term is not defined. The authors probably understand it to be an unspecified mishmash of needs, desires, usage requirements and directly formulated system requirements. If you are that imprecise with the terms, you will achieve imprecise results.

You can see for yourself where this lack of precision comes from: the standard thinks it can validate the fulfillment of PEMS requirements, in other words system requirements.

However, system requirements can be verified (in a system test) but not validated. 

By definition, however, you can validate usage requirements, but they do not appear, they are simply buried in the fog of user needs.

d) Incorrect belief that successful verification is a requirement for successful validation

The results of verification and validation can be independent of one another. It is entirely conceivable for the verification of the medical device to be successful and the validation not or vice versa, as the following examples show:

  • The specified 3000 V is generated on the pads on the defibrillator (verification successful). The patient’s heart is beating (purpose), but after the use of the device it is not (validation failed).

Instead of the specified 3000 V, only 200 V is generated on the pads on the defibrillator (verification failed). The patient’s heart is beating (purpose) and is still beating after the use of the device (validation successful).

Further information

Read more about the verification and validation of IVD here.

5. support

Successful and legally compliant verification and validation are some of the most important requirements for the authorization of medical devices. Manufacturers should therefore not make mistakes.

Numerous tools are available to avoid problems.

  • The free starter kit provides a rapid overview of the regulatory landscape and leads to the “authorization” of your medical device in six steps.
  • short article on software system requirements explains how these are formulated and therefore create the requirements for software system tests.
  • Another article describes what in particular needs to be taken into account in the validation of software as a medical device.
  • The pioneering seminar “Usability, Requirements and IEC 62366” by Thomas Geis explains how you can ensure the usability of medical devices and demonstrate this in the verification and validation. 
  • In our Usability Labs, you can have the usability of your products checked (verification and validation).
  • Several hundred training videos on the Audit Guarantee show how you can create and specify the requirements of your medical device and check them within the scope of verification and validation. Checklists and templates are also available to premium members, for example for a validation plan or the documentation of the validation results.

Get in touch!


Change history

  • 2021-09: Article fully revised, restructured, regulatory requirements updated.

Author:

Prof. Dr. Christian Johner

Starter-Kit_rot_dunkel

A quick overview: Our

Starter-Kit

Learn More Pfeil_weiß
blog_rot_dunkel

Always up to date: Our

Institutejournal

Learn More Pfeil_grau