When notified bodies identify discrepancies in clinical evaluations, in many cases, these relate to the clinical endpoints.
It is, therefore, essential for medical device manufacturers to specify the clinical endpoints precisely and to demonstrate that they have actually been achieved.
This article explains what manufacturers should pay particular attention to.
Manufacturers are required to demonstrate the safety, performance, and clinical benefit of their medical devices based on clinical data. To this end, manufacturers must specify the clinically relevant outcome parameters ("endpoints") with which they consider this evidence to be provided.
If a manufacturer claims that its medical device is capable of reducing pain, then he can quantify the pain reduction with the VAS score. If the clinical data shows that the score actually decreases by the claimed value, then the clinical benefit is proven.
The German Federal Institute for Drugs and Medical Devices defines (clinical) endpoints as follows:
Endpoints are the target variables of the study. All endpoints must be determined before the start of the study and written down in the study protocol. It must be defined which parameter (WHAT), at which time (WHEN), with which method, which device or survey instrument, etc. (HOW) will be recorded.
The definition of ISO 14155:2020:12 is similar:
Endpoint: <primary> major parameter used to provide evidence of clinical performance (3.11), effectiveness (3.20), or safety in a clinical investigation (3.8)
ISO 14155:2020:12, 3.22
Clinical endpoints are also referred to as clinically relevant (outcome) parameters.
A distinction is made between primary and secondary endpoints. The primary endpoint is the main objective of the study, and the secondary endpoints are the secondary endpoints. The primary endpoint is also referred to as the "hard" endpoint because it must be objective and measurable.
Secondary endpoints are "softer criteria" used to characterize the treatment under study further. For example, data on quality of life, pain perception, further interventions, and medications can be used.
The choice of endpoints and parameters is also critical in the planning of a study: study endpoints are those criteria used to measure the achievement of clinical research objectives, such as evidence of safety, performance, and patient-related benefit.
If safety of a device is to be demonstrated, the test plan must specify the metrics that will document and demonstrate safety. These could be, for example, the number of serious complications or the number and duration of specific deaths or hospital readmissions.
These parameters are objective and measurable.
In controlled studies (randomized controlled trials), the same parameters are measured and compared in both treatment groups.
The term "parameter" in this context is to be understood as an umbrella term for the term "endpoint." There are parameters that are not endpoints, e.g., material resistance, which would be determined as part of a bench test.
All relevant regulatory requirements require manufacturers to define clinically relevant endpoints. However, regulations differ in their definitions of terms and the granularity of requirements.
The determination of clinically relevant parameters is required by the MDR during the planning of the clinical evaluation:
MDR, ANNEX XIV, PART A – CLINICAL EVALUATION
Notified bodies review in their Clinical Evaluation Assessment Report (CEAR) whether a list of parameters is provided and whether the clinical benefit is evaluated using parameters.
Often the reasonableness of these parameters is not reviewed! Many auditors only review whether the manufacturers have defined the parameters. However, they do not question the meaningfulness of these parameters.
On the other hand, auditors insist that the choice of parameters MUST follow the state of the art. However, this does not make sense in some cases, as explained below (see surrogate endpoints).
The requirements can be found in, among other places
Many manufacturers are not aware that the MDR requirements for relevant concrete parameters for clinical output may be synonymous with trial endpoints. In terms of clinical investigations, the MDR states:
"The endpoints of the clinical investigation shall address the intended purpose, clinical benefits, performance and safety of the device. The endpoints shall be determined and assessed using scientifically valid methodologies. The primary endpoint shall be appropriate to the device and clinically relevant."
ANNEX XV CLINICAL INVESTIGATIONS CHAPTER I GENERAL REQUIREMENTS
Notified bodies review parameters against MDCG 2020-13 guidance, which requires:
“Has the manufacturer adequately described an indicative list and specification of parameters used to determine, based on the state of the art in medicine, the acceptability of the benefit-risk ratio for the various indications and for the intended purpose or purposes of the device?“
The fact that manufacturers do not adequately specify these parameters is one of the most frequent points in defect reports.
MEDDEV 2.7/1 rev.4 also requires that manufacturers define endpoints:
c. Quantification of benefit(s) to the patients
MEDDEV 2.7/1 rev.4 A7.2.
However, MEDDEV 2.7/1 rev.4 does not provide a definition or a more specific description of what the guideline means by an endpoint.
ISO 14155, the standard on clinical investigations, also requires specified endpoints:
(c) Primary and secondary endpoints with the rationale for their selection and measurement. Combined endpoints, if any, with the rationale for their choice and measurement. [...]
The MDR only requires that parameters supporting clinical benefit must be clinically relevant and should be measurable:
The primary endpoint shall be appropriate to the device and clinically relevant.
ANNEX XV CLINICAL INVESTIGATIONS CHAPTER I GENERAL REQUIREMENTS
It is up to the manufacturer to decide which endpoints are considered reasonable.
Conflicts of interest may arise in the selection of endpoints.
There are medical devices that do not have an intended medical purpose or where clinical data does not make sense. Examples include accessories and some Class I medical devices:
In these cases, no clinically relevant endpoints need to be collected, only technical parameters (performance parameters). This is why the MDR speaks of parameters and not endpoints.
The manufacturer should identify the relevant concrete parameters that allow the clinical output to be assessed and their acceptance criteria from the state of the art using a systematic literature search.
The acceptance criteria fix the limit values within which the endpoint is considered to be met. The manufacturers determine these from the literature, norms, and other standards.
Helpful for this activity is the article on systematic literature search.
According to Article 2 of the MDR, the "clinical performance" of a device [...] must meet the intended purpose stated by the manufacturer so that, when used as intended, a clinical benefit is achieved for patients according to the manufacturer's claims.
The MDR defines what it means by clinical benefit:
Refers to the positive impact of a device on a person's health as indicated by appropriate, measurable, and patient-relevant clinical outputs, including diagnostic results, or a positive impact on patient management or public health.
The clinical benefit must be considered individually for each device.
According to MDR Annex I, Chapter III, 23.4 (c), the clinical benefit must be stated by the manufacturer in the instructions for use.
There are different classes of clinical benefits and, thus, clinical endpoints:
While parameters should be appropriate and directly clinically relevant, in certain cases, a benefit can be assumed if validated surrogate endpoints are met (e.g., obtaining specific outputs from laboratory tests or measurements of anatomic or physiologic characteristics).
For drugs, one also chooses as classes for clinical endpoints:
Manufacturers must also determine safety and performance parameters, some of which are demonstrated through clinical investigation.
Not all parameters that need to be captured in the clinical evaluation for the review of the safety and performance of a device are clinically relevant.
In addition, there are safety and performance parameters that are not linked to endpoints in a clinical investigation with patients but require other experimental studies. These are nonetheless important for providing evidence in clinical evaluation.
For parameters for which clinical data are required, the manufacturer must determine the acceptance criteria. As with the clinical parameters (endpoints), he uses the literature, norms, and other standards for this purpose.
If clinically relevant outcome parameters are not specifically and measurably defined, it is not possible to design a trial or provide evidence in a clinical evaluation that these parameters have been achieved.
Problems with non-relevant surrogate endpoints
Instead of patient-relevant endpoints, so-called surrogate endpoints are often determined. These can usually be determined more quickly.
Surrogate endpoints are often physiological or biochemical measurements that are not of direct relevance to the patient. However, this often allows the number of cases to be reduced so that a smaller patient population can be selected and the costs of the studies can be reduced. However, the ethical justifiability is questionable.
Advantages of patient-relevant endpoints
Patient-relevant endpoints can be experienced directly by the patient. These include parameters such as mortality, morbidity, adverse events, and health-related quality of life.
Those responsible for clinical evaluation must critique endpoints and determine them from the state of the art.
If the design of a study is not planned correctly, the power of the measured endpoints may be limited. Examples that affect the determination of endpoints include:
The Johner Institute's clinical experts will help you determine your clinical strategy and formulate it into a Clinical Evaluation Plan (CEP). This includes:
This will provide you with the necessary input for the next steps while meeting all regulatory requirements for the CEP.
The clinical investigation can be planned ("designed") on the basis of the clinical strategy. The experts at the Johner Institutes will support you in this process. They also calculate the number of cases and ensure that the costs of the study remain as low as possible but that as many patients as necessary are included to prove the clinical benefit of the medical device.
This gives you a clear roadmap and allows you to estimate the duration and cost of clinical investigations. You avoid the worst-case scenario, namely that a notified body or authority rejects the outputs as insufficient after the study has ended.
Take advantage of the Johner Institute's support to help you achieve accurate clinical evaluations quickly and in a targeted manner, setting the stage for the quick commercialization of your devices.
Feel free to contact us here to clarify the next steps for this.
Determining the right parameters and endpoints is challenging. Therefore, it is not surprising that in the many inspections by notified bodies, the choice of these endpoints and their acceptance criteria leads to deficiency reports. This, in turn, delays or prevents the marketing of many medical devices.
Without the choice of relevant endpoints, no clinical investigation can be planned that is suitable for demonstrating the clinical benefit, safety, and performance of medical devices.
Therefore, as a manufacturer, you should plan the time necessary to do this and build or consult the necessary competencies. You owe this not only to yourself and the notified bodies but, above all, to your patients.