IVDR Performance Evaluation: Scientific Validity, Analytical Performance and Clinical Performance Under Annex XIII

Manufacturers arriving at IVDR 2017/746 from MDR 2017/745 often assume the clinical requirements transfer with a change of vocabulary. They do not. IVDR does not have a clinical evaluation; it has a performance evaluation, and the evidence it demands is built from three separate demonstrations with three separate reports, only one of which resembles anything in MDR.

The transition also caught a large number of manufacturers that had self-certified under Directive 98/79/EC and found themselves in class B or class C under IVDR classification, needing a notified body and an evidence file they had never built. This guide sets out Article 56, the structure of Annex XIII Part A, what has to be in each report, the update cycle, and what an EU authorised representative verifies in an IVD technical file.

Article 56: the basis of conformity

The opening paragraph sets out what conformity rests on. "Confirmation of conformity with relevant general safety and performance requirements set out in Annex I, in particular those concerning the performance characteristics referred to in Chapter I and Section 9 of Annex I, under the normal conditions of the intended use of the device, and the evaluation of the interference(s) and cross-reaction(s) and of the acceptability of the benefit-risk ratio referred to in Sections 1 and 8 of Annex I, shall be based on scientific validity, analytical and clinical performance data providing sufficient clinical evidence, including where applicable relevant data as referred to in Annex III." (IVDR Article 56(1))

The same paragraph then places the burden of calibration on the manufacturer. "The manufacturer shall specify and justify the level of the clinical evidence necessary to demonstrate conformity with the relevant general safety and performance requirements. That level of clinical evidence shall be appropriate in view of the characteristics of the device and its intended purpose." (IVDR Article 56(1))

There is no fixed evidence threshold in the Regulation. The manufacturer sets it, justifies it, and defends the justification to the notified body. That is a different exercise from meeting a stated requirement, and it is where most performance evaluation files are weakest: they present data without stating what level of evidence was judged necessary and why.

Paragraph 2 adds continuity. "The clinical evidence shall support the intended purpose of the device as stated by the manufacturer and be based on a continuous process of performance evaluation, following a performance evaluation plan." (IVDR Article 56(2))

The three pillars

Paragraph 3 is the structural heart of the whole subject. "A performance evaluation shall follow a defined and methodologically sound procedure for the demonstration of the following, in accordance with this Article and with Part A of Annex XIII: (a) scientific validity; (b) analytical performance; (c) clinical performance. The data and conclusions drawn from the assessment of those elements shall constitute the clinical evidence for the device. The clinical evidence shall be such as to scientifically demonstrate, by reference to the state of the art in medicine, that the intended clinical benefit(s) will be achieved and that the device is safe." (IVDR Article 56(3))

The three are not stages of one argument. They answer different questions.

Scientific validity asks whether the analyte or marker is associated with the clinical condition or physiological state at all. It is a question about biology and about the state of scientific knowledge, and it is not about your device. A perfectly engineered assay for a marker with no established association with the condition claimed fails at this pillar and cannot be rescued by the other two.

Analytical performance asks whether your device measures the analyte correctly — accuracy, precision, sensitivity, specificity, linearity, measuring range, cut-off. It is a question about the assay.

Clinical performance asks whether the results your device produces correlate with the clinical condition or physiological state in the target population — diagnostic sensitivity and specificity, predictive values, likelihood ratios. It is a question about the assay in use.

MDR has no equivalent of the first pillar and no separation between the second and third. This is the point at which experience with MDR files stops helping.

The performance evaluation plan

Annex XIII Part A opens by defining what the exercise is. "Performance evaluation of a device is a continuous process by which data are assessed and analysed to demonstrate the scientific validity, analytical performance and clinical performance of that device for its intended purpose as stated by the manufacturer. To plan, continuously conduct and document a performance evaluation, the manufacturer shall establish and update a performance evaluation plan. The performance evaluation plan shall specify the characteristics and the performance of the device and the process and criteria applied to generate the necessary clinical evidence." (IVDR Annex XIII, Part A, Section 1)

It then sets the standard the evaluation is held to. "The performance evaluation shall be thorough and objective, considering both favourable and unfavourable data. Its depth and extent shall be proportionate and appropriate to the characteristics of the device including the risks, risk class, performance and its intended purpose." (IVDR Annex XIII, Part A, Section 1)

Section 1.1 then lists what the plan must include as a general rule. Among the items that most often go missing: a specification of the analyte or marker to be determined by the device; identification of certified reference materials or reference measurement procedures to allow for metrological traceability; a clear identification of specified target patient groups with clear indications, limitations and contra-indications; a description of the state of the art including existing relevant standards, common specifications, guidance or best practice documents; and, for software qualified as a device, an identification and specification of reference databases and other sources of data used as the basis for its decision making.

The plan also requires "a specification of methods, including the appropriate statistical tools, used for the examination of the analytical and clinical performance of the device and of the limitations of the device and information provided by it" and "an outline of the different development phases including the sequence and means of determination of the scientific validity, the analytical and clinical performance, including an indication of milestones and a description of potential acceptance criteria" (IVDR Annex XIII, Part A, Section 1.1).

Acceptance criteria set in advance are what distinguish a plan from a description. A file where the acceptance criteria appear only in the reports, after the data, is not evidence of a planned evaluation.

The general methodology

Section 1.2 states the method that applies across all three pillars. "As a general methodological principle the manufacturer shall: identify through a systematic scientific literature review the available data relevant to the device and its intended purpose and identify any remaining unaddressed issues or gaps in the data; appraise all relevant data by evaluating their suitability for establishing the safety and performance of the device; generate any new or additional data necessary to address outstanding issues." (IVDR Annex XIII, Part A, Section 1.2)

Identify, appraise, generate. The gap analysis is explicit and it is the step that determines whether studies are needed. A file that moves straight from a literature list to a conclusion has skipped the appraisal and the gap identification, and a notified body reviewing it will start there.

Pillar one: scientific validity

"The manufacturer shall demonstrate the scientific validity based on one or a combination of the following sources: relevant information on the scientific validity of devices measuring the same analyte or marker; scientific (peer-reviewed) literature; consensus expert opinions/positions from relevant professional associations; results from proof of concept studies; results from clinical performance studies. The scientific validity of the analyte or marker shall be demonstrated and documented in the scientific validity report." (IVDR Annex XIII, Part A, Section 1.2.1)

Five permitted sources, used singly or in combination, and a named output document. For a well-established analyte this report is short and rests on literature and professional association positions. For a novel marker it is the hardest part of the file and may need proof of concept data of its own.

Pillar two: analytical performance

"The manufacturer shall demonstrate the analytical performance of the device in relation to all the parameters described in point (a) of Section 9.1 of Annex I, unless any omission can be justified as not applicable. As a general rule, the analytical performance shall always be demonstrated on the basis of analytical performance studies." (IVDR Annex XIII, Part A, Section 1.2.2)

Two things follow. The parameter list is not the manufacturer's to choose — it is Annex I Section 9.1(a), and every parameter is in scope unless an omission is justified as not applicable. And analytical performance is demonstrated by studies as a general rule, not by literature; this is the pillar where own data is normally unavoidable.

Section 1.2.2 then handles the novel marker case. "For novel markers or other markers without available certified reference materials or reference measurement procedures, it may not be possible to demonstrate trueness. If there are no comparative methods, different approaches may be used if demonstrated to be appropriate, such as comparison to some other well-documented methods or the composite reference standard. In the absence of such approaches, a clinical performance study comparing performance of the novel device to the current clinical standard practice is required." (IVDR Annex XIII, Part A, Section 1.2.2)

The escalation is explicit: no reference material, then comparative methods or a composite reference standard, and failing that a clinical performance study against current standard practice. The output is the analytical performance report.

Pillar three: clinical performance

"The manufacturer shall demonstrate the clinical performance of the device in relation to all the parameters described in point (b) of Section 9.1. of Annex I, unless any omission can be justified as not applicable. Demonstration of the clinical performance of a device shall be based on one or a combination of the following sources: clinical performance studies; scientific peer-reviewed literature; published experience gained by routine diagnostic testing. Clinical performance studies shall be performed unless due justification is provided for relying on other sources of clinical performance data." (IVDR Annex XIII, Part A, Section 1.2.3)

Three sources, and a default in favour of studies. Article 56(4) states the same rule from the other direction: clinical performance studies shall be carried out unless it is duly justified to rely on other sources. The justification is a document in the file, not an absence of one. The output is the clinical performance report.

Clinical evidence and the performance evaluation report

Section 1.3.1 defines what clinical evidence is. "The manufacturer shall assess all relevant scientific validity, analytical and clinical performance data to verify the conformity of its device with the general safety and performance requirements as referred to in Annex I. The amount and quality of that data shall allow the manufacturer to make a qualified assessment whether the device will achieve the intended clinical benefit or benefits and safety, when used as intended by the manufacturer. The data and conclusions drawn from this assessment shall constitute the clinical evidence for the device." (IVDR Annex XIII, Part A, Section 1.3.1)

Section 1.3.2 then assembles the reports. "The clinical evidence shall be documented in a performance evaluation report. This report shall include the scientific validity report, the analytical performance report, the clinical performance report and an assessment of those reports allowing demonstration of the clinical evidence." (IVDR Annex XIII, Part A, Section 1.3.2)

Four components: the three pillar reports plus an assessment of them. The assessment is the part that is doing the regulatory work, and it is the part most often reduced to a summary. Section 1.3.2 sets out six things the report must in particular include: "the justification for the approach taken to gather the clinical evidence; the literature search methodology and the literature search protocol and literature search report of a literature review; the technology on which the device is based, the intended purpose of the device and any claims made about the device's performance or safety; the nature and extent of the scientific validity and the analytical and clinical performance data that has been evaluated; the clinical evidence as the acceptable performances against the state of the art in medicine; any new conclusions derived from PMPF reports in accordance with Part B of this Annex." (IVDR Annex XIII, Part A, Section 1.3.2)

The second item is unusually specific. Not a literature review — a methodology, a protocol and a report, as three distinct artefacts. A file with search results and no protocol has not met it.

The fifth item is the one that ties the evidence to an external benchmark: acceptable performance against the state of the art in medicine. Performance data with no state of the art comparison does not establish acceptability, only measurement.

The update cycle

Section 1.3.3 keeps the file alive. "The clinical evidence and its assessment in the performance evaluation report shall be updated throughout the life cycle of the device concerned with data obtained from the implementation of the manufacturer's PMPF plan in accordance with Part B of this Annex, as part of the performance evaluation and the post-market surveillance system referred to in Article 10(9). The performance evaluation report shall be part of the technical documentation. Both favourable and unfavourable data considered in the performance evaluation shall be included in the technical documentation." (IVDR Annex XIII, Part A, Section 1.3.3)

The final sentence is the one to read twice. Unfavourable data goes into the technical documentation. A file containing only supportive data is not a complete file, and the omission is a finding in its own right.

Article 56 sets the frequency. "The performance evaluation and its documentation shall be updated throughout the life cycle of the device concerned with data obtained from implementation of the manufacturer's PMPF plan in accordance with Part B of Annex XIII and the post-market surveillance plan referred to in Article 79. The performance evaluation report for class C and D devices shall be updated when necessary, but at least annually, with the data referred to in the first subparagraph. The summary of safety and performance referred to in Article 29(1) shall be updated as soon as possible, where necessary." (IVDR Article 56(6))

Annual as a floor for class C and class D, with an additional when necessary trigger. For class A and class B there is no fixed interval, but the continuous update obligation in the first subparagraph still applies.

Article 56(5) places the report: "The performance evaluation report shall be part of the technical documentation, referred to in Annex II, relating to the device concerned." (IVDR Article 56(5)) Our guide to technical documentation covers the file it sits in.

PMPF, and the notified body's power to require studies

Post-market performance follow-up is the IVDR counterpart of PMCF, and it lives in Part B of Annex XIII. Annex III requires the post-market surveillance documentation to include "a PMPF plan as referred to in Part B of Annex XIII, or a justification as to why a PMPF is not applicable" — the same structure as MDR, where the justification is itself a document. Our guide to the PMCF plan and report covers the MDR equivalent, which is close enough in structure to be useful as a comparison.

One IVDR power has no obvious MDR analogue and is worth knowing before certification. "Notified bodies may impose restrictions to the intended purpose of a device to certain groups of patients or users or require manufacturers to undertake specific PMPF studies pursuant to Part B of Annex XIII." (IVDR Article 51(3)) A notified body can narrow the intended purpose or mandate a PMPF study as a condition of certification. A performance evaluation that is thin for a subgroup often results in that subgroup being excluded from the intended purpose rather than in a refusal.

What the authorised representative verifies

The authorised representative obligations under IVDR Article 11 mirror MDR Article 11: verify that the EU declaration of conformity and the technical documentation have been drawn up, keep a copy available, and provide competent authorities with the information and documentation necessary to demonstrate conformity. In an IVD file that means checking for a specific set of documents rather than a general clinical section:

A performance evaluation plan, with acceptance criteria set in advance, not reconstructed from the results.

Three named reports — scientific validity, analytical performance, clinical performance — as separate artefacts. A single combined document is the commonest structural defect in IVD files and is visible without reading the science.

A performance evaluation report that contains all three plus an assessment, and that includes a literature search methodology, protocol and report as distinct items.

Either clinical performance studies, or a documented justification under Article 56(4) for relying on other sources.

Unfavourable data present in the technical documentation, not filtered out.

For class C and class D devices, a performance evaluation report dated within the last twelve months.

A PMPF plan, or a documented justification that PMPF is not applicable.

Where the notified body imposed a restriction on intended purpose or required a PMPF study under Article 51(3), evidence that the labelling reflects the restriction and that the study is running.

The annual date check on class C and D is the one that recurs, and it is the one an authorised representative is positioned to track. Like the PSUR under MDR, the performance evaluation report is a document that expires on a calendar rather than on change, and it goes stale quietly. Our note on authorised representative liability covers the exposure that follows from certifying that a file is available when part of it is out of date.

If you place in vitro diagnostic devices on the EU market from outside the Union and need an authorised representative that reads Annex XIII files as Annex XIII files rather than as MDR files with different words, our authorised representative service covers IVDR portfolios across all four risk classes.

Medex Kurumsal as your EU authorised representative

If you place medical devices or in vitro diagnostics on the European market from outside the Union, Article 11 of MDR 2017/745 — and Article 11 of IVDR 2017/746 for in vitro diagnostics — requires a single authorised representative established in a Member State before those devices reach the market. Medex Kurumsal Danismanlik acts as EU authorised representative under both Regulations, registered in EUDAMED under SRN TR-AR-000057550.

What the mandate covers: verification of the EU declaration of conformity and the technical documentation before signature rather than after; the Article 11(3)(a) copy of the file kept available for the full retention period; EUDAMED actor and device registration and the Basic UDI-DI submission; the registered EU address printed on your label and instructions for use; handling of competent authority requests, samples and information in the language of the Member State concerned; and the vigilance interface under Articles 87 to 90.

We review the file before accepting a mandate, because Article 11(5) makes the authorised representative jointly and severally liable for defective devices where the manufacturer has not complied with Article 10. A file we have not read is a liability we cannot price. Our fee structure is published on the EU authorised representative cost page, and the scope of the service is set out under EU authorised representative. Send us the declaration of conformity and the device list and we will tell you within two working days whether the file is ready for a mandate and what is missing if it is not.