State of the Art

To comply with new European MDR and associated MEDDEV guidance document 2.7/1, which was released in revision 4 in 2016 (“MEDDEV 2.7.1 rev 4”), manufacturers are expected to demonstrate that they have conducted a thorough clinical evaluation of the device. The MDR defines this as systematic and planned process to continuously generate, collect, analyse and assess the clinical data pertaining to a device in order to verify the safety and performance, including clinical benefits, of the device when used as intended by the manufacturer.

As part of this process, manufacturers are required to demonstrate that they have conducted a thorough analysis of the current state of the art, to comply with the number one general requirement which states that devices “shall achieve performance […] taking into account the generally acknowledged state of the art” (MDR ANNEX I, Ch I, 1).

Preceding the publication of the MDR, MEDDEV 2.7/1 rev 4 has provided detailed guidance to what is referred to as “current knowledge/state of the art”. State of the art, or state-of-the-art, is commonly used to describe the best, cutting edge, or, per Merriam Webster, “the highest level of development (of a device, procedure, process, technique, or science) reached at any particular time, usually as a result of modern methods”.

MEDDEV 2.7/1 rev 4 not only clarifies the definition and purpose of establishing state of the art, but also includes guidance on methodology and required content. The revision expands substantially on the four very brief mentions of “state of the art” in the previous guidance (see Figure 1). A total of 39 mentions on 26 pages of the MEDDEV 2.7/1 rev 4 document not only provides a comprehensive understanding of the meaning and significance of state of the art, but also details requirements of how to establish and document this information.

The MDR stipulates that clinical evaluation planning should include “parameters to be used to determine, based on the state of the art in medicine, the acceptability of the benefit-risk ratio for the various indications and for the intended purpose or purposes of the device” (Annex XIV, Part A, 1).

Figure 2 summarizes several core roles of this analysis. Establishing and describing state of the art is not an isolated task but is central to the entire clinical evaluation. Defining the current, accepted best treatment options, and describing the risks and benefits of these options, provides essential information for multiple aspects of the clinical evaluation, including:

• Assessing safety and performance of a device: selection of endpoints. MEDDEV 2.7/1 rev 4 requires that the clinical evaluation incorporate specific and measurable objectives that are linked to specific safety and performance endpoints. Such endpoints should align with those used in the current therapeutic landscape.
• Assessing safety and performance of a device: data appraisal. Based on MEDDEV 2.7/1 rev 4 clarifications for demonstrating the scientific validity of data presented, it is essential to formulate a clear and methodologically sound plan for the identification, retrieval, appraisal and weighting of clinical data. Appraisal criteria must be consistent with currently accepted scientific standard, and must be justified based on state of the art. The evaluation of clinical data should be done quantitatively in comparison to state of the art, such as by comparing the frequency of side effects or number of incidents.
• Estimating risk and risk management. Data from existing technologies and state of the art therapeutic standards, including risks and benefits of current treatments, should inform risk estimation (for example, hazards due to substances or certain technologies) and approaches to minimize risk.
• Selecting criteria to demonstrate equivalence. Selecting suitable criteria to demonstrate equivalence to another device requires knowledge of current technologies.
• Determining the acceptability of the risk/benefit profile of the device. This must be done in the context of the state of the art, which describes the gold standard (highest level of protection of health and safety) and defines what are considered acceptable risks and side effects, what is considered beneficial, and which duration of effects is considered acceptable.
• Positioning of the device within the available treatment portfolio. This must be clearly defined in a MEDDEV 2.7/1 rev 4 CER. The device should have an improved or at least equivalent benefit-risk ratio compared to available alternatives. However, acceptability can also be justified if the device addresses a significant unmet clinical need or exhibits an acceptable profile in specific situations or patient subgroups.

A recommended approach to incorporate state of the art data into the clinical evaluation is to present this information in a dedicated section of the CER. Such a state of the art section, containing the necessary clinical background information relevant to the CER, can then be referred to and cross-referenced throughout the document.

Multiple sections of the MEDDEV 2.7/1 rev 4 compliant CER need to be supported by state of the art data (see also Figure 2):

• The executive summary should describe the acceptability of the benefit/risk profile in context of that which is state of the art.
• Analysis of clinical data and risk management documentation should address consistency with state of the art.
• Conclusions should address the acceptability of the benefit/risk profile in context of state of the art, and any inconsistencies.
• State of the art justifies the validity of criteria used to demonstrate equivalence.

The state of the art section should inform on the clinical background, other devices, medical alternatives, and their safety and performance profiles, based on clinical data. This includes numerical data on safety and performance endpoints against which clinical data obtained with the device can be compared.

Multiple sections of MEDDEV 2.7/1 rev 4 describe aspects of what content should be provided. A succinct list is given in section 3 of the proposed table of contents of a CER (see Figure 3).

• Identify medical fields and relevant medical conditions
• Summary and justification of methods used to identify, retrieve, select, and appraise information
• Applicable standards and guidance documents
• Medical condition
• Description, natural course, consequences
• Clinical forms, stages, severities
• Frequency in general population, by age group, gender, ethnicity, familiar predispositions, genetic aspects
• Therapeutic/management/diagnostic options
• Available options
• Historical context, developments
• Summary of advantages/disadvantages of options
• Benefit-risks profiles and acceptability
• Harms
• Technologies
• Hazards, risk reduction approaches
• Management of side effects
• Users
• Diverging opinions of professionals
• Unmet clinical needs

A main section of the state of the art is dedicated to the medical condition concerned. This section should cover each relevant condition and describe the condition, its natural course, consequences, and if applicable, different clinical forms or severities. As available, epidemiologic data should be cited to describe the frequency in the general population, and in specific subgroups such as by age or gender.

The central section of the state of the art should present a comprehensive analysis of available therapies for the medical conditions introduced. This section should include the therapy in which the device is used, and available alternatives. It should objectively analyze and present the advantages and disadvantages of the various options, and the acceptability of benefit risk profiles. This section should be supported by the highest quality evidence available in the field, and literature cited should be carefully appraised for scientific validity.

If available, data sources should include current guideline recommendations released by medical associations and the systematic literature reviews conducted to establish these recommendations. This section supports the choice of endpoints that are currently accepted and used in the medical field. The endpoint data cited here will be used as the reference against which device-specific data are compared in the clinical evaluation. Data on safety and performance of benchmark and other devices that constitute the current therapeutic landscape can be incorporated here, or in a subsequent section that informs on the maturity of available technologies. Technological information should include potential hazards associated with technologies, and strategies applied to mitigate these risks.

Because of the breadth of information that needs to be included, developing a state of the art section involves the identification of data for a wide array of topics, from the description of the medical condition, to various treatment approaches, to existing technology. As for the clinical literature review, it must be demonstrated that sound and objective methods are used to establish the state of the art review, and the data search and selection process must be justified and documented.

Database searches designed to identify state of the art data/literature are usually broader in scope than those for device-specific clinical literature and will also differ in respect to timeframes of included literature. Thus, separate searches will need to be conducted. Relevant sources of information also include applicable standards and guidance documents, including clinical practice guidelines and consensus statements. Screening citations in scientific literature and clinical practice guidelines represents an important approach to identify relevant literature for the state of the art. Thus, the state of the art literature review is a relatively complex process, and adequate documentation of the identification and selection process may therefore be challenging. As discussed in more detail below (practical example), web-based platforms can help configure, manage, and update the state of the data reviews.

When setting out to develop a new state of the art section, there is no one-size-fits-all recipe.

However, a general approach/process can be broadly grouped into 3 stages (see Figure 4), which are:

• Identification/planning/scoping
• Data collection/selection/drafting
• Analysis/assembly

Identify key content areas:

At the outset, key content areas for the state of the art review need to be identified and defined. This step is a prerequisite to developing appropriate search and selection strategies. The following should be defined:

• Medical field and relevant medical conditions (include all indications) in which the device is used (per instructions for use.)
• Patient population to be treated with the device. This may overlap with medical conditions (i.e., all patients having a specific condition) but may include key contraindications, specific age groups, or specific populations.
• Target therapy, i.e., the therapy in which the device is used. The target therapy may be a specific therapy within a hierarchy of broader therapy descriptions (such as interventional treatment -> surgical treatment-> specific surgical approach in which device is used.)
• Alternative therapies. These encompass all alternative therapies available to patients with the medical conditions specified (the patient population). Alternative therapies can include conservative strategies, pharmacological treatment, alternative interventional therapies, or similar interventional therapies.
• Benchmark devices, other devices. Defining state of the art includes describing the current development of the technology(ies) underlying the device, benchmark devices, and other devices that are available and used for the same condition and patient population, and for the same target therapy (in most cases). Data obtained with these devices will be used for comparison with the device in evaluation, and the selection of these devices should be justified in terms of similarities in technology, mechanism of action, materials, and other features.

Develop and define search and selection strategies:

• Define search strategies. Using the terms defined above, formulate strategies using systematic search strategies that are objective and non-biased, for example, patient, intervention, control, or outcome queries (PICO). Most likely, multiple searches will need to be performed to cover the diverse aspects needed, and search terms may need to be refined based on scoping searches. Depending on the medical conditions and its frequency and availability of clinical studies and data, search strategies may vary in time periods covered. Furthermore, clinical evidence supporting current treatment standards may accumulate over many years and relevant guidance documents may only be updated periodically. Thus, scoping searches and citation mining (such as of guidelines) are necessary to ensure appropriate time periods searched.
• Identify database sources (databases such as MEDLINE, EMBASE, internet searches) and search engines (PubMed)
• Perform scoping searches
• Based on scoping searches, refine search criteria
• Based on scoping searches, define inclusion and exclusion criteria. For example, for a rare condition, all clinical data from a defined period such as the past 10 years may be included, whereas for frequent conditions, data may need to be limited to higher-level evidence, shorter time periods, or clinical evidence supporting current guideline recommendations.
• Citation mining, i.e. analyzing references cited in other documents (for example, guidelines), is an important means of identifying relevant data for the state of the art and should be considered in the strategies.

Execute searches, retrieve literature:

• Search and selection process. This includes each search protocol (databases, search engine, search criteria, key words) and the corresponding results. Next, the selection process, i.e. inclusion and exclusion of references, with justifications. For example, only a certain level of evidence (randomized studies) may be included, or only references for a certain time period. As shown below (Best practices when establishing state of the art: challenges), web-based platforms offer one approach to manage references effectively while documenting essential processes.

Draft section outline:

• Using the suggested content for the state of the art section (see Figure 3) as a guideline, group the retrieved references according to the information provided (for example, medical background, epidemiological data, clinical trial data on target therapy, clinical practice guidelines, and data on other devices)
• Determine a logical structure for presentation, starting with main sections, such as medical background, treatment options, and device-specific information (technology). Next, based on the available data, identify relevant sub-categories and determine an optimal structure to present the evidence. How will information on multiple indications be presented, alternating or in sequence? How many alternative therapies will be discussed, and how do they relate (one or several main categories), or multiple therapies within one main umbrella? Sketching out several alternative bulleted outlines with key references can help visualize how to best present the data without confusing the reader.

Analysis and assembly:

1. Analyze the supporting references to determine if additional references need to be included (for instance, reference identified from citation mining) and added to the search and selection process
2. Compile tables. Extract safety and performance outcomes from the state of the art literature into data tables. Depending on data availability and use of endpoints this may involve multiple tables. These tables can be assembled manually, or, as discussed below (Best practices when establishing state of the art: a practical example—data extraction), be generated using data extraction on web-based platforms. These data tables inform concisely on safety and performance outcomes with available therapeutic options and serve as reference points for the evaluation of device specific data.
3. Develop the narrative

To avoid such pitfalls, it can be helpful to develop a conceptual outline or “anatomy” of the state of the art section early during the review (see Step 4 in Figure 4; Figure 5).

In order to effectively meet the requirements of the new regulations, it has become increasingly important to adopt a transparent, reproducible process for literature reviews—something that can be difficult to achieve with manual data management or even spreadsheets.

Notified bodies will be looking for documented evidence of the following elements in all CER literature reviews, including those used to establish state of the art.

• Manual data collation
• Manual report preparation
• Difficulty keeping track of articles and data
• Distribution and collection of materials
• Reliable storage of submitted/living reviews

• A handheld medical instrument used for wound closures.
• Descended from the 20th century suture, and used as early as the 22nd century. By the late
• 24th century this technology was laser-based (VOY: “The Cloud”).
• Function: Seal, close, and promote the healing of wounds from surgery or deep trauma by stimulating the patient’s own anabolism. Can be used to close incisions or heal knife wounds beyond the ability of a dermal regenerator.

Search strategies to establish state of the art for the autosuture will encompass multiple strategies, including database searches focused on the condition, interventions, history, as well as internet searches to identify applicable guidelines, obtain information on similar competitor devices, etc. To illustrate the use of DistillerSR in this practical example, we will focus on a single search using the PICO method. Applicable keywords would be the following:

• Patient – Surgical wound, incision
• Intervention – Primary closure, tissue adhesives, adhesives, acrylates, cyoanoacrylates, glue or glues, fibrin-tissue-adhesive, bycrylate, sutur*, surgical stapling, staple, tape
• Control – Spontaneous healing
• Outcome – Dehiscence, infection, cosmetic appearance, patient satisfaction, clinical satisfaction, time for wound closure

For this project, the screening form captured study design and patient attributes relevant to the inclusion or exclusion of references from the autosuture state of the art review. Specifically, the form screened for RCTs and systematic reviews and patients with surgical wounds or incisions.

Best practices dictate that two reviewers screen each reference. The software automatically compares answers submitted by different reviewers and identifies any discrepancies or conflicts so that they can be reviewed more closely.

Because this was a regulatory state of the art project, two screeners were used for each reference and the conflict detection tool was used to reach consensus on references where the reviewers initially disagreed.

Once the reviewers have reached consensus to include a reference, it is automatically moved to the next phase of the review. Even when a reference is excluded, however, it is kept on file and the answer(s) that determined it was not relevant are stored with it in case this data is needed to answer the question, “Why was this reference excluded?”

One of the most important things to note about DistillerSR’s forms and project workflows is that they are completely customizable to any review protocol. Everything from forms and highlighted keywords to user permissions and task assignments can be configured to create a prescriptive, reproducible process. Once a standard protocol has been established, it can easily be templated and copied for use in future projects. Doing the review in a compliant way, every time, is critical to producing results that will be accepted by the notified bodies.

After initial screening, full text documents associated with included references are typically retrieved and uploaded to DistillerSR for a second round of screening. Screening forms used in second level screening may be the same as those used in initial screening, or can be more elaborate.

In the autosuture project, the team retrieved freely available PDFs from PubMed links and acquired the remaining documents from EMBASE. The screening process was then repeated using the full text documents for references that successfully passed from the initial screening phase.

After full text screening, PRISMA Flow diagrams and included/excluded reference lists may be exported for reporting purposes. The software can also produce reports showing who completed the work, when it was completed and how long each step took.

For the autosuture project, study summary, safety and performance and appraisal forms were used to extract data pertaining to state of the art treatment approaches for surgical incision closure.

Reports can also be customized to capture any subset of the data specific to the state of the art review. Once configured, custom reports can be saved and rerun as new data is added to the review.

The update process typically involves four steps:

1. Uploading newly located references
2. De-duplication to ensure that references screened in an earlier iteration are automatically removed
3. Screening and data extraction of the new references
4. Producing updated reports to capture new data