The Process of Process Validation

Risk management is increasingly important with the newest versions of ISO 13485, the quality management standard for the medical device industry. Breaking news? No, not really, but many companies struggle with a good risk management process, not only the starters. Besides reading this article (and I love your comments on it), the latest revision of ISO 14971 is an excellent read.

Let’s start with why….

People don’t like risk. We want to trust that the products and services we receive do not contain any risk. Patients expect that medical devices function as intended. Maybe the expectations are even higher than for medicines. We all know drugs have side effects or may not solve your problem, but we expect the medical device to function flawlessly. Well, unfortunately, that is not possible Risks will never be eliminated.

Many risks can and, therefore, should be reduced. Two systems should be in place to ensure risks are considered and reduced.

  1. An internal well-functioning risk management process
  2. External auditing by notified body / competent authority / FDA

We are human, and most of us prefer to do less. An external party checking the medical device companies is vital. Nobody wants another PIP implant scandal……

In this article, we are now focusing on a company’s sound functioning risk management process.

And continue with how….

Three primary documents are essential for the Risk Management Process.

  • Risk management plan (RMP)
  • Risk analysis
  • Risk report

Risk Management Plan

A good template for a risk management plan supports the author in thinking about the following themes as a start of the risk management process:

  1. Scope
  2. product and process description
  3. risk management team
  4. requirements for the review of risk management activities
  5. risk management process
  6. risk evaluation and criteria for acceptance
  7. risk mitigation and verification actions

The scope describes essential choices that have been made. Is the RMP dealing with product risk, process risk, or both? Is the RMP dealing with one product and process or a process for multiple products? which steps are included? Design and development process, purchasing, incoming inspection, production, assembly, packaging, storage, sterilisation, transport, use by a patient, nurse or doctor, disposing of the device, service and/or repair of device end-of-life phases, etc.

Suppose a company has more than one product/process. In that case, it might be wise to have a general risk management process and analysis for general activities such as purchasing raw materials and storing finished goods. This risk analysis contains risks that are true for all processes. All choices are acceptable as long as all the processes and risks are captured in the analysis.

The scope is further detailed in the description of the product and process. However, a detailed description of every step or function can be recorded later in the risk analysis.

The selection of the risk management team is essential. This team should look at the process and product from all angles. The team should consist of a QA engineer with risk management experience who can lead the process and guide and train the members where needed. Further subject matter experts (SMEs) should be part of the process. These can be people from operations, logistics, quality control or the laboratory, and engineers. It is essential to have people from all layers of the organisation, not just the managers. Also, external team members can be needed, such as medical specialists, users, suppliers, contractors etc.

If there are requirements for reviewing the risk management activities, these requirements should be recorded upfront. A requirement could be that the risk analysis is done before or after a specific design and development stage or a particular stage of process validation.

Describe in the plan which risk activities you will take. What type of risk analysis, how will you collect the risk, who will be involved and when. What is the risk management process that you will follow?

You can collect hazards (failure modes) and give them a score (a rating). The scoring and evaluation of these scores need to be described in the risk evaluation and criteria for acceptance. What does individual scoring mean to the team? Is it acceptable, tolerable, or intolerable? For more details and an example, see below (Risk Analysis).

Finally, the team needs to consider how to deal with risk mitigation actions, implementation, and verification of these actions and review these risk mitigation actions that induced new risks.

Risk Analysis

There are numerous risk analysis techniques, and they are all allowed and have advantages and disadvantages. It is essential to ensure that all the hazards or failure modes are reviewed. So consider the following limited examples

  1. design risks
  2. production risk from purchasing raw materials to storage and transportation risks caused by human, machine, software, procedures or combination
  3. the risk to operators, nurses, surgeons or other employees
  4. the risk to patients over the lifetime of the device
  5. risk of ageing, degradation, maintenance, service, disposing
  6. unintended or unwanted use, re-use, or re-sterilization

In appendix C of ISO 14971:2007 (a newer revision is currently under development), questions have been formulated to define medical device characteristics that could impact safety. It is advised to answer all of these questions (some might be non-applicable) and incorporate them in the risk analysis

Failure Mode and Effect Analysis (FMEA) is the most commonly used risk analysis technique. The FMEA can be used for the design phase and the process (usually defined as dFMEA and pFMEA). The figure below shows an example of the FMEA form we use at Vosfox Medical.

Unless you have risk management software, I can advise you to use an excel sheet for the FMEA. The risk management team can decide to add or remove a column if deemed necessary. Let the RPN be calculated automatically and use conditional formatting to make the risk regions visible (e.g. green, negligible, yellow acceptable, orange undesired and red unacceptable). Make several tabs to keep the excel sheet clean and readable:

  1. Cover sheet providing information about the product, process, unique document number, date, sources of information, signature list of the contributing risk management team members etc.)
  2. FMEA history record. The FMEA will be reviewed several times, and it is wise to record the changes.
  3. Process flow. Before making the analysis, the process steps, functions, and risk evaluation scoring tables. The risk rating should be agreed upon with the whole risk management team.
  4. FMEA sheet (see picture above). In the FMEA, we list the process step (choose your steps small and wise, so you don’t skip hazard modes, do this for each production process step or life cycle stage of a product), the failure mode or hazard, the effect of the failure, the root cause of this failure and current controls to detect the failure. Then the risk is scored on Severity, occurrence, and detectability.
  • Severity (S). How bad would it be if the hazard or failure mode happened? Often a scoring scale of 1-10 is used, with 1 being a negligible negative effect and ten a catastrophic effect (e.g. death of a patient, high financial impact, or loss of certifications)
  • Occurrence (O). How likely will it occur? Also, a scoring scale of 1-10 is often used, with one being unlikely to happen until 10; for sure, it will happen.
  • Detectability (D). How well will you detect the failure or hazard? When using a scoring scale from 1-10 (although you also see a scale of 1-5), an one would mean that you will see it immediately before something goes bad, and a ten is that you most probably won’t detect the problem before it will be a huge problem (after implantation in the patient for example)

A risk priority number (RPN) can be calculated to evaluate the risk. This is the product of s, o, and D or RPN=S*O*D. The risk management team must determine how to evaluate the value of the risk region. One could choose three risk regions (acceptable, As low as reasonably possible (ALARP) and intolerable. Nowadays, four regions are often defined (Negligible, Tolerable, Undesirable and Intolerable). The team has to define the RPN numbers falling in one of these risk regions and how to deal with them. For example, with a tolerable risk, risk mitigation actions should always be considered, but if not possible to reduce risk, the team might consider the residual risk acceptable.

If risks are not negligible, a risk mitigation action will be proposed if possible and recorded on the sheet.

It is possible to have the whole risk management team together for all brainstorming sessions. Still, in my experience, a first meeting with the whole group agreeing about the process, the scoring, and the process steps, followed by sub-group meetings, thinking about specific specialized process steps, is much more effective. It is advised to have a QA specialist present at all times, ensuring the quality of the risk management process. A final (or several) group meetings with all members, reviewing and discussing the risk analysis might be useful. Of course, all team members will review the total risk analysis for comments and approval.

In the second revision of the document, the outcome of the recommended risk control actions is listed, and a reference to a document dealing with that action (e.g. validation report or part of the DHF. The risk management plan defines the product design process, development stage, or validation stage. The first version and the revision of the first version should be finished.

During the revision of the first version, the implemented risk control actions are recorded and reviewed to determine if these actions have induced new risks. The team will also review if new knowledge about the product or process records additional hazards or different scoring of the severity, occurrence, or detectability of previously defined hazards.

Microsoft Excel tip: Number the pages in the heading as # page/total #pages. When printing the Microsoft Excel workbook, choose safe as pdf, choose options and choose the entire workbook. Pages will continuously be numbered on your print.

Risk Management Report

At important milestones (e.g. before a clinical trial or after the validation of a process is finished), the risk management team should write the risk management report.

The following topics should be covered in a risk management report

  1. To make the document readable, a short introduction about the product, project, and/or process is useful, and references to related documents (e.g. risk management plan, risk analysis)
  2. Risk Analysis procedure. What was done, which risk analysis technique was used, and how was the risk scored and evaluated? If this differs from what is written in the risk management plan, a rationale should be written here.
  3. Risk evaluation. Review at least all risks that scored a higher RPN than negligible. Review the number of risks in a particular risk region
  4. Risk mitigation actions. Make an overview of the risk mitigation actions taken and the status (open or closed) if actions are open, list when they are closed and who is responsible. Discuss if the risk mitigation actions introduced new risks. Evaluate the effectiveness of the actions taken.
  5. Risk Evaluation.  After all risk mitigation actions are implemented, it is likely that some hazards (failure modes) still have a risk in a risk region other than negligible. For these risks, a risk evaluation should be done. Is the residual risk acceptable, is it related to safety, and is it very costly if this failure mode occurs? If the risk evaluation is negative, further risk mitigation actions should be taken. For some risks (especially safety-related risks), a risk-benefit analysis might be required. Disclosing significant residual risk(s) to users can be important so they can make informed decisions regarding the use of precautions for the device. The FDA has a nice document about patient-focused risk-benefit Assessment.
  6. In the norm EN-ISO-14971:2007 annexe C, a list of questions is available to support the risk analysis. In principle, these questions should have been covered in the risk analysis, but covering them in the risk report one by one (if applicable) is wise, and auditors do like to see this.
  7. Conclusion. The team should formulate a conclusion. Examples of conclusions are that the device is safe enough to start validation in a clinical trial or that the process is safe and stable enough that commercial production can commence. It is also important to state when a re-evaluation is required. For example, this could be in one year or after the clinical trial.
  8. Finally, attachments and appendices are listed, and the risk management team signs the document.