Have you implemented an ISO 13485 compliant training system, but are still struggling to determine how to evaluate the effectiveness of training actions? You’re not alone. The effectiveness aspect of ISO 13485 clause 6.2 is something that causes the most difficulties when implementing a medical device QMS.
You know your employees are trained and management has seen them do good work. But, how do you prove that your training actions were successful? This article will explain how you can implement written / multiple-choice tests, employee interviews, and recording work to meet this clause.
If you are wondering how to create an ISO 13485 training system from scratch, check out our article: ISO 13485 Training Requirements: A Complete Guide.
The Human Resources (6.2) clause of ISO 13485 does more than indicate that a procedure must be in place to handle the training of employees. Clause 6.2c also states that you must evaluate the effectiveness of the actions taken. Here are three ways that you can demonstrate that your training actions were effective:
Written / Multiple Choice Tests for Training Effectiveness
Create tests for each job role or similar job family. These tests can be heavily based on the material that you have defined as required training. This is one way to record the effectiveness of training actions for ISO 13485 training requirements.
Pros of Written Tests:
- Using the same type of training method for all job functions. You can create role-specific tests to evaluate the competence of production workers, QA personnel, engineers, customer service, etc.
- The tests can cover a wide variety of topics. This is especially useful if you have someone in a role that completes a significant amount of different processes. Instead of making them demonstrate competence in each process, you can make one test that covers all of the processes.
- Easy to include the Quality Policy and Quality Objectives in the test. Tests can make it easy to demonstrate clause 6.2d, as you can include questions related to how the individual’s activities contribute to the Quality Objectives and Quality Policy of the company.
- One and done. Once an individual has completed their necessary training, you can have them perform the written test. If they pass the test, you can consider their training complete. A test is much cleaner than recording the success rate of their work since you do not have to wait for completed production orders.
Cons of Written Tests:
- Tests can hinder employees who do not read well, or for whom the test is in a second language. Someone can be an excellent production worker or machinist. However, that does not mean that they will have a good understanding of abstract questions.
- Not the best approach for repetitive actions. An employee may be assembling the exact same medical device day in and day out. If so, a test may seem like a strange way of determining if they can successfully assemble the device. In this case, you might want to record their actual work. That way, you can demonstrate they are able to successfully assemble the device.
- Resource and paperwork burden. Your company may not have an automated system for recording the tests. The result is that you will need to keep records of each one of the tests. Additionally, you will need to record the answer keys and the employees’ results. Also, once any of the related procedures are updated, you will need to determine if the tests need to be updated as well.
Employee Interviews for Training Effectiveness
Similar to having the employee perform a test once their training is completed, you can have a supervisor or manager interview an employee to see if the training actions were effective.
You will still need to record what interview questions are being asked. However, you can have a little more leeway than just providing a multiple-choice test. This can be a bit more difficult to record compared to a test but also provides some advantages. Interviews are another great way to demonstrate the effectiveness of training for ISO 13485 training requirements.
Pros of Employee Interviews:
- Easier to break down any language barriers. While some employees may struggle to read and interpret questions, an interviewer can help guide them through the questions. The interviewer should never provide the answer. But they can offer some helpful hints to assist the interviewee in remembering the relevance of a particular question.
- A better understanding of training effectiveness and employee competence. Over the course of an interview, a well-informed interviewer will gain a much better understanding of what the employee is knowledgeable about. They will also learn what areas require more intensive training. Instead of having the employee reread a procedure, the supervisor can help guide the employee through the process. They can also provide explanations for any questions they may have missed during the initial interview.
- Relation of job function to Quality Objectives and Policy. Similar to written tests, an interview can gauge the employee’s understanding of how their job role contributes to the overall quality system.
Cons of Employee Interviews:
- Difficult to record evidence. An auditor may not just accept that training actions were effective because an interview was performed. They will need to see evidence of the interview, including the questions that were used (which should be consistent and established beforehand). Likewise, they will need a rough idea of how the employee answered. The supervisor/manager performing the interview must be able to take relevant notes. That way, the evidence of training effectiveness can be properly documented.
- Subjective. As opposed to a multiple-choice test, the answers to interview questions may be more subjective. You will have to determine if the given answer did a sufficient job of demonstrating knowledge of that question.
- Time intensive. Compared to a test that can be given and recorded by anyone, an interview must be performed by someone with extensive knowledge of the relevant area. If the supervisor or manager of the area is very busy or has a significant number of employees, their time may prove to be a bottleneck to completing employees’ training.
Recording Employee Work for Training Effectiveness
Based on the position, asking abstract questions may not be a good representation of the work being performed. In that case, why not record the results of the actual work? As an example, for a customer service role, you can record a specific amount of orders successfully created by the employee. This can be the best way to show the effectiveness of training for a specific process since the employee is clearly showing they can perform the process. Properly recording successful work can be a valuable way to show training effectiveness for ISO 13485 training requirements.
Pros of Recording Employee Work:
- Great for repetitive processes. If someone is performing the exact same process every single day, such as entering orders or assembling a device, recording the number of orders entered or created correctly does a brilliant job of establishing competence. How do you know they can assemble a device correctly? Someone watched and inspected a device they made ten times, and all ten devices were built successfully. This is a very clean way of providing evidence of training effectiveness.
- No language barrier. As stated in the written test portion, tests and interviews may cause issues with people who are not as familiar with the language or struggle with abstract questions. Recording their work eliminates this entirely, and can remove any barriers caused by language.
- Relevance to work. Demonstrating that the work being completed is correct and without issue can be the most direct way of determining that the training was effective.
Cons of Recording Employee Work:
- Difficult for complex job roles. If someone is doing the same thing every day, recording their work can be great. But if the job role is more complex, recording their work may become overbearing. What if the person is assembling 30 different devices? Are you going to record evidence that they can assemble every device? What about someone in a distribution role? Are you going to record evidence that they can properly pack every single thing you sell and that they can use five different shipping software? If the role is not consistent and repetitive, recording work will quickly become a burden.
- Lacking clear boundaries. When you have someone complete a test, you can consider the training complete once the employee has earned a passing grade on a test. If you are going to record work, you will have to determine how many orders must be successfully completed for the training to be deemed effective. Is it going to be the same for every process? Every job function? These questions must be answered before this type of effectiveness verification can be implemented into the training procedure.
- Difficult to record evidence of complex questions. If you are just recording someone’s work, how do you know they are aware of their relevance to the QMS? Where do you record knowledge about the Quality Policy?
Wrapping Up
If a company wants to have an ISO 13485 compliant training system, they need a way to show that training actions were effective. Hopefully, this article helped you learn several different ways that a company can demonstrate effectiveness. As with anything process related, there are pros and cons to each system. So, you will need to determine what is best for your company.
If you have any questions or opinions, please write them in the comment section below, or reach out from our contact page.