QSE 8 — Internal and External Assessments

Key Concepts

Key Concepts: This QSE addresses the organization’s internal quality assessment functions as well as processes to support external assessments by accreditors, health authorities, and regulators. This chapter also describes the need for the organization to engage in ongoing quality monitoring and utilization review.

Key Terms
Adverse Event: A complication. Adverse events may occur in relation to organization-defined activities.

Assessment: A systematic examination to determine whether actual activities comply with planned activities, are implemented effectively, and achieve objectives. Types of assessments include external assessments, internal assessments, peer review, and self-assessments.

Competent Authority: The agency responsible under its national law for regulations applicable to the organization.

Conformance: Fulfillment of requirements. Requirements may be defined by customers, practice standards, regulatory agencies, or law.

Corrective Action: Actions taken to address the root cause(s) of an existing nonconformance or other undesirable situation in order to reduce or eliminate recurrence.

Deviation: A departure from policies, processes, procedures, applicable regulations, standards, or specifications.

Nonconformance: Failure to meet requirements.

Preventive Action: An action taken to reduce or eliminate the potential for unexpected deviations, nonconformances, or other undesirable situations.

Quality Indicator Data: Information that may be collected and used to determine whether an organization is meeting its quality objectives as defined by executive management in its quality policy. Indicators are measured by data for movement or regression with regard to those quality intentions. The data used for monitoring a quality indicator may consist of single-source data or multiple-source data, as long as it is clear how the data will come together to define the indicator.

Root Cause(s): The underlying cause(s) of an event or nonconformance that, if eliminated, would prevent recurrence.

Examples of Objective Evidence:

• Policies, processes, and procedures related to this chapter.
• Records of internal assessments scheduled and conducted.
• Records of evidence that deficiencies discovered during assessments and inspections have been addressed, including changes to quality or operational functions.
• Records of external assessments being conducted.
• Quality indicator data collection and review.

8.0 Internal and External Assessments

The organization shall conduct assessments of operations and quality systems.

Guidance

Chapter 8 aims to ensure that the quality management system is effective and provides an opportunity for improvement. The laboratory has two obligations under the requirements in this chapter: internal and external assessments. (SS)

The committee revised standard 8.0 based on updates to the AABB Quality System Essentials. (SC)

☑ Actionable Checklist

• Develop and implement an internal audit schedule covering all operational and quality system areas.
  • Ensure audits are conducted at least annually.
  • Assign qualified personnel to perform audits.
• Document findings from internal audits using a standardized audit report form.
  • Include identified non-conformances and opportunities for improvement.
• Establish a process for conducting external assessments by accredited bodies.
  • Maintain records of all external assessment reports and findings.
• Implement a corrective and preventive action (CAPA) process to address audit findings.
  • Track and document the resolution of all identified issues.
• Review audit findings and CAPA outcomes in management review meetings.
  • Document discussions and decisions in meeting minutes.
• Update relevant SOPs and training materials based on assessment outcomes.

8.1 Internal Assessments

[Cord blood applicable]

The organization shall conduct internal assessments. Internal assessments shall be performed by personnel independent of those having direct responsibility for the activity being assessed.

Guidance

Internal assessments should be scheduled on the basis of the significance of the activity to be assessed. These CT Standards do not establish a specific schedule; the facility should review its operations and determine what schedule is appropriate. It is recommended that all activities that constitute the quality system be assessed once a year. During the first 2 to 3 years of implementation of a quality management system, however, it is recommended that the quality system be assessed more frequently (eg, at 3- to 6- month intervals). As a general rule, critical activities should be assessed frequently, and areas with consistent problems should be assessed often until there is clear resolution of the problems.

These internal assessments should evaluate both operational and quality system activities. It is acceptable to use the AABB Assessment Tools (eg, Cellular Therapy Services Standards Assessment Tool, available to members) as a guide for these assessments.

Ideally, individuals who assess specific activities should be independent from those having direct responsibility for the activity. However, this independence may not always be possible. Accordingly, independence may be interpreted as employees from the same department who assess one another’s work.

The facility should determine how often specific departments will be assessed and identify when they will be assessed. When the internal assessors have been identified, they need to be trained. External courses provide a training option, or a large organization may choose to provide internal training for a group of assessors. Regardless of where it takes place, assessment training should focus on the general structure of the quality system, understanding of auditing techniques, and good communication. Assessors should use a systems approach rather than depending on a predictable checklist of standard questions. Assessors can assess all departments at one time or may choose to develop a schedule in which departments are assessed, one at a time, throughout the year.

A “life of a product” assessment is another option. All procedures involved from the initial assessment/scheduling of the patient (eg, informed consent process) to the collection, preparation, labeling, any storage, and final disposition of the product, including reviewing of records related to quality assessments of personnel, products, and equipment used in the procedures, are audited. The program’s policies, processes, and procedures should describe the internal assessment process. (SS)

The committee revised standard 8.1 based on updates to the AABB Quality System Essentials. (SC)

☑ Actionable Checklist

• Develop an internal assessment schedule based on the significance of activities, ensuring all quality system activities are assessed annually.
  • Review operations to determine appropriate assessment frequency, increasing frequency during the first 2-3 years of QMS implementation.
• Assign internal assessors who are independent of the activities being assessed.
  • Train assessors on auditing techniques, quality system structure, and communication skills.
• Utilize the AABB Assessment Tools as a guide for conducting internal assessments.
• Document and maintain records of all internal assessments, including findings and corrective actions.
• Review and update SOPs to reflect the internal assessment process, ensuring alignment with AABB standards.
• Implement a “life of a product” assessment approach to evaluate end-to-end processes, from patient scheduling to product disposition.

8.2 External Assessments

[Cord blood applicable]

The organization shall participate in an external assessment program applicable to the activities performed in the organization.

Guidance

The facility must ensure that external assessments are obtained at appropriately defined intervals. External assessments may be provided by organizations such as AABB, The Joint Commission, and the College of American Pathologists. The results of the external assessments should be reviewed by executive management. (SS)

☑ Actionable Checklist

• Identify and enroll in an external assessment program relevant to the facility’s activities (e.g., AABB, The Joint Commission).
• Schedule external assessments at defined intervals as per the program requirements.
• Maintain a log of all scheduled and completed external assessments, including dates and outcomes.
• Review results of external assessments with executive management within 30 days of receipt.
  • Document the review meeting and any action items in meeting minutes.
• Develop and implement corrective action plans for any deficiencies identified during external assessments.
• Update SOPs and training materials based on findings from external assessments, as necessary.

8.3 Management of Assessment Results

[Cord blood applicable]

The results of assessments shall be:

1. Reviewed by the personnel having responsibility for the area assessed.
2. Evaluated to determine the need for corrective and preventive action.
3. Communicated to the appropriate staff.
4. Reported to executive management.

Guidance

The results of all assessments are required to be brought to the attention of executive management for review and sign-off and to individuals who have responsibility for the activity. Management personnel are responsible for implementing timely corrective and preventive actions for each identified nonconformance, as required in Chapter 9, Process Improvement. (SS)

☑ Actionable Checklist

• Review assessment results with personnel responsible for the assessed area.
  • Schedule review meetings with relevant staff.
  • Document meeting minutes and action items.
• Evaluate assessment results to determine the need for corrective and preventive actions (CAPAs).
  • Use CAPA forms to document findings and actions.
  • Update the CAPA log with new entries.
• Communicate assessment results to appropriate staff.
  • Distribute a summary report to all relevant team members.
  • Record acknowledgment of receipt by staff.
• Report assessment results to executive management.
  • Prepare a summary report for executive review.
  • Obtain executive sign-off on the assessment report.
• Implement corrective and preventive actions as identified.
  • Assign responsibility for each action item.
  • Track progress and completion in the CAPA log.

8.4 Quality Monitoring

The organization shall collect and evaluate quality indicator data on a scheduled basis, including adverse events.

Guidance

Executive management should define quality indicators on an ongoing basis. Data should be collected to support that quality indicators are met. Data should be collected and provided to executive management to ensure that the quality indicators are met. Periodic review of the data should be shared with the applicable quality review committee.

Examples of data that can be used as quality indicators for facility services may include, but are not limited to:

• Customer feedback.
• Audits.
• Deviations.
• Exceptions.
• Personnel training.
• Evaluations of personnel competency.
• Equipment calibration and validation.
• Vendor qualification of supplies and reagents.
• Standard operating procedure review and evaluation.
• Turnaround time for reported results.
• Product traceability (including chain of custody). (SS)

The ​committee reviewed this comment but did not feel that a change was needed at this time. The committee noted that standard 5.9 already covers what is requested in the comment. (SC)

☑ Actionable Checklist

• Define and document quality indicators in collaboration with executive management.
  • Review and update the list of quality indicators annually.
• Schedule regular data collection for each quality indicator.
  • Use designated forms or logs to record data consistently.
• Evaluate collected data against defined quality indicators.
  • Document findings in a quality monitoring report.
• Present quality monitoring data to executive management quarterly.
• Share periodic review outcomes with the quality review committee.
• Implement corrective actions for any indicators not meeting targets.
  • Document all CAPAs in the CAPA log.

8.4.1

The organization shall provide data generated to the personnel who have responsibility for the quality indicator data collected.

Guidance

The committee added standard 8.4.1 based on updates to the AABB Quality System Essentials. (SC)

☑ Actionable Checklist

• Identify personnel responsible for quality indicator data collection and management.
  • Create a list of personnel with roles and responsibilities related to quality data.
• Develop an SOP for the distribution of quality indicator data to relevant personnel.
  • Include timelines and methods for data distribution in the SOP.
• Implement a system for tracking the distribution of quality indicator data.
  • Maintain a log or record of data distribution events.
• Conduct training sessions for personnel on interpreting and utilizing quality indicator data.
  • Document attendance and competency evaluations for training sessions.
• Schedule regular meetings to review quality indicator data with responsible personnel.
  • Maintain minutes and action items from these meetings.
• Include quality indicator data review in the agenda of management review meetings.
  • Document discussions and decisions related to quality indicators in meeting minutes.

8.5 Monitoring Clinical Activities

[Cord blood applicable]

Facilities performing clinical activities shall have a program that addresses, evaluates, and monitors patient care practices for all cellular therapies. The following shall be monitored:

1. Ordering practices.
2. Patient identification.
3. Sample collection and labeling.
4. Medication errors.
5. Near-miss events.
6. Adverse events.
7. Ability of services to meet patient needs.
8. Compliance with peer-reviewed recommendations.
9. Adherence to research protocols or investigator’s brochures, if applicable.
10. Critical process points (eg, hand-offs, confirmation of patient identification before medical intervention) for conformance with policies, processes, procedures, and protocols.

Guidance

Executive management should define quality objectives on an ongoing basis. Data should be collected to support that the quality objectives are met. Quality indicator data should be collected and provided to executive management to ensure that the quality objectives are met. Periodic review of the data should be shared with the applicable quality review committee. (SS)

Excerpt of Reference Standard 6.2.9A Relevant to Internal and External Assessments

Standard	Record to Be Maintained	Quality System Records	Donor Eligibility/Management Issues	Unit/Recipient	Retention after Creation (C) or Final Disposition (F) of Related Product	Minimum Retention Time (in years)1
8.1	Internal assessments	X	X	X	C	10
8.2	External assessments	X	X	X	C	10
8.3	Management of assessment results	X	X	X	C	10
8.5	Monitoring of clinical activities	X	X	X	C	10

¹Applicable federal, state, or local law may exceed this period.

☑ Actionable Checklist

• Develop and implement an SOP for monitoring clinical activities, including ordering practices, patient identification, and sample collection.
  • Ensure SOP includes procedures for tracking medication errors, near-miss events, and adverse events.
• Establish a system for documenting and reviewing the ability of services to meet patient needs.
  • Include compliance with peer-reviewed recommendations and adherence to research protocols.
• Create a checklist for critical process points, such as hand-offs and patient identification confirmation, to ensure conformance with policies and procedures.
• Conduct regular audits to evaluate adherence to the monitoring program and document findings in quality system records.
• Maintain records of all monitoring activities for a minimum of 10 years, ensuring compliance with applicable laws.
• Schedule periodic reviews of quality indicator data with executive management and the quality review committee to ensure quality objectives are met.