About Performance Measures


The ultimate goal of EMS is to provide effective, high-quality healthcare to patients. But without performance measures, EMS providers, executives, and regulators have no way of knowing if they’re achieving those goals.

A consistent set of measures will allow EMS caregivers, managers, and government agencies to monitor performance and improve performance to ensure the best care for patients and the best service to the community.

Performance measures are critical for individuals, organizations, and the community—without them it is difficult to recognize:

Whether performance is improving or deteriorating

If an implemented change has had the intended impact

Whether the cost of a program or piece of equipment is worth the expense

In healthcare, the use of performance measures is not new, but their use has expanded tremendously in the last two decades: by hospitals to improve systems of care, by payers to determine reimbursement rates, and by communities to hold healthcare providers accountable. Having validated, patient-centered performance measures created for the EMS community, by the EMS community, is vital to ensuring that EMS is ready for the future healthcare environment.

History

The evolution of performance measurement in healthcare.

LEARN MORE

Resources

Additional information on performance measurement.

LEARN MORE

Using Available EMS Data

The ability to collect and manage data has spurred the increase in performance measurement. Never before have prehospital providers collected so much information in such a usable way, with electronic patient care reports and computer-aided dispatch systems creating vast databases of clinical and operational information.

The success of the National EMS Information System (NEMSIS) has led to standardized data collection in EMS systems throughout the country. But the next step is to use that data in a standardized way—and that’s what EMS Compass will make possible.

Members of the EMS community, from an individual ambulance service to local, state, and federal agencies, struggle with how to use these enormous quantities of data in a meaningful way. This is where performance measures play a role. For example:

an EMS quality improvement officer in a small city in Ohio can track how well his agency compares to the national average with a clinical measure on the treatment of patients with signs and symptoms of a heart attack;

an EMS director in rural Montana, knows whether a recent quality improvement and training initiative to reduce the number of medication errors has been successful, and how her agency compares to other rural systems;

and an EMS chief in a large city in Texas, who wants to compare billing rates to previous years, can use a standard performance measure to ensure the numbers are calculated the same way every year, and it stays consistent even after she retires.


How a Performance Measure is Created

EMS Compass established a system for creating performance measures based on the principles established by the National Quality Forum (NQF). The NQF is a highly regarded non-profit organization that works to improve healthcare by endorsing performance measures through a consensus-building, evidence-based process.

Through an initial month-long “Call for Measures,” members of the public submitted more than 400 measures to EMS Compass. With input from stakeholders, the EMS Compass leadership prioritized the measures for development. The EMS Compass Measurement Design Group worked to ensure the measures were designed in a way that was feasible to use and consistent with the best science and evidence available.

In order to design measures that were consistent with data being collected by EMS systems today, the Technology Developer Group reviewed the measures and the measures were tested in the field using actual data from EMS agencies. After testing and validation, the measures were approved by the EMS Compass Steering Committee.

The goal of EMS Compass is to establish an ongoing system of measure design, evaluation, implementation, and re-evaluation that lasts well beyond the initial two-year timeline of the initiative.

FAQ

What is a Performance Measure?
What is a performance measure?

A performance measure is an indicator that helps an organization know how well it is achieving its goals. Performance measures are specific and quantifiable, measuring both processes and results so organizations can see trends in performance.

What is the difference between an evidence-based guideline, a performance measure and a benchmark?

An evidence-based guideline is developed to provide guidance on how to treat a specific condition or situation based on the best available clinical research. For example, an evidence-based guideline on prehospital treatment of status seizures might recommend rapid administration of IV benzodiazepines, if that was indicated by the available published evidence. A performance measure assesses whether or not a specific process or outcome is being achieved. Using the example from above, one might establish a performance measure that asks what percentage of the time a benzodiazepine is administered to patients experiencing status seizures. A benchmark is an established goal or comparison used when assessing the results of performance measurement. An organization might establish 90% as its internal benchmark for the performance measure example above and then examine on a regular basis whether it is achieving that goal. Benchmarks can also be established by comparing an organization to other similar ones or to local or national averages.

What are the different types of performance measures?

While there are many ways to categorize performance measures, they are often divided into four categories: structural, process, outcome, and balancing measures. A structural measure evaluates infrastructure, such as percentage of ambulances staffed with advanced life support personnel. A process measure examines a process that is tied to a good outcome, but is not the outcome itself. Many clinical performance measures are process measures, such as percentage of chest pain patients who received a 12-lead ECG or aspirin. Sometimes, several process measures are examined together, to assess the provision of what is known as a “care bundle.” For example, a STEMI care bundle might include a 12-lead, aspirin, pain assessment, nitroglycerine, and analgesia. While each individual step in the bundle would be measured, the entire bundle itself would also be measured, to ensure each step was being performed on each individual patient. An outcome measure assesses the status of the ultimate goal, typically the patient’s status. The most commonly used outcome measure in EMS is survival-to-discharge with good neurological status for cardiac arrest patients. A balancing measure is used to ensure that trying to achieve one performance measure does not lead to other, unintended consequences. For example, a system that is focusing on managing patients’ pain and measuring change in pain scales during treatment and transport may want to also measure use of naloxone, incidents of respiratory depression, or other signs that may indicate a negative impact of managing pain with narcotics.

Are all performance measures clinical?

No. While it is important to develop clinical measures to help EMS providers provide the highest-quality care to patients, other types of measures are equally important. Operational measures will help agencies be more efficient. Safety measures will help track both patient and provider safety in order to reduce illness and injury. EMS Compass is using ten domains of measures, some of which are clinical but many of which are not.

Don’t we already have EMS performance measures?

Yes. There have been frequent and successful attempts to develop lists of performance measures, at local, state, and even national levels. Learn more about previous efforts to develop performance measures.  However, EMS Compass is the largest national effort ever undertaken to develop EMS performance measures and the first in more than a decade. EMS Compass also is working to create a process for performance measure development and evaluation that will be continuous, so that measures are frequently being reassessed and redesigned to meet the needs of the EMS community.

How does this help me?
How do performance measures work in the field?

EMS performance measures are used by EMS systems—including local agencies—to assess performance in many areas. The EMS Compass measures are being designed so any EMS system can use the data it already collects to evaluate how well it is achieving certain goals. While local agencies can determine how to use the measures, there are many resources available that describe the science of performance improvement and how to use measures as part of a quality improvement program.

Will EMS providers have to use the EMS Compass performance measures?

The EMS Compass initiative is working to create performance measures that will be able to be used by any EMS system in the country in order to help it assess and improve performance. But EMS Compass has no plans (and does not have the authority) to make the use of the measures mandatory.

Each EMS agency is different—are these measures relevant to all of them?

As much as possible, the EMS Compass team is trying to develop measures that are relevant to prehospital providers everywhere. The performance measures are being designed to assess things that patients and communities would value whether they live in rural or urban environments or are served by fire departments or private ambulances or paid or volunteer personnel. While some measures may be more relevant to some agencies than to others, the goal of EMS Compass is to make its measures useful and usable by all EMS agencies.

Is EMS Compass writing clinical protocols or changing how EMS agencies treat patients?

No, EMS Compass is not recommending specific clinical protocols. However, many of the EMS Compass performance measures will be assessing specific clinical processes and outcomes. These measures will be based on the best available evidence, but EMS practitioners should always follow their local protocols and medical direction.

Do generic performance measures take into account individual treatment plans and clinical judgment?

Performance measures strive to be evidence-based. For example, one measure may look at whether a certain treatment plan or care bundle was followed for a specific patient presentation. However, if the plan was not followed because of a documented, clinically sound reason, that case is often excluded from the measurement. For example, measures examining whether or not a patient with chest pain received aspirin would exclude cases where aspirin was not administered due to a documented allergy.

How are the measures developed?
How are EMS Compass performance measures developed?

EMS Compass has established a process to ensure that measures are relevant, evidence-based, and usable. Measure ideas were proposed by members of the public during the Call for Measures. The EMS Compass team reviewed each submission and also looked at past efforts to design performance measures, both in EMS and other areas of healthcare, when developing a list of measures. The measures were then prioritized based on several factors, including relevance, ease of use, availability of data, and the evidence base. Once prioritized, measures were designed by the Measurement Design Group. The Evidence Review Group ensured that the measures followed the best available scientific evidence and best practices, while the Technology Developers Group assessed the ability to use the measure with current technology and data standards. The Steering Committee than reviewed each measure before approving it.

What sources of data will be needed to use the performance measures?

EMS Compass was tasked to develop measures using NEMSIS data elements whenever possible, so the data should be available from an agency’s patient care reports and dispatch records. However, EMS Compass is also considering measures that may require outside data elements, such as outcomes from hospitals. Additionally, for domains such as EMS Workforce or EMS Fleet, the data may come from internal agency records or may not be readily available for some systems. These EMS Compass measures may help agencies start collecting data in a consistent and evidence-based manner in order to begin measuring and improving in these non-clinical but still vital domains.

What is a “measurement domain”?

The National Quality Forum categorizes measures using six domains: clinical process/effectiveness, patient and family engagement, patient safety, care coordination, population/public health, and efficient use of healthcare resources. These domains were based on the priorities set forth by the U.S. Department of Health and Human Services in its National Quality Strategy. EMS Compass added four other categories to address non-clinical aspects of performance: workforce, fleet, data, and finance. These ten domains are a way of sorting the measures and ensuring that all aspects of the EMS system are being assessed.

Performance measures on the EMS Compass website have a “status.” What does that mean?

The status of the measure indicates where in the EMS Compass process that measure currently is. There are four possible statuses for measures:

Proposed – These measures have been submitted by the public for consideration. The EMS Compass leadership team reviews each submission and assigns a priority using criteria based on the process established by the National Quality Forum. Highest priority measures will be moved to the next stage of the development process. You may submit additional measures (link to submit measures) for future consideration.

Public Comment – Measures in this stage are open for public comment on all aspects of the measure, including its appropriateness, feasibility and usefulness. EMS agencies are encouraged to use their own data to test these measures and provide feedback on the feasibility and utility of the measure.

Under Review – These measures have already been through the public comment period and are undergoing further testing and review by the EMS Compass working groups prior to consideration by the Steering Committee.

Approved – Approved measures have been thoroughly vetted and tested by the public and the EMS Compass team and have been approved by the EMS Compass Steering Committee.

Will the measures be tested before being approved?

The EMS Compass team encourages any EMS agency or organization to test measures during the public comment period and provide feedback on the ability to accurately define and compute the measures. In addition, the Technology Developers Group will be assessing the ability to use the measure with NEMSIS compliant patient care records.

How will the measures be used?
Is my agency’s performance on these measures going to be made public?

The EMS Compass initiative is focused on designing evidence-based measures that can be used to drive performance improvement across the EMS industry. How the measures are used—and whether the results are publicized—is up to individual agencies, communities, and state and local regulators.

Will these performance measures be used to determine reimbursement?

EMS Compass is not connected to any payor, such as private insurers or Medicare and Medicaid. The goal of EMS Compass is to create measures that lead to higher quality, safer patient care and more efficient and effective use of healthcare resources. However, EMS Compass also provides a chance for the EMS community to come together and create sound, valid measures, so that if and when payor tie reimbursement to quality measures, those measures already exist and were created by EMS providers for EMS providers.

Will regulators use the performance measures to rank or punish low performers?

There is no intent on the part of the EMS Compass initiative to use the measures to rank agencies or regions, or to punish those that do not meet certain levels of performance. The goal is to provide a method for EMS organizations to measure performance and find areas of improvement and to benchmark themselves against others.

How does EMS Compass work?
How can I get involved in EMS Compass?

There are several ways members of the EMS community and other interested stakeholders can contribute to EMS Compass. You can review proposed measures and submit comments. You can test those measures in your agency using real data and let the EMS Compass team know how it went and whether the measures need to be adjusted. If you want to be kept informed about other opportunities to participate and when measures will be available for public comment, sign up to receive updates.  If you have specific questions about how you can help, contact the EMS Compass Project Manager.

Who is in charge of the development of the EMS Compass measures?

EMS Compass is managed by the National Association of State EMS Officials (NASEMSO) through a cooperative agreement with the National Highway and Traffic Safety Administration (NHTSA) Office of EMS. NASEMSO worked closely with NHTSA and stakeholders across EMS to create a steering committee and work groups for the initiative who would represent the diverse EMS Community. The steering committee and its work groups are leading the development of the measures, while NASEMSO remains ultimately accountable for the success of the initiative.

Who is paying for EMS Compass?

EMS Compass is funded, through a cooperative agreement, by the National Highway Traffic and Safety Administration (NHTSA) Office of EMS. EMS Compass also relies heavily on the generosity of the many volunteer members of the steering committee and work groups who have given countless hours of time to make the initiative successful.

What is the relationship between EMS Compass and the National Quality Forum?

The National Quality Forum (NQF) is a non-profit organization that “works to catalyze improvements in healthcare,” mostly through collaborative efforts to facilitate performance improvement and endorse performance measures. EMS Compass has based many of its processes on those already established by NQF, and has reached out to NQF leadership to ensure that the EMS Compass measurement development process meets NQF standards.

Get the Latest Updates

Sign up here for more information and updates about EMS performance measures.


Fatal error: Call to undefined function mailchimpSF_signup_form() in C:\inetpub\wwwroot\wp-sqlsrv\EMSCompass\wp-content\themes\Florida_child_theme\footer.php on line 42