Unintended Consequences and Why Documentation Matters

Add bookmark

How many times have you reached the end of a quality or process improvement project only to be unsure of what has actually been accomplished? Here’s how better documentation can help, explain contributors John Moran and Elizabeth Pierson, and why it can also aid in better anticipating the unintended consequences that can be caused by process change.

In process improvement, it is not unusual to get so caught up in the act of problem solving that we forget to accurately document what we did, when we did it, and what it accomplished. A quality improvement team moves quickly on its quest to gather data and solve problems but can easily delay recording what the interventions actually accomplished. It is difficult to recall or recreate history since people involved in the project usually have sketchy and conflicting memories of what was done, when it was done and the associated impact.

The authors recommend that as soon as you have developed the AIM statement (a description of the quality improvement project a team is undertaking) you develop a process by which you can document what a quality improvement team changes. The following form can be developed in table format or excel and makes an easy way to document interventions and their results. Including the AIM statement at the top of the form should keep the quality improvement team focused on what is to be accomplished. All interventions should have an impact on the AIM statement; if not, they should be questioned as to why they are being implemented.

Creating An Intervention and Impact Form

AIM Statement Description:

1.

2.

3.

4.

5.

6.

7.

8.

Intervention Number

Date

What Was The Change?

How Did It Impact The AIM?

How Did Your Thinking Change?

How Did It Impact Your Procedures?

How Did It Impact Your Customer?

How Do You Know?

Measures

AIM Statement: A description of the quality improvement project a team is undertaking, the baseline measure they are starting from, and the goal they desire to achieve.

Example: : The XYZ Public Health Department quality improvement team will identify strategies to improve immunization rates for children in the target population (2-24 months) served in the Central and South clinics, demonstrated by an up to date rate of 90% for the fourth DTaP by June 20xx. The current up to date rate is 50%.

Column 1: Intervention number is the tracking identifier attached to each intervention starting at number 1.

Column 2: Date the intervention was made to the process, form, etc.

Example: The first intervention was made on March 20xx

Column 3: Detailed description of the change that was made. Detail how the change was made, where the change was made, who was involved with the change, and any problems encountered with making the change.

Example: Added reminder telephone call prior to scheduled appointments (this call is made the day before, & calls are made on Friday for Monday clinics).Clients also receive a post card the week before their scheduled appointments. Nurses in the clinics will assume responsibility for calls and postcards.

Column 4: Describe what impact the intervention had on the AIM Statement.

Example: This change was implemented to reduce the # of "No Shows" rate for scheduled appointments. Reducing the "No Shows" increased the probability of getting the population under study in for appointments on time and having their shots when required.

Column 5: Describe how this intervention may have changed the team’s thinking about the AIM Statement or the project in general.

Example: This change was relatively easy to implement. It was one of those low hanging fruit from our cause analysis and we were not sure why we had not introduced it sooner.

Column 6: Describe how the intervention impacted any current procedures in place.

Example: As the "no show" rate decreased the clinic had less room on the schedule to accommodate "walk ins."

Column 7: Describe how the intervention impacted the internal and external customers and where the intervention was made.

Example: The clinic now had less wiggle room and the staff concern was that clients who walked in for shots (no appointment) may get turned away and this would be a missed opportunity to bring infants up to date who may be behind on shots. This would impact both the drop off rate and overall immunization rates, which we are trying to increase.

Column 8: Detail out the quantitative or qualitative measures used to measure the impact and indicate the baseline of where the process was before the intervention was initiated.

Example: There was a decrease of 36% in the "No Show" rate in a one month time period. This [adoption of a different clinic schedule to accommodate more walk-in clients] change was implemented and will remain a part of the clinic protocol.

Dealing With Unintended Consequences

Often "Unintended Consequences" arise from some of the interventions. Many could not be foreseen since they are a result of the interaction of the intervention with the process where it is being implemented. Unintended consequences happen frequently in quality improvement projects and these need to be tracked along with the interventions. Some of these unintended consequences may result in the quality improvement team developing a sub-AIM statement which will also have to be tracked and monitored. The following columns can be added to the Intervention and Impact Form when needed to track the impact of unintended consequences.

9

10

11

12

13

14

15

Unintended Consequence Letter

Unintended Consequence Description

Date It Happened

Impact To Aim Statement

Need a Sub AIM Statement?

Impact to Customer

Modifications Made

Sub Aim Statements:

Related to Unintended Consequence

Column 9: The Unintended Consequence letter is the tracking identifier attached to each Unintended Consequence and it is best to use a letter starting at A so as not to confuse them with the Intervention numbers.

Column 10: Describe how the unintended consequence came about and its impact on the AIM statement.

Example: As the "no show" rate continued to decrease there was less room on the schedule to accommodate "walk ins."

Column 11: Date the Unintended Consequence was first observed. It is important to understand when this unintended consequence was first noticed to determine how long after the intervention this started to happen. It is a measure of the lag time and something to understand and watch as other interventions are made in the future.

Example: The lack of capacity to accommodate walk ins was first noticed two weeks after the intervention was put in place.

Column 12: Describe the unintended consequence’s impact on the original AIM statement.

Example: The Unintended Consequence with walk in clients was that clients who walked in for shots (no appointment) could get turned away and this would be a missed opportunity to bring infants up to date that they may have with them who may be behind on shots, this would reflect in the overall immunization rates

Column 13: Describe if a sub AIM Statement was required and what it was. The sub AIM statement should be directly tied to the original AIM statement.

Example: A sub improvement goal was developed for walk ins. A goal of "99% of all walk-in clients will be seen within one hour of presenting themselves without causing disruption of clinic flow, excluding flu or travel clients and also if this was not possible then they were scheduled to another time or day," was set by the quality improvement team.

Column 14: Describe the impact to the internal and external customer and where it is occurring.

Example: Since the "No Show" rate decreased dramatically the clinic staff now had less wiggle room with walk in clients to fit them into the schedule and this put some stress on the staff to accommodate those not expected. The client would be upset if they could not get service and had to come back another day. This increased the likelihood they may not return. It is best to provide the immunization service when the client presents with or without an appointment.

Column 15: Describe modifications that had to be made to overcome the Unintended Consequence from the intervention.

Example: The clinics had to juggle staff and clinic hours to accommodate the changes that were made. Support staff would keep a log of how many walk in clients presented to clinic and how many of them were seen and how many were turned away. Extending clinic hours enabled nurses to accommodate 6 extra clients, and a backup nurse was available most clinic days to help as needed.

It is easier to document in real time then to recreate history. Besides being a more accurate description of what has happened it also gives the quality improvement team a vehicle to start making predictions as to what Unintended Consequences may happen. Being prepared for Unintended Consequences makes the process of dealing with them easier and quicker.

Applications/Other Uses of Documentation

This tracking mechanism arose from a need to effectively and comprehensively document process changes and impact over the long-term. It has turned into an efficient method for tracking any changes and potential effects—serving as an awareness tool, a way to transfer knowledge and educate staff about quality improvement and programs. It can be used for day to day programming or outcome evaluation. There are multiple uses for this tool, and because of its simple table format, it can easily be adapted for other purposes.

When used appropriately and especially in conjunction with other quality improvement tools, the document can provide a clear history of change within a program or department and serve as a guide for the future.

[eventPDF]


Authors’ note: Documenting the impact of Quality Improvement interventions discussed in this article was developed as part of the Robert Wood Johnson Foundation Evaluation grant that was received by Franklin, Logan-Hocking, and Summit Counties in Ohio. The document "the impact of Quality Improvement interventions" was developed and utilized to keep track of the interventions that each of the three counties undertook to achieve their AIM statements. Participants from Logan-Hocking Health Department included: Doug Fisher, Kelly Taulbee, Jamie Funk, Lisa Castle. Participants from Summit County Heatlh District were: Gene Nixon, Gillian Solem, Anne Morse, Sheila Capone, Wendy Brolly, Leanne Beavers. Participants from Franklin County Public Health were: Milu Nguyen , Paula Mieseler, Katherine Suchy, Beth Pierson, Susan Tilgner. Susan Tilgner was the principle investigator and Beth Pierson served as project coordinator.


RECOMMENDED