Citation
Medication Use Performance Indicator Evaluation: A Systems Perspective

Material Information

Title:
Medication Use Performance Indicator Evaluation: A Systems Perspective
Creator:
SAUER, BRIAN C. ( Author, Primary )
Copyright Date:
2008

Subjects

Subjects / Keywords:
Databases ( jstor )
Drug evaluation ( jstor )
Drug interactions ( jstor )
Drug therapy ( jstor )
Educational evaluation ( jstor )
Health care industry ( jstor )
Health care organizations ( jstor )
Hospital admissions ( jstor )
Medications ( jstor )
Pharmacies ( jstor )

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
Copyright Brian C. Sauer. Permission granted to University of Florida to digitize and display this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
Embargo Date:
8/7/2004
Resource Identifier:
72696035 ( OCLC )

Downloads

This item has the following downloads:

sauer_b ( .pdf )

sauer_b_Page_162.txt

sauer_b_Page_127.txt

sauer_b_Page_099.txt

sauer_b_Page_012.txt

sauer_b_Page_003.txt

sauer_b_Page_104.txt

sauer_b_Page_017.txt

sauer_b_Page_019.txt

sauer_b_Page_154.txt

sauer_b_Page_092.txt

sauer_b_Page_060.txt

sauer_b_Page_126.txt

sauer_b_Page_073.txt

sauer_b_Page_065.txt

sauer_b_Page_091.txt

sauer_b_Page_148.txt

sauer_b_Page_093.txt

sauer_b_Page_036.txt

sauer_b_Page_081.txt

sauer_b_Page_054.txt

sauer_b_Page_004.txt

sauer_b_Page_068.txt

sauer_b_Page_014.txt

sauer_b_Page_134.txt

sauer_b_Page_064.txt

sauer_b_Page_118.txt

sauer_b_Page_024.txt

sauer_b_Page_039.txt

sauer_b_Page_155.txt

sauer_b_Page_149.txt

sauer_b_Page_079.txt

sauer_b_Page_044.txt

sauer_b_Page_030.txt

sauer_b_Page_062.txt

sauer_b_Page_056.txt

sauer_b_Page_048.txt

sauer_b_Page_025.txt

sauer_b_Page_002.txt

sauer_b_Page_078.txt

sauer_b_Page_046.txt

sauer_b_Page_028.txt

sauer_b_Page_158.txt

sauer_b_Page_076.txt

sauer_b_Page_018.txt

sauer_b_Page_063.txt

sauer_b_Page_129.txt

sauer_b_Page_137.txt

sauer_b_Page_113.txt

sauer_b_Page_043.txt

sauer_b_Page_096.txt

sauer_b_Page_109.txt

sauer_b_Page_146.txt

sauer_b_Page_135.txt

sauer_b_Page_143.txt

sauer_b_Page_153.txt

sauer_b_Page_009.txt

sauer_b_Page_037.txt

sauer_b_Page_163.txt

sauer_b_Page_122.txt

sauer_b_Page_010.txt

sauer_b_Page_080.txt

sauer_b_Page_108.txt

sauer_b_Page_145.txt

sauer_b_Page_088.txt

sauer_b_Page_008.txt

sauer_b_Page_053.txt

sauer_b_Page_144.txt

sauer_b_Page_045.txt

sauer_b_Page_114.txt

sauer_b_Page_124.txt

sauer_b_Page_098.txt

sauer_b_Page_011.txt

sauer_b_Page_150.txt

sauer_b_Page_170.txt

sauer_b_Page_117.txt

sauer_b_Page_123.txt

sauer_b_Page_131.txt

sauer_b_Page_152.txt

sauer_b_Page_049.txt

sauer_b_Page_111.txt

sauer_b_Page_022.txt

sauer_b_Page_067.txt

sauer_b_Page_156.txt

sauer_b_Page_130.txt

sauer_b_Page_167.txt

sauer_b_Page_116.txt

sauer_b_Page_050.txt

sauer_b_Page_033.txt

sauer_b_Page_139.txt

sauer_b_pdf.txt

sauer_b_Page_151.txt

sauer_b_Page_016.txt

sauer_b_Page_147.txt

sauer_b_Page_006.txt

sauer_b_Page_047.txt

sauer_b_Page_077.txt

sauer_b_Page_121.txt

sauer_b_Page_041.txt

sauer_b_Page_087.txt

sauer_b_Page_165.txt

sauer_b_Page_168.txt

sauer_b_Page_058.txt

sauer_b_Page_066.txt

sauer_b_Page_169.txt

sauer_b_Page_082.txt

sauer_b_Page_029.txt

sauer_b_Page_105.txt

sauer_b_Page_142.txt

sauer_b_Page_106.txt

sauer_b_Page_005.txt

sauer_b_Page_020.txt

sauer_b_Page_083.txt

sauer_b_Page_128.txt

sauer_b_Page_112.txt

sauer_b_Page_051.txt

sauer_b_Page_061.txt

sauer_b_Page_075.txt

sauer_b_Page_007.txt

sauer_b_Page_097.txt

sauer_b_Page_136.txt

sauer_b_Page_042.txt

sauer_b_Page_034.txt

sauer_b_Page_070.txt

sauer_b_Page_157.txt

sauer_b_Page_164.txt

sauer_b_Page_089.txt

sauer_b_Page_160.txt

sauer_b_Page_103.txt

sauer_b_Page_138.txt

sauer_b_Page_015.txt

sauer_b_Page_021.txt

sauer_b_Page_057.txt

sauer_b_Page_119.txt

sauer_b_Page_038.txt

sauer_b_Page_120.txt

sauer_b_Page_172.txt

sauer_b_Page_086.txt

sauer_b_Page_001.txt

sauer_b_Page_040.txt

sauer_b_Page_072.txt

sauer_b_Page_090.txt

sauer_b_Page_102.txt

sauer_b_Page_140.txt

sauer_b_Page_115.txt

sauer_b_Page_095.txt

sauer_b_Page_032.txt

sauer_b_Page_069.txt

sauer_b_Page_171.txt

sauer_b_Page_100.txt

sauer_b_Page_059.txt

sauer_b_Page_132.txt

sauer_b_Page_074.txt

sauer_b_Page_071.txt

sauer_b_Page_027.txt

sauer_b_Page_052.txt

sauer_b_Page_084.txt

sauer_b_Page_133.txt

sauer_b_Page_094.txt

sauer_b_Page_101.txt

sauer_b_Page_110.txt

sauer_b_Page_125.txt

sauer_b_Page_023.txt

sauer_b_Page_035.txt

sauer_b_Page_161.txt

sauer_b_Page_031.txt

sauer_b_Page_166.txt

sauer_b_Page_159.txt

sauer_b_Page_013.txt

sauer_b_Page_055.txt

sauer_b_Page_107.txt

sauer_b_Page_026.txt

sauer_b_Page_141.txt

sauer_b_Page_085.txt


Full Text












MEDICATION USE PERFORMANCE INDICATOR EVALUATION:
A SYSTEMS PERSPECTIVE















By

BRIAN C. SAUER


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2004

































Copyright 2004

by

Brian C. Sauer



































To my parents















ACKNOWLEDGMENTS

I would like to acknowledge my dissertation chair, Dr. Charles D. Hepler, for

taking me under his wings and sharing his knowledge with me. I have grown

tremendously over the past five years and Dr. Hepler has guided that process.

I thank the members of my supervising committee, Drs. Earlene Lipowski,

Abraham Hartzema, Murray Cote, and Richard Segal for their patience and guidance

through the process. I would like to thank Josue Rodas, Becky Cherney and Scott

Langdon for providing the support needed to make this study happen. I thank the Perry

Foote foundation for providing the grant that financed this project. I thank Dave Angaran

for listening and helping me understand and think through the clinical complexity of my

dissertation.

I would also like to thank Dr. Alan Spector for giving me the opportunity to work

in his laboratory as an undergraduate in biological psychology. This was an important

time in my academic development. Like Dr. Spector, I hope I will always be able to roll-

up my sleeves and loosen my tie when the critics are throwing heat. I also would like to

thank Mircea Garcea for teaching me, through example, to always pay attention to the

details and produce quality work.

Finally, I would like to thank the graduate students for producing a healthy

working environment and their friendship.
















TABLE OF CONTENTS



A C K N O W L E D G M E N T S ......... .................................................................................... iv

LIST OF TA BLES ............. .... .............................. .. ................ ... viii

LIST OF FIGURES ............................... ... ...... ... ................. .x

ABSTRACT .............. .................. .......... .............. xi

CHAPTER

1 IN TR OD U CTION ............................................... .. ......................... ..

Problem Statem ent .................. .............................. .. ........................ ..
O bj ectiv e ...................................................................................... . 2
Specific A im s........................................................ 2
Ju stific a tio n ............................................................................... 2
O v e rv iew .................................................................................. 3

2 CONCEPTUAL FRAMEWORK ..................................................... 6

A diverse Outcom es of D rug Therapy .........................................................................6
The M education U se System .......................................................................8
Levels of the H health System .................................................................. 9
H um an Error and System Failure ....................................................... 14
S u m m a ry .......................................................................................................1 6

3 LITERATURE REVIEW .............................................................. ...............18

Use of Administrative Databases for Researching PDRM .........................................18
Automated Methods for Detecting PDRM .............................................................22
D elp h i M eth o d ................................................................................................2 9
Cause-and-Effect A analysis ............. ..................... ........................................ 35

4 PRELIM IN ARY W ORK ........................................................................... 39

Prevalence of Preventable Drug Related Admissions ...............................................39
Drug Categories Involved in PDRA ......................................................... ....... 43
Apparent/Proximate Causes of PDRAs......... ............. ...................... 48


v









L atent C au ses of PD R A ......................................... .. .. ............................ ............... 52

5 M E T H O D S ........................................................................................................... 5 5

S p ecific A im s........................................................5 5
Step O ne: D database A nalysis........................................................... ............... 55
C laim s D ata T ypes ................................ .... .. ......................... .. .. ............ 55
Descriptive Analysis to Evaluate the Integrity of Claims Data.........................56
Population D em graphic ...................... .............. ..................... .... ........... 56
MU-PI Coding Concepts and Analysis .................................... ............... 57
Search algorithm s .............................................................. .. .... .. .... .. 57
Disease-drug interaction...................... ..... ............................ 57
D rug m monitoring ........................................... .. ...... .. ... ........ .... 58
D rug-drug interaction ......................................................... ............... 60
Prevalence Estimates for MU-PI Positives............................... ...............60
Population Based Explanations for Prevalence Findings............................... 61
Step Two: Node Identification of PDRM Scenarios .............................63
P ilo t T e stin g .................................. ............ ....... ............... ............... 6 3
Selection of PDRM Scenarios for Node Identification Study.............................64
D elp h i R ecruitm ent .................................................................. ....................64
N ode Identification Survey........................................... ........................... 65
Delphi Process ........................................... ............. ........ 65
N ode Identification A analysis: ............................................................ ............66
Selection of M U -PIs for Evaluation................................................................ 66
Step Three: Cause-and-Effect Analysis................................ ........................ 67
MU-PI Evaluation Team .......................................................................67
MU-PI Evaluation Process ............................ ...............67
Analysis for Common and Unique Cause Sequences ......................................70

6 R E S U L T S .......................................................................... 7 1

Step O ne. D database A analysis: ........................................................................ ... 7 1
Descriptive Analysis to Evaluate the Integrity of the Claims Database.............71
Check for missing and invalid data.............................................. ........72
L inks betw een data types ........................................ ......... ............... 72
Population D em graphics ............................................................................73
Prevalence of M U -PI Positives ................. ............... ....................................... 75
System Based Explanations for Prevalence Findings ......................................77
Distribution of m monitoring intervals.................................. ............... 78
Variables associated with PDRM positives ...............................................84
Step Two. Assignment of MU-PIs to Nodes of the MUS .......................................86
D elphi R ecruitm ent .................................................................... ....................86
Assignment of MU-PIs to Nodes of the MUS .................................................87
Step Three: Cause-and-Effect Analysis................................... ....................... 89
Evaluation of Monitoring Indicator .............. ..............................................91
Evaluation of Prescribing Indicator................................................................ 92
Cause Sequences Common to both the Monitoring and Prescribing Nodes.......94









7 DISCUSSION .................. ................................... ........... ............... 97

D atab ase A n aly sis............ ... ................................................................ ........ .. .... .. 97
Prevalence Findings ........................ ........ ... ............. .... ............. 97
System Explanations for Prevalence Findings .......................................... 99
Distribution of m monitoring intervals.................................. ............... 99
M multiple logistic regression analysis ....................................................... 100
N ode Identification of M U -PIs ....................................................... .... ........... 102
C ause-and-Effect A analysis ............................................... ............................ 103
C au se them es ................................................................. ............... 104
Conclusion ..................................... ................................. ......... 106
Lim stations ..................................... .................................... ......... 108
Significance and Theoretical Contribution...................... ...................... 109
C contribution to H health C are............................................................................... ... .... 110
Future A areas for Study .................. ..................................... .............. .. 111
In te rv e n tio n s ................................................................................................ 1 1 1
M U -PI Instrum ent Fidelity ...................................................... ..... .......... 113

APPENDIX

A MEDICATION INVOLVED IN DRA BY STUDY ...............................................115

B CLASSIFICATION OF DTPS INTO NODES OF THE MUS.............................116

C SUMMARY: CLASSIFICATION OF DTPS INTO NODES OF THE MUS......... 118

D MEDICATION USE PERFORMANCE INDICATOR DEFINITIONS..............119

E PROCEDURE CODES TO IDENTIFIY VISITS ...........................................126

F CODING SOLUTION FOR INDICATOR.............................................................127

G RECRUITMENT LETTER FOR NODE IDENTIFICATION STUDY.................. 128

H NODE IDENTIFICATION SURVEY ....................................................................130

I EVALUATION TEAM BRIEFING................................ 141

J M U-PI RESULTS .................. ...................................... .......... .. ..147

K SURVEY FOR COMMONALITY ................................... ....................................149

L NODE IDENTIFICATION BOX PLOTS.....................................................150

L IST O F R E FE R E N C E S ....................................................................... .................... 153

BIOGRAPHICAL SKETCH ............................................................. ............... 159
















LIST OF TABLES


Table page

3.1 Examples of DRM Screens by Type............. ..................... 25

3.2 Results of Leading DRM Screens ..................................... ............ ....... ........ 25

3.3 R results of D R M Screens ................................................ .............................. 26

3.4 R results of Top Five Indicators ........................................ ........................... 30

3.5 Percent Change in Average Group Error by Number of Group Members ..............32

4.1 Studies R reporting D R A and PD RA : ........................................................ .........44

4.2 Prevalence Estimates and Odds Ratio ........................................ ............... 45

4.3 L atent C auses of PD R A s............................................................... ....................54

6.1 Population Age and Gender Frequencies by Type of Claim................................74

6.2 Population D em graphics ............................................... ............................. 76

6.3 Frequency of Process and PDRM Positives .............................77

6.4 Number of PDRM Positives by Number of Indicators ................. ...................77

6.5 Eleven Most Prevalent PDRM Positives ................................. 78

6.6 Ten M ost Prevalent Process Positives................................................................... 78

6.7 MU-PI Categorized by Required Monitoring Interval ....................................79

6.8 Bivariate Tests of Association: PDRM Positives as Dependent Variable ..............85

6.9 Multicollinearity Assessment of Independent Variables ........................................86

6.10 Maximum likelihood Estimates: PDRM Positives as Dependent Variable ............86

6.11 Demographics of Delphi Panelists for Node Identification Study .........................87

6.12 Indicators Listed by Associated Node of the MUS...............................................89









6.13 Kruskal Wallis and Pairwise Comparisons for Indicator Assignment...................90

6.14 Monitoring Indicator: Affinity Table and Rating of Importance .............................92

6.15 Cause Sequence Agreem ent Results ............................................. ............... 93

6.16 Prescribing Indicator: Affinity Table and Rating of Importance .............................94
















LIST OF FIGURES


Figurege

2.1 M odel of the M education Use System ..................................... ........ ............... 14

2.2 Hierarchal Relationships among the System Levels..............................................15

2.3 Swiss Cheese Model of System Failure ...........................................................17

4.1 Involvement of Treatment Category to DRMs: .................................................49

4.2 Therapeutic Class Involvement in DRM:....................................................... 49

4.3 Nodes of the MUS Involved in PDRA..................................................................52

5.1 Disease-Drug Interaction Search Algorithm............................................................58

5.2 Process Search Algorithm for Indicators that Require Monitoring..........................59

5.3 Process and Outcome Search Algorithm...........................................................59

5.4 Example of a Process Positive but not a PDRM Positive .............................60

5.5 Drug-Drug Interaction Search Algorithm ..................................... .................60

5.6 Organization of Proposed Causes to System Levels .......................................69

6.1 O office V isits Claim s by M onth ........................................... ......................... 72

6.2 ED V isit Claim s by M onth................................ ........ ............... ............... 72

6.3 Facility Adm missions by M onth ..... ............................................. ............... 73

6.4 Pharm acy Claim s by M onth......................................... ............... ............... 73

6.5 First Lag for Indicator with One Month Monitoring Requirement........................80

6.6 First Lag for Indicator with Two Month Monitoring Requirement .........................81

6.7 First Lag for Indicator with Three Month Monitoring Requirement .......................82

6.8 First Lag for Indicator with Six Month Monitoring Requirement ...........................82















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

MEDICATION USE PERFORMANCE INDICATOR EVALUATION:
A SYSTEMS PERSPECTIVE
By

BRIAN C. SAUER

August, 2004

Chair: Charles Doug Hepler
Major Department: Pharmacy Health Care Administration

Background: Injury related to health care is a serious public health concern. The

prevalence of drug related admissions in the United States has been reported to be from

three to twelve percent of hospital admissions. Organizations interested in reducing drug

related morbidity need measurement techniques to gather baseline information about

preventable drug related morbidity (PDRMs), and methods to assess system related

causes to develop rational interventions that target the system failures

The objective of this dissertation is to better understand the relationship between

system design and patterns of care that can result in drug related injury.

Methods: This study was executed in three steps. In the first step, database

analysis, medication use performance indicators (MU-PI) were used to estimate the

number of PDRM positives in the managed care organization's administrative database.

Step two used the Delphi process to judge the degree of association between select MU-

PIs and specific nodes of the medication use system (MUS) where the process failure









may have originated. In step three, select MU-PIs from each node of the MUS were

submitted to cause-and-effect analysis (CEA).

Results: The period prevalence for process positives was 209.84 (206.37-213.34)

per 1,000 member years. The period prevalence for PDRM positives was 1.98 (1.6-2.4)

per 1,000 member years. Gender, number of office visits, number of drug classes and

number of medical conditions were independent risk factors for PDRM.

Fourteen indicators were selected for step two. Six of the fourteen indicators were

assigned to the prescribing node and seven were assigned to the monitoring node of the

MUS. One indicator did not reach significance within two rounds.

The cause-and-effect team identified twenty-nine cause sequences. They found

that twenty-three of the twenty-nine were common to both prescribing problems and

monitoring problems.

Conclusion: The MU-PIs proved to be a useful tool to identify possible cases of

PDRM and to initiate "system" thinking or organizational introspection to evaluate

system related causes for drug related injury. Four themes can be interpreted from

CEA: lack of necessary tools for adequate patient information and assessment, an

information system that can track patients and relay information to the providers,

pharmacist involvement in the MUS, and guideline adherence.














CHAPTER 1
INTRODUCTION

Problem Statement

Injury related to health care is a serious public health concern. The prevalence of

drug related admissions in the United States has been reported to be from three to twelve

percent of hospital admissions.1 The Institute of Medicine has proposed a fifty percent

reduction in errors by 2005.2 For this achievement in medications use, the following are

needed:

* Measurement techniques to gather baseline information about preventable drug
related morbidity (PDRMs)
* Assessment of system related causes and rational interventions that target the
system failures
* Follow-up measurements to gauge the effects of the interventions.

Automated data screening for PDRM has been proposed as a method to produce

baseline and longitudinal measurements. However, the underlying system problems that

contribute to PDRM in the ambulatory setting are largely unknown. Studies measuring

the prevalence of drug-related hospital admissions have identified nodes in the

medication use process where the drug therapy problems originated, i.e., prescribing,

monitoring, dispensing and patient adherence. The underlying system related causes of

node specific problems, however, have not systematically been evaluated. This research

will try to discover the system factors that contribute to node specific problems.









Objective

The objective of this dissertation is to better understand the relationship between

system design and patterns of care that can result in drug related morbidity.

Specific Aims

The specific aims of this study are as follows:
1. To establish the frequency of medication use performance indicator (MU-PI)
positives in the study population
2. To evaluate population based explanations for MU-PI positives
3. To identify the node of the medications use system (MUS) where indicator specific
patterns of care likely originated
4. To identify system related causes that are common among nodes of the MUS
5. To identify system related causes that are unique to nodes of the MUS

Justification

This study has both theoretical and practical implications. The theoretical

contribution of this study includes:

* Determining the node of the MUS where the pattern of care from specific MU-PIs
originated. This permits the evaluation of the indicators by processes rather than
medication or disease specific analysis.

* Providing information about how the various subsystems (patient, microsystem,
organization, and environment) within the health care enterprise interact to
influence the quality of medication use.

* Establishing system factors that contribute to node specific patterns of care or
process problems. Identifying system factors that appear to be node specific
problems will reveal leverage points for interventions that may span medication
classes and disease categories.

* Establishing system factors that contribute to patterns of care or process problems
from multiple nodes of the MUS. Identifying system factors that appear to be
common among the nodes of the MUS will reveal leverage points for interventions
that may span nodes of the MUS, medication classes and disease categories.

This study will demonstrate how the MU-PIs can be used to assess the quality of

medication use in a defined population. It will also demonstrate how the MU-PI findings









can be used to initiate activities that lead to the exploration of how system design

influences the quality of medication use, i.e., organizational introspection.

From a practical perspective, the health care purchasing group involved in this

study is interested in increasing the quality of care and decreasing costs for the purchasers

they represent. Understanding how system factors influence the quality of medication

use will help direct the development of interventions to reduce the prevalence of adverse

drug outcomes.

Overview

The health care industry is wasting billions of dollars3 and causing an unacceptable

amount of injury due to inappropriate medications use.1,2'4 Results from our meta-

analysis of fifteen studies indicated that approximately four percent of all hospital

admissions may be preventable.1 To place the reported prevalence estimate into

perspective, according to the National Hospital Discharge Survey, there were 31,827,000

admissions to U.S. hospitals in 1998. If the prevalence of preventable drug-related

hospital admissions (PDRAs) happened to have been four percent in 1998, there would

have been roughly 1.3 million PDRAs. This would have placed PDRAs among the top

causes of hospitalizations in the U.S. that year-above admissions related to congestive

heart failure (3.1%) and on par with pneumonia (4.2%).

Ambulatory health care is complex, with many opportunities for problems in drug

therapy to occur. The basic nodes in the MUS include prescribing, dispensing, (self)

administration, and monitoring. Of the fifteen studies analyzed in the above-mentioned

meta-analysis,1 eight described the types of drug therapy problems that led to PDRA.5-12









An analysis of these studies indicated that prescribing and monitoring problems were

highly associated with PDRA (median: 34% and 32%), respectively. Problems with

medications compliance (median: 22%) were also implicated as a main contributor to

PDRA.

The finding that prescribing and monitoring problems appeared to be the main

contributors to PDRA is disturbing because health care professionals are expected to

"first do no harm." Even though practitioners are expected to manage their patients

appropriately and "do it right the first time," the health care system should be designed in

a way that anticipates human error and does not allow it to progress to an adverse drug

event or PDRM. Ideally, the health care enterprise should be able to deliver care in a

highly reliable manner where performance problems and bad decisions are recognized

and corrected because adverse events occur. Results from the Institute of Medicine

report on medical errors2 and from our meta-analysis1 show that the American health care

system is currently far from highly reliable.

Before the rational development of interventions, a data stream must be established

for baseline and follow-up measurements, and underlying system related causes need to

be examined. A recent doctoral dissertation at the University of Florida used a Delphi

approach to validate forty nine MU-PIs; each paired an inappropriate process (violation

of a standard of care) with an adverse outcome.13 An example of these indicators

follows: Use of two or more NSAIDS concurrently for at least two weeks in patients 65

or older followed by gastritis and/or upper GI bleed.


a The analysis can be found in Chapter 4 (preliminary analysis)









The indicators were automated and used to screen an administrative database of a

large MCO in North Florida. The data set contained 11,711 patients, and 8.2% of them

screened positive for a PDRM. For further validation, the positive predictive value of

process to outcome (PTOV) was calculated to measure the strength of the association

between the inappropriate process and the corresponding adverse outcome for each

indicator. Ten of the nineteen indicators that had more than ten positive screens had

PTOV values greater that 0.74. This suggests that this instrument would be acceptable

for eliciting cases of PDRMs for system level analysis.

Cause-and-effect analysis is a retrospective method for identifying system failures,

and its popularity has largely advanced from its success in understanding industrial

accidents.14 Rooted in industrial psychology and human factors engineering, cause-and-

effect analysis has recently been adopted in the medical community to evaluate quality

problems.

The MU-PIs adopted from Faris (2001)13 will be used to screen the database of a

health care coalition in Florida. MU-PIs with high frequency will be selected and

assigned to the nodes of the MUS where the drug therapy problems appear to have

originated from. Cause-and-effect diagramming on selected MU-PI that represent

specific nodes of the MUS will be carried out to uncover system related problems that are

common among the nodes and unique to specific nodes of the MUS.














CHAPTER 2
CONCEPTUAL FRAMEWORK

The framework of this study was developed from Hepler's conceptualization of

PDRM and the MUS,15 Berwick's framework for the health care system16 and Reason's

accident theory.17 The premise is drug-related morbidity is largely preventable due to

failures in the management of drug-therapy. A model of the MUS is used to illustrate

how drug-therapy problems (DTPs) can develop into PDRM. Health care is a complex

system with many layers of embedded systems. A framework presented by Berwick16 is

used to simplify the complexity by referring to specific levels of the heath care system.

Finally, James Reason's accident theory 17 is used to discuss the types of failures and

conditions that affect the performance of the MUS and allow PDRM to occur.

Adverse Outcomes of Drug Therapy

The term drug-related morbidity (DRM) is used to discuss adverse outcomes of

drug therapy. DRM is a concept that includes (a) significant adverse effects of drug

therapy (e.g., hospital admissions or emergency department visits) (b) treatment failures

(i.e., occasions when drug therapy was attempted but did not achieve a realistic, intended

outcome in a reasonable time) and (c) occasions when a patient did not receive an

indicated or necessary drug therapy.18 It is important to appreciate the relationship

between the popular term adverse-drug event (ADE), which was used to describe adverse

outcomes of drug therapy in the IOM report "To Err is Human," and DRM. The term

ADE historically has been used to describe injury resulting from medical interventions

related to a drug (i.e., direct effects of the therapy),4'19-22 without recognition of indirect









injury resulting from sub-therapeutic dose or no drug while a valid indication is present.

Even though it does appear that the definition for an ADE has recently evolved to include

indirect injury, 23 the term DRM is used in this study because it has historically treated

indirect injury as a significant adverse outcome related to drug therapy.

A DTP that precedes a DRM can be classified into two types: potential DTPs and

actual DTPs. The first indicates a person has a theoretical problem associated with drug

therapy, and the latter signifies the problem has manifested (symptoms are present).15

Theoretical DTPs exist independently of individuals; they are the situations in care that

produce the risk for specific adverse effects. Potential DTPs occur when a theoretical

DTP is present in an individual, e.g., contra-indications, drug interactions and unjustified

violations of evidence-based medicine. Actual DTPs are denoted by observable or

patient reported symptoms. For example, concomitant prescribing of digoxin and

quinadine is a known theoretical DTP because this combination can potentially alter the

excretion of digoxin, thus increasing its serum concentration, which is especially

troubling because digoxin is known to have a very narrow therapeutic window.

Now imagine a patient sixty-five years old who is concurrently receiving digoxin

and quinidine. This patient has a potential DTP because a theoretical DTP is present in

his drug therapy. Now, suppose this patient begins to experience fatigue, weakness,

confusion, and diarrhea. Most likely he is experiencing an actual DTP because

predictable symptoms known to be associated with the theoretical DTP have manifested.

If he/she becomes hypokalemic and needs emergency care, and upon admission his/her

digoxin level is in toxic range, say > 50 mcg/kg, then he/she most likely would have

experienced a preventable DRM (PDRM).









A preventable DRM has the following attributes :1518

* The DRM was preceded by a recognizable DTP
* The DRM was reasonably foreseeable under the circumstance
* The cause of the DTP and the resulting DRM was identifiable
* The identified cause of the DTP and resulting DRM was controllable within the
context of therapy (i.e., without sacrificing essential therapeutic objectives).

In this case the DRM meets the four criteria for preventability because it was

preceded by a known drug interaction (potential DTP) and the manifest symptoms (actual

DTP) were commonly associated with digoxin toxicity. Under the circumstances

hospitalization was imminent because the symptoms represent the accumulation of

digoxin, the cause of the symptoms appears to be the result of too much digoxin, and

reducing the dose of digoxin and monitoring the patient's digoxin levels or switching to

an alternative therapy could have resolved the DTP and prevented the DRM. A DTP is

part of the process of care, it is a state of an individual in a medications use system, and it

is a possible precursor to a system failure, i.e., DRM.

The Medication Use System

For a DRM to be preventable a failure in the process of care must have occurred.

This becomes clearer when considering the typical sequence of actions that comprise the

MUS in the ambulatory care setting. Figure 2.1 below was adopted from Grainger-

Rousseau et al. and it illustrates the nodes in the medications use system.24 The episode

of care begins when the patient notices a problem and seeks professional medical

attention. Typically, the initiator (physician, physician's assistant or nurse practitioner)

then assesses the problem and develops a clinical impression or diagnosis. Next a plan is

devised and the decision to prescribe a medication is made. A prescription is either

written or not; if it is, the patient will typically present the prescription for filling at the









pharmacy. Before dispensing the medication the pharmacist should look for DTPs and

advise the patient how to use and self their new therapy. The patient then consumes (e.g.,

self-administers) the medication. Follow-up visits and monitoring are required to gauge

the effects of the medication in the individual and to determine how well the prescribed

medication is working towards the therapeutic objective. It is the information gathered

from the monitoring node that is used in the decision to continue the current treatment or

make alterations to the medical regimen.

Errors or problems at each step may occur during a passage through this sequence

of events. Examples include failure to recognize an indication for drug therapy, incorrect

patient assessment, incorrect diagnosis, prescribing, dispensing, and administration and

monitoring.25

Our meta-analysis (2002) estimated the prevalence of preventable drug related

admissions to be about four percent.1 Based on our review and the IOM report, it appears

the current MUS are not optimal. This may be because MUS are more of a virtual than

an actual microsystem. As described by Nelson et al., (1998)26 an essential element for a

microsystem is an information environment to support the work of care teams.

Unfortunately, the MUS in most ambulatory care settings is fragmented and clearly lacks

a reliable information exchange among providers, especially between initiator and co-

therapists. The development and adoption of communication systems such as electronic

medical records and computerized prescription order entry are intended to make the MUS

safer.

Levels of the Health System

The IOM defines a system as "a set of interdependent elements interacting to

achieve a common aim." The elements may be human and/or non-human. 2 The health









care enterprise consists of many embedded systems that transcend a multitude of

domains, ranging from the subsystems used to make and keep track of patient

appointments to the environment where accreditation and financial systems influence the

care of populations. All systems have subsystems or nested systems that interact with

one another. Some systems are in a hierarchical relationship in which higher order

systems influence the functioning of lower order subsystems. Identifying the nesting of

systems and their relationships to one another is key to understanding the mechanics of

any system.

Berwick (2002) published a paper that he described as a users manual for the IOM

report, "Crossing the Quality Chasm."16 In this paper he addressed the issue of

embedded systems by presenting a framework for the different levels within health care.

He separates the system into the following levels: the experience of the patients and

communities (Level A), the functioning of small units of care delivery called

microsystems (Level B), the functioning of organizations that house or otherwise support

microsystems (Level C) and the environment of policy, payment, regulation,

accreditation and other factors (Level D).

Level A is not actually framed in terms of a system; instead it represents the

patients' experience and perception of the care they received. Berwick states,

Rooted in the experiences of patients as the fundamental source of quality, the
report shows clearly that we should judge the quality of professional work, delivery
systems, organization and policies first and only by the cascade of effects back to
the individual patient and to the relief of suffering, the reduction of disability and
the maintenance of health.16

This is a significant paradigm shift from previous quality approaches that were

independent from patient outcomes. Determining quality based on the patients'

experience is allied with using outcomes of care to gauge quality. Donabedian (1978)









defines outcomes as the "primary changes in health status that can be attributed to that

care."27 Health status is determined by psychological factors and social performance

(subjective component), and physiological factors (objective component).

In this paradigm, the patients' experience is defined as the sole way of determining

the quality of other systems. The decision to place emphasis on the patient is strategic; it

acknowledges the patient's subjective experience and clinical outcomes are the primary

focus of quality rather than the dynamics of the microsystems.

The microsystems (Level B) are small units of work that actually gives care the

patient experiences.16 Clinical microsystems are basically "small organized groups of

providers and staff caring for a defined population of patients."28 They are composed of

patients who interact with clinical and support staff who perform various roles, e.g.,

physician, nurse, pharmacist, medical assistant, data managers, receptionist, etc. Nelson

et al. (1998) have described the essential elements of a microsystem as (a) a core team of

healthcare professionals; (b) the defined population they care for; (c) an information

environment to support the work of care team and patients; and (d) support staff,

equipment and work environment.26

In terms of medications management, the patients and clinical staff manage drug

therapy by engaging in direct care processes, which include recognizing, assessing and

diagnosing the patient's problem, along with developing a treatment plan, dispensing and

educating, and monitoring to make sure the treatment is progressing as planned. Direct

care processes are assisted by supporting processes that involve distinct tools and

resources such as medical records, scheduling, diagnostic tests, medications, billing, etc.









To move to a more reliable system of care, the performance of microsystems must

be optimized and the linkages between clinical microsystems must be seamless, timely,

and efficient. Change at the microsystem level is an important opportunity to focus on

the transformation of care at the front line of the health care service industry.29

Microsystems do not exist in a vacuum; they are embedded within the organizations that

help orchestrate their relationships with each other.

The health care organizations (Level C) are establishments that house and support

microsystems--they provide the necessary resources for microsystems to deliver care.

Health care delivery requires personnel, financing, technology, and facilities. Common

organizations include hospitals, provider groups, independent practice groups, nursing

homes, ambulatory surgery centers, and pharmacy benefit managers, all of which are

typically embedded within managed care organizations. Group practice, for example, is

an affiliation of three or more providers, usually physicians, who share income, expenses,

facilities, equipment, medical records, and support personnel in the provision of services,

through a legally constituted organization.30 They are embedded within managed care

organizations through such integrating mechanisms as referral arrangements, insurance

contracts, and in some cases direct ownership of practice.

Managed care is focused on controlling the functioning of microsystems, i.e.,

controlling provider and patient behaviors, and most of that control is realized through

the ambulatory care arena.30 Managed care has shifted toward an organizational form

with greater control over resources, providers, and patients within microsystems of care.

The health care environment (level D) includes multiple systems that influence

activities of the organizations and microsystems. Important environmental systems









include financing, regulation, industry, accreditation, policy, litigation, professional

education and social policy.16

Berwick (2002) describes the relationship among the four levels as the chain of

effect of improving health care quality. He states that,

The quality of the microsystem is its ability to achieve ever better care: safe,
effective, patient-centered, timely, efficient and equitable. The quality of the
organization is its capacity to help microsystems produce safe, effective and
efficient patient outcomes. And the quality of the environment--finance,
regulation, and professional education is its ability to support organizations that
can help microsystems achieve those aims.16

Figure 2.2 was designed to show the hierarchal relationship among the different

levels of health care, as well as the MUS's location within the microsystems. As

illustrated, the MUS intersects the "microsystems" and the patients' experience. The

spiral between the microsystems and the "patient level" indicates that the patients'

experience is the input into the MUS as well as the output, which is then used as feedback

to determine the performance of the microsystems. What was more difficult to illustrate

is the interaction among the microsystems. In an ideal MUS the interaction between

physicians, laboratory, pharmacists, case managers, etc., would be seamless with

information flowing freely among the microsystems and cooperation among the

professionals would be standard.

The hierarchical stacking from environment to organizations to microsystems to the

patient represents the main route of influence. It should be noted that this is not the only

direction of influence, organizations can use lobbyists to leverage influence over the

regulatory and financial environment. Likewise, microsystems can organize, like in the

case of provider groups, to counter the pressures from higher-up organizations such as

managed care.











RECOGNIZE ASSESS CLINICAL
PATIENT PATIENT 7 IMPRESSION Nl PLAN
PROBLEM PROBLEM (DIAGNOSIS) PRESCRIBE



IMPLEMENT
PLAN
DISPENSE &
FOLLOW-UP ADVISE
MONITORING


IMPLEMENT PLAN
ADMINISTER/CONSUME
Figure 2.1. Model of the Medication Use System


Human Error and System Failure

James Reason provides two views of how accidents occur within an organization -

the person approach and the system approach.17 These approaches are fundamentally

different, with each giving rise to contrasting philosophies of performance management.

The person approach (human error) focuses on unsafe acts. In medications use, the

attention would be on errors and procedural violations of health professionals and

patients. This approach views unsafe acts as divergent cognitive and behavioral

processes such as forgetfulness, inattention, carelessness, negligence and recklessness.

Historically, the most popular approach to countering human error has been to find

the operator or operators who committed the unsafe act and discipline them accordingly.

This is often referred to as blaming, and sometimes as "scapegoating." Even though

blaming may produce satisfaction it does nothing to correct the conditions that allowed

unsafe acts to progress to injury. Because errors are often made in the normal course of

providing care, traditional efforts at error reduction, which focused on individuals and









episodes by using training, rules and sanctions to improve performance, are considered

less effective than altering the system to remove or reduce the conditions that increase the

likelihood of adverse outcomes.31 Counter measures for reducing unwanted behaviors

are analogous to the "Whack-A-Mole" carnival game: you can whack a mole back into

his hole, but another is sure to challenge your skill by popping-up in a different location.


Figure 2.2. Hierarchal Relationships among the System Levels

The basic principle in the system approach is humans by their nature are fallible

and errors are expected even in the best organizations. From this perspective, injury

occurs because errors interact with flaws in the design of systems called latent

conditions.17


0-j









Latent conditions are the flaws in the design and organization of a system. In

health care they can arise from the decisions made by designers; however, it is believed

many latent conditions arise not from design but from self organization, which is an

adaptation to evolving systems and an unsettled environment.32 Latent conditions have

two types of adverse effects: they can produce error-provoking conditions and they can

produce longstanding holes or weaknesses in the defense mechanisms, i.e., error

detection mechanisms. When latent conditions combine with errors the opportunity for

PDRM occurs. In the Swiss cheese model presented in Figure 2.3, the holes in the

cheese are latent conditions that allow potential injury (i.e., drug therapy problems) to

manifest as drug related morbidity.

Summary

PDRM is a system problem. The nature of preventability means problems occurred

in the process of care. Patient injury occurs when human error interacts with latent

conditions of a system. The health care system is a complex network of interacting

systems and subsystems. Understanding how microsystems, organizations and

environment influence activities of the MUS and produce opportunities for human error

to result in PDRM will increase our ability to predict and resolve DTP before they

manifest as patient injury.


b This Swiss cheese was published by Hepler and it is a modification of Reason's Swiss cheese model15






















Figure 2.3. Swiss Cheese Model of System Failure














CHAPTER 3
LITERATURE REVIEW

This chapter will first discuss the use administrative data for scientific

investigation. The use of automated methods for detecting the incidence and/or

prevalence of PDRM will then be discussed. Next, experimental studies evaluating the

effectiveness of the Delphi process and its use in health care will be presented. Lastly,

techniques for cause-and-effect analysis will be presented.

Use of Administrative Databases for Researching PDRM

Administrative databases are derived from information produced by health care

providers and institutions in billing for products and services. Claims are filed from

institutions (i.e., inpatient hospital stays and outpatient visits) health professionals and

pharmacies for reimbursement of products and services from payer organizations such as

Medicaid, Medicare and private insurers. The claim form used depends on where

services were rendered and who provided them. Most hospital inpatients and acute care

outpatient services are submitted for payment with the uniform billing 92 (UB-92)

format. The UB-92 was derived from the Uniform Hospital Discharge Data Set, which

was formulated in 1972 by the National Committee on Vital and Health Statistics, U.S.

Department of Health, Education, and Welfare. The goal was to create a uniform but

minimum data set to facilitate investigation of cost and quality of hospital services across

populations.33 Health care services provided by single practitioners or practitioner groups

are submitted for payment in a format called the CMS-1500 (formally the HCFA-1500).

This is the common claim form for non-institutional providers that is updated by the









Center for Medicare Services (CMS) and is approved by the American Medical

Association Council on Medical Services. Both the UB-92 and the CMS-1500 forms

typically include details for each compensated service, including diagnosis, and

procedures, date, place of service, provider and patient identifiers and charges.

From a research and quality improvement perspective, the potential utility and

credibility of a database stems from its clinical content. In administrative data in the

United States, information on clinical diagnosis and conditions are documented with

International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-

CM) diagnostic codes. In addition, administrative data derived from hospital reports

(UB-92) use ICD-9-CM codes to document procedures, while procedures are

documented using Common Procedure Terminology (CPT) in data derived from

professional claims (CMS-1500).

The reliability and validity of coding is a serious issue when considering the use of

administrative data for research. In relation to hospital admissions, the two major steps in

coding an admission are first specifying the pertinent diagnosis and second ordering

them. The principal diagnosis in a UB-92 form defines the cause of admission. The

sequencing of diagnosis does impact reimbursement rates and mistakes or miscoding can

affect the accuracy of the data.

As previously mentioned, the validity of claims data is a cardinal issue when

considering its use for health services research. Threats to validity and two approaches

for evaluating the ability of claims data to accurately identify patients with specific

conditions will now be presented. The following threats to internal validity (i.e.,

misclassification bias) have been identified misspecification, resequencing, miscoding,









and clerical errors.33'34 Misspecification occurs when the attending physician selects an

incorrect diagnostic code for the principal diagnosis, listing of diagnoses or procedures.

Misspecification was identified as the leading source of error in a summary of studies that

evaluated coding quality.34 Resequencing is the substitution of a secondary diagnosis for

the correct principal diagnosis during the coding process and was found to be the second

most frequent source of misclassification.34 Miscoding is the coding of diagnoses or

procedures not attested to by the physician, misapplication of coding rules, or selection of

an unnecessarily vague diagnosis code.

Now that the types of misclassification bias have been presented, studies that

evaluated the validity of claims for research data will be discussed. Quam et al. (1993)

investigated the validity of claims data for epidemiologic research and found evidence

that supports its use.35 The researchers evaluated the ability of claims data to identify

patients with essential hypertension. The claims database used consisted of all

ambulatory, pharmacy and hospital claims from 1988 to 1989 in two large managed care

organizations. They identified essential hypertensive patients by three strategies. One

strategy was based on medical service claims alone (diagnosis without pharmacy claims),

another was based on only pharmacy claims (pharmacy claims without diagnosis), and

the third strategy was based on both medical services and pharmacy claims.

Diagnostic codes were validated by comparing findings with patient survey and

medical records. The strategies based on medical claims alone and pharmacy claims

alone exhibited low positive predicted values (PPV) with the medical record (47% and

50%, respectively) and the patient survey (43% and 63%, respectively).35 The









combination strategy, however, exhibited very high PPVs (96% for both the medial

record and patient survey).35

More recent information on the ability of administrative data to predict disease

outcomes is described in the Study of Clinically Relevant Indicators for Pharmacological

Therapy (SCRIPT) report.36 The objective of the SCRIPT project was to develop a core

set of valid and reliable performance measures to evaluate and improve the quality of

medication use. The common focus was cardiovascular disease outcomes and risk

factors. The candidate measures were tested in managed care organizations and practice

groups from eight states. The study evaluated patients with coronary artery disease

(CAD), heart failure (CHF), and atrial fibrillation (AFIB). The researchers evaluated the

PPV of specific ICD-9-CM and CPT codes for identifying the cardiovascular conditions

mentioned above. The requirement for the number of codes to identify specific

conditions was varied from one to three and the effects on yield and PPV were

determined.

The PPVs from the SCRIPT study averaged 89%, 89%, 84% for AFIB, CAD and

CHF, respectively. The state-specific values for AFIB ranged from 80% to 98%, the

values for CAD ranged from 81% to 95% and the values for CHF ranged from 78% to

94%. Requiring an additional code raised the PPV no more than four percentage points

on average and raised the worst PPV as much as eight percentage points. Requiring three

codes added little to PPV; however, it significantly reduced the yield. The most marginal

improvement in PPVs was found by requiring two codes vs. three codes. The two code

requirement lowered the yields by 13%, 14%, and 20%, respectively for AFIB CAD and

CHF









In summary, although administrative databases were not originally designed for

research, they have become a rich source of information for health services research.33

Nevertheless, threats to internal validity, specifically misclassification bias need to be

taken seriously when interpreting studies that used claims data. Studies that have

evaluated the ability to identify patients with specific conditions using administrative data

had good positive predictive ability. Quam et al. (1993) reached a PPV of 96% when

requiring medications used to treat essential hypertension and a diagnosis of essential

hypertension.35 The SCRIPT project was able to obtain good PPV while maintaining a

decent yield when they required two codes (ICD-9-CM or CPT) for specific

cardiovascular conditions.36

Automated Methods for Detecting PDRM

Chart review is the "gold standard" for PDRM ascertainment. The articles

reviewed by Winterstein et al. (2002)1 (discussed in the following chapter) used medical

chart review to measure the prevalence of drug related admissions (DRAs). This

typically included reviewing the medical records of all patients admitted to the hospital or

specific hospital units during a defined period of time to determine if the reason for

admission was drug related. The detection and classification method was often implicit,

which means the reviewers judged whether or not a DRA occurred by comparing the care

processes of that patient against his or her own knowledge, opinions, and beliefs about

how appropriate care should be carried out. Implicit methods may be more sensitive

because they allow experts to capture case-specific elements of information for the

judgment of preventability, however, their reliability is questionable.37 Furthermore,

chart-review is time and cost consuming and is not useful in the continuous improvement









paradigm where baseline and many follow-up measures are needed. Computerized

methods to identify DRM with explicit criteria are an alternative to chart-review. 3839

A PubMed search identified three studies with different approaches for detecting

ambulatory acquired DRM with computerized data. One method identified DRM with

search algorithms used to screen laboratory data and pharmacy records. Another method

used multiple types of search algorithms and databases to identify cases of ADEs. The

last method screened administrative claims data by using automated PDRM performance

indicators. These studies will be presented and special attention is given to the following

issues: the predictive validity of the instrument, the ability of the indicators to capture

preventable events, the scope of the database, and the sophistication of the software used

to run the indicators.

Jha et al. (2001)39 adapted published screening rules (i.e., search algorithms)

developed from the inpatient LDS study in Utah.40 The rules were used to estimate the

rate of DRA to the study hospital. Every day the computer generated a list of alerts. The

alerts were validated by reviewing each patient's medical record that screened positive

for a DRM on admission.

Jha et al's. (2001) screening rules can be separated into at least four categories:

antidotes, chemistry/drug levels, and physiological response to a specific therapy and

drug interaction. (Table 3.1) The information system used integrated pharmacy,

laboratory test results and a sophisticated physician order entry systems. Twenty two

rules required laboratory test results and nineteen screened for antidotes of known

adverse consequences of drug therapy (e.g., naloxone, a narcotic antagonist).









Table 3.2 shows the results of the most frequent screens. The overall PPV for the

instrument was 3.5% with a range from 2.0 to 100%. The screen for serum phenobarbital

levels < 45 mg/dl had a PPV of 100%. The screen for patients receiving charcoal

(activated) had the second highest PPV of 45%. All other screens had PPVs below 12%.

The overall PPV for this instrument and the individual screens' PPVs, excluding

phenobarbital levels, indicated that these screens picked up a tremendous amount of

noise. Furthermore, the majority of the indicators were directed toward toxic levels of

medications and abnormal blood chemistry levels. While these surrogate outcomes may

be able to indicate the presence of DRM, they do not provide information about the care

process that led to the event, i.e., preventability. Additional chart review or investigative

work is needed to judge preventability. Furthermore, the software used to search the data

was quite sophisticated (physician order entry and an event monitor). Nevertheless, less

sophisticated software such as SAS or Access could be used to write search algorithms

for the screens.

Honigman et al. (2001) took a more elaborate approach for using computerized

information to screen for ADEs.41 This approach used a sophisticated computer program

which consisted of four search methods: ICD-9 codes, allergy rules, computer event

monitoring rules, and an automated chart review using text searching.

ICD-9-CM codes associated with ADEs were used to screen the database. The

computer event monitoring rules were based on the same screening criteria as Jha et al.

(2001) mentioned above.39'40 Allergic reactions to medications were detected with a text

search of the medical records. The software program M2D2 was used to expand the

patients' drug allergy list to include product names, generics and ingredient. Patient










records were screened for allergies along with offending medications. The M2D2

program also matched terms in the medical record with known adverse effects of drugs

the patient was taking.

Table 3.1. Examples of DRM Screens by Type (Jha et al. 2001)

Antidotes Chemistry and Blood Levels Response to Drug and Interactions
Receiving betamethasone Receiving "nephrotoxin" AND blood creatinine
Serum digoxin > 1.7 ng/mL
dipropionate 0.05% has risen > 0.5 g/dL in last 1 day
Receiving charcoal Receiving ranitidine AND platelet count has fallen
Serum lidocaine > 5.0 mg/mL .
(activated) ......... to less than 50 of previous value
Receiving racemic Serum phenytoin results within
Receiving diphenoxylate with atropine
epinephrine hcl last I day are> 20 mg/mL
S> 10 m Receiving benzodiazepine AND receiving anti-
Receiving atropine sulfate Serum bilirubin > 10 mg/dL
epileptic
> 6.5 mmL Receiving phytonadione (vitamin K) AND order
Receiving naloxone Serum potassium > 6.5 mmol/L warfarin within last 14 days
for warfarin within last 14 days

Table 3.2. Results of Leading DRM Screens (Jha et al. 2001)
DRM screening rule Screen + True + PPV (%)
Patient receiving predinisone 313 12 3.8
Patient receiving diphenhydramine 300 9 3
Allergy entered 113 10 8.8
Patient receiving oral matronidazole/vancomycin 87 6 6.9
Serum digoxin > 2.0 ng/ml 61 5 8.2
Serum phenytoin > 20mg/dl 47 4 8.5
Patient receiving kaopectate 35 4 11.4
Patient receiving charcoal 11 5 45.5
Serum phenobarbital >45 mg/dl 4 4 100
Other 1,649 33 2
Total 2,620 92 3.5

Honigman et al. (2001)41 found allergy screens to have the highest PPV (49%), but

these screens did not detect the greatest number of ADEs (104). Text searching detected

the most ADEs (1,637) but it had many false positives (PPV = 7%). Searching for only

ICD-9-CM codes had the lowest PPV (2%) and it only detected five events. The event

monitoring rules detected 60 events with a PPV of 3.3%. The PPV for the composite tool

was 7.2%. (Table 3.3)









Table 3.3. Results of DRM Screens
Rule Class Screen positive true positive PPV, %
ICD-9-CM 248 5 2
Event Monitoring Rules 1,802 60 3.3
Allergy 214 104 48.6
Text Search 22,798 1,637 7.2
Composite 25062 1806 7.2

This approach was more comprehensive than Jha et al's. (2001) detection method.

The rules were more dynamic because they were able to identify less severe adverse drug

effects. The term adverse drug event was used instead of DRM when describing this

study because many of the effects screened for would not fit the definition of a DRM.

The events would be considered actual DTPs under Hepler's conceptual framework. For

example, text searches for signs and symptoms such as dizziness, fatigue, cough, and

over anticoagulation without bleeding were major components of the screening

instrument. Only nine percent of the adverse events identified required hospital

admissions.

The level of programming and database (administrative vs. clinical data)

sophistication needed to run the rules varied. The ICD-9-CM rules did not require

sophisticated software and were usable with administrative databases. The event

monitoring rules did not require sophisticated software either, programming could be

done with Access or a statistical program such as SAS. Nevertheless, clinical data (i.e.,

laboratory test results and medical records) was required. Most administrative records

only indicate that specific types of laboratory tests were requested or conducted. Patient

specific laboratory values are not documented. The allergy and text search rules were

more sophisticated and they required electronic medical records. Honigman et al's.

screening method had advantages over Jha et al's. method. Nevertheless, the level of









sophistication limits the use of these rules to organizations with access to electronic

clinical data and specific software.

The PPV of Honigman et al's. composite tool was relatively low (7.2%). The

screens were the most comprehensive and could detect minor adverse drug events as well

as events that led to hospital admission; however they were not designed to indicate

whether or not the events were avoidable or if the admissions were preventable. The

screens varied in their use of databases, they used administrative data, laboratory test

results and electronic clinical records. The software needed to run the text searches was

quite sophisticated.

Faris (2001) developed a different approach to electronically detect PDRM.13 His

approach automated forty-nine PDRM indicators that paired an inappropriate process of

care (potential DTP) with its predictable adverse outcome (DRM). The indicators were

originally developed by MacKinnon (1999)42 from a literature search of peer-reviewed

medical articles and reference texts from 1967-1998, and from a consensus panel of

seven experts in geriatric medicine.

Faris (2001) revalidated the PDRM indicator definitions through a Geriatric

medicine expert consensus panel. Forty-four of the fifty-two indicator definitions

accepted in MacKinnon's study were accepted for automation in the Faris study.

Additional indicators were proposed and accepted by the panel for a total of forty-nine

indicators.

Once the indicator definitions were accepted they were translated into a format that

could be applied to a managed care administrative database. The translation was

designed to capture the events described in the indicator definitions through billing









records from physician offices, hospitals, pharmacies, and other care sites in the managed

care network. Services are billed through a system of codes that represent different

treatments and activities. These codes included ICD-9-CMs, CPTs, and the National

Drug Codes (NDC). SAS was used to extract and analyze cases that fit the indicator

definitions.

Faris (2001) retrospectively sampled 11,711 elderly patients over a one-year period

and 966 had at least one positive for a PDRM indicator. Chart review was not done to

validate the performance indicators. As an alternative, the proportion of patterns of care

in the population that appeared to result in the corresponding adverse outcome was

calculated (PTOV). Ten of the nineteen indicators, which had more than ten positives,

had PTOV values greater that 0.74 percent. Of the five most prevalent indicators, three

had adjusted PTOVs greater than or equal to 75%, which means the process of care was

strongly associated with the corresponding adverse outcome. (Table 3.4.) For example,

Asthma exacerbation and/or status asthmaticus and/or ER visit/hospitalization due to

asthma was found 100% of the time when the following pattern of care occurred:

Diagnosis of moderate to severe asthma and use of a bronchodilator with no use of

maintenance corticosteroid.

The criterion validity of PDRM indicators developed by Faris (2001) showed

promise with ten indicators having adjusted PTOVs of 75% or greater, nevertheless the

PPV of the PDRM indicators is unknown and needs to be tested. MacKinnon (1999) did,

however, examine the criterion validity of two indicators that occurred frequently enough

for statistical validation and found relatively high PPVs, 82% and 34%.42









Unlike the approaches taken by Jha et al. (2001) and Honigman et al. (2001), these

indicators are specifically searching for preventable cases of DRM. The preventability is

represented by the inappropriate process and its predictable adverse outcome.

Unfortunately, only two of MacKinnon's PDRM indicators were validated with chart

review. Nevertheless, the PPV of the two indicators was decent and this method shows

promise as valid PDRM measures. Another advantage of this approach was it did require

sophisticated software. The search algorithms were produced with SAS. Furthermore,

clinical data is not needed-these indicators were designed for administrative data.

In summary, three studies with different computerized methods to detect PDRM

were found in the literature. The approaches varied in ability to detect true cases of

PDRM, and identify preventable events. The type of data needed, and the sophistication

of the software used to produce search criteria also varied. The approach used by Faris

(2001) will be adopted for this dissertation, because the PDRM performance indicators

from that study included preventability in their operationalization. Furthermore, the

indicators were intended for claims data and the search algorithms can be produced with

basic statistical software, i.e., SAS.

Delphi Method

The Delphi method is a systematic approach for the utilization of expert opinion

that was extensively studied by the RAND corporation in the 1960's.43 The concept of

the Delphi technique is simple; it is designed to define a single position (consensus) of a

group by systematically combining their opinions in a way that eliminates activity among

the experts to reduce dominance and group pressure.











Table 3.4. Results of Top Five Indicators
Positive Adjusted
PDRM Indicators Poiti d
Indicator PTOV
This outcome has occurred after the pattern of care below:
ER visit/hospitalization due to congestive heart failure
This is the pattern of care: 270 0.75
1. Diagnosis/history of congestive heart failure
2. Not on ACE inhibitor
This outcome has occurred after the pattern of care below:
ER visit/hospitalization due to congestive heart failure and or heart block
This is the pattern of care: 184 0.95
1.History/diagnosis of congestive heart failure with heart block or advanced bradycardia
2. Use of digoxin
This outcome has occurred after the pattern of care below:
ER visit/hospitalization due to hypothyroidism
This is the pattern of care: 129 0.12
1. Use of a thyroid or antithyroid agent (e.g.; levothyroxine, etc.)
2. T4/TSH not done before therapy starts and at least every 12 months
This outcome has occurred after the pattern of care below:
Gastritis and/or upper GI bleed and/or GI perforation and/or GI ulcer and anemia 103 0.45
This is the pattern of care:
1. Use of 2 or more NSAIDS concurrently
This outcome has occurred after the pattern of care below:
Asthma exacerbation and/or status asthmaticus and/or ER visit/hospitalization due to asthma
This is the pattern of care: 89 1
1. Diagnosis of moderate to severe asthma
2. Use of a bronchodilator
3. No use of maintenance corticosteroid

The Delphi technique is a modification of the traditional roundtable group decision

making process. The roundtable approach has inherent limitations because the final

decision or position may be an effect of the most outspoken person (i.e., the bandwagon

effect) or socially acceptable opinion rather than the group's true opinion.43 The Delphi

technique avoids these undesirable effects by replacing the group meeting with

anonymous response. Instead meaning the opinions of members are obtained by formal

questionnaire.

The procedures of the Delphi technique have the following key features :44

* Anonymous response-opinions of members are obtained by formal questionnaire
or other formal communication channels as a way to reduce the effects of dominant
individuals.

* Iteration and controlled feedback-the interaction among the members is supported
by a systematic exercise conducted in several iterations, with carefully controlled









feedback between rounds. The summary of the results from the previous round are
communicated to the participants

* Statistical group response-the group opinion is defined as an appropriate aggregate
of individual opinions in the final round. The use of a statistical definition of the
group response is a way of reducing group pressure for conformity; at the end of
the exercise there may still be a significant spread of opinion.

In the 1960's the RAND Corporation carried out a series of experiments to evaluate

Delphi procedures and to explore the nature of information processing.44 The

experiments involved fourteen groups of upper-class and graduate students from UCLA

and they ranged in size from eleven to thirty members. The three effects that researchers

examined were group size, a comparison of face-to-face discussion with the controlled-

feedback interaction, and the ability of controlled feedback as a means of improving

group estimates. They found increasing group size decreased the average group error and

increased reliability. The anonymous controlled feedback procedures of the Delphi

technique made the group estimates more accurate than face-to-face discussion. They

also found controlled feedback narrowed the dispersion around the median.

A large set of experimentally derived answers to factual questions was evaluated to

determine the relationship between group size and the mean accuracy of a group

response.44 In this study the experimenters knew the answers to the questions; however,

the subjects did not. The group error was calculated as the absolute value of the natural

algorithm of the group median divided by the true answer. Dalkey (1969) found the

gains in increasing group size were quite large from three to eleven members, but the

percent change in group error was less than five percent past eleven members. (Table

3.5.)









Table 3.5. Percent Change in Average Group Error by Number of Group Members.
(Extrapolated from Dalkey (1969))
Group members Average group error Percent change
1 1.194
3 0.8521 28.63%
5 0.6882 19.23%
7 0.6097 11.42%
9 0.572 6.18%
11 0.5539 3.16%
13 0.5453 1.56%
15 0.5411 0.76%
17 0.5391 0.37%
19 0.5382 0.18%

Dalkey (1969) also compared the reliability of two groups' opinion with various

numbers of members. Reliability was measured by the correlation between the answers

of the two groups over a set of questions. Dalkey found a definite and monotonic

increase in the reliability of the group responses with increasing group size. A mean

correlation of 0.8 was obtained with thirteen group members.

Even though these two studies showed increasing group size reduced the error of

the group and increase the reliability of the estimate, it appears these estimates were

obtained with traditional group consensus and not by a Delphi process. If these results

are generalizable to Delphi group estimates then having a Delphi group with between

eleven and fifteen members would seem reasonable because average group error only

decreased by approximately three percent from eleven to thirteen members and reliability

reached 80% with thirteen members.

In the main experiment the performance of groups using face-to-face discussion

was compared with Delphi groups. The first experiment involved two groups of five

graduate students, and twenty questions were presented in four blocks of five using the

ABBA design (A= face-to-face). The face-to-face group was instructed to follow a









specific procedure for each question and the Delphi procedure involved four rounds of

estimates, feedback of medians and quartiles and re-estimates. The median response of

the Delphi group was more accurate in thirteen cases, and the face-to-face group was

more accurate in seven cases.

The second experiment had a different study design. The Delphi group had twenty-

three members and from them the face-to-face groups were separated into seven groups

of three and one group of one. In this experiment they found that Delphi estimates were

similar to face-to-face estimates in terms of accuracy, however, face-to-face estimates

produced more changes from initial estimates that reduced accuracy.

Dalkey also examined the distributions of the answers between the first-round and

second-round of the Delphi groups. The second-round distribution shifted toward the

mean and Dalkey concluded the shift represents a convergence of answers toward the

group response. Nevertheless, the second-round distribution still had a large range,

indicating convergence was not complete.

Dalkey also evaluated the effects of iterations on the accuracy of responses. To do

this, changes with regards to individual questions were measured. The median improved

in accuracy for about 64%, while the median became less accurate in 31%. The

mechanism of improvement appeared to be from first-round feedback of the median

response. The tendency to change was determined by the distance of the first round

answer to the first round median. However, this did not explain the change in median

from round-one to round-two. Some respondents had to cross the median for a change in

median to occur.









To evaluate how the median value changed from the first to second round, the

researchers divided the groups into holdouts and swingers. Holdouts tended to cluster

around the median and they were more accurate than the total group in the first round.

The median of the group was between the median of the swingers and the median of the

holdouts. The shift in the group median occurred because the swingers shifted toward the

median, thus shifting the group median.

In a supplementary analysis Dalkey (1969) tested the effects of other forms of

feedback on the accuracy of the Delphi estimates.44 The experimental group documented

reasons for their decisions when their second round response was outside the interquartile

range of the first round. Formulating and feeding back reasons did not increase the

accuracy of the initial estimates or produce improvement on iteration.

In a follow-up study Dalkey et al., (1969) evaluated the use of self rating to

improve group estimates.45 This study used 282 University of California students.

Subjects were randomly assigned to sixteen groups with fifteen to twenty members per

group. Almanac type questions were used to assess the relationship between accuracy

and self rating. Subjects were given the questions and asked to rate each question from

one to five to indicate their knowledge of the content in question. Relative rates were

used, meaning subjects had to identify a question they were most knowledgeable about

and give it a five. They also had to identify a question they knew the least about and give

it a one. A clear association between average group self rating and group estimate

accuracy was found. This study also found a significant improvement in the

effectiveness of the Delphi procedures can be obtained by using self-rating information to

select more accurate subgroups.









The experimental studies on the Delphi process are important because they provide

information on how group processes affect the accuracy of group estimates to factual

questions. In this dissertation the Delphi method will be used to assign Faris' indicators

to nodes of the MUS where the process failure likely originated. In this case, a known

answer does not exist. The opinions of the group members will be aggregated to assign

select indicators to nodes of the MUS. Therefore the qualifications and diversity of

experts in domain specific knowledge will be important factors to consider when

constructing a Delphi panel.

Cause-and-Effect Analysis

To improve the performance of the MUS it is first necessary to understand error

etiology within the MUS and the underlying system design factors that contribute to

PDRM. Cause-and-Effect analysis (CEA) is a structured investigation that aims to

identify the true cause of a problem, and the actions necessary to eliminate it.46 CEA is

an integrated problem solving methodology that incorporates multiple tools and

strategies, which include process analysis, brainstorming and cause-and-effect

diagramming.46,47

First, process analysis is conducted to understand the processes involved in the

quality problem. Before brainstorming or cause-and-effect diagramming is conducted an

understanding of how the system works is needed. Process analysis should include flow

diagrams that represent different nodes of the system. The steps needed to construct a

flow diagram include:47

1. Defining the basic nodes of a system
2. Further defining the process, breaking each node down to specific steps needed to
complete the process
3. Following the objects through the process a number of times to verify the process
by observation









4. Discussing the process representation with the project team and reaching consensus
on the underlying process.

Brainstorming is a possible cause generation technique.47 Brainstorming helps a

group to generate many thoughts or ideas in a very short period of time.46 The important

aspect of brainstorming is ideas are to be generated without judgment or discussion until

all the ideas are presented. In a typical group discussion someone presents a thought and

others comment on it, or judge it. The problem is group members may focus on one idea

before all other possible ideas have a chance to emerge.46 Brainstorming encourages the

flow and fluency of group member thoughts.

Brainstorming has three steps: generation, clarification and evaluation.46 In the

generation phase the leader clearly states and writes the question or purpose and then

invites and records responses. This step can either be structured or unstructured. In the

structured approach, known as round-robin, each participant in turn launches one idea,

this insures equal participation, but is less spontaneous and may limit the possibility for

building on one another's ideas.46 In unstructured brainstorming everyone can freely

launch ideas, this a spontaneous process, but it can become confusing and it may lead to

one or more persons dominating the activity.46

The second step in brainstorming is clarification. After all ideas have been

generated, the group reviews them to make sure everyone is clear about their meaning.

The focus of the generation phase was quantity and not quality, therefore, during

clarification group members are encouraged to question the meaning of the generated

items.

The final step of brainstorming is the evaluation of clarified items. Here the group

considers the list and rules out duplications, and irrelevant ideas. During the evaluation









phase affinity diagramming or other methods for grouping ideas into similar concepts

may be employed to make sense out of the generated items.

Cause-and-effect diagramming is the heart of CEA. It is a tool used to define and

illustrate the relationships between an effect and an outcome, or a problem and beliefs

about the possible causes or factors contributing to it.46 It combines the product of

brainstorming with a systematic analysis to organize and evaluate causes to determine

which are most likely contributing factors or causes.46 Cause-and-effect diagramming

can be done using fishbone or tree diagrams. Fishbone diagrams are the traditional way

of illustrating cause-and-effect relationships. Nevertheless, tree diagramming, or the five

whys analysis, is an effective way to graphically show the breakdown of large problems

into their increasingly more detailed elements.46 Tree diagrams help the group members

to move from general to specific, or vice versa, in an organized manner, and they show

the logical connections among the relationships.

Accreditation organizations such as the Joint Commission on Accreditation of

HealthCare Organizations (JCAHO) have recognized the importance of CAE and have

incorporated the use of root cause analysis (RCA), which is a more specific form of CAE,

into their quality criteria. Currently, JCAHO only requires RCA for inpatient sentinel

events. Causes of DRM in the ambulatory setting have not received much attention and

information on CAE for adverse outcomes of drug therapy in the outpatient setting to my

knowledge is non-existent.

In the Joint Commission paradigm, the data used as input for RCA are the

collection of activities that led to the specific incident. Activities are determined by

interviews with providers involved in that particular patient's therapy and by reviewing









the patient's medical chart. While this approach is ideal for sentinel events in the

inpatient setting because the quality team has access to providers involved in the

particular event and the patients' medical record, it is not as useful for rate based events

that occur in ambulatory care. A patient who experienced a DRM in the ambulatory

setting may have participated in a number of microsystems which are geographically

separated, such as the primary care, specialist, pharmacy, and laboratory systems; or

personal events. Managed care organizations and health care coalitions, typically do not

have the same leverage and influence over these microsystems as, say, hospital

administrators who have access to the medical records and have the influence to require

participation from professionals involved in the care of a patient who experienced a

PDRM.

In summary, literature that evaluates the utility of administrative data for health

services research was presented. Different methodologies for electronically screening

health care data were also discussed. The history, purpose and methods of the Delphi

technique were presented. Finally, the process for cause-and-effect analysis was

discussed















CHAPTER 4
PRELIMINARY WORK

This section first addresses the prevalence of PDRM in the ambulatory care setting.

Following is an evaluation of drug treatment categories frequently involved in

preventable drug related admissions (PDRAs). Next, is a discussion of problem areas in

the MUS that were found to be proximate causes of PDRAs. This section will end with

an evaluation of latent causes of PDRAs.

Prevalence of Preventable Drug Related Admissions

The Harvard Medical Practice Study (HMPS) is historic because it was the first

large scale study to look at medical injury as a result of errors or problems in the process

of medical care.48 Prior to the HMPS, research on drug related injury focused mainly on

adverse drug reactions (ADRs), which are typically thought of as the inherent risk

associated with medications use and not problems with drug therapy management. Even

though the HMPS was published in the early 1990s, medication errors and PDRM did not

gain widespread appreciation and public awareness until the Institute of Medicine's

(IOM) Report, "To Err is Human," in 2000.2

The IOM report revisited the HMPS and a large study from Utah that focused

exclusively on inpatient medical errors and adverse events. The report gave much less

attention to ambulatory events. Nevertheless, they did suggest DRM causes a significant


a The term proximate cause is being used to describe the initial error or problem that started the events that
lead to a PDRM. The term is defined as: "That which in ordinary natural sequence produces a specific
result, no independent disturbing agencies intervening." Webster's Revised UnabridgedDictionary, C 1996, 1998
MICRA









number of admissions to inpatient facilities, and they reported the proportion of

preventable admissions is unknown. Winterstein et al. (2001) tried to address this issue

by producing a meta-analytic analysis of drug related hospital admissions (DRA).1 The

objective was to estimate the prevalence of PDRA and to explore the relationship

between study characteristics and prevalence estimates. Unfortunately, a generalizable

prevalence estimate was not produced because the studies did not have similar protocols

and there was too much heterogeneity in prevalence estimates to summarize the findings

with a mean value. Instead, a median and range were presented to display the

distribution of studies and meta-regression coefficients were calculated to evaluate the

association of various study characteristics with prevalence estimates. The methods and

results are presented below.

Since the goal was to produce a meta-analytic summary estimate of PDRA, a

criterion was established to select only studies that attempted to conduct a comprehensive

surveillance of drug therapy as a cause of preventable patient injury. Studies were

excluded that limited their scope to specific drug treatments, indications, injuries, or to

only one drug-therapy problem. The studies had to discuss the relationship between

pharmacotherapy and patient morbidity, and preventability, along with the necessary

information to calculate prevalence.

Fifteen studies5-12'49-55 published between 1980 and 1999 met the inclusion criteria.

All studies were conducted in industrialized countries: eight in Europe, four in the US,

two in Australia, and one in Canada. (Table 4.1) The median DRA prevalence was 7.1%

(IQR 5.7-16.2%) and the median PDRA prevalence was 4.3% (IQR 3.1-9.5%). Overall,

the median preventability rate was 59% (IQR 50-73%).









Since the protocols were not homogeneous across the studies and the Cochran's Q

test indicated extreme heterogeneity (Q: 176; df: 14; p<0.001), an analysis of the

association between different study characteristics and PDRA prevalence estimates was

warranted. The following characteristics have been discussed in the literature and were

considered potential contributors to the heterogeneity of the findings: (Table 4.2)

* Inclusion/ exclusion of first hospital admission (re-admission studies)
* Planned admissions, and transfers from other units or hospitals
* Mean sample age (>70 vs <70); country (US/other)
* Selection of hospital units vs. inclusion of entire hospitals
* Publication year (>1992 vs. < 1992)
* Inclusion/ exclusion of indirect drug-related morbidity (lack of drug effectiveness,
and no access)
* Inclusion/ exclusion of MD and patient interview
* Explicit criteria for judging preventability

The study characteristics were analyzed using meta-regression models, the

dependent variable was the transformed PDRA estimate and the independent variable

was a specific study characteristic for each model. These regressions were fit with

random-effects weights using restricted maximum likelihood estimation of the between

study variance to account for study heterogeneity. The meta-regression models produced

point estimates and 95% confidence intervals of prevalence odds ratios comparing studies

that differ with respect to each study characteristic (Table 4.2). The inclusion or

exclusion offirst admissions (i.e. re-admission studies) was the study characteristic most

strongly associated with PDRA prevalence (OR: 3.7; 95% CI: 1.5-8.9).

The mean of age of admissions was the next most strongly associated characteristic

with PDRA prevalence (OR: 2.0; 95% CI: 0.95-4.2). The inclusion of indirect DRM was

also associated with higher prevalence estimates (OR: 1.9; 95% CI: 0.92-3.9). The









remaining study characteristics listed in Table 4.2 had little or no apparent association

with PDRA prevalence.

The results of this systematic review suggest that PDRAs represent a significant

public health concern in ambulatory care (Table 4.1). In most studies, more than half of

DRAs were preventable. That is, they were not considered acceptable consequences of

therapeutic risk-benefit considerations but rather caused by inappropriate care and

medication errors. The data in Table 4.1 give the impression of a widespread and long-

standing problem in the quality of drug therapy management because the studies

represent problems from 1980 to 1999 and nine of the fifteen studies had PDRA

prevalence estimates above four percent.

To place the reported prevalence estimates in perspective, according to the National

Hospital Discharge Survey, there were 31.8 million admissions to U.S. hospitals in

1998.56 The top six primary diagnostic categories (heart disease, delivery, neoplasms,

pneumonia, psychosis, and cerebrovascular disease) each accounted for three to twelve

percent of these admissions. Any of the studies, or all of them combined, suggest that

inappropriate management of drug therapy may be a leading cause of hospital admissions

in developed countries.

The meta-regression analysis indicated that sampling methodology does affect

prevalence finding. Limiting the sample population to patients previously hospitalized

produced much higher PDRA prevalence estimates than not restricting the sample to re-

admissions. The causes of these findings are not clear. Patients re-admitted may be a

sicker sub-population who had been hospitalized with more risk factors for DRM, e.g.,

more drugs, more diseases, more prescribers, etc. A need for research specifically









designed to evaluate the transition of drug therapy management from the hospital to the

ambulatory setting is clearly warranted.

DRM is a concept, and the way in which it is operationalized affects its prevalence.

As one would expect, studies that included indirect injury (e.g., lack of therapeutic effect,

and lack of access to medications) were associated with higher prevalence findings. The

limited view of drug-relatedness found in one-third of the articles used in this analysis

may be an artifact of an outdated approach to evaluating adverse outcomes of drug

therapy. For an assessment of DRM to be considered comprehensive it would have to

include both direct and indirect drug-related injury. The IOM report suggested, "There is

evidence indicating that [adverse drug events] account for a sizeable number of

admissions to inpatient facilities." This systematic review confirms their suspicion and

suggests that PDRM in ambulatory care is at least as significant and prevalent as in

inpatient care environment.

Drug Categories Involved in PDRA

The same fifteen drug-related admission studies from Table 4.1 were used to assess

the medications commonly involved in DRAs. Three of the studies5'12'54 were not

included because they did not provide information on specific medications or therapeutic

classes involved in DRAs. Medications involved in DRAs were the unit of analysis

rather than medications involved in PDRAs because fewer than five studies provided

drug specific preventability.










Table 4.1: Studies Reporting DRA and PDRA: Sample Characteristics and Prevalence
Estimates

Reference Country Study year & DRA PDRA Preventability of
period Prevalence Prevalence DRA
period Prevalence Prevalence DRA


Darchy
1999
Ng
1999
Raschetti
1999
Cunningham
1997
Nelson
1996
Courtman
1995
Dartnell
1995
Hallas
1992
Lindley
1992
Nikolaus
1992
Bero
1991
Bigby
1987
Lakshmanan
1986
Trunet
1985
Trunet
1980


1994
France
12 months
1996
Australia
3 weeks
94/95
Italy
Italy 12 weeks

4 weeks each
UK
unit
1993
USA
1 month
92/93
Canada
5 months
1994
Australia
1 month

Denmark 1988/89


10 weeks


87/90
Germany 36 months
36 months


USA


Missing


83/84
USA
24 months
1984
USA
2 months
78/81
France
33 months
78/79
France
12 months


b Modified table from Winterstein et al. (2002)


41/623
-6.6%
31/172
(18%)
45/1833
(2.5%)
54/1011
(5.3%)
73/450
(16.2%)
21/150
(14%)
55/965
(5.7%)
8.0%
(n= 1999)
26/416
(6.3%)
22/87
(25.3%)
45/224
(21.1%)
73/686
(10.6%)
35/834
(4.2%)
97/1651
(5.9%)
23/325
(7.1%)


30/623
(4.8%)
10/172
(5.8%)
25/1833
(1.4%)
43/1011
(4.3%)
43/450
(9.5%)
18/150
(12%)
36/965
(3.7%)

3.8%

13/416
(3.1%)
32082
(12.6%)
34/224
(15.2%)
43/686
(6.3%)
19/834
(2.3%)
43/1651
(2.6%)
14/325
(4.3%)


30/41
(73%)
11963
(32%)
25/45
(55.6%)
43/54
(79.6%)
43/73
(58.9%)
18/21
(86%)
36/55
(65.5%)
67/143
(47%)
13/26
(50%)
38313
(50%)
34/45
(76%)
43/73
(58.9%)
19/35
(54%)
43/97
(44.3%)
14/23
(61%)










Table 4.2. Prevalence Estimates and Odds Ratio per Stratum/Specific Study Groups

Number of Average PDRA prevalence odds
Study characteristic Category Studies PRA1 ratios (95% C)2
Studies PDRA (%) ratios (95% CI)

Excluded 2 14
First hospital adm mission ...... ...........................
Included 13 4.2 3.7 (1.5- 8.9)
> 70 6 7.6
Mean age <70 7 3.9 2.0 (0.95 4.2)
Missing 2
Indirect drug-related Included 10 6.1
morbidity Excluded 5 3.3 1.9 (0.92 3.9)
USA 4 6.9
C o u n t r y ..................................................................................................................................................................................................................................................................................................................................................
Other 11 4.4 1.6 (0.73 3.6)
Transfers from other units Excluded 8 5.9
or hospitals Included 7 4.1 1.5 (0.71 3.0)
Excluded 6 6
Planned admissions.........
Included 9 5 1.3 (0.61 2.8)
Entire hospital 8 5.8
H hospital U nits...................
Selected units 7 4.5 1.1 (0.57- 2.3)
< 1992 8 5
Year of publication.........n
> 1992 7 5 1.0 (0.47-2.1)
Yes 8 5.8
M D / patient interview .........................................
No 7 4.2 1.4 (0.67- 2.9)
Specified criteria for Yes 6 5
preventability judgment* No 9 4.9 1.0 (0.47- 2.2)


The purpose was to find the treatment categories that were involved in DRAs and

to report their median frequency. This was done by classifying specific medications into

their therapeutic class and then organizing therapeutic classes into treatment categories.

Variability existed n the way medications involved in the DRAs were presented. Studies

either listed the specific medications involved in the DRAs, the number of events

involving a specific therapeutic class or only the number of events related to a treatment

category. Therapeutic classes were kept as long as they were mentioned in at least two

studies. If only one study mentioned a particular therapeutic class it was categorized into

a more general group, e.g., cardiovascular/other.









The cardiovascular treatment category had the highest median DRA prevalence

(33.3%) and this category was represented in twelve studies. The median prevalence for

the Anti-inflammatory category was 11.4% (represented in eleven studies). The median

prevalence of the Anti-diabetic category was 12.15% (represented in eight studies). The

median prevalence of the Psychotropic category was 8.75% (represented in eight studies).

The median prevalence of the Anti-infective category was 8% (represented in seven

studies). The median prevalence of the Non-specific miscellaneous category was 12%

(represented in ten studies).d See Figure 4.1 for bar charts of treatment categories and

their median prevalence that were mentioned in at least six studies. The actual data from

the studies can be found in Appendix A.

The therapeutic classes specifically addressed in at least six studies are presented in

Figure 4.2. The therapeutic class Diuretics was most often mentioned (10 studies).

Antihypertensives, Hypoglycemic, and NSAIDs were mentioned in eight studies.

Antibiotics were specifically mentioned in seven studies.

These findings are not surprising because they represent therapeutic categories for

common disease states. Cardiovascular disease is the leading cause of hospital

admissions56 and medications used to treat cardiovascular disease were involved in a





Prevalence was calculated by using the total number of medications involved in DRAs from a particular
treatment category as the numerator and total number of medications involved in DRAs for the
denominator. The prevalence for each treatment category was calculated individually for each study and
the median values was used to represent the prevalence of that treatment category.

d The Miscellaneous category (also described as the "other" category was often used in the articles to group
medications that were infrequently involved in DRAs. This category is not mutually exclusive from the
other treatment categories. For example; one study may have found that 15% of the medications involved
in DRAs were NSAIDs while another study only found 1% was from NSAIDs. The study that found only
1% if medications involved were NSAIDs probably would have put them in the "other" category.









large portion of DRAs. Admissions due to diabetes, infections, and psychosis are also

among the leading causes of hospital admissions.

From the results of this analysis it appears that medications most often prescribed

tend to be the treatments most often associated with DRAs. Cardiovascular,

psychotherapeutic anti-infective, analgesics, and anti-diabetic agents were among the top

ten therapeutic classes by retail sales share in 2001.e This means that DRAs appear to be

happening most frequently in patients being treated for common conditions with common

and well-accepted therapies.

From a measurement perspective this information can be very useful. It can be

used to focus the development of medication specific performance indicators and it can

also be used to help select indicators for use that have already been developed. DRAs in

the twelve studies used in this analysis were determined by medical record review,

however, which is currently the "gold standard" for measuring the prevalence of DRM.

Medical record review is a costly and inefficient process. The use of automated

performance indicators is an alternative for measuring the prevalence of DRM. This

information is useful for developing and selecting performance indicators. Any

instrument used to measure the prevalence of DRM in a population should try to address

the treatment categories found to represent a significant proportion of medications

involved in DRM i.e., cardiovascular, psychotropic, anti-inflammatory, analgesic, ant-

diabetic and anti-infective agents.


e Source NDC Pharmaceutical audit Suite. http://www.ndchealth.com/epharma/YIR/pharmatrends.htm









Apparent/Proximate Causes of PDRAs

As discussed in chapter two, DRM occurs when an error interacts with the latent

conditions of a system to produce injury. Errors are produced at the provider-system

interface. In the MUS, the "sharp-edges" where professionals interact with the patient

and with other health professionals happens at the prescribing, dispensing, (self)

administration, and monitoring nodes of the system.

The same fifteen articles listed in Table 4.1 were analyzed to determine the

proximate causes of the DTPs, i.e., DTPs were classified into the nodes of the MUS

where the errors likely occurred. A classification template for mapping DTPs to the

nodes of the MUS was developed to judge error etiology. When DTPs were not

classified into the nodes of the MUS the judgment of three reviewers were used to

classify them (Dr Charles D Hepler, Sooyeon Kwon, and Brian C Sauer). Classification

required 100% agreement, each reviewer judged the DTPs location in the MUS

independently. The reviewers discussed their individual decisions and when

discrepancies occurred they stated their opinions and worked through the differences

until agreement was obtained.

The authors focus, as well as any additional information mentioned in the article,

was considered when assigning the DTPs to nodes of the MUS. When an author in some

cases appeared to include both dose prescribed and dose administered in the same

"overdose" group, the case was classified into a category labeled non-identifiable process

problem. See Appendix B for a complete list of DTPs and information used to make

judgments for the placement of DTPs into nodes of the MUS.












Miscellaneous

Anti-diabetic

Anti-inflammatory

Psychotropic

Antibiotic

0 5 10 15 20 25 30 35
Figure 4.1: Involvement of Treatment Category to DRMs: Median prevalence for
Treatment Categories Mentioned in at Least 6 studies


Diuretics

Hypoglycemic

NSAIDs

Antihypertensive

Antibiotics


-I-I-I-I-


-- -

-- -

-- -


10 12


Figure 4.2: Therapeutic Class Involvement in DRM: The Number of Studies that
Specifically Mentioned Therapeutic Class









Since the purpose was to identify process related problemsf that led to PDRA, non-

preventable ADRs were not included in the analysis. ADRs are a special type of DTPs;

they typically represent an outcome and not a process of drug therapy. They have a

gradient of severity; the less severe ADRs would be considered an actual DTP, e.g.,

diarrhea. The severe ADRs would be considered DRMs, e.g., hospitalization from

dehydration due to frequent diarrhea. ADRs have historically been treated as the inherent

risk of medication use and not as a failure of drug-therapy management. Preventable

hospital admissions due to ADRs, nevertheless, may represent a process failure, because

somewhere in the MUS actions were not taken to detect and correct the ADR from

developing into a serious injury. Preventable ADRs were classified into the nodes of the

MUS according to the authors' description. If no description of the process failure was

presented then, as default, preventable ADRs were assigned to the monitoring node.

To be included, studies had to specify the type of DTPs, or the node in the MUS

where the DTP was initiated, that led to PDRAs. Eight studies linked DTPs to

preventability,5-12 (Appendix B) the other seven presented DTPs in relation to DRM, but

did not partition DTPs by preventable admissions.

The median values and inter-quartile range (IQR) for each node in the MUS were:

prescribing (33.93%; IQR: 5.73-54.86%), dispensing (0%), self-administration (22.29%;

IQR: 14.19-25.51%), monitoring (31.80%; IQR: 20.13-45.22), non-identifiable process

(5.93%; IQR: 0-16.39). (Appendix C and Figure 4.3)



fThe phrase, "problems with the process of care" is being used as an alternative to represent the same
concept as error. When discussing the etiology of DTPs terms related to process failures and process
problems will often be used since the term error has the ability to produce a defensive stance. When
examining the causes of DRM it is important to reduce tension and the defensive armory of professionals to
uncover factual events and true beliefs about the processes of care delivered.









The finding that prescribing and monitoring problems were major contributors to

PDRAs in these studies is troubling because it clearly indicates the MUS failed at the

professional/patient interface. Another important finding was problems in the monitoring

node were as prominent as problems in prescribing. This is interesting because

inappropriate prescribing tends to receive more attention than monitoring. Many

resources are being directed toward improving prescribing decisions, as seen through the

popular interest in computerized prescription order entry systems. From these findings

one could argue that interventions towards improving systematic monitoring and follow-

up of drug therapy should receive at least as much attention as prescribing.

(Self) administration or patient non-compliance was also found to be a substantial

contributor to PDRAs. Patients and caregivers are participants in microsystems of care

and they have a critical role in the MUS. (Self) administration is a patient behavior and

its overlap with medical error and professional responsibility can be debated. Non-

compliance is a result of intended and unintended actions. It has been shown that patients

make poor decisions because of rule and knowledge based mistakes.57 Health

professionals should be able to help patients correct their misunderstandings. They can

also help patients find creative ways to prevent unintentional non-compliance by

reducing slips and lapses. Nevertheless, patients typically choose to initiate the processes

involved in the MUS and in doing so they incur some of the responsibility for their health

outcomes. From a systems perspective, finding methods to reduce the occurrence of

PDRM is not limited to directing interventions at health care professionals and provider

specific processes. Patient directed interventions, and reconfigurations directed toward

patient behavior would be included in the systems umbrella.










Median


34%, 16% Prescribing
% Administration
iMonitoring
ONon-identifiable process
24%

Figure 4.3: Nodes of the MUS Involved in PDRA

Prescribing, monitoring and patient (self) administration appear to be the nodes

where most DTPs were generated. Because prescribing, monitoring and (self)

administration nodes have different characteristics it is possible error provoking

conditions upstream and faulty corrective mechanisms downstream (i.e., latent failures)

have unique system design flaws. Any "rational intervention" directed at a specific node

in the MUS would need to address the system conditions that increase the chance of an

error being created and progressing undetected.

Latent Causes of PDRA

Latent causes are flaws in the design and organization of systems that allowed

errors to occur, go undetected and result in patient injury. None of the fifteen studies

provided a comprehensive assessment of latent causes. Nevertheless, eight studies5-8'10-12

did mention, in the discussion, possible latent causes. (Table 4.3)

Lack of adequate knowledge was mentioned most frequently as an underlying

cause for errors in the prescribing node of the MUS (seven studies). Three of these

studies mentioned patient knowledge was a probable reason for problems in the (self)

administration node. Education was considered the best intervention for both prescriber

and patient knowledge deficits.

Three studies mentioned lack of communication and coordination among

professionals, especially the physician and pharmacist, was an underlying cause of









PDRAs. Two studies implied the lack of pharmacists' involvement in the MUS was a

factor and one study mentioned inadequate monitoring procedures were an underlying

cause.

Unlike many inpatient DRM studies, none of the fifteen PDRA studies

systematically evaluated the latent causes or empirically tried to locate system failures or

determine their relationship to higher order or competing systems. Without an adequate

evaluation of these influences and interactions, the development of meaningful

interventions will likely be difficult. Instead, corrective approaches will probably result

in generic interventions, those that the organization know how to do, for example send a

letter to the physicians who seem to be producing a portion of the problems.

For meaningful improvements to occur, the development of "rational interventions"

are necessary. The term rational is being used to represent interventions that are directed

at specific underlying system related causes. These are typically design and organization

issues. If lack of prescribing knowledge was determined to be a main cause of

prescribing errors, then a rational intervention may include finding better ways to provide

therapeutic information to prescribers at the time therapeutic decisions are being made.

From this analysis it is clear that a formal cause-and-effect analysis is needed to uncover

conditions of the MUS that make it prone to problems within the medications use system.

To summarize, injury as a result of drug therapy is a serious problem in ambulatory care.

A review of fifteen studies showed the median PDRA prevalence was approximately four

percent. Many common treatment categories were involved in these DRA.

Cardiovascular and anti-inflammatory agents were involved in many DRAs. Errors in

the MUS are the apparent causes of PDRM and problems in the prescribing, monitoring










and (self) administration nodes occurred frequently. None of the studies systematically

evaluated the latent conditions that provide the opportunity for proximate errors to occur

and go undetected. Lack of prescriber and patient knowledge was mentioned as latent

causes for prescribing and (self) administration problems. Nevertheless, a planned

systematic analysis is warranted. A better understanding of latent conditions could be

used to develop rational interventions. CEA is a promising method for uncovering latent

conditions.

Table 4.3. Latent Causes of PDRAs
Prescriber's Patient's Monitor Involve Inter-professional
Education Education pharmacist Communication
Darchy, 1999 /
Raschetti,1998 /
Cunningham, 1997 /
Nelson, 1996 / / / /
Dartnell, 1996 / / /
Courtman, 1995 / / /
Bero, 1991 / /
Trunet, 1980 /














CHAPTER 5
METHODS

This study was executed in three steps. In the first step, database analysis, MU-PIs

were used to estimate the number of PDRM positives in the study database. In step two

the degree of association between selected MU-PIs and specific nodes of the MUS where

the process failure may have originated was evaluated. In step three, selected MU-PIs

from each node of the MUS were submitted to cause-and-effect analysis (CEA).

Specific Aims

1. To establish the frequency of MU-PI positives in the study population.

2. To evaluate population based explanations for MU-PI positives.

3. To identify the node of the medications use system (MUS) where the pattern of
care from the MU-PI originated.

4. To identify system related causes that are common among nodes of the MUS

5. To identify system related causes that are unique to nodes of the MUS

Step One: Database Analysis

Claims Data Types

The MU-PIs were used to screen the study database. Each MU-PI was translated

into the administrative billing codes that represent the process and outcome components

of the indicators.13 The billing codes included the International Classification of Disease

9th edition (ICD-9) and Current Procedural Terminology (CPT) and National Drug

Codes (NDC).

The population included patients of all ages who were enrolled in a preferred

provider organization (PPO) health plan that managed employees from a member of the









health care coalition. The primary data files include Professional Claims (Center for

Medicare Services (CMS) 1500 form), Facility Claims (Universal Billing (UB) 92 form)

and Pharmacy Claims. The CMS 1500 and UB 92 forms are provided at the following

web address: http://www.cms.hhs.gov/providers/edi/edi5.asp#Form%20CMS-1500

Descriptive Analysis to Evaluate the Integrity of Claims Data

The number of office visit, emergency department (ED) visit, hospital admission

and pharmacy claims were compared month by month. If any month showed a large drop

in claims, this would suggest large numbers of claims might be missing from the

database. Office visit and ED visit claims were identified in the professional claims

database with specific CPTs.a Primary diagnosis, NDC, procedure codes, and service

dates were evaluated to identify missing or invalid codes. Links between databases were

established to ensure members could be traced across claim types.

Population Demographic

Average length of time enrolled in the health plan, average number of office visits,

age and gender frequencies were calculated to describe the study population. Frequency

of age by category, gender, number of office visits, and number of different drug classes,b

different pharmacies, different conditions, and different prescribers were used in logistic

regression analysis to better understand the MU-PI findings from information available in

the database.





a See Appendix E for a list of the codes used to identify office visits and ED visits
b The Universal System of Classification (USC) was used to identity different drug classes and routes of
administration. Unique USC codes identified drug classes (e.g., H2 blockers vs. GI proton pump) as well
as different routes of administration (e.g., sumatriptan oral vs. sumatriptan nasal). Drug class was
identified by unique USC codes.









MU-PI Coding Concepts and Analysis

Faris' 2001 translation of the PDRM scenarios into medical event codes was used

to construct the MU-PIs.13 In Faris' study, two medical record coders had independently

selected all possible ICD-9 and CPT codes that represented the PDRM scenarios. The

codes identified by the medical record coders had also been reviewed by a physician for

clinical judgment about whether each code was consistent with the PDRM scenario.

Please see Faris 2001 for more detail on the coding methodology.13 A list of the 40 MU-

PIsC used in this study can be found in Appendix D.

Search algorithms

Each MU-PI had the following format, an outcome of care and a specific pattern of

prior care. For example, ED visit or hospital admission for Hemorrhagic event AND use

of warfarin AND prothrombin time/INR not done every month. The pattern of care is the

process component of the indicator and it represents a potential DTP. An ED visit or

hospitalization was required for the outcome of care. An indicator positive (possible

PDRM) required both the process failure and the outcome component of the indicator.

Process failures fell into three categories: disease-drug interaction, drug

monitoring, and drug-drug interactions. Each required different types of search

algorithms. SAS version 8.2 was used to code and run the indicators in the database.

The type of search algorithm used for each indicator is presented in Appendix F.

Disease-drug interaction

The disease-drug algorithm required the presence of specific diagnosis codes and

specific pharmacy claims prior to a claim for the associated outcome. As illustrated in


Only 40 of Faris' 49 indicators were run. Search algorithms were to complex for nine and they were not
used for this analysis.









Figure 5.1, time moves from right to left by the blue arrows. The diagnosis claim had to

be present before the use of the medication (pharmacy claim). The green arrow

represents the lag, i.e., the duration of time from the outcome date to the last drug date

prior to the outcome. The cut off for the disease-drug interaction lag was set at 100 days,

in order to accommodate 90-day mail-order pharmacy refill cycles and to include patients

who may have been taking the medication in proximity to the outcome. An indicator

positive required the outcome, the disease code(s) and a pharmacy claim for the drugs)

within 100 days prior to the outcome.


Prior Diagnosis Last Drug Date Out e
Date Prior to Outcome

Lag



Figure 5.1. Disease-Drug Interaction Search Algorithm

Drug monitoring

Drug monitoring algorithms required the presence of specific pharmacy claims and

CPT claims, which indicate whether specific laboratory analyses were conducted. Two

coding solutions were required for monitoring indicators: one for the process component

and another for the process and outcome simultaneously. As illustrated in Figure 5.2,

time moves from right to left by the blue arrows. Once specific drug claims initiate the

process, lags are calculated to represent the interval of time from drug to first CPT claim

(lag 1), CPT claim to CPT claim (lag 2), and last drug claim date to last CPT claim date

(lag 3). Patients were considered at risk and included in the analysis when they appeared

to be taking the medication for at least as long as the defined lag time. Monitoring

intervals (i.e., lag times) vary according to MU-PIs. If any lag exceeded the defined









monitoring interval the member screened positive for the process component of the MU-

PI.

LAG 2a LAG 2b

Fs Du DCPT CPT ./ nth CPT
First Drug Date -- < tPT /a \ date 2 / date Last Drug D


LAG 1 LAG 3

Figure 5.2. Process Search Algorithm for Indicators that Require Monitoring

PDRM positives were identified when members had specified pharmacy claims and

the lag between the outcome and the last CPT claim prior to the outcome exceeded the

defined monitoring interval. If the lag was less than four days, the lag between the

outcome date and second to last CPT date was used to avoid false negatives. The

assumption is that monitoring within four days of hospitalization identified the

impending injury and need for emergency or hospital care. (Figure 5.3)

Drug Date <, Last CPT > OutcomeDate



LAG


Figure 5.3. Process and Outcome Search Algorithm for Indicators that Require
Monitoring

The determined monitoring interval for the warfarin indicator is 30 days. The

example in Figure 5.4 would register as a process positive, but not a PDRM positive

because the pattern of care was stabilized before the outcome occurred. The outcome

may indeed be drug related, but it is not represented by the MU-PI indicators. The lag

had to exceed the defined monitoring interval from outcome date to last CPT claim for a

PDRM positive.










50 days 25 days


INR INR nth INR
First Warfarin INOutcome Date
Claim W Claim 1 Claim 2 Claim OutcomeDate
date date date


40 days 25 days
Figure 5.4. Example of a Process Positive but not a PDRM Positive

Drug-drug interaction

The third search algorithm considered multiple drugs. This included drug-drug

interactions, drug-no drug situations and overuse of one drug and under use of the other.

The process component is indicator specific. If the MU-PI is a simple drug-drug

interaction with Drug A being a chronic medication and Drug B acute process positives

are recorded when a claim for Drug B occurred between the first and last claims for Drug

A. (Figure 5.5) PDRM positives are recorded when pharmacy claims for Drugs A and B

occur within the specified number of days for each lag.

Drug A Drug B Drug A
Diagnosis Date First Drug Date Last Date Last Drug Date



ALAG

Outcome Date
LAG 2
Figure 5.5. Drug-Drug Interaction Search Algorithm

Prevalence Estimates for MU-PI Positives

Specific Aim 1. To establish the frequency of MU-PI positives in the study
population.

Research Question 1. What is the period prevalence for process positives in the
study population?
Research Question 2. What is the period prevalence for the PDRM positives in
the study population?









This is a cross-sectional (period prevalence) analysis. The population included

patients of all ages who were enrolled in a PPO health plan from January 1st 1999 to

September 11th 2001. The analysis was limited to one process positive per indicator;

however, no limits were placed on PDRM positives.

Demographic Explanations for Prevalence Findings

Specific Aim 2. To evaluate demographic based explanations for MU-PI
positives.

Information from the database was used to better understand indicator positives.

Descriptive analyses of monitoring intervals were conducted to determine if cases on the

margin of the monitoring interval were being identified as process positives, and

multivariate logistic regression was conducted to explore the relationship between PDRM

positives and demographic/system related variables.

Distribution of Monitoring Intervals

Research Question 3. Do many process positives appear to be captured from the
"margin" of the defined monitoring intervals for indicators
that require drug therapy monitoring?

Due to the nature of the MU-PIs, the quality of drug-therapy monitoring can be

grouped in one of two categories. A case can either screen positive or negative for the

inappropriate monitoring interval. When a case screens positive it means an individual

did not have claims for the defined laboratory tests within the specified monitoring

interval. One limitation of this approach is the strict cut-offs-those on the margins are

categorized as process failures even if they were only a few days or weeks over the

required time. To develop a more comprehensive picture of how cases deviated from the

defined monitoring intervals, an analysis of the first monitoring interval (i.e., the lag time

from the first pharmacy claim for the drug of interest to the first claim for laboratory tests









of interest) was presented for indicators grouped by the indicator defined monitoring

interval. This was a descriptive analysis. The intervals were determined by dividing the

monitoring requirement in half. For example, if required monitoring for the indicators

was one-month, the categories for descriptive analyses were based on fifteen day

intervals. Analysis was only carried out for indicators with defined monitoring intervals

of one month or greater.

Variables Associated with PDRM Positives

Research Question 4. What demographic variables are associated with MU-PI
findings?

Available information from the database was used to explain the prevalence

findings. Bivariate and multivariate logistic regression analyses were conducted.

Demographic variables age, gender, and number of drug class, pharmacies and

prescribers were used as independent variables, while PDRM positives were the

dependent variables.

Multicollinearity was evaluated with tolerance and variance inflation statistics.

Tolerance is 1-R2 for the regression of a given independent variable on all other

independents, ignoring the dependent. The higher the intercorrelation of the

independents, the more the tolerance will approach zero. If tolerance is less than 0.20 a

problem with multicollinearity is indicated. After multicollinearity was assessed,

stepwise variable selection was used to build the regression model.

The likelihood ratio test was used to test the overall significance of the model.

Max-rescaled R2 was used to assess the predictive power of the model. The significance

of the variables in the model was assessed by the Wald Chi-Squared test and confidence

intervals (CIs). Hosmer-Lemeshow statistics were calculated to determine the goodness-









of-fit of the final multivariate model. Odds ratios (ORs) and Wald's CIs were calculated

directly from the estimated regression coefficients and their standard errors. All P-values

were set at 0.05. All analysis were performed using SAS Version 8.2

Step Two: Node Identification of Select MU-PIs

Specific Aim 3. To MUS where the pattern of care from PDRM scenarios
originated.

The ultimate goal of this dissertation was to identify system factors (i.e., latent

conditions of the health care system) that appear to be common or unique to nodes of the

MUS. This requires an exploration of the association between selected indicators and

specific nodes of the MUS. To do this, selected MU-PIs were sent to a Delphi panel of

clinicians and researchers for their judgment on the node of the MUS in which the

process component of the indicator likely originated.

Pilot Testing

The node identification panel survey was pilot tested with graduate students and

faculty in Pharmacy Health Care Administration at the University of Florida. All five

participants were pharmacists, three had received their pharmacy degrees in the United

States, one had received her degree in South Korea and the other had received his degree

in Germany.

The purpose of the pilot test was to evaluate the understandability of the survey and

the node identification scoring method. Organization of content, directions, and scoring

system were modified based on comments from the first round of the pilot test. No

significant problems or need for changes were identified in the second round of pilot

testing.









Selection of MU-PIs for Node Identification Study

MU-PIs used in the pilot were not the same as those used in the Delphi study.

Indicators were chosen based on preliminary results from the MU-PI study. MU-PIs

were primarily chosen based on indicator frequency.d

Delphi Recruitment

The snowball sampling or chain referral sampling was used to identify participants

for the Node Identification Delphi study. This method has been widely used in

qualitative research. It yields a study sample of people with specific characteristics, e.g.,

expertise. People with desired characteristics mention others who may also be likely

candidates for study participation.58

The first wave of recruitment letters was mailed with stamped return envelopes to

researchers who have published extensively in the field of medication error research or

who were referred by faculty members in Pharmacy Health Care Administration at the

University of Florida. The letter specified the goal of the study, asked the contacts to

participate in the Delphi panel, and invited them to recommend others who are trusted

experts in medication error or adverse drug events researcher I telephoned the first round

contacts between one and two weeks after the recruitment letter was mailed to answer

questions the contacts may have had and to try to convince them to participate.f

Sample size and power for the statistical tests were considered prior to the

recruitment process. When setting alpha at 0.05 and beta at 0.8, it was determined that a

d Scenarios were selected based on preliminary indicator findings, changes were made to the MU-PIs and
not all scenarios selected for the Delphi study had PDRM indicator positives.

e The term adverse drug event was used for the Delphi study instead of drug related morbidity because it is
the standard term in the medical community and I wanted to avoid confusion among the panel members.

f Please see Appendix G for a copy of the recruitment letter.









sample size of thirty-two was needed to identify a node as statistically different from the

others when its mean value was at least a six and the standard deviation was less than or

equal to three.

Node Identification Survey

Surveys were mailed to members who agreed to participate. The survey provided

detailed instructions that included definitions for the nodes of the MUS for the

assignment task. Specific demographic information was requested and it included name,

occupation (researcher and/or licensed health professional), whether they were practicing

or not, level of education, age, and gender, belief about the significance of PDRM

The panel members were provided fourteen MU-PIs and were asked to allocate the

number of points from 1-10 into the nodes of the MUS where they believe the error or

drug therapy problem originated for each of the fourteen indicators. They were asked to

answer based on a population perspective, meaning they were asked to identify the node

that explained the origin of the majority of cases that might fit the indicator definition.

Panel members were also asked to document their decision logic and provide comments

when they deemed necessary.g

Delphi Process

Results (panelists' scores) for each MU-PI were analyzed after the first round. If

an indicator had received an average score for one node that was significantly higher than

its average score for all other nodes, it was "assigned" to that node and deleted from the

second round of the Delphi study. In the second round, panel members received

information about each remaining indicator, including summary distributions of


g See Appendix H for a copy of the Node Identification Survey.









responses, median and interquartile ranges for each node, the particular Delphi member's

response to each item, and a summary of the decision logic statements for the panel

(grouped into like statements). Delphi participates were asked to reconsider their

opinions and to state their decision logic regardless of whether they changed their belief.

Node Identification Analysis

Research Question 5. What nodes of the MUS did the selected PDRM scenarios
likely originate from?

Hypothesis: Ho: Prescribing = Dispensing = Administration = Monitoring
Ha: At least one node is significantly greater than all other nodes.

The Kruskall Wallis test (KW), an analysis of variance test for non-parametric data,

was used to determine if differences in rank order means existed among the nodes of the

MUS. Dwass, Steel, Critchlow-Fligner (DSC) multiple comparison post hoc tests were

used to determine which nodes were significantly different (P values=0.05). StatsDirect

Version 2.3.7 software was used for the KW and DSC tests.

Step Three. MU-PI System Level Evaluation

Specific Aim 5. To identify common cause sequences among nodes of the MUS.

Specific Aim 6. To identify unique cause sequences for specific nodes of the MUS.

Selection of MU-PIs for Evaluation

The fourteen MU-PIs evaluated in step two were found to be problems that

originated in the prescribing or monitoring nodes. Two MU-PIs were selected from the

prescribing and monitoring nodes. Selection was limited to MU-PIs that were evaluated

in the Node Identification Study and was based on frequency of MU-PI positives.h



h Indicator number 28 was evaluated in the Delphi study and had the second highest frequency for the
prescribing indicators, however it was not selected because the indicator did not appear to differentiate
those with congestive hearth failure (CHF) and heart block from those with CHF only.









Step Three: Cause-and-Effect Analysis

MU-PI Evaluation Team

Members of the MU-PI evaluation team were identified and invited to participate

by the CEO of the health care coalition. They consisted of a Medical Director, PharmD,

nurse practitioner, consumer advocate, laboratory manager and a director of a health

maintenance organization. The session was scheduled for four hours. Members of the

evaluation team were paid $100 per hour.

One week prior to the meeting the evaluation team was mailed a briefing that

contained the goals for the study and background on the MU-PI results, cause-and-effect

analysis and the levels of the health care system. It also provided the MU-PIs that would

be evaluated and asked them to begin generating ideas. A copy of the brief is available in

Appendix I.

MU-PI Evaluation Process

The evaluation process began with a brief introduction and slideshow presentation

to orient them to the study purpose and database findings. The two MU-PIs formally

evaluated were the asthma "prescribing" indicator and the warfarin "monitoring"

indicator, indicators 39 and 30, respectively. (Appendix D)

The following six steps were carried out for each MU-PI that was formally

evaluated: brainstorm causes, clarify and organize causes, prioritize causes, tree-

diagramming, and establish causes that are common to multiple nodes of the MUS and

causes that are unique to specific nodes of the MUS.

The moderator provided an introduction and background of the study objectives

and indicator prevalence findings. Indicator specific finding were presented for the









warfarin "monitoring" indicator and the asthma "prescribing" indicator. The question to

guide the brainstorming process was presented for each indicator.

The evaluation team brainstormed possible system related problems that may have

contributed to the MU-PI indicators. The round robin process was used to elicit causes

from each participant. Proposed causes were projected on a screen and documented in

Path Maker Version 5.5 software.

The evaluation team then clarified and organized the proposed system causes into

an affinity table. The affinity table was organized by the levels of the health care system

(patient, professional, organization, environmental levels). Proposed causes were

organized into the level of the system the problem was associated with. A category for

measurement error called "artifact" was also permitted. Proposed causes were organized

into affinity tables with Path Maker Version 5.5 software

The evaluation team then prioritized the proposed system cause and the top five

causes were selected for tree-diagramming. The facilitator went through the list of

proposed causes from the affinity table and the evaluation team voted by raising their

hands for each of the five causes they believed contributed the most to the prevalence of

the indicator findings. Scores were tallied for each cause and the five receiving the most

votes were selected for tree diagramming.

The evaluation team then used tree diagramming to identify relationships among

the different levels of the health care, as they relate to the proposed cause. Tree diagram

consisted of branches that had each level of the health care system (patient, professional,

organization, environmental levels). One at a time, the top five proposed causes were

placed into the system levels associated with the cause. For example; if the proposed









cause was lack of MDs providing patient education it would go into the professional

(microsystem) level. The follow-up question would have been; what at the

organizational level may influence MDs ability to provide patient information? The

proposed organizational influence would be placed in the organization node of the same

branch of the tree. (Figure 5.6) Excel Version XP was used to create the tree diagrams

and document the findings.

Proposed Cause
No Medication Management System What at the organizational
Guidelines unclear level may influence MDs
ability to provide patient

Branch

SPatient level Professional level Organization / Environment

1 MDs don't provide patient

1 H H H





Figure 5.6. Organization of Proposed Causes to System Levels

The evaluation team then established commonality and uniqueness within and

between nodes of the MUS. An indicator assigned to the monitoring node was tree

diagrammed. The evaluation team was given a questionnaire to determine if each cause

sequence (i.e., each branch of the tree) would apply to another indicator assigned to the

monitoring node. They also had to judge whether each cause sequences would apply to

an indicator assigned to the prescribing node. The same procedure was applied to the

prescribing indicator that was formally evaluated, i.e., tree diagrammed. (Appendix K)






70


Analysis for Common and Unique Cause Sequences

Research Question 6. What cause sequences were common within the Prescribing
Node?
Research Question 7. What cause sequences were common within the Monitoring
Node?
Research Question 8. What cause sequences were common to the Prescribing and
Monitoring Nodes?

Agreement was determined when five out of six (83%) or four out of five (80%) of

judges agreed.














CHAPTER 6
RESULTS

Step One. Database Analysis

Descriptive Analysis to Evaluate the Integrity of the Claims Database

To evaluate the completeness of the database the numbers of claims were compared

by month. The dates of office visit claims ranged from January, 1999 to September,

2001. As illustrated in Figure 6.1, many office visit claims appear to be missing on the

tail ends of the data collection period, from January, 1999 to June, 1999 and from July,

2001 to September, 2001. Emergency Department (ED) visit claim dates ranged from

January, 1999 to June, 2001. As illustrated in figure 6.2, many ED visit claims appear to

be missing from January, 1999 until June, 1999. Facility admission claim dates ranged

from January, 1999 to June, 2001. No claims appeared to be missing during the study

period (Figure 6.3). Pharmacy claim dates ranged from January, 1999 to June, 2001. As

illustrated in figure 6.4, pharmacy claims appeared to be missing from January, 1999 to

June, 1999 and in September, 1999. Missing pharmacy claims from September, 1999 are

not followed by an increase in claims. It appears claims during this month were missing

from the database.

The decision was made to only include claims from July, 1999 to June, 2001 for

analysis. This decision was made because the number of claims was consistent during

this time frame. Even though pharmacy claims appeared to be missing in September,

1999 the decision to include the previous months was made because the benefit of

keeping July, August, and September in the database outweighs censoring the time.







72



Check for missing and invalid data

Macro level analysis found no missing or invalid codes for primary diagnosis,


procedures or service dates. Missing or uninterpretable drug codes were found in 530 of


the 572,010 pharmacy claims (571,480 usable pharmacy claims).



12000

10000 -

8000

6000

4000

2000
6000 0 ---- T 9-------------------------------


O^ O^ O^ O O^ O ^ ^ N JN N N N




Figure 6.1. Office Visits Claims by Month




14000

12000

10000

8000

6000

4000

2000

o n .. .





Figure 6.2. ED Visit Claims by Month

Links between data types

To ensure that members were identifiable across claim types links were established


between datasets to identify the number of members that had multiple types of claims.











4,227 of the 4,589 members who had a facility admission claim also had a professional

claim. 28,478 of the 33,891 persons with a pharmacy claim also had a professional claim



350

300

250

200

150

100

50

0





Figure 6.3. Facility Admissions by Month


35000
3000ooo0

25000oo
Q Nooo I N
2moo






60 0 ,0 ...... .





Figure 6.4. Pharmacy Claims by Month

Population Demographics

There were 47,053 members who contributed 58,719 member years during the


sample period of July, 1999 to June, 1999. The mean length of enrollment was 467.47

days with a standard deviation of(SD) 225.53 days. The 37,063 members with a medical









claim (pharmacy or professional) contributed a total of 51,892 member-years; mean

length of enrollment was 511.04 days with a SD of 201.73 days.

Age and gender distributions for the different claim types are presented in Table

6.1. The mean age ranged from thirty-four to thirty-seven across the databases. The

majority of members with a claim were female. Females ranged from sixty-five percent

to seventy-two percent across the claim types. The fifteen to forty-four year olds were

the most frequent age group, their frequencies ranged across the data types from thirty-

seven to forty-eight percent. The sixty-five and older group accounted for a small

fraction of the claims, their frequencies ranged from about two to eight percent.

Table 6.1. Population Age and Gender Frequencies by Type of Claim
Professional Facility D v Pharmacy
ED visit
Claim Admission Claim


mean age(SD) 33.97 (18.23) 34.48 (22.12) 34.71 (18.54) 37.03 (18.34)

By Category
frequency, [percent]
age <15 6,622 [19.07] 1,084 [18.51] 1,263 [17.61] 4,956 [14.3]
15<=age<45 16,519 [47.58] 2,143 [36.6] 3,399 [47.39] 16,461 [47.48]
45<=age<65 10,743 [30.94] 2,144 [36.62] 2,270 [31.65] 11,421 [32.94]
65<=age 835 [2.41] 484 [8.27] 240 [3.35] 1830 [5.28]

Female 22,861 [65.85] 3243 [71.84] 4,659 [64.96] 23,499 [67.78]
Male 11,858 [34.15] 1271 [28.16] 2,513 [35.04] 11,169 [32.22]
0 0 0 0


Table 6.2 displays additional population characteristics. There were 220,967 office

visit claims during the twenty-three month period with an average of 6.64 visits, which

were approximately three visits per member per year. Pharmacy claims were submitted

for 33,891 members. The average number of pharmacy claims per member during the

twenty-three month time period was 16.05 with a SD of 22.9, which were approximately









eight pharmacy claims per year. Members went to an average of 1.55 different

pharmacies and an average of 2.13 different prescribers during the study period.

Members averaged 5.8 conditions and used an average 4.7 prescriptions from different

drug classes.

Prevalence of MU-PI Positives

There were a total of 103 PDRM positives from eighty members. The prevalence

of PDRM positives in members with a medical claim from July, 1999 to June, 2001, was

1.98 (1.60 to 2.37)a per 1,000 member years. There were a total of 10,889 process

positives in 5,741 members. The prevalence of process positives from July, 1999 to June,

2001 was 209.84 (206.37-213.34) per 1,000 member years. Table 6.3 shows that, 0.22%

of the population with a medical claim screened positive for both the process and

outcome component of the MU-PIs, i.e., PDRM positives. There were forty-three PDRM

positives in thirty-eight members where the outcome was identified by an ED visit and

forty-one PDRM positives in twenty-six members where the outcome was identified by

hospital admissions (HA). Nineteen PDRM positives in eighteen members involved a

hospital admission from the ED. PDRM positives accounted for 0.75% (95% CI: 0.74%-

0.76%) of hospital admissions, and 0.56% (95% CI: 0.55%-0.57%) of ED visits. Table

6.4 provides a summary of the number of PDRM positives by number of indicators.

Twenty-two of the forty MU-PIs had no PDRM positives. Of the remaining eighteen

indicators, the range of PDRM positives per indicator was from zero to twenty-six. The

eleven indicators with the most frequent PDRM positives are listed in Table 6.5.


a 95% Confidence interval











Table 6.2. Population Demographics
Characteristics Frequency Percent


No of Enrollees
No of Members with claim
Length of time enrolled
mean (SD)
No. ED visits
No. Hospital Admissions
No. of outpatient physician visits
mean (SD)
No. of outpatient physician visits (members)


9-10
>10
No. of Conditions (diseases)
mean (SD)
LE 5
6-10
11-15
16-20
GE 20
No. of members with pharmacy claim
average per member (SD)
No. of drug classes
average per member (SD)


No of different pharmacies
average per member (SD)
1


6 or more
No of different prescribers
average per member (SD)
1


7 or more


47,053
37,063


511.04 (201.73)
10,048
5,855
220,967
6.64 (7.05)
33,271
10,281
6,781
4,480
3,167
2,295
6,267
34,938
5.8 (4.8)
20,714
9,161
3,382
1,030
651
33,891
13.93(22.61)
543,959
4.7 (5.31)
25,135
7,305
2,809
978
836


1.55 (1.45)
17,981
8,238
3,919
1,989
1,033
731

2.13(2.07)
12,482
8,281
5,302
3,206
1,989
1,117
1,514


All analysis are for the 23 month time window


30.9
20.38
13.47
9.52
6.9
18.84


59.29
26.22
9.68
2.95
1.86





67.82
19.71
7.58
2.64
2.26


53.06
24.31
11.56
5.87
3.05
2.16


36.83
24.43
15.64
9.46
5.87
3.3
4.47









Table 6.3. Frequency of Process and PDRM Positives
No. of Members with Population % No. of Positives (unit:
Positives (unit: person) (denominator=37,063) each event)
Process 5,741 15.49% 10,889
PDRM_ED 38 0.10% 43
PDRM_HA 26 0.07% 41
PDRMEDHA 18 0.05% 19
Total PDRM 80 0.22% 103

Table 6.4. Number of PDRM Positives by Number of Indicators
Freq. of PDRM Positive No. of Indicators Percent
0 22 55.0
1 3 7.5
2 4 10.0
3 3 7.5
5 3 7.5
9 2 5.0
12 2 5.0
26 1 2.5

Table 6.6 lists the indicators with the ten most prevalent process positives. Thirty-

eight of the forty MU-PIs screened positive for the process component of the indicator.

The range of process positives was from zero to 2,282. Approximately fifty-four percent

of the indicators had less than one hundred process positives, and ninety-two percent of

the indicators had less than 1,000 process positives.

System Based Explanations for Prevalence Findings

Information from the database was used to better understand indicator positives.

Descriptive analyses of monitoring intervals were conducted to determine if cases on the

margin of the monitoring interval were being identified as process positives, and

multivariate logistic regression was conducted to explore the relationship between PDRM

positives and demographic/system related variables.










Distribution of monitoring intervals


Table 6.5. Eleven Most Prevalent PDRM Positives
Ind Mnemonic Hx Drug_A Drug_B Risk Process PDRM
39 Asthma< 28 CHF< 37 CHF< 1 Dep< 30 Hemr< 19 Hypoth< 29 ARF< 31 MI 7 Dep< 14 ActRespFaik 16 CHF<
Table 6.6. Ten Most Prevalent Process Positives
Ind Mnemonic Hx Drug_A Drug_B Risk Process PDRM
29 ARF< 23 HyprK< 16 CHF< 19 Hypoth< 17 HypoK< 37 CHF< 1 Dep< 5 G< 30 Hemr< 27 G< Legend for Tables 6.5 and 6.6
* See Appendix D to link indicator numbers to the indicator scenarios.
* The 'Mnemonic' is an abbreviated version of the MU-PI scenario
* Hx = the number of members who had an ICD-9 code for the particular history of disease or diagnosis
of interest
* Drug A and Drug B = the number of members who had a prescription claim for the drugs of interest.
* Risk = the number of members who met the criteria for analysis. For example; if the process component
includes a drug and six month monitoring interval, then the members at risk would be those who were
using the medication of interest for at least six months.
* Process = the number of members who had the pattern of care represented in the MU-PI
* PDRM = the number of members who had both the process component and the outcome component of
the MU-PI within the specified time frame.
* Fourteen indicators were sent to a Delphi process for node identification, they were numbered
consecutively from one to fifteen. This column provides a cross reference to the MU-PI number

Due to the nature of the MU-PIs, the quality of drug-therapy monitoring can be

grouped in one of two categories. A case can either screen positive or negative for the









inappropriate monitoring interval. When a case screens positive it means an individual

did not have a laboratory test claim within the specified time frame. One limitation of

this approach is the strict cut-offs-those on the margins would be categorized as process

failures even if they were only a few days or weeks over the required time. To develop a

more comprehensive picture of how cases deviated from the specified monitoring

intervals, an analysis of the first monitoring interval (i.e., the lag time from the first

pharmacy claim for the drug of interest to the first laboratory test claim of interest) is

presented for indicators grouped by the required monitoring interval. Analysis was only

carried out for indicators with defined monitoring intervals of one month or greater.

Table 6.7 provides the indicator numbers for each monitoring interval. For

example: Only indicator 30 had a one month monitoring requirement, while indicators 9

and 25 had a three month monitoring requirement.

Table 6.7. MU-PI Categorized by Required Monitoring Interval
one month two months three months six months twelve months
30 12 9 2 19
35 25 8
11
18
23
24
26
29
33

Indicator 30 (Hemm warfarin INR (1)) was the only indicator with a required

monitoring interval of one month. Figure 6.5 shows forty-one percent of the members

who were taking warfarin had a laboratory test claim for an International Normalization

Ratio (INR) within a month from their first warfarin pharmacy claim. Only about 3.8%

had laboratory test claim on the "margin" within forty-five days of their first warfarin










claim. The majority, 50.4 percent did not have a laboratory test claim for an INR within

120 days of their first claim for warfarin.











100
80
60

40 3.79% 1.75%
20 0.87 0.87% 0.87%

days<30 30 lag time

Figure 6.5. First Lag for Indicator with One Month Monitoring Requirement

Indicators 35 and 12 had a required monitoring interval of two months. Figure 6.6

shows 0.56% of members at risk for these indicators had the specified laboratory test

claims (blood pressure and cell blood counts) within two months of their first pharmacy

claim, while 99.4% did not have the specified laboratory test claim within 240 days from

their first pharmacy claim. Indicator 35 accounted for all but one process positives.

Indicator 35 required blood pressure monitoring it is likely that blood pressure

monitoring is not billed separately and the process positives are false positives. This

indicator was removed from all other analysis (including the previous prevalence

findings) because the CPT codes used to identify blood pressure monitoring were not

sensitive to blood pressure monitoring.

Indicators 9 and 25 had a required monitoring interval of three months. Figure 6.7

shows 4.35% of members at risk for these indicators had the specified laboratory test










claims (lithium levels and BUN) within three months of their first pharmacy claim for the

defined medication, while 84.78% did not have the laboratory test claim within 360 days

from their first pharmacy claim. No lags were identified on the margin of the required

interval.

Indicators 2, 8, 11, 23, 24, 26, 29, 33 had a required monitoring interval of six

months. Figure 6.8 shows 24.78%of members at risk for these indicators had the

specified laboratory test claims within six months of their first pharmacy claim for the

defined medication. Some process positives were captured from the margin, 6.53% had

the laboratory test claim between 180 and 270 days.



99.44%


1.30 '/o


days <=60


0 0% 0% 0%


90

150 lag tim


Figure 6.6. First Lag for Indicator with Two Month Monitoring Requirement


































days<=90


4.35% 2.1


0% 2.17% 0% 2.17%


135

225 lag time '


Figure 6.7. First Lag for Indicator with Three Month Monitoring Requirement


Figure 6.8. First Lag for Indicator with Six Month Monitoring Requirement


ArO%/


















2954


1 2 Q


days <=365 365 lag time

Figure 6.9. First Lag for Indicator with Twelve Month Monitoring Requirement

The majority of members who met the requirements to be considered for the MU-

PIs with six month monitoring requirements (57.97%) did not have the specified

laboratory test claim within 720 days from their first pharmacy claim.

Indicator 19 had a required monitoring interval of twelve months. Figure 6.9

shows the majority of the members who appeared to be using thyroid therapies (52.25%)

had their thyroid tests (laboratory test claims) within twelve months of their first

pharmacy claim for the defined medication. About thirteen percent had the specified

laboratory test claim between 365 and 545 days, about six percent had a claim between

545 and 730 days from the pharmacy claim, and about thirty percent did not have the

laboratory test claim within 730 days of the first pharmacy claim for the required

medication.

In summary, the majority of process positives did not result from the margins of the

required lag interval. The majority of process positives resulted from the "extreme" of









the distribution. The indicators that required one month, six month and twelve month

monitoring intervals were picking up some process positives from the margins, 3.78%

6.53%, 12.58%, respectively.

Variables associated with PDRM positives

Personal variables associated with PDRM positives were identified with bivariate

logistic regression. Table 6.8 displays the OR and CI for each dependent variable. Age,

and number of office visits, prescribers, pharmacies, drug classes and conditions were

significantly associated with PDRM positives. Gender was not significant (odds ratio

OR: 1.00; confidence interval CI: 0.63 1.59). Number of pharmacies and number of

prescribers were the strongest bivariate predictors (OR: 1.48; 95% CI: 1.43-1.63) and

(OR: 1.48; 95% CI: 1.40 -1.57), respectively.

Number of drug class and number of conditions had the next strongest association

(OR: 1.19; 95% CI: 1.16-1.21) and (OR: 1.20; 95% CI: 1.18- 1.23), respectively. Number

of office visits and age had significant but weak bivariate associations (OR: 1.07; 95%

CI: 1.06-1.09) and (OR: 1.06; 95% CI: 1.05-1.08), respectively.

Multivariate multicollinearity was assessed using the tolerance score. As shown in

table 6.9, no tolerance values were below 0.20. The lowest tolerance score was 0.256,

meaning 0.256 of the variance in number of office visits was unexplained by the other

independent variables.

A multivariate logistic regression was conducted to explore the relationship

between PDRM positives and demographic/system related variables. Logistic regression

with PDRM positives as the dependent variable and the seven demographic/system

related variables as the independent variables used stepwise variable selection to produce

the model. A four-variable risk model was produced. The model indicated that males









Table 6.8. Bivariate Tests of Association: PDRM Positives as Dependent Variable
Independent Variable Odds Ratio 95% Wald CI
Age 1.06 1.05 1.08
Gender F vs. M 1.00 0.63 1.59
number of Office visits 1.07 1.06 1.09
number of Prescribers 1.48 1.40 1.57
number of Pharmacies 1.48 1.34 1.63
number of Drug class 1.19 1.16 1.21
number of Conditions 1.20 1.18 1.23

were about 50% more likely to have a PDRM positive after adjusting for the other

variables in the equation (OR: 0.45; 95% CI: 0.28-0.74). Number of office visits had a

negative influence on the odds of having a PDRM positive (OR: 0.93; CI: 0.9-0.96).

That is, each additional office visit reduced the probability of having a PDRM positive by

approximately seven percent, after adjusting for the other variables in the model.

Number of drug classes and number of conditions increased the odds of having a PDRM

positive. When drug class was held constant at zero, each additional medical condition

increased the odds of being PDRM positive by about twenty-nine percent. When medical

conditions were held constant at zero, each additional drug class increased the odds of

having a PDRM positive by about twenty-two percent. Number of drug classes and

number of medical conditions had a significant interaction with a negative beta-

coefficient (B coefficient: -0.004; p-value: 0.0014). Increases in both medical conditions

and drug classes resulted in decreased odds of having a PDRM positive.

The overall significance of the equation by the likelihood ratio had a chi-square of

306.007 (p< 0.0001) with four degrees of freedom (df). The proportion of variance

explained by the model (max-rescaled R2: 0.279). Overall the model fit well to the data

(Hosmer-Lemeshow: 4.01; p=0.779; df=7).










Table 6.9. Multicollinearity Assessment of Independent Variables
Parameter Standard Variance
Variable DF me t value Pr > Itl Tolerance ia
Estimate Error inflation
Intercept 1 -0.04072 0.00394 -10.35 <.0001 0
Age 1 -0.0000551 0.00007498 -0.73 0.4624 0.84942 1.17728
number of Conditions 1 0.00724 0.00020712 34.96 <.0001 0.26857 3.72342
number of Drug Classes 1 0.0066 0.00015911 41.46 <.0001 0.31993 3.12572
number of Office Visit 1 -0.00421 0.00014943 -28.19 <.0001 0.25561 3.9122
number of Pharmacy 1 -0.00345 0.00055755 -6.18 <.0001 0.78954 1.26657
number of Prescribers 1 -0.01089 0.00045741 -23.8 <.0001 0.41653 2.40079

Table 6.10. Maximum likelihood Estimates: PDRM Positives as Dependent Variable
Standard
Parameter DF Estimate ard Wald chi-Sq Pr > ChiSq
Error
Intercept 1 -9.229 0.404 521.676 <.0001
Gender, F 1 -0.398 0.126 10.014 0.0016
number of Conditions 1 0.254 0.031 66.933 <.0001
number of Drug Classes 1 0.200 0.024 69.664 <.0001
number of Office Visits 1 -0.074 0.017 19.389 <.0001
Cond*Drug class 1 -0.004 0.001 10.156 0.0014

Step Two. Assignment of MU-PIs to Nodes of the MUS

The ultimate goal of this dissertation was to identify system factors (i.e., latent

conditions of the health care system) that appear to be common or unique to nodes of the

MUS. To do this, the node of the MUS where the MU-PIs' process problem originated

was determined. Selected MU-PIs were sent to a Delphi panel of clinicians and

researchers for their judgment on where in MUS the process component of the indicator

likely originated.

Delphi Recruitment

A total of fifty-seven recruitment letters were mailed out. Twenty-nine contacts

agreed to participate, seven declined, and twenty-one did not respond. Twenty-nine

contacts were e-mailed or mailed the survey (based on individual request) and eighteen

completed and returned the first round. The Delphi panelists consisted of five medical

doctors (MDs), one nurse, five doctors of pharmacy (PharmDs), six bachelors of










pharmacy (BA pharm), and one masters in health service research (MHSA). Fifteen of

the seventeen members with clinical degrees were licensed in the US and eight were

currently practicing. All of the panelists with clinical degrees had additional graduate

level degrees. (Table 6.11).

Table 6.11. Demographics of Delphi Panelists for Node Identification Study
Other Licensed in
Clinical Degress Dther Gender censed inPracticing
Degree the US
2 MPH
MD 5 1MS
1PhD
Yes =17 Nurse 1 1MPA
.............. .......... .......6 F e m a le 1
PharmD 5 1 PhD 12 Male
1 MBA PhD
BA Pharm 6
4 PhD
No = 1 1 MHSA
Total Number of Delphi Panel = 18

Assignment of MU-PIs to Nodes of the MUS

The Node Identification study went through two rounds of the Delphi process. An

indicator was assigned to a node and not returned for the second round when the average

score for a node was significantly greater than the average scores for all other nodes.

Eleven indicators reached significance within the first round. Three of the indicators

were returned for the second round and two achieved significance. Based on the Delphi

panelist responses, problems with the specificity of the indicators were the reason they

did not reach significance in the first round. Changes in the indicators were made and

they were sent back to the Delphi panel. The panelists were informed of the changes

made to the indicator scenarios. They were also provided with their score, the groups'

average score and a summary of the panelists responses for the three indicators. The

changes made to the three indicators included,









* Changing "sympathomimetic decongestants" to "prescription sympathomimetic
decongestants" for Delphi indicator 9.

* Removing "INR not done before therapy starts" from Delphi indicator 12 (the
monitoring requirement was one-month)

* Changing "antibiotic use" to use of"macrolide or sulfonamide antibiotics" for
Delphi indicator 14

The indicators were judged to be primarily associated with either the prescribing or

monitoring node of the MUS. None of the indicators were found to be primarily

associated with the dispensing or administration nodes of the MUS. Six indicators were

assigned to the prescribing node, seven to the monitoring node and one did not reach

significance within the two rounds. Table 6.12 provides a summary of the indicator

assignments.b The MU-PI column in Table 6.12 is the MU-PI number and it provides a

cross reference to the Delphi indicator number. Table 6.13 lists the statistics for each

indicator analysis. Mean scores for each node, Kruskal Wallis statistics and p-values are

presented for each indicator.

Significance of the Dwass, Steel, Critchlow-Flinger pairwise comparisons are

represented by an underscore under the abbreviated node. Nodes connected with an

underscore were not statistically different. Delphi indicators 1-6, 8, 10, 11,13, and 15

reached significance in the first round, all p-values <0.001 for KW statistic and pairwise

comparisons. Indicators 9, and 12 reached significance in the second round, p-values

<0.001 for KW and pairwise comparisons. Indicator 14 did not reach significance and

therefore it was not assigned to a node. Box plots for the mean, median and mode for

each indicator are provided in Appendix L.

b Delphi indicator number seven was not included because it was a repeat of Delphi indicator
number four.




Full Text

PAGE 1

MEDICATION USE PERFORMANCE INDICATOR EVALUATION: A SYSTEMS PERSPECTIVE By BRIAN C. SAUER A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2004

PAGE 2

Copyright 2004 by Brian C. Sauer

PAGE 3

To my parents

PAGE 4

ACKNOWLEDGMENTS I would like to acknowledge my dissertation chair, Dr. Charles D. Hepler, for taking me under his wings and sharing his knowledge with me. I have grown tremendously over the past five years and Dr. Hepler has guided that process. I thank the members of my supervising committee, Drs. Earlene Lipowski, Abraham Hartzema, Murray Cote, and Richard Segal for their patience and guidance through the process. I would like to thank Josue Rodas, Becky Cherney and Scott Langdon for providing the support needed to make this study happen. I thank the Perry Foote foundation for providing the grant that financed this project. I thank Dave Angaran for listening and helping me understand and think through the clinical complexity of my dissertation. I would also like to thank Dr. Alan Spector for giving me the opportunity to work in his laboratory as an undergraduate in biological psychology. This was an important time in my academic development. Like Dr. Spector, I hope I will always be able to roll-up my sleeves and loosen my tie when the critics are throwing heat. I also would like to thank Mircea Garcea for teaching me, through example, to always pay attention to the details and produce quality work. Finally, I would like to thank the graduate students for producing a healthy working environment and their friendship. iv

PAGE 5

TABLE OF CONTENTS page ACKNOWLEDGMENTS .................................................................................................iv LIST OF TABLES ...........................................................................................................viii LIST OF FIGURES .............................................................................................................x ABSTRACT .......................................................................................................................xi CHAPTER 1 INTRODUCTION........................................................................................................1 Problem Statement........................................................................................................1 Objective.......................................................................................................................2 Specific Aims................................................................................................................2 Justification...................................................................................................................2 Overview.......................................................................................................................3 2 CONCEPTUAL FRAMEWORK.................................................................................6 Adverse Outcomes of Drug Therapy............................................................................6 The Medication Use System.........................................................................................8 Levels of the Health System.........................................................................................9 Human Error and System Failure...............................................................................14 Summary.....................................................................................................................16 3 LITERATURE REVIEW...........................................................................................18 Use of Administrative Databases for Researching PDRM.........................................18 Automated Methods for Detecting PDRM.................................................................22 Delphi Method............................................................................................................29 Cause-and-Effect Analysis.........................................................................................35 4 PRELIMINARY WORK............................................................................................39 Prevalence of Preventable Drug Related Admissions................................................39 Drug Categories Involved in PDRA...........................................................................43 Apparent/Proximate Causes of PDRAs......................................................................48 v

PAGE 6

Latent Causes of PDRA..............................................................................................52 5 METHODS.................................................................................................................55 Specific Aims..............................................................................................................55 Step One: Database Analysis......................................................................................55 Claims Data Types..............................................................................................55 Descriptive Analysis to Evaluate the Integrity of Claims Data...........................56 Population Demographic.....................................................................................56 MU-PI Coding Concepts and Analysis...............................................................57 Search algorithms.........................................................................................57 Disease-drug interaction...............................................................................57 Drug monitoring...........................................................................................58 Drug-drug interaction...................................................................................60 Prevalence Estimates for MU-PI Positives..........................................................60 Population Based Explanations for Prevalence Findings....................................61 Step Two: Node Identification of PDRM Scenarios..................................................63 Pilot Testing.........................................................................................................63 Selection of PDRM Scenarios for Node Identification Study.............................64 Delphi Recruitment.............................................................................................64 Node Identification Survey..................................................................................65 Delphi Process.....................................................................................................65 Node Identification Analysis:..............................................................................66 Selection of MU-PIs for Evaluation....................................................................66 Step Three: Cause-and-Effect Analysis......................................................................67 MU-PI Evaluation Team.....................................................................................67 MU-PI Evaluation Process..................................................................................67 Analysis for Common and Unique Cause Sequences.........................................70 6 RESULTS...................................................................................................................71 Step One. Database Analysis:....................................................................................71 Descriptive Analysis to Evaluate the Integrity of the Claims Database..............71 Check for missing and invalid data..............................................................72 Links between data types.............................................................................72 Population Demographics...................................................................................73 Prevalence of MU-PI Positives...........................................................................75 System Based Explanations for Prevalence Findings.........................................77 Distribution of monitoring intervals.............................................................78 Variables associated with PDRM positives.................................................84 Step Two. Assignment of MU-PIs to Nodes of the MUS.........................................86 Delphi Recruitment.............................................................................................86 Assignment of MU-PIs to Nodes of the MUS.....................................................87 Step Three: Cause-and-Effect Analysis......................................................................89 Evaluation of Monitoring Indicator.....................................................................91 Evaluation of Prescribing Indicator.....................................................................92 Cause Sequences Common to both the Monitoring and Prescribing Nodes.......94 vi

PAGE 7

7 DISCUSSION.............................................................................................................97 Database Analysis.......................................................................................................97 Prevalence Findings.............................................................................................97 System Explanations for Prevalence Findings....................................................99 Distribution of monitoring intervals.............................................................99 Multiple logistic regression analysis..........................................................100 Node Identification of MU-PIs.................................................................................102 Cause-and-Effect Analysis.......................................................................................103 Cause themes..............................................................................................104 Conclusion................................................................................................................106 Limitations................................................................................................................108 Significance and Theoretical Contribution...............................................................109 Contribution to Health Care......................................................................................110 Future Areas for Study..............................................................................................111 Interventions......................................................................................................111 MU-PI Instrument Fidelity................................................................................113 APPENDIX A MEDICATION INVOLVED IN DRA BY STUDY................................................115 B CLASSIFICATION OF DTPS INTO NODES OF THE MUS................................116 C SUMMARY: CLASSIFICATION OF DTPS INTO NODES OF THE MUS.........118 D MEDICATION USE PERFORMANCE INDICATOR DEFINITIONS................119 E PROCEDURE CODES TO IDENTIFIY VISITS...................................................126 F CODING SOLUTION FOR INDICATOR..............................................................127 G RECRUITMENT LETTER FOR NODE IDENTIFICATION STUDY..................128 H NODE IDENTIFICATION SURVEY.....................................................................130 I EVALUATION TEAM BRIEFING.........................................................................141 J MU-PI RESULTS.....................................................................................................147 K SURVEY FOR COMMONALITY..........................................................................149 L NODE IDENTIFICATION BOX PLOTS................................................................150 LIST OF REFERENCES.................................................................................................153 BIOGRAPHICAL SKETCH...........................................................................................159 vii

PAGE 8

LIST OF TABLES Table page 3.1 Examples of DRM Screens by Type........................................................................25 3.2 Results of Leading DRM Screens............................................................................25 3.3 Results of DRM Screens..........................................................................................26 3.4 Results of Top Five Indicators.................................................................................30 3.5 Percent Change in Average Group Error by Number of Group Members...............32 4.1 Studies Reporting DRA and PDRA: .......................................................................44 4.2 Prevalence Estimates and Odds Ratio .....................................................................45 4.3 Latent Causes of PDRAs..........................................................................................54 6.1 Population Age and Gender Frequencies by Type of Claim....................................74 6.2 Population Demographics........................................................................................76 6.3 Frequency of Process and PDRM Positives.............................................................77 6.4 Number of PDRM Positives by Number of Indicators............................................77 6.5 Eleven Most Prevalent PDRM Positives..................................................................78 6.6 Ten Most Prevalent Process Positives......................................................................78 6.7 MU-PI Categorized by Required Monitoring Interval.............................................79 6.8 Bivariate Tests of Association: PDRM Positives as Dependent Variable...............85 6.9 Multicollinearity Assessment of Independent Variables.........................................86 6.10 Maximum likelihood Estimates: PDRM Positives as Dependent Variable.............86 6.11 Demographics of Delphi Panelists for Node Identification Study...........................87 6.12 Indicators Listed by Associated Node of the MUS..................................................89 viii

PAGE 9

6.13 Kruskal Wallis and Pairwise Comparisons for Indicator Assignment.....................90 6.14 Monitoring Indicator: Affinity Table and Rating of Importance.............................92 6.15 Cause Sequence Agreement Results........................................................................93 6.16 Prescribing Indicator: Affinity Table and Rating of Importance.............................94 ix

PAGE 10

LIST OF FIGURES Figure page 2.1 Model of the Medication Use System......................................................................14 2.2 Hierarchal Relationships among the System Levels................................................15 2.3 Swiss Cheese Model of System Failure...................................................................17 4.1 Involvement of Treatment Category to DRMs:.......................................................49 4.2 Therapeutic Class Involvement in DRM:.................................................................49 4.3 Nodes of the MUS Involved in PDRA.....................................................................52 5.1 Disease-Drug Interaction Search Algorithm............................................................58 5.2 Process Search Algorithm for Indicators that Require Monitoring..........................59 5.3 Process and Outcome Search Algorithm..................................................................59 5.4 Example of a Process Positive but not a PDRM Positive........................................60 5.5 Drug-Drug Interaction Search Algorithm................................................................60 5.6 Organization of Proposed Causes to System Levels................................................69 6.1 Office Visits Claims by Month................................................................................72 6.2 ED Visit Claims by Month.......................................................................................72 6.3 Facility Admissions by Month.................................................................................73 6.4 Pharmacy Claims by Month.....................................................................................73 6.5 First Lag for Indicator with One Month Monitoring Requirement..........................80 6.6 First Lag for Indicator with Two Month Monitoring Requirement.........................81 6.7 First Lag for Indicator with Three Month Monitoring Requirement.......................82 6.8 First Lag for Indicator with Six Month Monitoring Requirement...........................82 x

PAGE 11

Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy MEDICATION USE PERFORMANCE INDICATOR EVALUATION: A SYSTEMS PERSPECTIVE By BRIAN C. SAUER August, 2004 Chair: Charles Doug Hepler Major Department: Pharmacy Health Care Administration Background: Injury related to health care is a serious public health concern. The prevalence of drug related admissions in the United States has been reported to be from three to twelve percent of hospital admissions. Organizations interested in reducing drug related morbidity need measurement techniques to gather baseline information about preventable drug related morbidity (PDRMs), and methods to assess system related causes to develop rational interventions that target the system failures The objective of this dissertation is to better understand the relationship between system design and patterns of care that can result in drug related injury. Methods: This study was executed in three steps. In the first step, database analysis, medication use performance indicators (MU-PI) were used to estimate the number of PDRM positives in the managed care organizations administrative database. Step two used the Delphi process to judge the degree of association between select MU-PIs and specific nodes of the medication use system (MUS) where the process failure xi

PAGE 12

may have originated. In step three, select MU-PIs from each node of the MUS were submitted to cause-and-effect analysis (CEA). Results: The period prevalence for process positives was 209.84 (206.37-213.34) per 1,000 member years. The period prevalence for PDRM positives was 1.98 (1.6-2.4) per 1,000 member years. Gender, number of office visits, number of drug classes and number of medical conditions were independent risk factors for PDRM. Fourteen indicators were selected for step two. Six of the fourteen indicators were assigned to the prescribing node and seven were assigned to the monitoring node of the MUS. One indicator did not reach significance within two rounds. The cause-and-effect team identified twenty-nine cause sequences. They found that twenty-three of the twenty-nine were common to both prescribing problems and monitoring problems. Conclusion: The MU-PIs proved to be a useful tool to identify possible cases of PDRM and to initiate system thinking or organizational introspection to evaluate system related causes for drug related injury. Four themes can be interpreted from CEA: lack of necessary tools for adequate patient information and assessment, an information system that can track patients and relay information to the providers, pharmacist involvement in the MUS, and guideline adherence. xii

PAGE 13

CHAPTER 1 INTRODUCTION Problem Statement Injury related to health care is a serious public health concern. The prevalence of drug related admissions in the United States has been reported to be from three to twelve percent of hospital admissions. 1 The Institute of Medicine has proposed a fifty percent reduction in errors by 2005. 2 For this achievement in medications use, the following are needed: Measurement techniques to gather baseline information about preventable drug related morbidity (PDRMs) Assessment of system related causes and rational interventions that target the system failures Follow-up measurements to gauge the effects of the interventions. Automated data screening for PDRM has been proposed as a method to produce baseline and longitudinal measurements. However, the underlying system problems that contribute to PDRM in the ambulatory setting are largely unknown. Studies measuring the prevalence of drug-related hospital admissions have identified nodes in the medication use process where the drug therapy problems originated, i.e., prescribing, monitoring, dispensing and patient adherence. The underlying system related causes of node specific problems, however, have not systematically been evaluated. This research will try to discover the system factors that contribute to node specific problems. 1

PAGE 14

2 Objective The objective of this dissertation is to better understand the relationship between system design and patterns of care that can result in drug related morbidity. Specific Aims The specific aims of this study are as follows: 1. To establish the frequency of medication use performance indicator (MU-PI) positives in the study population 2. To evaluate population based explanations for MU-PI positives 3. To identify the node of the medications use system (MUS) where indicator specific patterns of care likely originated 4. To identify system related causes that are common among nodes of the MUS 5. To identify system related causes that are unique to nodes of the MUS Justification This study has both theoretical and practical implications. The theoretical contribution of this study includes: Determining the node of the MUS where the pattern of care from specific MU-PIs originated. This permits the evaluation of the indictors by processes rather than medication or disease specific analysis. Providing information about how the various subsystems (patient, microsystem, organization, and environment) within the health care enterprise interact to influence the quality of medication use. Establishing system factors that contribute to node specific patterns of care or process problems. Identifying system factors that appear to be node specific problems will reveal leverage points for interventions that may span medication classes and disease categories. Establishing system factors that contribute to patterns of care or process problems from multiple nodes of the MUS. Identifying system factors that appear to be common among the nodes of the MUS will reveal leverage points for interventions that may span nodes of the MUS, medication classes and disease categories. This study will demonstrate how the MU-PIs can be used to assess the quality of medication use in a defined population. It will also demonstrate how the MU-PI findings

PAGE 15

3 can be used to initiate activities that lead to the exploration of how system design influences the quality of medication use, i.e., organizational introspection. From a practical perspective, the health care purchasing group involved in this study is interested in increasing the quality of care and decreasing costs for the purchasers they represent. Understanding how system factors influence the quality of medication use will help direct the development of interventions to reduce the prevalence of adverse drug outcomes. Overview The health care industry is wasting billions of dollars 3 and causing an unacceptable amount of injury due to inappropriate medications use. 1,2,4 Results from our meta-analysis of fifteen studies indicated that approximately four percent of all hospital admissions may be preventable. 1 To place the reported prevalence estimate into perspective, according to the National Hospital Discharge Survey, there were 31,827,000 admissions to U.S. hospitals in 1998. If the prevalence of preventable drug-related hospital admissions (PDRAs) happened to have been four percent in 1998, there would have been roughly 1.3 million PDRAs. This would have placed PDRAs among the top causes of hospitalizations in the U.S. that year-above admissions related to congestive heart failure (3.1%) and on par with pneumonia (4.2%). Ambulatory health care is complex, with many opportunities for problems in drug therapy to occur. The basic nodes in the MUS include prescribing, dispensing, (self) administration, and monitoring. Of the fifteen studies analyzed in the above-mentioned meta-analysis, 1 eight described the types of drug therapy problems that led to PDRA. 5-12

PAGE 16

4 An analysis of these studies a indicated that prescribing and monitoring problems were highly associated with PDRA (median: 34% and 32%), respectively. Problems with medications compliance (median: 22%) were also implicated as a main contributor to PDRA. The finding that prescribing and monitoring problems appeared to be the main contributors to PDRA is disturbing because health care professionals are expected to first do no harm. Even though practitioners are expected to manage their patients appropriately and do it right the first time, the health care system should be designed in a way that anticipates human error and does not allow it to progress to an adverse drug event or PDRM. Ideally, the health care enterprise should be able to deliver care in a highly reliable manner where performance problems and bad decisions are recognized and corrected because adverse events occur. Results from the Institute of Medicine report on medical errors 2 and from our meta-analysis 1 show that the American health care system is currently far from highly reliable. Before the rational development of interventions, a data stream must be established for baseline and follow-up measurements, and underlying system related causes need to be examined. A recent doctoral dissertation at the University of Florida used a Delphi approach to validate forty nine MU-PIs; each paired an inappropriate process (violation of a standard of care) with an adverse outcome. 13 An example of these indicators follows: Use of two or more NSAIDS concurrently for at least two weeks in patients 65 or older followed by gastritis and/or upper GI bleed. a The analysis can be found in Chapter 4 (preliminary analysis)

PAGE 17

5 The indicators were automated and used to screen an administrative database of a large MCO in North Florida. The data set contained 11,711 patients, and 8.2% of them screened positive for a PDRM. For further validation, the positive predictive value of process to outcome (PTOV) was calculated to measure the strength of the association between the inappropriate process and the corresponding adverse outcome for each indicator. Ten of the nineteen indicators that had more than ten positive screens had PTOV values greater that 0.74. This suggests that this instrument would be acceptable for eliciting cases of PDRMs for system level analysis. Cause-and-effect analysis is a retrospective method for identifying system failures, and its popularity has largely advanced from its success in understanding industrial accidents. 14 Rooted in industrial psychology and human factors engineering, cause-and-effect analysis has recently been adopted in the medical community to evaluate quality problems. The MU-PIs adopted from Faris (2001) 13 will be used to screen the database of a health care coalition in Florida. MU-PIs with high frequency will be selected and assigned to the nodes of the MUS where the drug therapy problems appear to have originated from. Cause-and-effect diagramming on selected MU-PI that represent specific nodes of the MUS will be carried out to uncover system related problems that are common among the nodes and unique to specific nodes of the MUS.

PAGE 18

CHAPTER 2 CONCEPTUAL FRAMEWORK The framework of this study was developed from Heplers conceptualization of PDRM and the MUS, 15 Berwicks framework for the health care system 16 and Reasons accident theory. 17 The premise is drug-related morbidity is largely preventable due to failures in the management of drug-therapy. A model of the MUS is used to illustrate how drug-therapy problems (DTPs) can develop into PDRM. Health care is a complex system with many layers of embedded systems. A framework presented by Berwick 16 is used to simplify the complexity by referring to specific levels of the heath care system. Finally, James Reasons accident theory 17 is used to discuss the types of failures and conditions that affect the performance of the MUS and allow PDRM to occur. Adverse Outcomes of Drug Therapy The term drug-related morbidity (DRM) is used to discuss adverse outcomes of drug therapy. DRM is a concept that includes (a) significant adverse effects of drug therapy (e.g., hospital admissions or emergency department visits) (b) treatment failures (i.e., occasions when drug therapy was attempted but did not achieve a realistic, intended outcome in a reasonable time) and (c) occasions when a patient did not receive an indicated or necessary drug therapy. 18 It is important to appreciate the relationship between the popular term adverse-drug event (ADE), which was used to describe adverse outcomes of drug therapy in the IOM report To Err is Human, and DRM. The term ADE historically has been used to describe injury resulting from medical interventions related to a drug (i.e., direct effects of the therapy), 4,19-22 without recognition of indirect 6

PAGE 19

7 injury resulting from sub-therapeutic dose or no drug while a valid indication is present. Even though it does appear that the definition for an ADE has recently evolved to include indirect injury, 23 the term DRM is used in this study because it has historically treated indirect injury as a significant adverse outcome related to drug therapy. A DTP that precedes a DRM can be classified into two types: potential DTPs and actual DTPs. The first indicates a person has a theoretical problem associated with drug therapy, and the latter signifies the problem has manifested (symptoms are present). 15 Theoretical DTPs exist independently of individuals; they are the situations in care that produce the risk for specific adverse effects. Potential DTPs occur when a theoretical DTP is present in an individual, e.g., contra-indications, drug interactions and unjustified violations of evidence-based medicine. Actual DTPs are denoted by observable or patient reported symptoms. For example, concomitant prescribing of digoxin and quinadine is a known theoretical DTP because this combination can potentially alter the excretion of digoxin, thus increasing its serum concentration, which is especially troubling because digoxin is known to have a very narrow therapeutic window. Now imagine a patient sixty-five years old who is concurrently receiving digoxin and quinidine. This patient has a potential DTP because a theoretical DTP is present in his drug therapy. Now, suppose this patient begins to experience fatigue, weakness, confusion, and diarrhea. Most likely he is experiencing an actual DTP because predictable symptoms known to be associated with the theoretical DTP have manifested. If he/she becomes hypokalemic and needs emergency care, and upon admission his/her digoxin level is in toxic range, say > 50 mcg/kg, then he/she most likely would have experienced a preventable DRM (PDRM).

PAGE 20

8 A preventable DRM has the following attributes : 15,18 The DRM was preceded by a recognizable DTP The DRM was reasonably foreseeable under the circumstance The cause of the DTP and the resulting DRM was identifiable The identified cause of the DTP and resulting DRM was controllable within the context of therapy (i.e., without sacrificing essential therapeutic objectives). In this case the DRM meets the four criteria for preventability because it was preceded by a known drug interaction (potential DTP) and the manifest symptoms (actual DTP) were commonly associated with digoxin toxicity. Under the circumstances hospitalization was imminent because the symptoms represent the accumulation of digoxin, the cause of the symptoms appears to be the result of too much digoxin, and reducing the dose of digoxin and monitoring the patients digoxin levels or switching to an alternative therapy could have resolved the DTP and prevented the DRM. A DTP is part of the process of care, it is a state of an individual in a medications use system, and it is a possible precursor to a system failure, i.e., DRM. The Medication Use System For a DRM to be preventable a failure in the process of care must have occurred. This becomes clearer when considering the typical sequence of actions that comprise the MUS in the ambulatory care setting. Figure 2.1 below was adopted from Grainger-Rousseau et al. and it illustrates the nodes in the medications use system. 24 The episode of care begins when the patient notices a problem and seeks professional medical attention. Typically, the initiator (physician, physicians assistant or nurse practitioner) then assesses the problem and develops a clinical impression or diagnosis. Next a plan is devised and the decision to prescribe a medication is made. A prescription is either written or not; if it is, the patient will typically present the prescription for filling at the

PAGE 21

9 pharmacy. Before dispensing the medication the pharmacist should look for DTPs and advise the patient how to use and self their new therapy. The patient then consumes (e.g., self-administers) the medication. Follow-up visits and monitoring are required to gauge the effects of the medication in the individual and to determine how well the prescribed medication is working towards the therapeutic objective. It is the information gathered from the monitoring node that is used in the decision to continue the current treatment or make alterations to the medical regimen. Errors or problems at each step may occur during a passage through this sequence of events. Examples include failure to recognize an indication for drug therapy, incorrect patient assessment, incorrect diagnosis, prescribing, dispensing, and administration and monitoring. 25 Our meta-analysis (2002) estimated the prevalence of preventable drug related admissions to be about four percent. 1 Based on our review and the IOM report, it appears the current MUS are not optimal. This may be because MUS are more of a virtual than an actual microsystem. As described by Nelson et al., (1998) 26 an essential element for a microsystem is an information environment to support the work of care teams. Unfortunately, the MUS in most ambulatory care settings is fragmented and clearly lacks a reliable information exchange among providers, especially between initiator and co-therapists. The development and adoption of communication systems such as electronic medical records and computerized prescription order entry are intended to make the MUS safer. Levels of the Health System The IOM defines a system as a set of interdependent elements interacting to achieve a common aim. The elements may be human and/or non-human. 2 The health

PAGE 22

10 care enterprise consists of many embedded systems that transcend a multitude of domains, ranging from the subsystems used to make and keep track of patient appointments to the environment where accreditation and financial systems influence the care of populations. All systems have subsystems or nested systems that interact with one another. Some systems are in a hierarchical relationship in which higher order systems influence the functioning of lower order subsystems. Identifying the nesting of systems and their relationships to one another is key to understanding the mechanics of any system. Berwick (2002) published a paper that he described as a users manual for the IOM report, Crossing the Quality Chasm. 16 In this paper he addressed the issue of embedded systems by presenting a framework for the different levels within health care. He separates the system into the following levels: the experience of the patients and communities (Level A), the functioning of small units of care delivery called microsystems (Level B), the functioning of organizations that house or otherwise support microsystems (Level C) and the environment of policy, payment, regulation, accreditation and other factors (Level D). Level A is not actually framed in terms of a system; instead it represents the patients experience and perception of the care they received. Berwick states, Rooted in the experiences of patients as the fundamental source of quality, the report shows clearly that we should judge the quality of professional work, delivery systems, organization and policies first and only by the cascade of effects back to the individual patient and to the relief of suffering, the reduction of disability and the maintenance of health. 16 This is a significant paradigm shift from previous quality approaches that were independent from patient outcomes. Determining quality based on the patients experience is allied with using outcomes of care to gauge quality. Donabedian (1978)

PAGE 23

11 defines outcomes as the primary changes in health status that can be attributed to that care. 27 Health status is determined by psychological factors and social performance (subjective component), and physiological factors (objective component). In this paradigm, the patients experience is defined as the sole way of determining the quality of other systems. The decision to place emphasis on the patient is strategic; it acknowledges the patients subjective experience and clinical outcomes are the primary focus of quality rather than the dynamics of the microsystems. The microsystems (Level B) are small units of work that actually gives care the patient experiences. 16 Clinical microsystems are basically small organized groups of providers and staff caring for a defined population of patients. 28 They are composed of patients who interact with clinical and support staff who perform various roles, e.g., physician, nurse, pharmacist, medical assistant, data managers, receptionist, etc. Nelson et al. (1998) have described the essential elements of a microsystem as (a) a core team of healthcare professionals; (b) the defined population they care for; (c) an information environment to support the work of care team and patients; and (d) support staff, equipment and work environment. 26 In terms of medications management, the patients and clinical staff manage drug therapy by engaging in direct care processes, which include recognizing, assessing and diagnosing the patients problem, along with developing a treatment plan, dispensing and educating, and monitoring to make sure the treatment is progressing as planned. Direct care processes are assisted by supporting processes that involve distinct tools and resources such as medical records, scheduling, diagnostic tests, medications, billing, etc.

PAGE 24

12 To move to a more reliable system of care, the performance of microsystems must be optimized and the linkages between clinical microsystems must be seamless, timely, and efficient. Change at the microsystem level is an important opportunity to focus on the transformation of care at the front line of the health care service industry. 29 Microsystems do not exist in a vacuum; they are embedded within the organizations that help orchestrate their relationships with each other. The health care organizations (Level C) are establishments that house and support microsystems--they provide the necessary resources for microsystems to deliver care. Health care delivery requires personnel, financing, technology, and facilities. Common organizations include hospitals, provider groups, independent practice groups, nursing homes, ambulatory surgery centers, and pharmacy benefit managers, all of which are typically embedded within managed care organizations. Group practice, for example, is an affiliation of three or more providers, usually physicians, who share income, expenses, facilities, equipment, medical records, and support personnel in the provision of services, through a legally constituted organization. 30 They are embedded within managed care organizations through such integrating mechanisms as referral arrangements, insurance contracts, and in some cases direct ownership of practice. Managed care is focused on controlling the functioning of microsystems, i.e., controlling provider and patient behaviors, and most of that control is realized through the ambulatory care arena. 30 Managed care has shifted toward an organizational form with greater control over resources, providers, and patients within microsystems of care. The health care environment (level D) includes multiple systems that influence activities of the organizations and microsystems. Important environmental systems

PAGE 25

13 include financing, regulation, industry, accreditation, policy, litigation, professional education and social policy. 16 Berwick (2002) describes the relationship among the four levels as the chain of effect of improving health care quality. He states that, The quality of the microsystem is its ability to achieve ever better care: safe, effective, patient-centered, timely, efficient and equitable. The quality of the organization is its capacity to help microsystems produce safe, effective and efficient patient outcomes. And the quality of the environment--finance, regulation, and professional education is its ability to support organizations that can help microsystems achieve those aims. 16 Figure 2.2 was designed to show the hierarchal relationship among the different levels of health care, as well as the MUSs location within the microsystems. As illustrated, the MUS intersects the microsystems and the patients experience. The spiral between the microsystems and the patient level indicates that the patients experience is the input into the MUS as well as the output, which is then used as feedback to determine the performance of the microsystems. What was more difficult to illustrate is the interaction among the microsystems. In an ideal MUS the interaction between physicians, laboratory, pharmacists, case managers, etc., would be seamless with information flowing freely among the microsystems and cooperation among the professionals would be standard. The hierarchical stacking from environment to organizations to microsystems to the patient represents the main route of influence. It should be noted that this is not the only direction of influence, organizations can use lobbyists to leverage influence over the regulatory and financial environment. Likewise, microsystems can organize, like in the case of provider groups, to counter the pressures from higher-up organizations such as managed care.

PAGE 26

14 Figure 2.1. Model of the Medication Use System Human Error and System Failure James Reason provides two views of how accidents occur within an organization the person approach and the system approach. 17 These approaches are fundamentally different, with each giving rise to contrasting philosophies of performance management. The person approach (human error) focuses on unsafe acts. In medications use, the attention would be on errors and procedural violations of health professionals and patients. This approach views unsafe acts as divergent cognitive and behavioral processes such as forgetfulness, inattention, carelessness, negligence and recklessness. Historically, the most popular approach to countering human error has been to find the operator or operators who committed the unsafe act and discipline them accordingly. This is often referred to as blaming, and sometimes as scapegoating. Even though blaming may produce satisfaction it does nothing to correct the conditions that allowed unsafe acts to progress to injury. Because errors are often made in the normal course of providing care, traditional efforts at error reduction, which focused on individuals and

PAGE 27

15 episodes by using training, rules and sanctions to improve performance, are considered less effective than altering the system to remove or reduce the conditions that increase the likelihood of adverse outcomes. 31 Counter measures for reducing unwanted behaviors are analogous to the Whack-A-Mole carnival game: you can whack a mole back into his hole, but another is sure to challenge your skill by popping-up in a different location. Figure 2.2. Hierarchal Relationships among the System Levels The basic principle in the system approach is humans by their nature are fallible and errors are expected even in the best organizations. From this perspective, injury occurs because errors interact with flaws in the design of systems called latent conditions. 17

PAGE 28

16 Latent conditions are the flaws in the design and organization of a system. In health care they can arise from the decisions made by designers; however, it is believed many latent conditions arise not from design but from self organization, which is an adaptation to evolving systems and an unsettled environment. 32 Latent conditions have two types of adverse effects: they can produce error-provoking conditions and they can produce longstanding holes or weaknesses in the defense mechanisms, i.e., error detection mechanisms. When latent conditions combine with errors the opportunity for PDRM occurs. In the Swiss cheese model b presented in Figure 2.3, the holes in the cheese are latent conditions that allow potential injury (i.e., drug therapy problems) to manifest as drug related morbidity. Summary PDRM is a system problem. The nature of preventability means problems occurred in the process of care. Patient injury occurs when human error interacts with latent conditions of a system. The health care system is a complex network of interacting systems and subsystems. Understanding how microsystems, organizations and environment influence activities of the MUS and produce opportunities for human error to result in PDRM will increase our ability to predict and resolve DTP before they manifest as patient injury. b This Swiss cheese was published by Hepler and it is a modification of Reasons Swiss cheese model 15

PAGE 29

17 Patient injury Figure 2.3. Swiss Cheese Model of System Failure

PAGE 30

CHAPTER 3 LITERATURE REVIEW This chapter will first discuss the use administrative data for scientific investigation. The use of automated methods for detecting the incidence and/or prevalence of PDRM will then be discussed. Next, experimental studies evaluating the effectiveness of the Delphi process and its use in health care will be presented. Lastly, techniques for cause-and-effect analysis will be presented. Use of Administrative Databases for Researching PDRM Administrative databases are derived from information produced by health care providers and institutions in billing for products and services. Claims are filed from institutions (i.e., inpatient hospital stays and outpatient visits) health professionals and pharmacies for reimbursement of products and services from payer organizations such as Medicaid, Medicare and private insurers. The claim form used depends on where services were rendered and who provided them. Most hospital inpatients and acute care outpatient services are submitted for payment with the uniform billing 92 (UB-92) format. The UB-92 was derived from the Uniform Hospital Discharge Data Set, which was formulated in 1972 by the National Committee on Vital and Health Statistics, U.S. Department of Health, Education, and Welfare. The goal was to create a uniform but minimum data set to facilitate investigation of cost and quality of hospital services across populations. 33 Health care services provided by single practitioners or practitioner groups are submitted for payment in a format called the CMS-1500 (formally the HCFA-1500). This is the common claim form for non-institutional providers that is updated by the 18

PAGE 31

19 Center for Medicare Services (CMS) and is approved by the American Medical Association Council on Medical Services. Both the UB-92 and the CMS-1500 forms typically include details for each compensated service, including diagnosis, and procedures, date, place of service, provider and patient identifiers and charges. From a research and quality improvement perspective, the potential utility and credibility of a database stems from its clinical content. In administrative data in the United States, information on clinical diagnosis and conditions are documented with International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnostic codes. In addition, administrative data derived from hospital reports (UB-92) use ICD-9-CM codes to document procedures, while procedures are documented using Common Procedure Terminology (CPT) in data derived from professional claims (CMS-1500). The reliability and validity of coding is a serious issue when considering the use of administrative data for research. In relation to hospital admissions, the two major steps in coding an admission are first specifying the pertinent diagnosis and second ordering them. The principal diagnosis in a UB-92 form defines the cause of admission. The sequencing of diagnosis does impact reimbursement rates and mistakes or miscoding can affect the accuracy of the data. As previously mentioned, the validity of claims data is a cardinal issue when considering its use for health services research. Threats to validity and two approaches for evaluating the ability of claims data to accurately identify patients with specific conditions will now be presented. The following threats to internal validity (i.e., misclassification bias) have been identified misspecification, resequencing, miscoding,

PAGE 32

20 and clerical errors. 33,34 Misspecification occurs when the attending physician selects an incorrect diagnostic code for the principal diagnosis, listing of diagnoses or procedures. Misspecification was identified as the leading source of error in a summary of studies that evaluated coding quality. 34 Resequencing is the substitution of a secondary diagnosis for the correct principal diagnosis during the coding process and was found to be the second most frequent source of misclassification. 34 Miscoding is the coding of diagnoses or procedures not attested to by the physician, misapplication of coding rules, or selection of an unnecessarily vague diagnosis code. Now that the types of misclassification bias have been presented, studies that evaluated the validity of claims for research data will be discussed. Quam et al. (1993) investigated the validity of claims data for epidemiologic research and found evidence that supports its use. 35 The researchers evaluated the ability of claims data to identify patients with essential hypertension. The claims database used consisted of all ambulatory, pharmacy and hospital claims from 1988 to 1989 in two large managed care organizations. They identified essential hypertensive patients by three strategies. One strategy was based on medical service claims alone (diagnosis without pharmacy claims), another was based on only pharmacy claims (pharmacy claims without diagnosis), and the third strategy was based on both medical services and pharmacy claims. Diagnostic codes were validated by comparing findings with patient survey and medical records. The strategies based on medical claims alone and pharmacy claims alone exhibited low positive predicted values (PPV) with the medical record (47% and 50%, respectively) and the patient survey (43% and 63%, respectively). 35 The

PAGE 33

21 combination strategy, however, exhibited very high PPVs (96% for both the medial record and patient survey). 35 More recent information on the ability of administrative data to predict disease outcomes is described in the Study of Clinically Relevant Indicators for Pharmacological Therapy (SCRIPT) report. 36 The objective of the SCRIPT project was to develop a core set of valid and reliable performance measures to evaluate and improve the quality of medication use. The common focus was cardiovascular disease outcomes and risk factors. The candidate measures were tested in managed care organizations and practice groups from eight states. The study evaluated patients with coronary artery disease (CAD), heart failure (CHF), and atrial fibrillation (AFIB). The researchers evaluated the PPV of specific ICD-9CM and CPT codes for identifying the cardiovascular conditions mentioned above. The requirement for the number of codes to identify specific conditions was varied from one to three and the effects on yield and PPV were determined. The PPVs from the SCRIPT study averaged 89%, 89%, 84% for AFIB, CAD and CHF, respectively. The state-specific values for AFIB ranged from 80% to 98%, the values for CAD ranged from 81% to 95% and the values for CHF ranged from 78% to 94%. Requiring an additional code raised the PPV no more than four percentage points on average and raised the worst PPV as much as eight percentage points. Requiring three codes added little to PPV; however, it significantly reduced the yield. The most marginal improvement in PPVs was found by requiring two codes vs. three codes. The two code requirement lowered the yields by 13%, 14%, and 20%, respectively for AFIB CAD and CHF

PAGE 34

22 In summary, although administrative databases were not originally designed for research, they have become a rich source of information for health services research. 33 Nevertheless, threats to internal validity, specifically misclassification bias need to be taken seriously when interpreting studies that used claims data. Studies that have evaluated the ability to identify patients with specific conditions using administrative data had good positive predictive ability. Quam et al. (1993) reached a PPV of 96% when requiring medications used to treat essential hypertension and a diagnosis of essential hypertension. 35 The SCRIPT project was able to obtain good PPV while maintaining a decent yield when they required two codes (ICD-9-CM or CPT) for specific cardiovascular conditions. 36 Automated Methods for Detecting PDRM Chart review is the gold standard for PDRM ascertainment. The articles reviewed by Winterstein et al. (2002) 1 (discussed in the following chapter) used medical chart review to measure the prevalence of drug related admissions (DRAs). This typically included reviewing the medical records of all patients admitted to the hospital or specific hospital units during a defined period of time to determine if the reason for admission was drug related. The detection and classification method was often implicit, which means the reviewers judged whether or not a DRA occurred by comparing the care processes of that patient against his or her own knowledge, opinions, and beliefs about how appropriate care should be carried out. Implicit methods may be more sensitive because they allow experts to capture case-specific elements of information for the judgment of preventability, however, their reliability is questionable. 37 Furthermore, chart-review is time and cost consuming and is not useful in the continuous improvement

PAGE 35

23 paradigm where baseline and many follow-up measures are needed. Computerized methods to identify DRM with explicit criteria are an alternative to chart-review. 38,39 A PubMed search identified three studies with different approaches for detecting ambulatory acquired DRM with computerized data. One method identified DRM with search algorithms used to screen laboratory data and pharmacy records. Another method used multiple types of search algorithms and databases to identify cases of ADEs. The last method screened administrative claims data by using automated PDRM performance indicators. These studies will be presented and special attention is given to the following issues: the predictive validity of the instrument, the ability of the indicators to capture preventable events, the scope of the database, and the sophistication of the software used to run the indicators. Jha et al. (2001) 39 adapted published screening rules (i.e., search algorithms) developed from the inpatient LDS study in Utah. 40 The rules were used to estimate the rate of DRA to the study hospital. Every day the computer generated a list of alerts. The alerts were validated by reviewing each patients medical record that screened positive for a DRM on admission. Jha et als. (2001) screening rules can be separated into at least four categories: antidotes, chemistry/drug levels, and physiological response to a specific therapy and drug interaction. (Table 3.1) The information system used integrated pharmacy, laboratory test results and a sophisticated physician order entry systems. Twenty two rules required laboratory test results and nineteen screened for antidotes of known adverse consequences of drug therapy (e.g., naloxone, a narcotic antagonist).

PAGE 36

24 Table 3.2 shows the results of the most frequent screens. The overall PPV for the instrument was 3.5% with a range from 2.0 to 100%. The screen for serum phenobarbital levels < 45 mg/dl had a PPV of 100%. The screen for patients receiving charcoal (activated) had the second highest PPV of 45%. All other screens had PPVs below 12%. The overall PPV for this instrument and the individual screens PPVs, excluding phenobarbital levels, indicated that these screens picked up a tremendous amount of noise. Furthermore, the majority of the indicators were directed toward toxic levels of medications and abnormal blood chemistry levels. While these surrogate outcomes may be able to indicate the presence of DRM, they do not provide information about the care process that led to the event, i.e., preventability. Additional chart review or investigative work is needed to judge preventability. Furthermore, the software used to search the data was quite sophisticated (physician order entry and an event monitor). Nevertheless, less sophisticated software such as SAS or Access could be used to write search algorithms for the screens. Honigman et al. (2001) took a more elaborate approach for using computerized information to screen for ADEs. 41 This approach used a sophisticated computer program which consisted of four search methods: ICD-9 codes, allergy rules, computer event monitoring rules, and an automated chart review using text searching. ICD-9-CM codes associated with ADEs were used to screen the database. The computer event monitoring rules were based on the same screening criteria as Jha et al. (2001) mentioned above. 39,40 Allergic reactions to medications were detected with a text search of the medical records. The software program M 2 D 2 was used to expand the patients drug allergy list to include product names, generics and ingredient. Patient

PAGE 37

25 records were screened for allergies along with offending medications. The M 2 D 2 program also matched terms in the medical record with known adverse effects of drugs the patient was taking. Table 3.1. Examples of DRM Screens by Type (Jha et al. 2001) AntidotesChemistry and Blood LevelsResponse to Drug and InteractionsReceiving betamethasone di p ro p ionate 0.05%Serum digoxin > 1.7 ng/mLReceiving nephrotoxin AND blood creatinine has risen > 0.5 m g /dL in last 1 da y Receiving charcoal ( activated ) Serum lidocaine > 5.0 mg/mLReceiving ranitidine AND platelet count has fallen to less than 50% of p revious valueReceiving racemic e p ine p hrine hclSerum phenytoin results within last 1 da y are > 20 m g /mLReceiving diphenoxylate with atropineReceiving atropine sulfateSerum bilirubin > 10 mg/dLReceiving benzodiazepine AND receiving anti-e p ile p ticReceiving naloxoneSerum potassium > 6.5 mmol/LReceiving phytonadione (vitamin K) AND order for warfarin within last 14 days Table 3.2. Results of Leading DRM Screens (Jha et al. 2001) DRM screening ruleScreen +True +PPV (%)Patient receiving predinisone313123.8Patient receiving diphenhydramine30093Allergy entered113108.8Patient receiving oral matronidazole/vancomycin8766.9Serum digoxin > 2.0 ng/ml6158.2Serum phenytoin > 20mg/dl4748.5Patient receiving kaopectate35411.4Patient receiving charcoal11545.5Serum phenobarbital >45 mg/dl44100Other1,649332Total2,620923.5 Honigman et al. (2001) 41 found allergy screens to have the highest PPV (49%), but these screens did not detect the greatest number of ADEs (104). Text searching detected the most ADEs (1,637) but it had many false positives (PPV = 7%). Searching for only ICD-9-CM codes had the lowest PPV (2%) and it only detected five events. The event monitoring rules detected 60 events with a PPV of 3.3%. The PPV for the composite tool was 7.2%. (Table 3.3)

PAGE 38

26 Table 3.3. Results of DRM Screens ICD-9-CM24852Event Monitoring Rules1,802603.3Allergy21410448.6Text Search22,7981,6377.2Composite2506218067.2Rule ClassScreen positivetrue positivePPV, % This approach was more comprehensive than Jha et als. (2001) detection method. The rules were more dynamic because they were able to identify less severe adverse drug effects. The term adverse drug event was used instead of DRM when describing this study because many of the effects screened for would not fit the definition of a DRM. The events would be considered actual DTPs under Heplers conceptual framework. For example, text searches for signs and symptoms such as dizziness, fatigue, cough, and over anticoagulation without bleeding were major components of the screening instrument. Only nine percent of the adverse events identified required hospital admissions. The level of programming and database (administrative vs. clinical data) sophistication needed to run the rules varied. The ICD-9-CM rules did not require sophisticated software and were usable with administrative databases. The event monitoring rules did not require sophisticated software either, programming could be done with Access or a statistical program such as SAS. Nevertheless, clinical data (i.e., laboratory test results and medical records) was required. Most administrative records only indicate that specific types of laboratory tests were requested or conducted. Patient specific laboratory values are not documented. The allergy and text search rules were more sophisticated and they required electronic medical records. Honigman et als. screening method had advantages over Jha et als. method. Nevertheless, the level of

PAGE 39

27 sophistication limits the use of these rules to organizations with access to electronic clinical data and specific software. The PPV of Honigman et als. composite tool was relatively low (7.2%). The screens were the most comprehensive and could detect minor adverse drug events as well as events that led to hospital admission; however they were not designed to indicate whether or not the events were avoidable or if the admissions were preventable. The screens varied in their use of databases, they used administrative data, laboratory test results and electronic clinical records. The software needed to run the text searches was quite sophisticated. Faris (2001) developed a different approach to electronically detect PDRM. 13 His approach automated forty-nine PDRM indicators that paired an inappropriate process of care (potential DTP) with its predictable adverse outcome (DRM). The indicators were originally developed by MacKinnon (1999) 42 from a literature search of peer-reviewed medical articles and reference texts from 1967-1998, and from a consensus panel of seven experts in geriatric medicine. Faris (2001) revalidated the PDRM indicator definitions through a Geriatric medicine expert consensus panel. Forty-four of the fifty-two indicator definitions accepted in MacKinnons study were accepted for automation in the Faris study. Additional indicators were proposed and accepted by the panel for a total of forty-nine indicators. Once the indicator definitions were accepted they were translated into a format that could be applied to a managed care administrative database. The translation was designed to capture the events described in the indicator definitions through billing

PAGE 40

28 records from physician offices, hospitals, pharmacies, and other care sites in the managed care network. Services are billed through a system of codes that represent different treatments and activities. These codes included ICD-9-CMs, CPTs, and the National Drug Codes (NDC). SAS was used to extract and analyze cases that fit the indicator definitions. Faris (2001) retrospectively sampled 11,711 elderly patients over a one-year period and 966 had at least one positive for a PDRM indicator. Chart review was not done to validate the performance indicators. As an alternative, the proportion of patterns of care in the population that appeared to result in the corresponding adverse outcome was calculated (PTOV). Ten of the nineteen indicators, which had more than ten positives, had PTOV values greater that 0.74 percent. Of the five most prevalent indicators, three had adjusted PTOVs greater than or equal to 75%, which means the process of care was strongly associated with the corresponding adverse outcome. (Table 3.4.) For example, Asthma exacerbation and/or status asthmaticus and/or ER visit/hospitalization due to asthma was found 100% of the time when the following pattern of care occurred: Diagnosis of moderate to severe asthma and use of a bronchodilator with no use of maintenance corticosteroid. The criterion validity of PDRM indicators developed by Faris (2001) showed promise with ten indicators having adjusted PTOVs of 75% or greater, nevertheless the PPV of the PDRM indicators is unknown and needs to be tested. MacKinnon (1999) did, however, examine the criterion validity of two indicators that occurred frequently enough for statistical validation and found relatively high PPVs, 82% and 34%. 42

PAGE 41

29 Unlike the approaches taken by Jha et al. (2001) and Honigman et al. (2001), these indicators are specifically searching for preventable cases of DRM. The preventability is represented by the inappropriate process and its predictable adverse outcome. Unfortunately, only two of MacKinnons PDRM indicators were validated with chart review. Nevertheless, the PPV of the two indicators was decent and this method shows promise as valid PDRM measures. Another advantage of this approach was it did require sophisticated software. The search algorithms were produced with SAS. Furthermore, clinical data is not neededthese indicators were designed for administrative data. In summary, three studies with different computerized methods to detect PDRM were found in the literature. The approaches varied in ability to detect true cases of PDRM, and identify preventable events. The type of data needed, and the sophistication of the software used to produce search criteria also varied. The approach used by Faris (2001) will be adopted for this dissertation, because the PDRM performance indicators from that study included preventability in their operationalization. Furthermore, the indicators were intended for claims data and the search algorithms can be produced with basic statistical software, i.e., SAS. Delphi Method The Delphi method is a systematic approach for the utilization of expert opinion that was extensively studied by the RAND corporation in the 1960s. 43 The concept of the Delphi technique is simple; it is designed to define a single position (consensus) of a group by systematically combining their opinions in a way that eliminates activity among the experts to reduce dominance and group pressure.

PAGE 42

30 Table 3.4. Results of Top Five Indicators PDRM IndicatorsPositive IndicatorAdjusted PTOVThis outcome has occurred after the p attern of care below: ER visit/hospitalization due to congestive heart failureThis is the pattern of care:1. Diagnosis/history of congestive heart failure2. Not on ACE inhibitorThis outcome has occurred after the p attern of care below: ER visit/hospitalization due to congestive heart failure and or heart block This is the pattern of care:1.History/diagnosis of congestive heart failure with heart block or advanced bradycardia2. Use of digoxinThis outcome has occurred after the pattern of care below: ER visit/hos p italization due to h yp oth y roidis m This is the p attern of care:1. Use of a th y roid or antith y roid a g ent ( e. g .; levoth y roxine, etc. ) 2. T4/TSH not done before thera py starts and at least ever y 12 months This outcome has occurred after the p attern of care below: Gastritis and/or upper GI bleed and/or GI perforation and/or GI ulcer and anemiaThis is the pattern of care:1. Use of 2 or more NSAIDS concurrentl y This outcome has occurred after the p attern of care below:Asthma exacerbation and/or status asthmaticus and/or ER visit/hospitalization due to asthmaThis is the pattern of care:1. Diagnosis of moderate to severe asthma2. Use of a bronchodilator3. No use of maintenance corticosteroid2700.751030.458911840.951290.12 The Delphi technique is a modification of the traditional roundtable group decision making process. The roundtable approach has inherent limitations because the final decision or position may be an effect of the most outspoken person (i.e., the bandwagon effect) or socially acceptable opinion rather than the groups true opinion. 43 The Delphi technique avoids these undesirable effects by replacing the group meeting with anonymous response. Instead meaning the opinions of members are obtained by formal questionnaire. The procedures of the Delphi technique have the following key features : 44 Anonymous responseopinions of members are obtained by formal questionnaire or other formal communication channels as a way to reduce the effects of dominant individuals. Iteration and controlled feedbackthe interaction among the members is supported by a systematic exercise conducted in several iterations, with carefully controlled

PAGE 43

31 feedback between rounds. The summary of the results from the previous round are communicated to the participants Statistical group responsethe group opinion is defined as an appropriate aggregate of individual opinions in the final round. The use of a statistical definition of the group response is a way of reducing group pressure for conformity; at the end of the exercise there may still be a significant spread of opinion. In the 1960s the RAND Corporation carried out a series of experiments to evaluate Delphi procedures and to explore the nature of information processing. 44 The experiments involved fourteen groups of upper-class and graduate students from UCLA and they ranged in size from eleven to thirty members. The three effects that researchers examined were group size, a comparison of face-to-face discussion with the controlled-feedback interaction, and the ability of controlled feedback as a means of improving group estimates. They found increasing group size decreased the average group error and increased reliability. The anonymous controlled feedback procedures of the Delphi technique made the group estimates more accurate than face-to-face discussion. They also found controlled feedback narrowed the dispersion around the median. A large set of experimentally derived answers to factual questions was evaluated to determine the relationship between group size and the mean accuracy of a group response. 44 In this study the experimenters knew the answers to the questions; however, the subjects did not. The group error was calculated as the absolute value of the natural algorithm of the group median divided by the true answer. Dalkey (1969) found the gains in increasing group size were quite large from three to eleven members, but the percent change in group error was less than five percent past eleven members. (Table 3.5.)

PAGE 44

32 Table 3.5. Percent Change in Average Group Error by Number of Group Members. (Extrapolated from Dalkey (1969)) Group membersAverage group errorPercent change11.19430.852128.63%50.688219.23%70.609711.42%90.5726.18%110.55393.16%130.54531.56%150.54110.76%170.53910.37%190.53820.18% Dalkey (1969) also compared the reliability of two groups opinion with various numbers of members. Reliability was measured by the correlation between the answers of the two groups over a set of questions. Dalkey found a definite and monotonic increase in the reliability of the group responses with increasing group size. A mean correlation of 0.8 was obtained with thirteen group members. Even though these two studies showed increasing group size reduced the error of the group and increase the reliability of the estimate, it appears these estimates were obtained with traditional group consensus and not by a Delphi process. If these results are generalizable to Delphi group estimates then having a Delphi group with between eleven and fifteen members would seem reasonable because average group error only decreased by approximately three percent from eleven to thirteen members and reliability reached 80% with thirteen members. In the main experiment the performance of groups using face-to-face discussion was compared with Delphi groups. The first experiment involved two groups of five graduate students, and twenty questions were presented in four blocks of five using the ABBA design (A= face-to-face). The face-to-face group was instructed to follow a

PAGE 45

33 specific procedure for each question and the Delphi procedure involved four rounds of estimates, feedback of medians and quartiles and re-estimates. The median response of the Delphi group was more accurate in thirteen cases, and the face-to-face group was more accurate in seven cases. The second experiment had a different study design. The Delphi group had twenty-three members and from them the face-to-face groups were separated into seven groups of three and one group of one. In this experiment they found that Delphi estimates were similar to face-to-face estimates in terms of accuracy, however, face-to-face estimates produced more changes from initial estimates that reduced accuracy. Dalkey also examined the distributions of the answers between the first-round and second-round of the Delphi groups. The second-round distribution shifted toward the mean and Dalkey concluded the shift represents a convergence of answers toward the group response. Nevertheless, the second-round distribution still had a large range, indicating convergence was not complete. Dalkey also evaluated the effects of iterations on the accuracy of responses. To do this, changes with regards to individual questions were measured. The median improved in accuracy for about 64%, while the median became less accurate in 31%. The mechanism of improvement appeared to be from first-round feedback of the median response. The tendency to change was determined by the distance of the first round answer to the first round median. However, this did not explain the change in median from round-one to round-two. Some respondents had to cross the median for a change in median to occur.

PAGE 46

34 To evaluate how the median value changed from the first to second round, the researchers divided the groups into holdouts and swingers. Holdouts tended to cluster around the median and they were more accurate than the total group in the first round. The median of the group was between the median of the swingers and the median of the holdouts. The shift in the group median occurred because the swingers shifted toward the median, thus shifting the group median. In a supplementary analysis Dalkey (1969) tested the effects of other forms of feedback on the accuracy of the Delphi estimates. 44 The experimental group documented reasons for their decisions when their second round response was outside the interquartile range of the first round. Formulating and feeding back reasons did not increase the accuracy of the initial estimates or produce improvement on iteration. In a follow-up study Dalkey et al., (1969) evaluated the use of self rating to improve group estimates. 45 This study used 282 University of California students. Subjects were randomly assigned to sixteen groups with fifteen to twenty members per group. Almanac type questions were used to assess the relationship between accuracy and self rating. Subjects were given the questions and asked to rate each question from one to five to indicate their knowledge of the content in question. Relative rates were used, meaning subjects had to identify a question they were most knowledgeable about and give it a five. They also had to identify a question they knew the least about and give it a one. A clear association between average group self rating and group estimate accuracy was found. This study also found a significant improvement in the effectiveness of the Delphi procedures can be obtained by using self-rating information to select more accurate subgroups.

PAGE 47

35 The experimental studies on the Delphi process are important because they provide information on how group processes affect the accuracy of group estimates to factual questions. In this dissertation the Delphi method will be used to assign Faris indicators to nodes of the MUS where the process failure likely originated. In this case, a known answer does not exist. The opinions of the group members will be aggregated to assign select indicators to nodes of the MUS. Therefore the qualifications and diversity of experts in domain specific knowledge will be important factors to consider when constructing a Delphi panel. Cause-and-Effect Analysis To improve the performance of the MUS it is first necessary to understand error etiology within the MUS and the underlying system design factors that contribute to PDRM. Cause-and-Effect analysis (CEA) is a structured investigation that aims to identify the true cause of a problem, and the actions necessary to eliminate it. 46 CEA is an integrated problem solving methodology that incorporates multiple tools and strategies, which include process analysis, brainstorming and cause-and-effect diagramming. 46,47 First, process analysis is conducted to understand the processes involved in the quality problem. Before brainstorming or cause-and-effect diagramming is conducted an understanding of how the system works is needed. Process analysis should include flow diagrams that represent different nodes of the system. The steps needed to construct a flow diagram include: 47 1. Defining the basic nodes of a system 2. Further defining the process, breaking each node down to specific steps needed to complete the process 3. Following the objects through the process a number of times to verify the process by observation

PAGE 48

36 4. Discussing the process representation with the project team and reaching consensus on the underlying process. Brainstorming is a possible cause generation technique. 47 Brainstorming helps a group to generate many thoughts or ideas in a very short period of time. 46 The important aspect of brainstorming is ideas are to be generated without judgment or discussion until all the ideas are presented. In a typical group discussion someone presents a thought and others comment on it, or judge it. The problem is group members may focus on one idea before all other possible ideas have a chance to emerge. 46 Brainstorming encourages the flow and fluency of group member thoughts. Brainstorming has three steps: generation, clarification and evaluation. 46 In the generation phase the leader clearly states and writes the question or purpose and then invites and records responses. This step can either be structured or unstructured. In the structured approach, known as round-robin, each participant in turn launches one idea, this insures equal participation, but is less spontaneous and may limit the possibility for building on one anothers ideas. 46 In unstructured brainstorming everyone can freely launch ideas, this a spontaneous process, but it can become confusing and it may lead to one or more persons dominating the activity. 46 The second step in brainstorming is clarification. After all ideas have been generated, the group reviews them to make sure everyone is clear about their meaning. The focus of the generation phase was quantity and not quality, therefore, during clarification group members are encouraged to question the meaning of the generated items. The final step of brainstorming is the evaluation of clarified items. Here the group considers the list and rules out duplications, and irrelevant ideas. During the evaluation

PAGE 49

37 phase affinity diagramming or other methods for grouping ideas into similar concepts may be employed to make sense out of the generated items. Cause-and-effect diagramming is the heart of CEA. It is a tool used to define and illustrate the relationships between an effect and an outcome, or a problem and beliefs about the possible causes or factors contributing to it. 46 It combines the product of brainstorming with a systematic analysis to organize and evaluate causes to determine which are most likely contributing factors or causes. 46 Cause-and-effect diagramming can be done using fishbone or tree diagrams. Fishbone diagrams are the traditional way of illustrating cause-and-effect relationships. Nevertheless, tree diagramming, or the five whys analysis, is an effective way to graphically show the breakdown of large problems into their increasingly more detailed elements. 46 Tree diagrams help the group members to move from general to specific, or vice versa, in an organized manner, and they show the logical connections among the relationships. Accreditation organizations such as the Joint Commission on Accreditation of HealthCare Organizations (JCAHO) have recognized the importance of CAE and have incorporated the use of root cause analysis (RCA), which is a more specific form of CAE, into their quality criteria. Currently, JCAHO only requires RCA for inpatient sentinel events. Causes of DRM in the ambulatory setting have not received much attention and information on CAE for adverse outcomes of drug therapy in the outpatient setting to my knowledge is non-existent. In the Joint Commission paradigm, the data used as input for RCA are the collection of activities that led to the specific incident. Activities are determined by interviews with providers involved in that particular patients therapy and by reviewing

PAGE 50

38 the patients medical chart. While this approach is ideal for sentinel events in the inpatient setting because the quality team has access to providers involved in the particular event and the patients medical record, it is not as useful for rate based events that occur in ambulatory care. A patient who experienced a DRM in the ambulatory setting may have participated in a number of microsystems which are geographically separated, such as the primary care, specialist, pharmacy, and laboratory systems; or personal events. Managed care organizations and health care coalitions, typically do not have the same leverage and influence over these microsystems as, say, hospital administrators who have access to the medical records and have the influence to require participation from professionals involved in the care of a patient who experienced a PDRM. In summary, literature that evaluates the utility of administrative data for health services research was presented. Different methodologies for electronically screening health care data were also discussed. The history, purpose and methods of the Delphi technique were presented. Finally, the process for cause-and-effect analysis was discussed

PAGE 51

CHAPTER 4 PRELIMINARY WORK This section first addresses the prevalence of PDRM in the ambulatory care setting. Following is an evaluation of drug treatment categories frequently involved in preventable drug related admissions (PDRAs). Next, is a discussion of problem areas in the MUS that were found to be proximate causes a of PDRAs. This section will end with an evaluation of latent causes of PDRAs. Prevalence of Preventable Drug Related Admissions The Harvard Medical Practice Study (HMPS) is historic because it was the first large scale study to look at medical injury as a result of errors or problems in the process of medical care. 48 Prior to the HMPS, research on drug related injury focused mainly on adverse drug reactions (ADRs), which are typically thought of as the inherent risk associated with medications use and not problems with drug therapy management. Even though the HMPS was published in the early 1990s, medication errors and PDRM did not gain widespread appreciation and public awareness until the Institute of Medicines (IOM) Report, To Err is Human, in 2000. 2 The IOM report revisited the HMPS and a large study from Utah that focused exclusively on inpatient medical errors and adverse events. The report gave much less attention to ambulatory events. Nevertheless, they did suggest DRM causes a significant a The term proximate cause is being used to describe the initial error or problem that started the events that lead to a PDRM. The term is defined as: That which in ordinary natural sequence produces a specific result, no independent disturbing agencies intervening. Webster's Revised Unabridged Dictionary, 1996, 1998 MICRA 39

PAGE 52

40 number of admissions to inpatient facilities, and they reported the proportion of preventable admissions is unknown. Winterstein et al. (2001) tried to address this issue by producing a meta-analytic analysis of drug related hospital admissions (DRA). 1 The objective was to estimate the prevalence of PDRA and to explore the relationship between study characteristics and prevalence estimates. Unfortunately, a generalizable prevalence estimate was not produced because the studies did not have similar protocols and there was too much heterogeneity in prevalence estimates to summarize the findings with a mean value. Instead, a median and range were presented to display the distribution of studies and meta-regression coefficients were calculated to evaluate the association of various study characteristics with prevalence estimates. The methods and results are presented below. Since the goal was to produce a meta-analytic summary estimate of PDRA, a criterion was established to select only studies that attempted to conduct a comprehensive surveillance of drug therapy as a cause of preventable patient injury. Studies were excluded that limited their scope to specific drug treatments, indications, injuries, or to only one drug-therapy problem. The studies had to discuss the relationship between pharmacotherapy and patient morbidity, and preventability, along with the necessary information to calculate prevalence. Fifteen studies 5-12,49-55 published between 1980 and 1999 met the inclusion criteria. All studies were conducted in industrialized countries: eight in Europe, four in the US, two in Australia, and one in Canada. (Table 4.1) The median DRA prevalence was 7.1% (IQR 5.7-16.2%) and the median PDRA prevalence was 4.3% (IQR 3.1-9.5%). Overall, the median preventability rate was 59% (IQR 50-73%).

PAGE 53

41 Since the protocols were not homogeneous across the studies and the Cochrans Q test indicated extreme heterogeneity (Q: 176; df: 14; p<0.001), an analysis of the association between different study characteristics and PDRA prevalence estimates was warranted. The following characteristics have been discussed in the literature and were considered potential contributors to the heterogeneity of the findings: (Table 4.2) Inclusion/ exclusion of first hospital admission (re-admission studies) Planned admissions, and transfers from other units or hospitals Mean sample age (>70 vs < 70); country (US/other) Selection of hospital units vs. inclusion of entire hospitals Publication year (>1992 vs. < 1992) Inclusion/ exclusion of indirect drug-related morbidity (lack of drug effectiveness, and no access) Inclusion/ exclusion of MD and patient interview Explicit criteria for judging preventability The study characteristics were analyzed using meta-regression models, the dependent variable was the transformed PDRA estimate and the independent variable was a specific study characteristic for each model. These regressions were fit with random-effects weights using restricted maximum likelihood estimation of the between study variance to account for study heterogeneity. The meta-regression models produced point estimates and 95% confidence intervals of prevalence odds ratios comparing studies that differ with respect to each study characteristic (Table 4.2). The inclusion or exclusion of first admissions (i.e. re-admission studies) was the study characteristic most strongly associated with PDRA prevalence (OR: 3.7; 95% CI: 1.5-8.9). The mean of age of admissions was the next most strongly associated characteristic with PDRA prevalence (OR: 2.0; 95% CI: 0.95-4.2). The inclusion of indirect DRM was also associated with higher prevalence estimates (OR: 1.9; 95% CI: 0.92-3.9). The

PAGE 54

42 remaining study characteristics listed in Table 4.2 had little or no apparent association with PDRA prevalence. The results of this systematic review suggest that PDRAs represent a significant public health concern in ambulatory care (Table 4.1). In most studies, more than half of DRAs were preventable. That is, they were not considered acceptable consequences of therapeutic risk-benefit considerations but rather caused by inappropriate care and medication errors. The data in Table 4.1 give the impression of a widespread and long-standing problem in the quality of drug therapy management because the studies represent problems from 1980 to 1999 and nine of the fifteen studies had PDRA prevalence estimates above four percent. To place the reported prevalence estimates in perspective, according to the National Hospital Discharge Survey, there were 31.8 million admissions to U.S. hospitals in 1998. 56 The top six primary diagnostic categories (heart disease, delivery, neoplasms, pneumonia, psychosis, and cerebrovascular disease) each accounted for three to twelve percent of these admissions. Any of the studies, or all of them combined, suggest that inappropriate management of drug therapy may be a leading cause of hospital admissions in developed countries. The meta-regression analysis indicated that sampling methodology does affect prevalence finding. Limiting the sample population to patients previously hospitalized produced much higher PDRA prevalence estimates than not restricting the sample to re-admissions. The causes of these findings are not clear. Patients re-admitted may be a sicker sub-population who had been hospitalized with more risk factors for DRM, e.g., more drugs, more diseases, more prescribers, etc. A need for research specifically

PAGE 55

43 designed to evaluate the transition of drug therapy management from the hospital to the ambulatory setting is clearly warranted. DRM is a concept, and the way in which it is operationalized affects its prevalence. As one would expect, studies that included indirect injury (e.g., lack of therapeutic effect, and lack of access to medications) were associated with higher prevalence findings. The limited view of drug-relatedness found in one-third of the articles used in this analysis may be an artifact of an outdated approach to evaluating adverse outcomes of drug therapy. For an assessment of DRM to be considered comprehensive it would have to include both direct and indirect drug-related injury. The IOM report suggested, There is evidence indicating that [adverse drug events] account for a sizeable number of admissions to inpatient facilities. This systematic review confirms their suspicion and suggests that PDRM in ambulatory care is at least as significant and prevalent as in inpatient care environment. Drug Categories Involved in PDRA The same fifteen drug-related admission studies from Table 4.1 were used to assess the medications commonly involved in DRAs. Three of the studies 5,12,54 were not included because they did not provide information on specific medications or therapeutic classes involved in DRAs. Medications involved in DRAs were the unit of analysis rather than medications involved in PDRAs because fewer than five studies provided drug specific preventability.

PAGE 56

44 Table 4.1: Studies Reporting DRA and PDRA: Sample Characteristics and Prevalence Estimates b ReferenceCountryStudy year & periodDRA PrevalencePDRA PrevalencePreventability of DRADarchy199441/ 62330/41199912 months-6.6%(73%)Ng199631/17210/1721196319993 weeks(18%)(5.8%)(32%)Raschetti 94/9545/ 183325/183325/45199912 weeks(2.5%)(1.4%)(55.6%)Cunningham 54/1011 1997(5.3%)Nelson199373/450 19961 month(16.2%)Courtman92/9321/15018/15018/2119955 months(14%)(12%)(86%)Dartnell199455/96536/96519951 month(5.7%)(3.7%)Hallas8.0%67/143 1992(n= 1999)(47%)Lindley26/41613/ 261992(6.3%)(50%)Nikolaus87/9022/873208238313199236 months(25.3%)(12.6%)(50%)Bero45/22434/22434/451991(21.1%)(15.2%)(76%)Bigby83/8473/686198724 months(10.6%)Lakshmanan 198435/83419/3519862 months(4.2%)(54%)Trunet 78/8197/165143/165143/97198533 months(5.9%)(2.6%)(44.3%)Trunet 78/7923/32514/32514/23198012 months(7.1%)(4.3%)(61%)France30/623 (4.8%)AustraliaItaly43/54 (79.6%)USA43/450 (9.5%)43/73 (58.9%)UK4 weeks each unit43/1011 (4.3%)CanadaAustralia36/55 (65.5%)Denmark 1988/893.8%UK10 weeks13/416 (3.1%)GermanyUSAMissingUSA43/686 (6.3%)FranceFrance43/73 (58.9%)USA19/834 (2.3%) b Modified table from Winterstein et al. (2002)

PAGE 57

45 Table 4.2. Prevalence Estimates and Odds Ratio per Stratum/Specific Study Groups Study characteristicCategoryNumber of StudiesAverage PDRA1 (%)PDRA prevalence odds ratios (95% CI)2Excluded214Included134.23.7 (1.5 8.9)> 7067.6 7073.92.0 (0.95 4.2)Missing2Included106.1Excluded53.31.9 (0.92 3.9)USA46.9Other114.41.6 (0.73 3.6)Excluded85.9Included74.11.5 (0.71 3.0)Excluded66Included951.3 (0.61 2.8)Entire hospital85.8Selected units74.51.1 (0.57 2.3) 199285> 1992751.0 (0.47 2.1)Yes85.8No74.21.4 (0.67 2.9)Yes65No94.91.0 (0.47 2.2)MD/ patient interview*Specified criteria for preventability judgment*Transfers from other units or hospitalsPlanned admissionsHospital UnitsYear of publicationFirst hospital admissionMean ageIndirect drug-related morbidityCountry The purpose was to find the treatment categories that were involved in DRAs and to report their median frequency. This was done by classifying specific medications into their therapeutic class and then organizing therapeutic classes into treatment categories. Variability existed n the way medications involved in the DRAs were presented. Studies either listed the specific medications involved in the DRAs, the number of events involving a specific therapeutic class or only the number of events related to a treatment category. Therapeutic classes were kept as long as they were mentioned in at least two studies. If only one study mentioned a particular therapeutic class it was categorized into a more general group, e.g., cardiovascular/other.

PAGE 58

46 The cardiovascular treatment category had the highest median DRA prevalence (33.3%) c and this category was represented in twelve studies. The median prevalence for the Anti-inflammatory category was 11.4% (represented in eleven studies). The median prevalence of the Anti-diabetic category was 12.15% (represented in eight studies). The median prevalence of the Psychotropic category was 8.75% (represented in eight studies). The median prevalence of the Anti-infective category was 8% (represented in seven studies). The median prevalence of the Non-specific miscellaneous category was 12% (represented in ten studies). d See Figure 4.1 for bar charts of treatment categories and their median prevalence that were mentioned in at least six studies. The actual data from the studies can be found in Appendix A. The therapeutic classes specifically addressed in at least six studies are presented in Figure 4.2. The therapeutic class Diuretics was most often mentioned (10 studies). Antihypertensives, Hypoglycemic, and NSAIDs were mentioned in eight studies. Antibiotics were specifically mentioned in seven studies. These findings are not surprising because they represent therapeutic categories for common disease states. Cardiovascular disease is the leading cause of hospital admissions 56 and medications used to treat cardiovascular disease were involved in a c Prevalence was calculated by using the total number of medications involved in DRAs from a particular treatment category as the numerator and total number of medications involved in DRAs for the denominator. The prevalence for each treatment category was calculated individually for each study and the median values was used to represent the prevalence of that treatment category. d The Miscellaneous category (also described as the other category was often used in the articles to group medications that were infrequently involved in DRAs. This category is not mutually exclusive from the other treatment categories. For example; one study may have found that 15% of the medications involved in DRAs were NSAIDs while another study only found 1% was from NSAIDs. The study that found only 1% if medications involved were NSAIDs probably would have put them in the other category.

PAGE 59

47 large portion of DRAs. Admissions due to diabetes, infections, and psychosis are also among the leading causes of hospital admissions. From the results of this analysis it appears that medications most often prescribed tend to be the treatments most often associated with DRAs. Cardiovascular, psychotherapeutic anti-infective, analgesics, and anti-diabetic agents were among the top ten therapeutic classes by retail sales share in 2001. e This means that DRAs appear to be happening most frequently in patients being treated for common conditions with common and well-accepted therapies. From a measurement perspective this information can be very useful. It can be used to focus the development of medication specific performance indicators and it can also be used to help select indicators for use that have already been developed. DRAs in the twelve studies used in this analysis were determined by medical record review, however, which is currently the gold standard for measuring the prevalence of DRM. Medical record review is a costly and inefficient process. The use of automated performance indicators is an alternative for measuring the prevalence of DRM. This information is useful for developing and selecting performance indicators. Any instrument used to measure the prevalence of DRM in a population should try to address the treatment categories found to represent a significant proportion of medications involved in DRM i.e., cardiovascular, psychotropic, anti-inflammatory, analgesic, ant-diabetic and anti-infective agents. e Source NDC Pharmaceutical audit Suite. http://www.ndchealth.com/epharma/YIR/pharmatrends.htm

PAGE 60

48 Apparent/Proximate Causes of PDRAs As discussed in chapter two, DRM occurs when an error interacts with the latent conditions of a system to produce injury. Errors are produced at the provider-system interface. In the MUS, the sharp-edges where professionals interact with the patient and with other health professionals happens at the prescribing, dispensing, (self) administration, and monitoring nodes of the system. The same fifteen articles listed in Table 4.1 were analyzed to determine the proximate causes of the DTPs, i.e., DTPs were classified into the nodes of the MUS where the errors likely occurred. A classification template for mapping DTPs to the nodes of the MUS was developed to judge error etiology. When DTPs were not classified into the nodes of the MUS the judgment of three reviewers were used to classify them (Dr Charles D Hepler, Sooyeon Kwon, and Brian C Sauer). Classification required 100% agreement, each reviewer judged the DTPs location in the MUS independently. The reviewers discussed their individual decisions and when discrepancies occurred they stated their opinions and worked through the differences until agreement was obtained. The authors focus, as well as any additional information mentioned in the article, was considered when assigning the DTPs to nodes of the MUS. When an author in some cases appeared to include both dose prescribed and dose administered in the same overdose group, the case was classified into a category labeled non-identifiable process problem. See Appendix B for a complete list of DTPs and information used to make judgments for the placement of DTPs into nodes of the MUS.

PAGE 61

49 05101520253035Antibiotic PsychotropicAnti-inflammatoryAnti-diabeticMiscellaneousCardiovascular Figure 4.1: Involvement of Treatment Category to DRMs: Median prevalence for Treatment Categories Mentioned in at Least 6 studies 02468101AntibioticsAntihypertensive NSAIDsHypoglycemicDiuretics 2 Figure 4.2: Therapeutic Class Involvement in DRM: The Number of Studies that Specifically Mentioned Therapeutic Class

PAGE 62

50 Since the purpose was to identify process related problems f that led to PDRA, non-preventable ADRs were not included in the analysis. ADRs are a special type of DTPs; they typically represent an outcome and not a process of drug therapy. They have a gradient of severity; the less severe ADRs would be considered an actual DTP, e.g., diarrhea. The severe ADRs would be considered DRMs, e.g., hospitalization from dehydration due to frequent diarrhea. ADRs have historically been treated as the inherent risk of medication use and not as a failure of drug-therapy management. Preventable hospital admissions due to ADRs, nevertheless, may represent a process failure, because somewhere in the MUS actions were not taken to detect and correct the ADR from developing into a serious injury. Preventable ADRs were classified into the nodes of the MUS according to the authors description. If no description of the process failure was presented then, as default, preventable ADRs were assigned to the monitoring node. To be included, studies had to specify the type of DTPs, or the node in the MUS where the DTP was initiated, that led to PDRAs. Eight studies linked DTPs to preventability, 5-12 (Appendix B) the other seven presented DTPs in relation to DRM, but did not partition DTPs by preventable admissions. The median values and inter-quartile range (IQR) for each node in the MUS were: prescribing (33.93%; IQR: 5.73-54.86%), dispensing (0%), self-administration (22.29%; IQR: 14.19-25.51%), monitoring (31.80%; IQR: 20.13-45.22), non-identifiable process (5.93%; IQR: 0-16.39). (Appendix C and Figure 4.3) f The phrase, problems with the process of care is being used as an alternative to represent the same concept as error. When discussing the etiology of DTPs terms related to process failures and process problems will often be used since the term error has the ability to produce a defensive stance. When examining the causes of DRM it is important to reduce tension and the defensive armory of professionals to uncover factual events and true beliefs about the processes of care delivered.

PAGE 63

51 The finding that prescribing and monitoring problems were major contributors to PDRAs in these studies is troubling because it clearly indicates the MUS failed at the professional/patient interface. Another important finding was problems in the monitoring node were as prominent as problems in prescribing. This is interesting because inappropriate prescribing tends to receive more attention than monitoring. Many resources are being directed toward improving prescribing decisions, as seen through the popular interest in computerized prescription order entry systems. From these findings one could argue that interventions towards improving systematic monitoring and follow-up of drug therapy should receive at least as much attention as prescribing. (Self) administration or patient non-compliance was also found to be a substantial contributor to PDRAs. Patients and caregivers are participants in microsystems of care and they have a critical role in the MUS. (Self) administration is a patient behavior and its overlap with medical error and professional responsibility can be debated. Non-compliance is a result of intended and unintended actions. It has been shown that patients make poor decisions because of rule and knowledge based mistakes. 57 Health professionals should be able to help patients correct their misunderstandings. They can also help patients find creative ways to prevent unintentional non-compliance by reducing slips and lapses. Nevertheless, patients typically choose to initiate the processes involved in the MUS and in doing so they incur some of the responsibility for their health outcomes. From a systems perspective, finding methods to reduce the occurrence of PDRM is not limited to directing interventions at health care professionals and provider specific processes. Patient directed interventions, and reconfigurations directed toward patient behavior would be included in the systems umbrella.

PAGE 64

52 Figure 4.3: Nodes of the MUS Involved in PDRA Prescribing, monitoring and patient (self) administration appear to be the nodes where most DTPs were generated. Because prescribing, monitoring and (self) administration nodes have different characteristics it is possible error provoking conditions upstream and faulty corrective mechanisms downstream (i.e., latent failures) have unique system design flaws. Any rational intervention directed at a specific node in the MUS would need to address the system conditions that increase the chance of an error being created and progressing undetected. Latent Causes of PDRA Latent causes are flaws in the design and organization of systems that allowed errors to occur, go undetected and result in patient injury. None of the fifteen studies provided a comprehensive assessment of latent causes. Nevertheless, eight studies 5-8,10-12 did mention, in the discussion, possible latent causes. (Table 4.3) Lack of adequate knowledge was mentioned most frequently as an underlying cause for errors in the prescribing node of the MUS (seven studies). Three of these studies mentioned patient knowledge was a probable reason for problems in the (self) administration node. Education was considered the best intervention for both prescriber and patient knowledge deficits. Three studies mentioned lack of communication and coordination among professionals, especially the physician and pharmacist, was an underlying cause of

PAGE 65

53 PDRAs. Two studies implied the lack of pharmacists involvement in the MUS was a factor and one study mentioned inadequate monitoring procedures were an underlying cause. Unlike many inpatient DRM studies, none of the fifteen PDRA studies systematically evaluated the latent causes or empirically tried to locate system failures or determine their relationship to higher order or competing systems. Without an adequate evaluation of these influences and interactions, the development of meaningful interventions will likely be difficult. Instead, corrective approaches will probably result in generic interventions, those that the organization know how to do, for example send a letter to the physicians who seem to be producing a portion of the problems. For meaningful improvements to occur, the development of rational interventions are necessary. The term rational is being used to represent interventions that are directed at specific underlying system related causes. These are typically design and organization issues. If lack of prescribing knowledge was determined to be a main cause of prescribing errors, then a rational intervention may include finding better ways to provide therapeutic information to prescribers at the time therapeutic decisions are being made. From this analysis it is clear that a formal cause-and-effect analysis is needed to uncover conditions of the MUS that make it prone to problems within the medications use system. To summarize, injury as a result of drug therapy is a serious problem in ambulatory care. A review of fifteen studies showed the median PDRA prevalence was approximately four percent. Many common treatment categories were involved in these DRA. Cardiovascular and anti-inflammatory agents were involved in many DRAs. Errors in the MUS are the apparent causes of PDRM and problems in the prescribing, monitoring

PAGE 66

54 and (self) administration nodes occurred frequently. None of the studies systematically evaluated the latent conditions that provide the opportunity for proximate errors to occur and go undetected. Lack of prescriber and patient knowledge was mentioned as latent causes for prescribing and (self) administration problems. Nevertheless, a planned systematic analysis is warranted. A better understanding of latent conditions could be used to develop rational interventions. CEA is a promising method for uncovering latent conditions. Table 4.3. Latent Causes of PDRAs Prescribers EducationPatients EducationMonitorInvolve pharmacistInter-professional CommunicationDarchy, 1999Raschetti,1998Cunningham, 1997Nelson, 1996Dartnell, 1996Courtman, 1995Bero, 1991Trunet, 1980

PAGE 67

CHAPTER 5 METHODS This study was executed in three steps. In the first step, database analysis, MU-PIs were used to estimate the number of PDRM positives in the study database. In step two the degree of association between selected MU-PIs and specific nodes of the MUS where the process failure may have originated was evaluated. In step three, selected MU-PIs from each node of the MUS were submitted to cause-and-effect analysis (CEA). Specific Aims 1. To establish the frequency of MU-PI positives in the study population. 2. To evaluate population based explanations for MU-PI positives. 3. To identify the node of the medications use system (MUS) where the pattern of care from the MU-PI originated. 4. To identify system related causes that are common among nodes of the MUS 5. To identify system related causes that are unique to nodes of the MUS Step One: Database Analysis Claims Data Types The MU-PIs were used to screen the study database. Each MU-PI was translated into the administrative billing codes that represent the process and outcome components of the indicators. 13 The billing codes included the International Classification of Disease 9th edition (ICD-9) and Current Procedural Terminology (CPT) and National Drug Codes (NDC). The population included patients of all ages who were enrolled in a preferred provider organization (PPO) health plan that managed employees from a member of the 55

PAGE 68

56 health care coalition. The primary data files include Professional Claims (Center for Medicare Services (CMS) 1500 form), Facility Claims (Universal Billing (UB) 92 form) and Pharmacy Claims. The CMS 1500 and UB 92 forms are provided at the following web address: http://www.cms.hhs.gov/providers/edi/edi5.asp#Form%20CMS-1500 Descriptive Analysis to Evaluate the Integrity of Claims Data The number of office visit, emergency department (ED) visit, hospital admission and pharmacy claims were compared month by month. If any month showed a large drop in claims, this would suggest large numbers of claims might be missing from the database. Office visit and ED visit claims were identified in the professional claims database with specific CPTs. a Primary diagnosis, NDC, procedure codes, and service dates were evaluated to identify missing or invalid codes. Links between databases were established to ensure members could be traced across claim types. Population Demographic Average length of time enrolled in the health plan, average number of office visits, age and gender frequencies were calculated to describe the study population. Frequency of age by category, gender, number of office visits, and number of different drug classes, b different pharmacies, different conditions, and different prescribers were used in logistic regression analysis to better understand the MU-PI findings from information available in the database. a See Appendix E for a list of the codes used to identify office visits and ED visits b The Universal System of Classification (USC) was used to identity different drug classes and routes of administration. Unique USC codes identified drug classes (e.g., H2 blockers vs. GI proton pump) as well as different routes of administration (e.g., sumatriptan oral vs. sumatriptan nasal). Drug class was identified by unique USC codes.

PAGE 69

57 MU-PI Coding Concepts and Analysis Faris 2001 translation of the PDRM scenarios into medical event codes was used to construct the MU-PIs. 13 In Faris study, two medical record coders had independently selected all possible ICD-9 and CPT codes that represented the PDRM scenarios. The codes identified by the medical record coders had also been reviewed by a physician for clinical judgment about whether each code was consistent with the PDRM scenario. Please see Faris 2001 for more detail on the coding methodology. 13 A list of the 40 MU-PIs c used in this study can be found in Appendix D. Search algorithms Each MU-PI had the following format, an outcome of care and a specific pattern of prior care. For example, ED visit or hospital admission for Hemorrhagic event AND use of warfarin AND prothrombin time/INR not done every month. The pattern of care is the process component of the indicator and it represents a potential DTP. An ED visit or hospitalization was required for the outcome of care. An indicator positive (possible PDRM) required both the process failure and the outcome component of the indicator. Process failures fell into three categories: disease-drug interaction, drug monitoring, and drug-drug interactions. Each required different types of search algorithms. SAS version 8.2 was used to code and run the indicators in the database. The type of search algorithm used for each indicator is presented in Appendix F. Disease-drug interaction The disease-drug algorithm required the presence of specific diagnosis codes and specific pharmacy claims prior to a claim for the associated outcome. As illustrated in c Only 40 of Faris 49 indicators were run. Search algorithms were to complex for nine and they were not used for this analysis.

PAGE 70

58 Figure 5.1, time moves from right to left by the blue arrows. The diagnosis claim had to be present before the use of the medication (pharmacy claim). The green arrow represents the lag, i.e., the duration of time from the outcome date to the last drug date prior to the outcome. The cut off for the disease-drug interaction lag was set at 100 days, in order to accommodate 90-day mail-order pharmacy refill cycles and to include patients who may have been taking the medication in proximity to the outcome. An indicator positive required the outcome, the disease code(s) and a pharmacy claim for the drug(s) within 100 days prior to the outcome. Prior Diagnosis Date Last Drug Date Prior to Outcome Outcome Date Lag Figure 5.1. Disease-Drug Interaction Search Algorithm Drug monitoring Drug monitoring algorithms required the presence of specific pharmacy claims and CPT claims, which indicate whether specific laboratory analyses were conducted. Two coding solutions were required for monitoring indicators: one for the process component and another for the process and outcome simultaneously. As illustrated in Figure 5.2, time moves from right to left by the blue arrows. Once specific drug claims initiate the process, lags are calculated to represent the interval of time from drug to first CPT claim (lag 1), CPT claim to CPT claim (lag 2), and last drug claim date to last CPT claim date (lag 3). Patients were considered at risk and included in the analysis when they appeared to be taking the medication for at least as long as the defined lag time. Monitoring intervals (i.e., lag times) vary according to MU-PIs. If any lag exceeded the defined

PAGE 71

59 monitoring interval the member screened positive for the process component of the MU-PI. Figure 5.2. Process Search Algorithm for Indicators that Require Monitoring PDRM positives were identified when members had specified pharmacy claims and the lag between the outcome and the last CPT claim prior to the outcome exceeded the defined monitoring interval. If the lag was less than four days, the lag between the outcome date and second to last CPT date was used to avoid false negatives. The assumption is that monitoring within four days of hospitalization identified the impending injury and need for emergency or hospital care. (Figure 5.3) Figure 5.3. Process and Outcome Search Algorithm for Indicators that Require Monitoring The determined monitoring interval for the warfarin indicator is 30 days. The example in Figure 5.4 would register as a process positive, but not a PDRM positive because the pattern of care was stabilized before the outcome occurred. The outcome may indeed be drug related, but it is not represented by the MU-PI indicators. The lag had to exceed the defined monitoring interval from outcome date to last CPT claim for a PDRM positive. CPT date 1 First Drug Date n t h CPT date CPT date 2 LAG 2a LAG 2b Last Drug Date LAG 1 LAG 3 Last CPT Outcome Date Drug Date LAG

PAGE 72

60 Figure 5.4. Example of a Process Positive but not a PDRM Positive Drug-drug interaction The third search al gorithm considered multiple drugs. This included drug-drug interactions, drug-no drug situations and overuse of one drug and under use of the other. The process component is indicator specific. If the MU-PI is a simple drug-drug interaction with Drug A being a chronic medication and Drug B acute process positives are recorded when a claim for Drug B occurred between the first and last claims for Drug A. (Figure 5.5) PDRM positives are recorded when pharmacy claims for Drugs A and B occur within the specified number of days for each lag. Figure 5.5. Drug-Drug Interaction Search Algorithquency of MU-PI positives in the study esearch Question 1. What is the period prevalence for process positives in the Research Question 2. prevalence for the PDRM positives in INR Claim 1 date First Warfarin Claim nt h INR Claim date INR Claim 2 date 50da y s25 days40 days 25 days Outcome Date LAG 2 Drug A Firsat t Drug D e Drug A Lasat t Drug D e Drug B L ast Date Outcome Date LAG 1 Diagnosis Date m Prevalence Estimates for MU-PI Positives Specific Aim 1. To establish the fre population. R study population? What is the period the study population?

PAGE 73

61 This is a cross-sectional (period prevalence) analysis. The population included patients of all ages who were enrolled in a PPO health plan from January 1 st 1999 to September 11 th 2001. The analysis was limited to one process positive per indicator; however, no limits were placed on PDRM positives. Demographic Explanations for Prevalence Findings Specific Aim 2. To evaluate demographic based explanations for MU-PI positives. Information from the database was used to better understand indicator positives. Descriptive analyses of monitoring intervals were conducted to determine if cases on the margin of the monitoring interval were being identified as process positives, and multivariate logistic regression was conducted to explore the relationship between PDRM positives and demographic/system related variables. Distribution of Monitoring Intervals Research Question 3. Do many process positives appear to be captured from the margin of the defined monitoring intervals for indicators that require drug therapy monitoring? Due to the nature of the MU-PIs, the quality of drug-therapy monitoring can be grouped in one of two categories. A case can either screen positive or negative for the inappropriate monitoring interval. When a case screens positive it means an individual did not have claims for the defined laboratory tests within the specified monitoring interval. One limitation of this approach is the strict cut-offsthose on the margins are categorized as process failures even if they were only a few days or weeks over the required time. To develop a more comprehensive picture of how cases deviated from the defined monitoring intervals, an analysis of the first monitoring interval (i.e., the lag time from the first pharmacy claim for the drug of interest to the first claim for laboratory tests

PAGE 74

62 of interest) was presented for indicators grouped by the indicator defined monitoring interval. This was a descriptive analysis. The intervals were determined by dividing the monitoring requirement in half. For example, if required monitoring for the indicators was one-month, the categories for descriptive analyses were based on fifteen day intervals. Analysis was only carried out for indicators with defined monitoring intervals of one month or greater. Variables Associated with PDRM Positives Research Question 4. What demographic variables are associated with MU-PI findings? Available information from the database was used to explain the prevalence findings. Bivariate and multivariate logistic regression analyses were conducted. Demographic variables age, gender, and number of drug class, pharmacies and prescribers were used as independent variables, while PDRM positives were the dependent variables. Multicollinearity was evaluated with tolerance and variance inflation statistics. Tolerance is 1-R 2 for the regression of a given independent variable on all other independents, ignoring the dependent. The higher the intercorrelation of the independents, the more the tolerance will approach zero. If tolerance is less than 0.20 a problem with multicollinearity is indicated. After multicollinearity was assessed, stepwise variable selection was used to build the regression model. The likelihood ratio test was used to test the overall significance of the model. Max-rescaled R 2 was used to assess the predictive power of the model. The significance of the variables in the model was assessed by the Wald Chi-Squared test and confidence intervals (CIs). Hosmer-Lemeshow statistics were calculated to determine the goodness

PAGE 75

63 of-fit of the final multivariate model. Odds ratios (ORs) and Walds CIs were calculated directly from the estimated regression coefficients and their standard errors. All P-values were set at 0.05. All analysis were performed using SAS Version 8.2 Step Two: Node Identification of Select MU-PIs Specific Aim 3. To MUS where the pattern of care from PDRM scenarios originated. The ultimate goal of this dissertation was to identify system factors (i.e., latent conditions of the health care system) that appear to be common or unique to nodes of the MUS. This requires an exploration of the association between selected indicators and specific nodes of the MUS. To do this, selected MU-PIs were sent to a Delphi panel of clinicians and researchers for their judgment on the node of the MUS in which the process component of the indicator likely originated. Pilot Testing The node identification panel survey was pilot tested with graduate students and faculty in Pharmacy Health Care Administration at the University of Florida. All five participants were pharmacists, three had received their pharmacy degrees in the United States, one had received her degree in South Korea and the other had received his degree in Germany. The purpose of the pilot test was to evaluate the understandability of the survey and the node identification scoring method. Organization of content, directions, and scoring system were modified based on comments from the first round of the pilot test. No significant problems or need for changes were identified in the second round of pilot testing.

PAGE 76

64 Selection of MU-PIs for Node Identification Study MU-PIs used in the pilot were not the same as those used in the Delphi study. Indicators were chosen based on preliminary results from the MU-PI study. MU-PIs were primarily chosen based on indicator frequency. d Delphi Recruitment The snowball sampling or chain referral sampling was used to identify participants for the Node Identification Delphi study. This method has been widely used in qualitative research. It yields a study sample of people with specific characteristics, e.g., expertise. People with desired characteristics mention others who may also be likely candidates for study participation. 58 The first wave of recruitment letters was mailed with stamped return envelopes to researchers who have published extensively in the field of medication error research or who were referred by faculty members in Pharmacy Health Care Administration at the University of Florida. The letter specified the goal of the study, asked the contacts to participate in the Delphi panel, and invited them to recommend others who are trusted experts in medication error or adverse drug events research. e I telephoned the first round contacts between one and two weeks after the recruitment letter was mailed to answer questions the contacts may have had and to try to convince them to participate. f Sample size and power for the statistical tests were considered prior to the recruitment process. When setting alpha at 0.05 and beta at 0.8, it was determined that a d Scenarios were selected based on preliminary indicator findings, changes were made to the MU-PIs and not all scenarios selected for the Delphi study had PDRM indicator positives. e The term adverse drug event was used for the Delphi study instead of drug related morbidity because it is the standard term in the medical community and I wanted to avoid confusion among the panel members. f Please see Appendix G for a copy of the recruitment letter.

PAGE 77

65 sample size of thirty-two was needed to identify a node as statistically different from the others when its mean value was at least a six and the standard deviation was less than or equal to three. Node Identification Survey Surveys were mailed to members who agreed to participate. The survey provided detailed instructions that included definitions for the nodes of the MUS for the assignment task. Specific demographic information was requested and it included name, occupation (researcher and/or licensed health professional), whether they were practicing or not, level of education, age, and gender, belief about the significance of PDRM The panel members were provided fourteen MU-PIs and were asked to allocate the number of points from 1 into the nodes of the MUS where they believe the error or drug therapy problem originated for each of the fourteen indicators. They were asked to answer based on a population perspective, meaning they were asked to identify the node that explained the origin of the majority of cases that might fit the indicator definition. Panel members were also asked to document their decision logic and provide comments when they deemed necessary. g Delphi Process Results (panelists scores) for each MU-PI were analyzed after the first round. If an indicator had received an average score for one node that was significantly higher than its average score for all other nodes, it was assigned to that node and deleted from the second round of the Delphi study. In the second round, panel members received information about each remaining indicator, including summary distributions of g See Appendix H for a copy of the Node Identification Survey.

PAGE 78

66 responses, median and interquartile ranges for each node, the particular Delphi members response to each item, and a summary of the decision logic statements for the panel (grouped into like statements). Delphi participates were asked to reconsider their opinions and to state their decision logic regardless of whether they changed their belief. Node Identification Analysis Research Question 5. What nodes of the MUS did the selected PDRM scenarios likely originate from? Hypothesis: Ho: Prescribing = Dispensing = Administration = Monitoring Ha: At least one node is significantly greater than all other nodes. The Kruskall Wallis test (KW), an analysis of variance test for non-parametric data, was used to determine if differences in rank order means existed among the nodes of the MUS. Dwass, Steel, Critchlow-Fligner (DSC) multiple comparison post hoc tests were used to determine which nodes were significantly different (P values=0.05). StatsDirect Version 2.3.7 software was used for the KW and DSC tests. Step Three. MU-PI System Level Evaluation Specific Aim 5. To identify common cause sequences among nodes of the MUS. Specific Aim 6. To identify unique cause sequences for specific nodes of the MUS. Selection of MU-PIs for Evaluation The fourteen MU-PIs evaluated in step two were found to be problems that originated in the prescribing or monitoring nodes. Two MU-PIs were selected from the prescribing and monitoring nodes. Selection was limited to MU-PIs that were evaluated in the Node Identification Study and was based on frequency of MU-PI positives. h h Indicator number 28 was evaluated in the Delphi study and had the second highest frequency for the prescribing indicators, however it was not selected because the indicator did not appear to differentiate those with congestive hearth failure (CHF) and heart block from those with CHF only.

PAGE 79

67 Step Three: Cause-and-Effect Analysis MU-PI Evaluation Team Members of the MU-PI evaluation team were identified and invited to participate by the CEO of the health care coalition. They consisted of a Medical Director, PharmD, nurse practitioner, consumer advocate, laboratory manager and a director of a health maintenance organization. The session was scheduled for four hours. Members of the evaluation team were paid $100 per hour. One week prior to the meeting the evaluation team was mailed a briefing that contained the goals for the study and background on the MU-PI results, cause-and-effect analysis and the levels of the health care system. It also provided the MU-PIs that would be evaluated and asked them to begin generating ideas. A copy of the brief is available in Appendix I. MU-PI Evaluation Process The evaluation process began with a brief introduction and slideshow presentation to orient them to the study purpose and database findings. The two MU-PIs formally evaluated were the asthma prescribing indicator and the warfarin monitoring indicator, indicators 39 and 30, respectively. (Appendix D) The following six steps were carried out for each MU-PI that was formally evaluated: brainstorm causes, clarify and organize causes, prioritize causes, tree-diagramming, and establish causes that are common to multiple nodes of the MUS and causes that are unique to specific nodes of the MUS. The moderator provided an introduction and background of the study objectives and indicator prevalence findings. Indicator specific finding were presented for the

PAGE 80

68 warfarin monitoring indicator and the asthma prescribing indicator. The question to guide the brainstorming process was presented for each indicator. The evaluation team brainstormed possible system related problems that may have contributed to the MU-PI indicators. The round robin process was used to elicit causes from each participant. Proposed causes were projected on a screen and documented in Path Maker Version 5.5 software. The evaluation team then clarified and organized the proposed system causes into an affinity table. The affinity table was organized by the levels of the health care system (patient, professional, organization, environmental levels). Proposed causes were organized into the level of the system the problem was associated with. A category for measurement error called artifact was also permitted. Proposed causes were organized into affinity tables with Path Maker Version 5.5 software The evaluation team then prioritized the proposed system cause and the top five causes were selected for tree-diagramming. The facilitator went through the list of proposed causes from the affinity table and the evaluation team voted by raising their hands for each of the five causes they believed contributed the most to the prevalence of the indicator findings. Scores were tallied for each cause and the five receiving the most votes were selected for tree diagramming. The evaluation team then used tree diagramming to identify relationships among the different levels of the health care, as they relate to the proposed cause. Tree diagram consisted of branches that had each level of the health care system (patient, professional, organization, environmental levels). One at a time, the top five proposed causes were placed into the system levels associated with the cause. For example; if the proposed

PAGE 81

69 cause was lack of MDs providing patient education it would go into the professional (microsystem) level. The follow-up question would have been; what at the organizational level may influence MDs ability to provide patient information? The proposed organizational influence would be placed in the organization node of the same branch of the tree. (Figure 5.6) Excel Version XP was used to create the tree diagrams and document the findings. Figure 5.6. Organization of Proposed Causes to System Levels Proposed Cause MDs don't provide patient education N o Medication Management System Guidelines unclear Patient level Professional level Organization level Environment 1 MDs don't provide patient education 2 3 4 Branch What at the organizational level may influence MDs ability to provide patient education? The evaluation team then established commonality and uniqueness within and between nodes of the MUS. An indicator assigned to the monitoring node was tree diagrammed. The evaluation team was given a questionnaire to determine if each cause sequence (i.e., each branch of the tree) would apply to another indicator assigned to the monitoring node. They also had to judge whether each cause sequences would apply to an indicator assigned to the prescribing node. The same procedure was applied to the prescribing indicator that was formally evaluated, i.e., tree diagrammed. (Appendix K)

PAGE 82

70 Analysis for Common and Unique Cause Sequences Research Question 6. What cause sequences were common within the Prescribing Node? Research Question 7. What cause sequences were common within the Monitoring Node? Research Question 8. What cause sequences were common to the Prescribing and Monitoring Nodes? Agreement was determined when five out of six (83%) or four out of five (80%) of judges agreed.

PAGE 83

CHAPTER 6 RESULTS Step One. Database Analysis Descriptive Analysis to Evaluate the Integrity of the Claims Database To evaluate the completeness of the database the numbers of claims were compared by month. The dates of office visit claims ranged from January, 1999 to September, 2001. As illustrated in Figure 6.1, many office visit claims appear to be missing on the tail ends of the data collection period, from January, 1999 to June, 1999 and from July, 2001 to September, 2001. Emergency Department (ED) visit claim dates ranged from January, 1999 to June, 2001. As illustrated in figure 6.2, many ED visit claims appear to be missing from January, 1999 until June, 1999. Facility admission claim dates ranged from January, 1999 to June, 2001. No claims appeared to be missing during the study period (Figure 6.3). Pharmacy claim dates ranged from January, 1999 to June, 2001. As illustrated in figure 6.4, pharmacy claims appeared to be missing from January, 1999 to June, 1999 and in September, 1999. Missing pharmacy claims from September, 1999 are not followed by an increase in claims. It appears claims during this month were missing from the database. The decision was made to only include claims from July, 1999 to June, 2001 for analysis. This decision was made because the number of claims was consistent during this time frame. Even though pharmacy claims appeared to be missing in September, 1999 the decision to include the previous months was made because the benefit of keeping July, August, and September in the database outweighs censoring the time. 71

PAGE 84

72 Check for missing and invalid data Macro level analysis found no missing or invalid codes for primary diagnosis, procedures or service dates. Missing or uninterpretable drug codes were found in 530 of the 572,010 pharmacy claims (571,480 usable pharmacy claims). 020004000600080001000012000Jan-99Mar-99May-99Jul-99Sep-99Nov-99Jan-00Mar-00May-00Jul-00Sep-00Nov-00Jan-01Mar-01May-01Jul-01Sep-01 Figure 6.1. Office Visits Claims by Month 02000400060008000100001200014000Jan-99Mar-99May-99Jul-99Sep-99Nov-99Jan-00Mar-00May-00Jul-00Sep-00Nov-00Jan-01Mar-01May-01 Figure 6.2. ED Visit Claims by Month Links between data types To ensure that members were identifiable across claim types links were established between datasets to identify the number of members that had multiple types of claims.

PAGE 85

73 4,227 of the 4,589 members who had a facility admission claim also had a professional claim. 28,478 of the 33,891 persons with a pharmacy claim also had a professional claim 050100150200250300350Jan-99Mar-99May-99Jul-99Sep-99Nov-99Jan-00Mar-00May-00Jul-00Sep-00Nov-00Jan-01Mar-01May-01 Figure 6.3. Facility Admissions by Month 05000100001500020000250003000035000Jan-99Mar-99May-99Jul-99Sep-99Nov-99Jan-00Mar-00May-00Jul-00Sep-00Nov-00Jan-01Mar-01May-01 Figure 6.4. Pharmacy Claims by Month Population Demographics There were 47,053 members who contributed 58,719 member years during the sample period of July, 1999 to June, 1999. The mean length of enrollment was 467.47 days with a standard deviation of (SD) 225.53 days. The 37,063 members with a medical

PAGE 86

74 claim (pharmacy or professional) contributed a total of 51,892 member-years; mean length of enrollment was 511.04 days with a SD of 201.73 days. Age and gender distributions for the different claim types are presented in Table 6.1. The mean age ranged from thirty-four to thirty-seven across the databases. The majority of members with a claim were female. Females ranged from sixty-five percent to seventy-two percent across the claim types. The fifteen to forty-four year olds were the most frequent age group, their frequencies ranged across the data types from thirty-seven to forty-eight percent. The sixty-five and older group accounted for a small fraction of the claims, their frequencies ranged from about two to eight percent. Table 6.1. Population Age and Gender Frequencies by Type of Claim mean age(SD)By Categoryfrequency, [percent]age <156,622 [19.07]1,084[18.51]1,263[17.61]4,956[14.3]15<=age<4516,519 [47.58]2,143[36.6]3,399[47.39]16,461[47.48]45<=age<6510,743 [30.94]2,144[36.62]2,270[31.65]11,421[32.94]65<=age835 [2.41]484[8.27]240[3.35]1830[5.28]Female22,861 [65.85]3243[71.84]4,659[64.96]23,499[67.78]Male11,858 [34.15]1271[28.16]2,513[35.04]11,169[32.22]000033.97 (18.23)34.48 (22.12)34.71 (18.54)37.03 (18.34)Professional ClaimFacility AdmissionED visitPharmacy Claim Table 6.2 displays additional population characteristics. There were 220,967 office visit claims during the twenty-three month period with an average of 6.64 visits, which were approximately three visits per member per year. Pharmacy claims were submitted for 33,891 members. The average number of pharmacy claims per member during the twenty-three month time period was 16.05 with a SD of 22.9, which were approximately

PAGE 87

75 eight pharmacy claims per year. Members went to an average of 1.55 different pharmacies and an average of 2.13 different prescribers during the study period. Members averaged 5.8 conditions and used an average 4.7 prescriptions from different drug classes. Prevalence of MU-PI Positives There were a total of 103 PDRM positives from eighty members. The prevalence of PDRM positives in members with a medical claim from July, 1999 to June, 2001, was 1.98 (1.60 to 2.37) a per 1,000 member years. There were a total of 10,889 process positives in 5,741 members. The prevalence of process positives from July, 1999 to June, 2001 was 209.84 (206.37-213.34) per 1,000 member years. Table 6.3 shows that, 0.22% of the population with a medical claim screened positive for both the process and outcome component of the MU-PIs, i.e., PDRM positives. There were forty-three PDRM positives in thirty-eight members where the outcome was identified by an ED visit and forty-one PDRM positives in twenty-six members where the outcome was identified by hospital admissions (HA). Nineteen PDRM positives in eighteen members involved a hospital admission from the ED. PDRM positives accounted for 0.75% (95% CI: 0.74%-0.76%) of hospital admissions, and 0.56% (95% CI: 0.55%-0.57%) of ED visits. Table 6.4 provides a summary of the number of PDRM positives by number of indicators. Twenty-two of the forty MU-PIs had no PDRM positives. Of the remaining eighteen indicators, the range of PDRM positives per indicator was from zero to twenty-six. The eleven indicators with the most frequent PDRM positives are listed in Table 6.5. a 95% Confidence interval

PAGE 88

76 Table 6.2. Population Demographics No of Enrollees47,053No of Members with claim37,063Length of time enrolledmean (SD) 511.04 (201.73)No. ED visits10,048No. Hospital Admissions5,855No. of outpatient ph y sician visits220,967mean (SD) 6.64 (7.05)No. of outpatient ph y sician visits ( members ) 33,271 1-210,28130.9 3-46,78120.38 5-64,48013.477-83,1679.529-102,2956.9>106,26718.84No. of Conditions ( diseases ) 34,938mean (SD) 5.8 (4.8)LE 520,71459.296-109,16126.2211-153,3829.6816-201,0302.95GE 206511.86No. of members with pharmac y claim33,891 average per member (SD)13.93 (22.61)No. of dru g classes 543,959 average per member (SD)4.7 (5.31)25,13567.827,30519.712,8097.589782.648362.26No of different pharmacies average per member (SD)1.55 ( 1.45 ) 117,98153.0628,23824.3133,91911.5641,9895.8751,0333.056 or more7312.16No of different prescribers average per member (SD)2.13 ( 2.07 ) 112,48236.8328,28124.4335,30215.6443,2069.4651,9895.8761,1173.37 or more1,5144.47All analysis are for the 23 month time windowCharacteristicsFrequencyPercent

PAGE 89

77 Table 6.3. Frequency of Process and PDRM Positives No. of Members with Positives (unit: person)Population % (denominator=37,063)No. of Positives (unit: each event)Process5,74115.49%10,889PDRM_ED380.10%43PDRM_H A 260.07%41PDRM_ED_H A 180.05%19Total_ PDRM800.22%103 Table 6.4. Number of PDRM Positives by Number of Indicators Freq. of PDRM PositiveNo. of IndicatorsPercent02255.0132410.03353921225.02612.5 7.57.57.55.0 Table 6.6 lists the indicators with the ten most prevalent process positives. Thirty-eight of the forty MU-PIs screened positive for the process component of the indicator. The range of process positives was from zero to 2,282. Approximately fifty-four percent of the indicators had less than one hundred process positives, and ninety-two percent of the indicators had less than 1,000 process positives. System Based Explanations for Prevalence Findings Information from the database was used to better understand indicator positives. Descriptive analyses of monitoring intervals were conducted to determine if cases on the margin of the monitoring interval were being identified as process positives, and multivariate logistic regression was conducted to explore the relationship between PDRM positives and demographic/system related variables.

PAGE 90

78 Distribution of monitoring intervals Table 6.5. Eleven Most Prevalent PDRM Positives IndMnemonicHxDrug_ADrug_BRiskProcessPDRM39AsthmaHx-Bron-NoSteroid2,1533,7043,8781902628CHFHx/CHF_HB-Dig645309791237CHFHx/CHF-NoACEI6454,059398121DepHx/Dep-Benzo1,8982,689321930HemrWar-INR(1)343256919HypothThy-T4TSH(12)2,6151,510915529ARFACEI-BUN(6)4,0592,4092,282531MIHx/MI--ASA-BB**1812,6688957DepHx/Dep-Symp1,8982,971136314ActRespFailCOPD-Benzo6412,689102316CHFHx/HBP-NSAID6,0017,2831,4273 Table 6.6. Ten Most Prevalent Process Positives IndMnemonicHxDrug_ADrug_BRiskProcessPDRM29ARFACEI-BUN(6)4,0592,4092,282523HyprKACEI-ElctroCBC(6)4,0592,4092,209216CHFHx/HBP-NSAID6,0017,2831,427319HypothThy-T4TSH(12)2,6151,510915517HypoKKwd-NoK-Elctro(2)1,810837814643037CHFHx/CHF-NoACEI6454,059398121DepHx/Dep-Benzo1,8982,68932195GIHx/GI-NSAID2,5265,899316030HemrWar-INR(1)343256927GIHx/GI-Ocort2,5264,0752721 Legend for Tables 6.5 and 6.6 See Appendix D to link indicator numbers to the indicator scenarios. The Mnemonic is an abbreviated version of the MU-PI scenario Hx = the number of members who had an ICD-9 code for the particular history of disease or diagnosis of interest Drug A and Drug B = the number of members who had a prescription claim for the drugs of interest. Risk = the number of members who met the criteria for analysis. For example; if the process component includes a drug and six month monitoring interval, then the members at risk would be those who were using the medication of interest for at least six months. Process = the number of members who had the pattern of care represented in the MU-PI PDRM = the number of members who had both the process component and the outcome component of the MU-PI within the specified time frame. Fourteen indicators were sent to a Delphi process for node identification, they were numbered consecutively from one to fifteen. This column provides a cross reference to the MU-PI number Due to the nature of the MU-PIs, the quality of drug-therapy monitoring can be grouped in one of two categories. A case can either screen positive or negative for the

PAGE 91

79 inappropriate monitoring interval. When a case screens positive it means an individual did not have a laboratory test claim within the specified time frame. One limitation of this approach is the strict cut-offsthose on the margins would be categorized as process failures even if they were only a few days or weeks over the required time. To develop a more comprehensive picture of how cases deviated from the specified monitoring intervals, an analysis of the first monitoring interval (i.e., the lag time from the first pharmacy claim for the drug of interest to the first laboratory test claim of interest) is presented for indicators grouped by the required monitoring interval. Analysis was only carried out for indicators with defined monitoring intervals of one month or greater. Table 6.7 provides the indicator numbers for each monitoring interval. For example: Only indicator 30 had a one month monitoring requirement, while indicators 9 and 25 had a three month monitoring requirement. Table 6.7. MU-PI Categorized by Required Monitoring Interval one monthtwo monthsthree monthssix monthstwelve months301292193525811182324262933 Indicator 30 (Hemm warfarin INR (1)) was the only indicator with a required monitoring interval of one month. Figure 6.5 shows forty-one percent of the members who were taking warfarin had a laboratory test claim for an International Normalization Ratio (INR) within a month from their first warfarin pharmacy claim. Only about 3.8% had laboratory test claim on the margin within forty-five days of their first warfarin

PAGE 92

80 claim. The majority, 50.4 percent did not have a laboratory test claim for an INR within 120 days of their first claim for warfarin. 020406080100120140160180 days<=3030120lag timefrequency 50 4 % 0. 2 9% 0. 87% 0. 87% 3. 79% 1 75% 0. 87% 41 1 % Figure 6.5. First Lag for Indicator with One Month Monitoring Requirement Indicators 35 and 12 had a required monitoring interval of two months. Figure 6.6 shows 0.56% of members at risk for these indicators had the specified laboratory test claims (blood pressure and cell blood counts) within two months of their first pharmacy claim, while 99.4% did not have the specified laboratory test claim within 240 days from their first pharmacy claim. Indicator 35 accounted for all but one process positives. Indicator 35 required blood pressure monitoring it is likely that blood pressure monitoring is not billed separately and the process positives are false positives. This indicator was removed from all other analysis (including the previous prevalence findings) because the CPT codes used to identify blood pressure monitoring were not sensitive to blood pressure monitoring. Indicators 9 and 25 had a required monitoring interval of three months. Figure 6.7 shows 4.35% of members at risk for these indicators had the specified laboratory test

PAGE 93

81 claims (lithium levels and BUN) within three months of their first pharmacy claim for the defined medication, while 84.78% did not have the laboratory test claim within 360 days from their first pharmacy claim. No lags were identified on the margin of the required interval. Indicators 2, 8, 11, 23, 24, 26, 29, 33 had a required monitoring interval of six months. Figure 6.8 shows 24.78%of members at risk for these indicators had the specified laboratory test claims within six months of their first pharmacy claim for the defined medication. Some process positives were captured from the margin, 6.53% had the laboratory test claim between 180 and 270 days. 050100150200250300350400 days <=6090
PAGE 94

82 0510152025303540 days<=90135
PAGE 95

83 0100200300400500600700800 days <=365365730lag timefrequency 29.54 5.63% 12.58 52.25 Figure 6.9. First Lag for Indicator with Twelve Month Monitoring Requirement The majority of members who met the requirements to be considered for the MU-PIs with six month monitoring requirements (57.97%) did not have the specified laboratory test claim within 720 days from their first pharmacy claim. Indicator 19 had a required monitoring interval of twelve months. Figure 6.9 shows the majority of the members who appeared to be using thyroid therapies (52.25%) had their thyroid tests (laboratory test claims) within twelve months of their first pharmacy claim for the defined medication. About thirteen percent had the specified laboratory test claim between 365 and 545 days, about six percent had a claim between 545 and 730 days from the pharmacy claim, and about thirty percent did not have the laboratory test claim within 730 days of the first pharmacy claim for the required medication. In summary, the majority of process positives did not result from the margins of the required lag interval. The majority of process positives resulted from the extreme of

PAGE 96

84 the distribution. The indicators that required one month, six month and twelve month monitoring intervals were picking up some process positives from the margins, 3.78% 6.53%, 12.58%, respectively. Variables associated with PDRM positives Personal variables associated with PDRM positives were identified with bivariate logistic regression. Table 6.8 displays the OR and CI for each dependent variable. Age, and number of office visits, prescribers, pharmacies, drug classes and conditions were significantly associated with PDRM positives. Gender was not significant (odds ratio OR: 1.00; confidence interval CI: 0.63 1.59). Number of pharmacies and number of prescribers were the strongest bivariate predictors (OR: 1.48; 95% CI: 1.43-1.63) and (OR: 1.48; 95% CI: 1.40 -1.57), respectively. Number of drug class and number of conditions had the next strongest association (OR: 1.19; 95% CI: 1.16-1.21) and (OR: 1.20; 95% CI: 1.181.23), respectively. Number of office visits and age had significant but weak bivariate associations (OR: 1.07; 95% CI: 1.06-1.09) and (OR: 1.06; 95% CI: 1.05-1.08), respectively. Multivariate multicollinearity was assessed using the tolerance score. As shown in table 6.9, no tolerance values were below 0.20. The lowest tolerance score was 0.256, meaning 0.256 of the variance in number of office visits was unexplained by the other independent variables. A multivariate logistic regression was conducted to explore the relationship between PDRM positives and demographic/system related variables. Logistic regression with PDRM positives as the dependent variable and the seven demographic/system related variables as the independent variables used stepwise variable selection to produce the model. A four-variable risk model was produced. The model indicated that males

PAGE 97

85 Table 6.8. Bivariate Tests of Association: PDRM Positives as Dependent Variable Independent VariableOdds RatioAge1.061.051.08Gender F vs. M1.000.631.59number of Office visits1.071.061.09number of Prescribers1.481.401.57number of Pharmacies1.481.341.63number of Drug class1.191.161.21number of Conditions 1.201.181.23 95% Wald CI were about 50% more likely to have a PDRM positive after adjusting for the other variables in the equation (OR: 0.45; 95% CI: 0.28-0.74). Number of office visits had a negative influence on the odds of having a PDRM positive (OR: 0.93; CI: 0.9-0.96). That is, each additional office visit reduced the probability of having a PDRM positive by approximately seven percent, after adjusting for the other variables in the model. Number of drug classes and number of conditions increased the odds of having a PDRM positive. When drug class was held constant at zero, each additional medical condition increased the odds of being PDRM positive by about twenty-nine percent. When medical conditions were held constant at zero, each additional drug class increased the odds of having a PDRM positive by about twenty-two percent. Number of drug classes and number of medical conditions had a significant interaction with a negative beta-coefficient ( coefficient: -0.004; p-value: 0.0014). Increases in both medical conditions and drug classes resulted in decreased odds of having a PDRM positive. The overall significance of the equation by the likelihood ratio had a chi-square of 306.007 (p< 0.0001) with four degrees of freedom (df). The proportion of variance explained by the model (max-rescaled R 2 : 0.279). Overall the model fit well to the data (Hosmer-Lemeshow: 4.01; p=0.779; df=7).

PAGE 98

86 Table 6.9. Multicollinearity Assessment of Independent Variables VariableDFParameter EstimateStandard Errort valuePr > |t|Tolerance V ariance inflationIntercept1-0.040720.00394-10.35<.0001.0Age1-0.00005510.00007498-0.730.46240.849421.17728number of Conditions10.007240.0002071234.96<.00010.268573.72342number of Drug Classes10.00660.0001591141.46<.00010.319933.12572number of Office Visit1-0.004210.00014943-28.19<.00010.255613.9122number of Pharmacy1-0.003450.00055755-6.18<.00010.789541.26657number of Prescribers1-0.010890.00045741-23.8<.00010.416532.40079 Table 6.10. Maximum likelihood Estimates: PDRM Positives as Dependent Variable Intercept1-9.2290.404521.676<.0001Gender, F1-0.3980.12610.0140.0016number of Conditions10.2540.03166.933<.0001number of Drug Classes10.2000.02469.664<.0001number of Office Visits1-0.0740.01719.389<.0001Cond*Drug class1-0.0040.00110.1560.0014Pr > ChiSqParameterDFEstimateStandard ErrorWald chi-Sq Step Two. Assignment of MU-PIs to Nodes of the MUS The ultimate goal of this dissertation was to identify system factors (i.e., latent conditions of the health care system) that appear to be common or unique to nodes of the MUS. To do this, the node of the MUS where the MU-PIs process problem originated was determined. Selected MU-PIs were sent to a Delphi panel of clinicians and researchers for their judgment on where in MUS the process component of the indicator likely originated. Delphi Recruitment A total of fifty-seven recruitment letters were mailed out. Twenty-nine contacts agreed to participate, seven declined, and twenty-one did not respond. Twenty-nine contacts were e-mailed or mailed the survey (based on individual request) and eighteen completed and returned the first round. The Delphi panelists consisted of five medical doctors (MDs), one nurse, five doctors of pharmacy (PharmDs), six bachelors of

PAGE 99

87 pharmacy (BA pharm), and one masters in health service research (MHSA). Fifteen of the seventeen members with clinical degrees were licensed in the US and eight were currently practicing. All of the panelists with clinical degrees had additional graduate level degrees. (Table 6.11). Table 6.11. Demographics of Delphi Panelists for Node Identification Study Other DegreeGenderLicensed in the USPracticingMD52 MPH 1MS 1PhDYes = 17Nurse11MPAPharmD51 PhDBA Pharm61 MBA PhD 4 PhDNo = 11 MHSATotal Number of Delphi Panel = 18Clinical Degress1586 Female 12 Male Assignment of MU-PIs to Nodes of the MUS The Node Identification study went through two rounds of the Delphi process. An indicator was assigned to a node and not returned for the second round when the average score for a node was significantly greater than the average scores for all other nodes. Eleven indicators reached significance within the first round. Three of the indicators were returned for the second round and two achieved significance. Based on the Delphi panelist responses, problems with the specificity of the indicators were the reason they did not reach significance in the first round. Changes in the indicators were made and they were sent back to the Delphi panel. The panelists were informed of the changes made to the indicator scenarios. They were also provided with their score, the groups average score and a summary of the panelists responses for the three indictors. The changes made to the three indicators included,

PAGE 100

88 Changing sympathomimetic decongestants to prescription sympathomimetic decongestants for Delphi indicator 9. Removing INR not done before therapy starts from Delphi indicator 12 (the monitoring requirement was one-month) Changing antibiotic use to use of macrolide or sulfonamide antibiotics for Delphi indicator 14 The indicators were judged to be primarily associated with either the prescribing or monitoring node of the MUS. None of the indicators were found to be primarily associated with the dispensing or administration nodes of the MUS. Six indicators were assigned to the prescribing node, seven to the monitoring node and one did not reach significance within the two rounds. Table 6.12 provides a summary of the indicator assignments. b The MU-PI column in Table 6.12 is the MU-PI number and it provides a cross reference to the Delphi indicator number. Table 6.13 lists the statistics for each indicator analysis. Mean scores for each node, Kruskal Wallis statistics and p-values are presented for each indicator. Significance of the Dwass, Steel, Critchlow-Flinger pairwise comparisons are represented by an underscore under the abbreviated node. Nodes connected with an underscore were not statistically different. Delphi indicators 1-6, 8, 10, 11,13, and 15 reached significance in the first round, all p-values <0.001 for KW statistic and pairwise comparisons. Indicators 9, and 12 reached significance in the second round, p-values <0.001 for KW and pairwise comparisons. Indicator 14 did not reach significance and therefore it was not assigned to a node. Box plots for the mean, median and mode for each indicator are provided in Appendix L. b Delphi indicator number seven was not included because it was a repeat of Delphi indicator number four.

PAGE 101

89 Table 6.12. Indicators Listed by Associated Node of the MUS Indicator No. given to Delphi PanelNode Assigned by Delphi Panel1(16)CHFHx/HBP-NSAIDPrescribing2(37)CHFHx/CHF-NoACEIPrescribing3(28)CHFHx/CHF_HB-DigPrescribing4(5)GIHx/GI-NSAIDPrescribing5(17)HypoKKwd-NoK-Elctro(2)Monitoring6(8)SeizAntconv-DrugLvl(3)Monitoring8(9)LiToxLi-DrugLvl(3)Monitoring9(22)TachyHx/HTN-DeconPrescribing10(23)HyprKACEI-ElctroCBC(6)Monitoring11(33)ARFAllop-BUN/Scr(6)Monitoring12(30)HemrWar-INR(1)Monitoring13(39)AsthmaHx-Bron-NoSteroidPrescribing14(42)HemrWar-Antibiot-NoLab(5dys)No agreement reached15(29)ARFACEI-BUN(6)MonitoringMU-PI Step Three: Cause-and-Effect Analysis Indicators that had been evaluated in the node identification (Delphi) study were eligible to be selected for cause-and-effect analysis. The MU-PI evaluation team consisted of a Medical Director, PharmD, nurse practitioner, consumer advocate, laboratory manager and a director of a health maintenance organization. Indicator 30 (warfarin indicator) and indicator 39 (asthma indicator) were formally evaluated. Proposed causes were brainstormed, organized into affinity tables, and placed into tree diagrams (see methods).

PAGE 102

90 Table 6.13. Kruskal Wallis and Pairwise Comparisons for Indicator Assignment to Nodes of the MUS Indicator No.NodeMean ScoreKW statisticp valueMean ScoreKW statisticp valueP61.39D25.89P > M > D > APrescribingA16.22M42.50P60.78D24.78P > M > D > APrescribingA23.11M37.33P54.41D22.85P > M > D > APrescribingA18.74M42.00P63.36D28.50P > M > D > APrescribingA21.89M32.25P41.36D25.67M > P > D > AMonitoringA18.44M60.52P36.30D21.33M > P > A > DMonitoringA25.11M63.25P36.81D21.19M > P > A > DMonitoringA24.89M63.11P43.2551.37D37.19P > A > D > M29.10P > D > M > APrescribingA40.5019.37M25.0622.17P32.94D22.03M > P > D > AMonitoringA19.06M55.97P35.59D20.75M > P > D > AMonitoringA18.25M55.41P41.0329.80D24.24M > P > D > A20.17M > P > D > AMonitoringA21.7119.03M51.0353.00P57.59D26.27P > A > M > DPrescribingA27.12M27.03P45.3243.70D31.50P > M > D > A26.93P > M > D > AA17.0012.10M44.1839.27P34.94D19.38M > P > A > DMonitoringA20.59M55.09*Delphi indicator number (MU-PI number)**Dwass-Steel-Chritchlow-Fligner (connecting lines = not significant differences between nodes)No Decision made15 (29)42.121<0.000114 (42)24.548<0.000130.742<0.00012nd round13 (39)32.276<0.000112 (30)29.174<0.000140.57<0.00012nd round11(33)44.497<0.0001<0.00012nd round10 (23)44.57<0.00019 (22)*8.4950.03732.228 (9)48.427<0.00016 (8)49.587<0.00015 (17)45.071<0.00014 (5)46.139<0.00013 (28)41.724<0.00012 (37)45.124<0.00011 (16)*51.583<0.0001First Round Second Round Kruskal Wallis testPairwise Comparison**1st round decision Kruskal Wallis testPairwise Comparison1st round decision

PAGE 103

91 Evaluation of Monitoring Indicator The following question initiated the brainstorming for the warfarin monitoring indicator, What system factors may explain the number of people using warfarin who appear to be missing monthly INRs? Nineteen causes were proposed. Table 6.14 lists the eleven proposed causes that remained after clarifying, grouping and organizing them into an affinity table. Four were deemed artifacts of the indicator. They included: difficulty titrating warfarin doses, no claim submitted for INR because (a) cash payment for laboratory test, (b) INR was not specifically coded in bill and lack of claim even though INR had been conducted. The evaluation team voted on the causes they believed contributed most to the lack of monthly INRs for patients taking warfarin. The four causes with the most votes were selected for cause-and-effect diagramming. Lack of patient understanding about the importance of INR monitoring, patient non-compliance with orders to have laboratory tests done, lack of physician follow-up to determine if labs are being done, and lack of a medication management system were judged to be the most important contributors to the lack of monthly INRs in patients taking warfarin. The evaluation team was asked to fit the four causes deemed most influential to the warfarin monitoring problem into cause-and-effect sequences, i.e., tree branches. They identified fifteen sequences (Table 6.19). After sequencing, team members were given a written questionnaire to determine if the cause sequences would be contributing factors to another type of monitoring indicator (indicator 29 c ). When five out of the six team members (83%) agreed that each branch listed problems that were common the other c Indicator number 29 ( ARF ACE-I no BUN (6)) was assigned to the monitoring node of the MUS in the Delphi study.

PAGE 104

92 monitoring indicator that cause sequence was identified as a system problem that was common to the monitoring node of the MUS. Table 6.14. Monitoring Indicator: Affinity Table and Rating of Importance RatingPatient level7patient education (pt does't understand the need to monitor drug therapy) 0difficult titrate dose 5patient is not compliant with physicians order to have lab doneProfessional level4delay b/t titration and prescribing 1pt comprehension assessment is lacking 3responsibility (lack of claiming responsibility), multiple physician 7lack of followup by physician to make sure labs are being done and dose is appropriate Organization level5medication management system lacking to detect those not being monitored3prescription supply exceeds the INR check interval 1billing process does not capture INR specifically Environment Artifactdifficult to titrate dose cash payers so no claim for INR (bundled claims)billing process does not capture INR specifically lack of billing (claim) eventhough lab was conducted To evaluate whether the cause sequences elicited for the warfarin monitoring indicator would exhibit problems with the prescribing node as well as the monitoring node the team was asked whether each sequence would explain problems found by the asthma prescribing indicator. Twelve of the fifteen cause sequences from the warfarin monitoring indicator were judged to be problems that were common the asthma prescribing indicator. Table 6.15 presents the agreement among the team members for each branch. Evaluation of Prescribing Indicator The same process was applied for the evaluation of the asthma prescribing indicator. The following question initiated the brainstorming for the asthma

PAGE 105

93 prescribing indicator, What system factors may explain the number of asthmatics who did not appear to be using maintenance therapy? Eighteen causes were proposed during brainstorming. Sixteen remained after clarifying, grouping and organizing them into an affinity table. Patient education, patient does not understand function of steroid, multiple physicians, and lack of a medication management system were judged to be the most important contributors to the under-use of maintenance therapy. Table 6.16 lists the affinity table and ratings for each proposed cause. Table 6.15. Cause Sequence Agreement Results BranchM-MM-PBranchP-PP-M183%100%167%40%283%100%267%40%383%83%383%80%483%100%4100%80%5100%83%567%80%683%100%683%80%783%50%7100%80%8100%83%883%80%9100%67%9100%100%10100%67%10100%100%11100%83%11100%100%12100%100%12100%80%13100%100%13100%100%1483%100%1467%60%15100%83%M-M = monitoring to monitoring indicato r M-P = monitoring to prescribing indicato r P-P = prescribing to prescribing indicato r P-M = prescribing to monitoring indicato r one member did not vote so acceptance was 80%Monitoring IndicatorPrescribing Indicator Fourteen additional cause-and-effect sequences (i.e., tree branches) were identified for the four proposed causes. The cause-and-effect sequences for patient education identified during the monitoring indicator evaluation were judged to be common to the prescribing indicator and they were not diagrammed again. Ten of the fourteen branches were judged to be common to another prescribing (indicator 37). Eleven of the

PAGE 106

94 fourteen cause sequences from the asthma prescribing indicator were judged to be problems that were common the warfarin monitoring indicator. Table 6.15 presents the agreement among the team members for each branch. Table 6.16. Prescribing Indicator: Affinity Table and Rating of Importance RatingPatient level 4pt may not appreciate the function of steroid 1non compliance -intentional 1lack of tools (self assessment tool) 0taste bad 0adverse effect 2misperception of adverse effects of steroid 2caregivers not supervising med use Professional level 3medication copay misdiagnosis 6pt education 0Lack of objective measures 0lack of appropriate beta agonist use 3guideline adherence 4multiple physicians Organizational level 4medication management Environment 3compliance (related to embarrassment, barriers e.g., school, peer issues) Cause Sequences Common to both the Monitoring and Prescribing Nodes Table 6.17 displays the cause sequences from both the monitoring and prescribing indicator analysis. The first column labeled Branch identifies the node of the indicator formally evaluated and the branch number. For example: the M_1 sequence was discovered when the warfarin monitoring indicator was evaluated. A yes in the M&P column means the cause sequence was judged to be a common problem to both the monitoring and prescribing nodes. An M in the Node column signifies the cause sequence was judged to be unique to the monitoring node of the MUS.

PAGE 107

95 Twenty-three cause sequences were judge d to be common to both the monitoring and prescribing node. Two were unique to th e monitoring node and four did not reach consensus on either. The three main themes that appeared to be common to both the monitoring and prescribing node were patien t education, patient co mpliance (laboratory tests), and medication management systems.

PAGE 108

Table 6.17. Tree-diagram for Warfarin Monitoring and Asthma Prescribing Indicators Patient levelProfessional levelOrganization level EnvironmentBranchM & PNodeM_1patient educationpatients' knowledge not assesseddevelop tools (AHRQ)yesM_2patient educationlack of adequate teaching toolsMCO unification / standard assessmentyesM_3patient educationlack of adequate teaching toolspharm companies yesM_4patient educationdifficult to get education in Hospitaldischarge planningyesM_5patient educationdifficult to get education in Hospitallack of consistencyyesM_6patient educationtime issuesmarket forcesyesM_7patient educationcomorbidity takes more time yesM_8lack of information technologyyesM_9lack of systematic follow-up mechanismMM_10disease focused not drugMM_11not compliant with labMCO don't provide incentives to patientyesM_12not compliant with labtracking mechanism to see who gets labscentralized records/ reminder systemyesM_13not compliant with labMCO contact patientsyesM_14not compliant with labdisease managementyesM_15not compliant with labaccess/ convenience access/ convenienceyesP_1Not sensitive to medication copaylack of unfied formulary systemP_2Not sensitive to medication copaylack of software supportP_3some will adopt new techyesP_4pharmacists may helpyesP_5cultural disparityP_6physicians are afraid to adopt techmedication management to track patientsyesP_7pharm can take a role in med managementmedication management to track patientsyesP_8pharm may respond to the frequent usemedication management to track patientsyesP_9pharmacist involvementmedication management to track patientsyesP_10pharmacist involvementno paymentyesP_11following guidelineno rewardyesP_12data owner can disseminate infoyesP_13multiple physiciansunknown gatekeeperyesP_14multiple physiciansdata doesn't follow patient care 96

PAGE 109

CHAPTER 7 DISCUSSION The overall objective of this study was to explore the relationship between system design, patterns of drug therapy and drug related morbidity. This study had three major parts. The first part consisted of a database analysis. In it, we applied a set of medication use performance indicators (MU-PIs) to an administrative database in order to assess the overall quality of medications use in the population represented by the database. In the second part, a panel of experts described the association between selected indicators and the nodes of the medication use system (MUS). In the third part, an evaluation team carried out a cause-and-effect analysis on select indicator positives that had originated in the prescribing and monitoring nodes of the MUS. The first part replicated work by MacKinnon 42 and Faris. 13 The second and third parts, however, go beyond their work. The major contribution of this study was exploratory in two respects. It was the first attempt to associate medication use indicators with specific parts of the medication use system, and it was the first study to evaluate how the four levels of the health care system (patient, professional microsystems, organization, and environment) interact and contribute to indicator and node specific process problems. Database Analysis Prevalence Findings Perhaps the most notable finding from the database analysis was that the prevalence of PDRM positives was much lower than would be expected from Mackinnon 97

PAGE 110

98 and Faris studies. The period prevalence of PDRM positives (in members with a medical claim) for this study was 1.98 per 1,000 patient years, while Faris 13 found a prevalence rate of 62.5 per 1,000 patients and Mackinnon 59 found a prevalence rate of 28.8 per 1,000 patients. The lower prevalence found in the current study may be explained by differences in population demographics and differences in search algorithms for the MU-PIs. The previous studies used the MU-PIs to screen the database of an elderly Medicare population. Medicare patients tend to take more medications and the number of medications are associated with and increase risk of drug related injury. 59-61 This study required all events (i.e., PDRM positives) to have an ED visit or hospitalization for the outcome, and only the primary diagnosis was used for outcome identification. The previous studies did not have the same requirements. Their search algorithms did not require ED visits or hospitalizations for more than half of the indicators, and their analysis was not limited to primary diagnosis codes to signify the reason for seeking medical care (cause of ED visit or admission). Another difference in search algorithms between the studies was the attention given to the proximity of the process to the outcome. The search algorithms in this study required claims for the offending medication to have occurred shortly before the outcome--in most cases the lag was set at 100 days. Proximity requirements were also established for indicators that required monitoring. t The prevalence of PDRM positives that required hospital admission was much lower (0.75%; CI: 0.74%-0.76%) than the findings from our systematic review of t Refer to methods for search algorithms.

PAGE 111

99 preventable drug related admissions (median: 4.3%; IQR: 3.1%-9.5%). 1 The articles in the systematic review used a chart review process to identify PDRAs, while the current study employed a computer screening method. It has been shown that different methods of PDRM ascertainment yield different results. Jha et al. (1998), in an inpatient ADE study, compared yields of chart review, automated computer screening, and stimulated voluntary report. 62 This study found that each method identified different types of ADEs and the computer screening method only identified about forty-five percent of the total ADEs. Computer screening methods are believed to increase efficiency and the reliability of detecting PDRM, however, the yield is limited to the explicit PDRM scenarios. Implicit review methods allow for more flexibility and comprehensive assessments of PDRM. With the implicit chart review method, the limiting step is the knowledge and thoroughness of the reviewers and not the predefined search algorithms. Demographic Explanations for Prevalence Findings Information from the database was used to better understand indicator positives. Descriptive analyses of monitoring intervals were conducted to determine if cases on the margin of the monitoring interval were being identified as process positives, and multivariate logistic regression was conducted to explore the relationship between PDRM positives and demographic/system related variables. Distribution of monitoring intervals The MU-PIs that require drug-therapy monitoring identify cases as positive or negative for the inappropriate monitoring interval. When a case screens positive it means an individual did not have a laboratory test claim within the specified time frame. These strict cut-offs could allow monitoring intervals on the margins of appropriate monitoring to be categorized as process failures even if they had been only a few days or

PAGE 112

100 weeks over the required time. To develop a more comprehensive picture of how cases deviated from the specified monitoring intervals, an analysis of the first monitoring interval was conducted. The distributions of monitoring intervals from the first pharmacy claim to the first laboratory test claim for the indicators that required monitoring revealed that few process positives were captured from the margins of the required monitoring intervals. The indicators that required six-month and twelve-month monitoring intervals did, however, identify more cases on the margin than the others (6.5% and 12.6%, respectively). Because the categorized intervals were a function of the monitoring requirement u the indicators with longer monitoring intervals had a larger interval for the descriptive analysis and a more liberal interval on the margin of the required monitoring interval. For example: the categories for indicators with a one-year monitoring requirement were set at 180 day intervals, while the categories for indicators with a one-month monitoring requirement were set at fifteen days. Further investigation is needed to drill down the margins of the six and twelve month monitoring indicator positives. Nevertheless, the majority of process positives resulted from lag times that were on the extreme of the distribution, in many cases, no claims for the specified labs were observed. Multiple logistic regression analysis In the exploratory analysis, age, number of office visits, pharmacies, prescribers, drug class, and number of medical conditions were all significant bivariate predictors of PDRM positives. In multivariate analysis, however, only gender, number of office visits, drug classes and medical conditions were independent predictors. Males, number of drug u Descriptive categories were defined by dividing the defined monitoring interval by two. See methods.

PAGE 113

101 classes and number of medical conditions had a positive relationship with PDRM positives, while number of office visits and the interaction between drug classes and medical conditions were negatively associated with PDRM positives. Interestingly, gender was not significantly associated with PDRM positives in bivariate analysis; however, in multivariate analysis gender was an independent risk factor. The only system related variable that stayed in the multivariate model was number of office visits. After adjusting for the other variables in the model, as the number of office visits increased the risk of PDRM decreased in this population. Following up with patients who appear to be missing office visits or who have not had an office visit for an extended period of time may reduce the probability of drug related injury. Number of different pharmacies and number of different prescribers were not independent predictors of PDRM positives. Another interesting finding was that an increase in the number of medical conditions (with number of drug classes held constant at zero) increased the odds for PDRM by about twenty-nine percent. Likewise, an increase in the number of drug classes (with medical conditions held constant zero) increased the odds for PDRM by about twenty-two percent. The interaction effect of medical condition and number of drug classes, however, had a significant but weak negative association, meaning as number of medical conditions and number of drug classes increase together the odds of PDRM decreased. This seems counterintuitive at first; however, a reasonable explanation exists. Increases in drug therapy without an increase in medical conditions may increase the risk of PDRM through over treatment, while an increase in medical conditions without an increase in drug therapy may increase the risk of PDRM due to

PAGE 114

102 under-treatment. Increases in medical conditions with complementary increases in drug therapy may indicate appropriate treatment and therefore reduce the risk for PDRM. Most studies that have looked at independent predictors of PDRM have done so in the inpatient setting. These studies have also found number of medications and number of diseases as independent risk factors, but have failed to find age or gender as independent predicators. 61,63 Three studies have used multivariate statistics to evaluate predictors of PDRM in the ambulatory care setting. Mackinnon (2003), found four or more prescribers, four or more medical conditions, female gender, antihypertensive drug use and six or more prescription medications were risk factors for PDRM. Faris (2001) confirmed these findings. In addition, a recent study by Gandhi et al (2003), found that number of medications were the only independent risk factor of adverse drug events in ambulatory care. 60 Considering the finding from this study in the context of previous research, it would seem prudent to closely monitor patients who have multiple medical conditions or who require multiple medications for signs and symptoms that would indicate problems with drug therapy. Node Identification of MU-PIs The ultimate goal of this dissertation was to identify system factors that appeared to be common or unique to nodes of the MUS. To do this, the location in the MUS where the indicators process failure originated had to be assessed. Fourteen indicators were sent to Delphi members to identify the node of the MUS where the process failure from specific indicators originated. The Delphi study began before the database analysis was completed. The fourteen indicators were selected based

PAGE 115

103 on preliminary prevalence findings. The final results of the MU-PIs differed from the preliminary findings and only nine of the fourteen had PDRM positives. Thirteen of the fourteen indicators were assigned to either the prescribing or the monitoring node of the MUS. The assignment followed a clear pattern. All indicators that required drug therapy monitoring except, Delphi indicator 14 v (which did not reach consensus) were assigned to the monitoring node. Indicators where the process problem involved disease-drug interactions or lack of needed therapy were assigned to the prescribing node. The process component of Delphi indicator 14 included the use of inappropriate medications in patients taking warfarin and no lab for INRs done within five days, the node assignment was split between the prescribing and monitoring node. Cause-and-Effect Analysis The cause-and-effect evaluation team accomplished multiple tasks. First, it identified likely causes for the process failures and PDRMs described by select MU-PIs. Second, it sketched out some cause sequences across the four system levels (patient, microsystem, organization, and environment). Out of the apparent complexity of causes and system levels, however, the team found that many cause sequences explained more than one indicator and problems from more than one node of the MUS. The final task was a discussion of possible corrective actions. During that discussion, four themes emerged that would explain some general latent failures and some broad strategies about how to improve medication use. The evaluation team was able to use the information from the MU-PIs to reflect on how the different levels of the health care system interact to influence patient care v MU-PI number 42

PAGE 116

104 processes. The team identified twenty-nine cause sequences. Furthermore, they found that twenty-three of the twenty-nine were common to both prescribing problems and monitoring problems. That is, the team decided that the same cause sequences could contribute to prescribing process failures and to monitoring process failures. The team did not reach consensus on four sequences. (See Table 6.19) The cause sequences involving patient is not compliant with getting labs (M11-M16 from Table 6.19) were originally proposed as a partial explanation for a monitoring process failure from the warfarin indicator. Somewhat surprisingly, the evaluation team found that this sequence also could be relevant to prescribing process failures. The evaluation team may have focused on the compliance component of the proposed cause and the higher level (professional, organizational and environmental) causes in the sequence when judging their relationship to the prescribing node. Cause Themes In the latter part of the analysis, four themes emerged from the cause sequences that were common to the prescribing and monitoring nodes: patient education (teaching and assessment tools), information demands and medication management, pharmacist involvement and guideline adherence. Many cause sequences included, lack of patient education. These sequences involved lack of adequate teaching and assessment tool, insufficient discharge education, and time constraints. At the professional level, lack of adequate teaching tools and lack of tools to assess patients understanding of their drug therapy were identified as contributing to patient knowledge deficits. The evaluation team believed that managed care has not promoted and reinforced standard methods for education and assessment (organizational level). They also believed that pharmaceutical manufacturers and

PAGE 117

105 agencies such as the Agency for Health Care Research and Quality (environmental level) are not investing in the development and dissemination of valid tools to educate and assess patient knowledge. According to the evaluation team, deficits in patients understanding of their drug therapy may occur when patients are discharged from the hospital with new medications. Lack of discharge planning and consistency in medication consults were the organizational level explanations for this. Time constraints due to market forces (organizational level) were also considered reasons for patient education deficits. Lack of a technology infrastructure with the ability to record patient information through the continuum of care was considered a latent condition that allowed problems to occur and persist in the prescribing and monitoring nodes. Lack of medication management systems that could be used to determine if patients are refilling medications, having their laboratory tests ordered, completed and evaluated was also considered a common problem to the prescribing and monitoring nodes. An effective medication management system would include access to centralized records that could support a tracking mechanism that providers could use to evaluate their patients drug use. The team also concluded that the managed care organizations (MCOs) affiliated with the coalition were not actively evaluating patient behavior in response to medication refills and compliance with laboratory test orders. The evaluation team believed the MCOs should identify members who are not adhering to their medication or laboratory test orders and contact them to discuss the consequences of their decisions. Lack of pharmacist involvement was also considered a system related cause that allowed problems to persist from the prescribing and monitoring nodes. The evaluation

PAGE 118

106 team believed that pharmacists could take a more significant role in medication management by responding to overuse of medications and by supporting drug monitoring. They believed pharmacists are not providing consultative services because they are not paid for consultative services. Failure to follow guidelines also contributed to problems that originate in the prescribing and monitoring nodes of the MUS. The MU-PIs were evidence based and the process component of the indicator represents deviations from guidelines. The evaluation team believed the deviations from the guidelines were because providers have no financial incentive to adhere to guidelines. One of the benefits of the cause-and-effect analysis was that it promoted the recognition that indicator-specific findings may be inter-related with common system failures, i.e., latent conditions. The team recognized the close relationships among the four causal themes described above and possible corrective interventions. The system related themes that emerged from the cause-and-effect analysis are extremely complex. Further investigation into them is warranted before interventions to promote organizational change are developed and implemented. Conclusion The MU-PIs were used to screen the study database. The period prevalence for process positives was 209.84 (206.37-213.34) per 1,000 member years. The period prevalence for PDRM positives was much lower, 1.98 (1.6-2.4) per 1,000 member years. The PDRM positive rate from this study was much lower than the previous studies using the MU-PIs. The lower rate may be due to the much younger study population and differences in search algorithms.

PAGE 119

107 Descriptive analyses of monitoring intervals were conducted to determine if cases on the margin of the monitoring interval were being identified as process positives. A small portion of process positives resulted from the margins of indicators with six-month and twelve-month monitoring requirements. The majority of process positives, however, were a result of intervals that were at least three times greater than the required interval. Multivariate logistic regression was conducted to explore the relationship between PDRM positives and demographic/system related variables. Gender, number of office visits, number of drug classes and number of medical conditions were independent risk factors for PDRM. Males were about 50% more likely to have a PDRM positive after adjusting for the other variables. Each additional office visit decreased the risk of PDRM by about seven percent. When drug class is held constant, each additional medical condition increased the chance of being PDRM positive by about twenty-nine percent. When medical conditions are held constant, each additional drug class increased the chance of having a PDRM positive by about twenty-two percent. Number of drug classes and number of medical conditions had a significant interaction with a negative beta-coefficient. Increases in both medical conditions and drug classes resulted in a decreased risk for a PDRM positive. Indicators were assigned to different nodes of the MUS. To do this, a Delphi panel judged the degree of association between select indicators and specific nodes of the MUS where the process failure may have originated. Six of the fourteen indicators were assigned to the prescribing node and seven were assigned to the monitoring node of the MUS. One indicator did not reach significance within the two round Delphi study.

PAGE 120

108 The MU-PIs proved to be a useful tool to initiate system thinking or organizational introspection. The team identified twenty-nine cause sequences. Furthermore, they found that twenty-three of the twenty-nine were common to both prescribing problems and monitoring problems. Four themes emerged from the cause-and-effect analysis and they included lack of necessary tools for adequate patient information and assessment, an information system that can track patients and relay information to the providers, pharmacist involvement in the MUS and guideline adherence. The evaluation team determined that interventions should address these themes. Further investigation is needed to design interventions that would have the greatest impact. Limitations The MU-PIs are content validated, however; they have not been criterion validated. Therefore, MU-PI positives represent possible PDRM, but specificity and sensitivity are not known. This should be considered when interpreting the indicator findings. Published literature shows that claims data are reasonably valid for identifying patients with specific disease, especially when pharmacy claims that would be consistent with the disease are included, 35 nevertheless; no studies were identified that evaluated the validity of using CPT codes as proxies for lab monitoring. In the database analysis, the majority of process positives for indicators that required drug therapy monitoring resulted from lag times that were on the extreme of the distribution, in many cases, no claims for the specified labs were observed. Because of this, further analysis is needed to determine if labs were conducted but no claims were submitted. During the cause-and-effect analysis, one of the team members argued that lab claims may be bundled and therefore specific lab claims would not be observed.

PAGE 121

109 The search algorithms for the MU-PIs were limited to severe outcome, i.e., ED visit or hospitalization. Studies have shown that a large portion of PDRM is considered significant but not severe. It is unclear if the latent conditions identified in this study would be involved in the production of less severe PDRM. The cause-and-effect analysis was essentially a case study. It is possible that an evaluation team with a different skill set or expertise may have identified different system failures. Nevertheless, it would not invalidate the findings of this study. Additional studies may be useful to develop a more comprehensive picture of all possible latent conditions. Significance and Theoretical Contribution This study included developing a modified version of Faris search algorithm, identifying indicators that represent different nodes of the MUS, identifying latent conditions of the health care system that are associated with both prescribing and monitoring problems, and demonstrating that the MU-PIs can be used to initiate organizational introspection. The search algorithms were refined from earlier versions by paying more attention to the time relationships between the process and outcome. The search algorithms were intended to identify cases that were currently taking the offending medication within proximity to the adverse outcome. The search algorithm also had more strict limitations on the indicators that required drug therapy monitoring. This study demonstrated that medication and disease specific indicators can be grouped according to the nodes of the system where the process failure originated. Node specific indicators can be used to probe into the MUS to evaluate node-specific quality, e.g, the quality of monitoring. This study also established that problems that originate in

PAGE 122

110 the monitoring and prescribing phase often have common system characteristics that allow the process failures to eventually end in adverse outcomes. Lastly, this project demonstrated that the MU-PI can be used to initiate dialog and introspection about how the design of the organization contributes to the prevalence of process failures and PDRM. The MU-PIs can function as an important tool for the improvement of medication use. Contribution to Health Care The connection between medication use quality measurement and improvements in the MUS can take two routes: 1) improvements through selection and 2) improvement through changes in care. 64 The use of performance measurement by organizations such as the National Committee for Quality Assurance (NCQA) are designed to provide a report card for employers so they can select the providers and organizations that appear to have the highest level of quality. The mechanism of improvement through selection works when the employers can make choices among health plans or providers. Selection as a mechanism for improvement does not change the basic distribution of performance. Instead it improves outcomes by shifting business to providers and plans that are performing better. 64 Improvements through changes in care have the potential to shift the underlying distribution of care itself. Before changes can be made to the MUS the organization needs to understand the process involved in the production of poor quality. Once the performance is measured and the influence of system designs on quality is understood organizations and individuals can improve performance through changing the processes of care. This study has contributed to health care by providing a method to initiate the

PAGE 123

111 change process by identifying latent conditions of the system and areas for process improvement interventions. Future Areas for Study Interventions The next step in the change process will focus on the development of interventions to address the system related problems identified in the cause-and-effect analysis. A more in-depth analysis of the organization and its ability to pursue various interventions is needed. Information from the literature can help us frame the latent conditions to better understand them. The evaluation team believed that issues related to patient education were important reasons for members neglecting to use their medications appropriately. They believed methods should be developed and implemented to reduce the educational deficits. Research on decision aids for patients facing health treatment decisions may provide valuable insight and tools for educating patients about the risks and benefits related to their drug therapy. It may, as well, assess their competency about medication specific issues. Decision aids are intended to help patients understand the probable outcomes of different behavioral decisions, e.g., not complying with ordered drug therapy monitoring, make an informed decision and communicate their preferences to their provider. 65 Decision aids have been found to improve knowledge, reduce decisional conflicts, and stimulate patients to be more active in decision making without increasing their anxiety. 66 Since lack of patient education and assessment was an important latent condition, future research should be committed to evaluating how education and assessment tools, such as decision aids can reduce the PDRM problem.

PAGE 124

112 Guideline adherence was believed to be a system problem that was common to the prescribing and monitoring nodes as well. The evaluation team believed the lack of a reward system tied to guideline adherence was a reason for the deviations from guidelines. Cubana et al (1999), conducted a systematic review of barriers to physician adherence to practice guidelines and found multiple barriers exist for physicians to follow guidelines. 67 Lack of awareness and lack of familiarity with guidelines affect physician knowledge of guidelines. Lack of agreement with guidelines, self-efficacy, outcome expectancy, and inertia from previous practice can pose attitude barriers. Patient and environmental barriers were common external barriers. Any attempt to improve guideline adherence should consider the common barriers. Interventions need to address the specific cognitive and behavioral barriers to guideline adherence. Lack of pharmacists recognizing and responding to drug therapy problems was also considered a common system failure. Hepler and Strand have led the movement to expand the role of community pharmacists from simply packaging and dispensing to detecting drug therapy problems and consulting with patients and other health professionals. 18 Financial barriers and business model are believed to have limited the adoption of pharmaceutical care. 68 The Medicare prescription drug improvement and modernization act of 2003 may remove these barriers. In the bill, pharmacists are designated as eligible Medicare providers by virtue of the plan paying for their services. Starting in 2006, pharmacists will be able to bill for medication therapy management in Medicare patients. Organizations, such as the coalition, should consider how they can take advantage of this financial arrangement and extend pharmacists medication therapy management to employer sponsored health plans.

PAGE 125

113 The lack of a centralized electronic infrastructure that would allow records to be shared across the continuum of care and used for medication management was also considered an important contributor to process problems from the prescribing and monitoring nodes. Development and implementation of electronic infrastructure is logistically difficult and expensive. Many health care systems are hesitant to invest in an electronic infrastructure. In the 1990s many health stystems invested hundreds of millions of dollars in information systems that were supposed to link providers and institutions, but often failed to achieve their objectives. 64 Nevertheless, decision makers and politicians are becoming interested in developing an electronic clinical infrastructure. Senator, Edward Kennedy introduced a bill to the senate on May 13, 2004 to modernize health care. A major component of the bill was to provide funding for the development of electronic medical records. Furthermore, President Bush recently called for all patient information to be available electronically by 2014. He claims the federal government will set the standards for the conversion to electronic records. Once this technology is adopted, careful attention must be given to measuring barriers to use and unintentional consequence of the technology. MU-PI Fidelity Further work is needed to refine the technical aspects of the medication use performance indicators. This would include updating the MU-PI to include newer therapies and newer treatment evidence, evaluating the criterion validity of the MU-PIs, and adapting the search algorithms to include clinical data along with claims data. New drugs have been entering the market and new evidence for medical treatment is becoming more abundant. Indicators need to be specifically constructed to represent nodes other than prescribing and monitoring. Indicators that represent problems in the administration

PAGE 126

114 and dispensing nodes were not represented in this study and are needed for a comprehensive assessment of the medication use system. Careful and meaningful analysis of the more recent literature is needed to add new performance indicators and to remove existing indicators that do not represent failures in the management of drug therapy. The criterion validity of the indicators is unknown. To validate the MU-PIs each indicator would need enough positives for statistical evaluation. This population should include a representative proportion of patients over sixty-five year old. Attention should be given to determining the positive predictive value, sensitivity and specificity of the indictors. A risk factor study is needed to better understand the population and system variables most associated with having process failures and PDRM. The analysis should include a separate evaluation for process positives and PDRM positives to determine what characteristics produce greater risk for drug related injury. The analysis should also be separated by nodes of the MUS. It would be interesting to determine if the characteristics that predict PDRM for monitoring problems are different from the characteristics that predict PDRM related to problems in the prescribing node.

PAGE 127

APPENDIX A MEDICATION INVOLVED IN DRA BY STUDY

PAGE 128

Cardiovascular Diuretics916 -5.613.51316.763.21212 -10.512.510Digoxin -16.7 -44 -43Nitrates -5.6 -4.2 -1 -4.23Antihypertensive -9 -2.8 -18.514.69.921022.9 -9.958Anticoagulants/Antithrombotics -13 -5.66.3 -18 -15.8135Cardiovascular / other16 -40 -10.89.3 -1185.7 -10.872538401424.346.458.573.1206228.626.333.312PsychotropicAntipsychotic -1.33 -2.152Antidepressant -4.62 -3.32Lithium -6.3 -1 -3.652Anxiolitic, Sedative, Hypnotic -2.8 -7 -4.92Phychotropic/other -20 -9.5 -18 -15.89.55 -202.89.5 -6.35.9148 -15.88.758Anti-inflammatoryCorticosteroids -3.313.9 -11.1 -5 -11.4 -11.15NSAIDs -131041.75.49.310.42.614 -10.28Anti-inflammatory -3.3910 -7.98.454 -1313.355.65.420.410.45.9281011.47.911.411AnalgesicsNarcotic Analgesics -8.3 -3.966 -3.965 -8.3 -3.966 -3.965Anti-diabeticHypoglycemic agents27 -16.25.620.8 -10414.32.612.15827 -16.25.620.8 -10414.32.612.158Anti-infectiveAntibiotics -136.7 -12.2 -4.2 -8 -5.713.287Anti-asthmatics -3.3 -9.3 -1.3 -5.7 -4.54GI -10 -9.5 -9.752Antineaplastic -3.3 -2.79.3 -17.16.366.365Anti-epileptics -2.79.3 -11.4 -9.33Non-specificOther48363.319.417.6 -9.914105.823.615.81010010099.9100.1100.1100.3100.210010010010099.6631Dartnell 1996Hallas 1991MedianCountCourtman 1995Lindley 1992Bigby 1987Lakshmanan 1986Trunet 1986Darchy 1999Nelson 1996Raschetti 1999Ng 1999Cunningham 1999 115

PAGE 129

APPENDIX B CLASSIFICATION OF DTPS INTO NODES OF THE MUS

PAGE 130

117 Dartnel, DTPsFrequenc y % occupiedClassDartnelPrescribing factor1130.56%PP30.56%Non compliance1541.67%ACDEadverse drug reaction (preventable)1027.78%MAC41.67%36100.00%M27.78%non-id-p100.00%AD R Lakshmanan, DTPsFrequenc y % occupiedClass7 admissions related to antihypertensive agents*741.18%M2 aminophylline and phenytoin**211.76%MP5.88%2 aminophylline and phenytoin***211.76%MDE1 high dose of prednisone in a known diabetic who developed ketoacidosis15.88%PAC11.76%1 rotator cuff tear after five intra articular steroid injections15.88%Non-id-pM76.47%2 hypoglycemia secondary to taking too much insulin****211.76%ACnon-id-p5.88%2 admissions, results of a patient taking chlorambucil on an unsupervised basis over several years211.76%M100.00%17100.00%AD R 35 cases were drug mediated adverse effects. This article does not differentiate 'drug-mediated adverse effects' and 'drug mediated adverse reactions.** Toxic reaction where levels were available and medication continued*** Toxic reaction could have been detected with the patients examined as outpatients Courtman, DTPsFrequenc y % occupiedClassCourtmanInappropriate dose3558.33%PP61.67%Avoidable or possibly avoidable ADR*1423.33%M DEnon-compliance915.00%ACAC15.00%Drug interaction**11.67%PM23.33%Lack of therap y 11.67%Pnon-id-p60100.00%100.00%* Among 16 ADR, author reported that 14 cases were avodable or possibly avoidable. We classify avoidable/possibly avoidable ADRs as M.AD R *** Author cited Strand et al (1990) and said, 'the patient is taking a drug for whichthere is no medical indication; and, the patient is receiving the wrong drug or drug product could not be accurately assessed from the medical chart alone at the time of admission and were, therefore, not recorded. Author gave examples of avoidable cases as inappropriate or contraindicated drug therapy, non-compliance. ** monitoring for drug interactions and ensuring drugs are discontinued by the prescribing physician when they are no longer required by the patient.LakshmananAuthor says that 19 cases are preventable drug related admissions. But only 17 case information is provided. evolving over weeks to months such that the victims could have been examined as outpatients and had their therapy altered****1 had been having frequent symptoms but did not contact a physician, 1 was having frequent symptoms that were neither volunteered nor e

PAGE 131

APPENDIX C SUMMARY: CLASSIFICATION OF DTPS INTO NODES OF THE MUS NodesDartnellLakshmananCourtmanBeroHallasBigbyDarchyNikolausOf the MUS19991986199519911992198719991992Prescribing30.56%5.88%61.67%59.46%37.31%5.26%53.33%0.00%Dispensing0.00%0.00%0.00%0.00%0.00%0.00%0.00%0.00%Administration41.67%11.76%15.00%27.03%20.90%25.00%0.00%23.68%Monitoring27.78%76.47%23.33%0.00%35.82%44.74%46.67%10.53%Non-identifiable process0.00%5.88%0.00%13.51%5.97%25.00%0.00%65.79%Non-preventable ADR34.55%0.00%3.23%22.92%74.06%0.00%26.83%0.00%Total100.00%100.00%100.00%100.00%100.00%100.00%100.00%100.00% 118

PAGE 132

APPENDIX D MEDICATION USE PERFORMANCE INDICATOR DEFINITIONS 1. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to depression This is the pattern of care: 1. History/diagnosis of depression 2. Use of long-acting benzodiazepine (e.g., Librium, Valium, Centrax, Paxipam, Dalmane, Azaene/Tranxene, etc.) 2. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to theophylline toxicity This is the pattern of care: 1. Use of theophylline 2. Drug level not done at least every 6 months 3. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to bipolar disorder. This is the pattern of care: 1. Diagnosis/history of bipolar disorder 2. Use of lithium 3. Drug level not done at least every three months 4. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to major and/or minor hemorrhagic event This is the pattern of care: 1. Use of IV heparin 2. PTT not done at least every day 5. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to gastritis and/or upper GI bleed and/or upper GI perforation and/or GI ulcers and anemia This is the pattern of care: 1. History/diagnosis of ulcers and/or GI bleeding 2. NSAID use for at least 1 month (not including Cox-2) 119

PAGE 133

120 6. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to depression This is the pattern of care: 1. History/diagnosis of depression 2. Use of a barbiturate (e.g., butalbital) 7. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to depression This is the pattern of care: 1. History/diagnosis of depression 2. Use of a sympatholytic antihypertensive (e.g., reserpine, methyldopa, clonidine, etc.) 8. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to seizure activity This is the pattern of care: 1. Use of anticonvulsant requiring drug level monitoring (e.g., phenytoin, carbamazepine, valproic acid) 2. Drug level not done at least every 6 months 9. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to lithium toxicity This is the pattern of care: 1. Use of lithium 2. Lithium level not done every 3 months 10. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to a major/minor hemorrhagic event This is the pattern of care: 1. Warfarin use 2. NSAID use (e.g., diclofenac, ibuprofen, ketoprofen, etc.) 3. INR not done within 10 days 11. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to hypothyroidism This is the pattern of care: 1. Lithium use for at least 6 months 2. TSH not done at least every 6 months

PAGE 134

121 12. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to blood dyscrasias/thrombocytopenia This is the pattern of care: 1. Use of ticlopidine (Ticlid) 2. CBC/platelets not done every 2 months 13. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to acute urinary retention This is the pattern of care: 1. Diagnosis/history of bladder atony due to diabetes 2. Use of imipramine 14. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to acute respiratory failure This is the pattern of care: 1. History/diagnosis of severe COPD 2. Use of medium to long-acting benzodiazepines 15. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to acute urinary retention This is the pattern of care: 1. History/diagnosis of benign prostatic hypertrophy (BPH) 2. Use of an anticholinergic agent 16. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to congestive heart failure and/or fluid overload This is the pattern of care: 1. History/diagnosis of high blood pressure (over 140/90) and/or congestive heart failure 2. NSAID use for at least 3 months 17. This outcome has occurred after the pattern of care below: ER visit/hospitalization for hypokalemia This is the pattern of care: 1. Use of NONpotassium SPARING diuretic (e.g., hydrochlorothiazide, etc.) 2. No concurrent use of potassium supplement 3. Electrolytes not checked at least every 2 months initially, then every 6 months 18. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to anticonvulsant drug toxicity

PAGE 135

122 This is the pattern of care: 1. Use of an anticonvulsant requiring drug level monitoring (e.g., phenytoin, carbamazepine, valproic acid) 2. Drug level not done at least every 6 months 19. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to hypothyroidism This is the pattern of care: 1. Use of thyroid or antithyroid agent (e.g., levothyroxine, propylthiouricil, etc.) 2. T4/TSH not done at least every 12 months 20. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to aminoglycoside toxicity (acute renal failure and/or renal insufficiency and/or vestibular damage and/or auditory damage) This is the pattern of care: 1. Use of an aminoglycoside 2. Serum creatinine not done before and after therapy (and if therapy longer than 7 days, not done at least every 7 days) 3. At least one drug level not done 21. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to COPD This is the pattern of care: 1. Diagnosis/history of COPD 2. Use of a beta-blocker (e.g., propranolol, etc.) 22. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to Hypertension/tachycardia This is the pattern of care: 1. History of hypertension 2. Prescription use of sympathomemetic decongestants 23. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to hyperkalemia This is the pattern of care: 1. Use of ACE inhibitor 2. Electrolytes/CBC not done at least every 6 months 24. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to blood dyscrasias and/or hyponatremia and/or excessive water retention and/or syndrome of inappropriate antidiuretic hormone (SIADH)

PAGE 136

123 This is the pattern of care: 1. Use of carbamazepine 2. Electrolytes/CBC not done at least every 6 months 25. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to acute renal failure and/or renal insufficiency This is the pattern of care: 1. Use of lithium 2. BUN/serum creatinine not done at least every 3 months 26. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to digoxin toxicity This is the pattern of care: 1. Use of digoxin 2. BUN/serum creatinine not done at least every 6 months 3. Digoxin level not done at least every 6 months 27. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to gastritis and/or upper GI bleeds and/or GI perforations and/or GI ulcers and anemia This is the pattern of care: 1. History/diagnosis of ulcers and/or gastrointestinal bleeding 2. Use of an oral corticosteroid (e.g., prednisone) for at least 3 months 28. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to congestive heart failure and/or heart block This is the pattern of care: 1. History/diagnosis of congestive heart failure with heart block or advanced bradycardia 2. Use of digoxin 29. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to acute renal failure and/or renal insufficiency This is the pattern of care: 1. Use of an ACE inhibitor 2. BUN/serum creatinine not done at least every 6 months 30. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to major and/or minor hemorrhagic event

PAGE 137

124 This is the pattern of care: 1. Use of warfarin 2. INR not done at least every month 31. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to secondary myocardial infarction This is the pattern of care: 1. History/diagnosis of myocardial infarction 2. No use of ASA and/or beta blocker (e.g., metoprolol) 32. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to a major/minor hemorrhagic event This is the pattern of care: 1. Warfarin use 2. Antibiotic use (e.g., Bactrim, etc.) 3. PT not done within 5 days 33. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to acute renal failure and/or renal insufficiency This is the pattern of care: 1. Use of allopurinol 2. BUN/serum creatinine not done at least every 6 months 34. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to depression and/or increase in dosage of antidepressant This is the pattern of care: 1. History/diagnosis of depression 2. Use of moderate to high lipophilic beta-adrenergic blocking agent (e.g., propranolol, pindolol, etc.) 35. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to patient fall and hip fracture This is the pattern of care: 1. Patient on alpha blocker 2. Standing BP not checked within 2 months of initiation of therapy 36. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to fall and/or hip fracture and/or other bone fracture and/or bone break

PAGE 138

125 This is the pattern of care: 1. 65 years or older 2. Use of long half-life hypnotic/anxiolytic (e.g., flurazepam, diazepam, chlordiazepoxide, etc.) 37. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to congestive heart failure This is the pattern of care: 1. Diagnosis/history of congestive heart failure 2. Not on an ACE inhibitor or ARB (e.g., captopril, enalapril, etc.) 38. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to congestive heart failure This is the pattern of care: 1. History/diagnosis of congestive heart failure 2. Use of an antiarrhythmic agent (e.g., disopyramide, procainamide, etc.) 39. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to asthma This is the pattern of care: 1. Diagnosis of moderate to severe asthma 2. Use of a bronchodilator 3. No use of maintenance therapy (e.g., beclomethasone, etc.) 40. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to fall and/or hip fracture and/or other bone fracture and/or bone break. This is the pattern of care: 1. 65 years of older 2. Use of tricyclic antidepressant (e.g., amitriptyline, doxepin, imipramine, etc.)

PAGE 139

APPENDIX E PROCEDURE CODES USED TO IDENTIFIY OFFICE VISITS AND EMERGENCY DEPARTMENT VISITS FROM PROFESSIONALS CLAIMS DATA CPT Code Office VisitCPT CodeEmergency Department Visits99201Office Visit,New,Focused,10#99281Emerg Dept.Visit, Focused99202Office Visit,New,Expanded,20#99282Emerg Dept.Visit,Expanded,Low Complexity99203Office Visit,New,Detailed,30#99283Emerg Dept.Visit,Expanded,Lo-Mod. Complexity99204Office Visit,New,Mod. Complex,45#99284Emerg Dept.Visit, Detailed99205Office Visit,New,Hi Complex,60#99285Emerg Dept.Visit, Comprehensive99211Office Visit,Estab,Minimal,5#99212Office Visit,Estab,Focused,10#99213Office Visit,Estab,Expanded,15#99214Office Visit,Estab,Detailed,25#99215Office Visit,Estab,Comprehens,40#99241Office Consultation, Focused,15#99242Office Consultation, Expanded,30#99243Office Consultation, Detailed,40#99244Office Consultation,Mod. Complex,60#99245Office Consultation,Hi Complex,80# 126

PAGE 140

APPENDIX F CODING SOLUTION FOR EACH MEDICATION USE PERFORMANCE INDICATOR Drug-Disease InteractionMonitoringDrug-Drug InteractionInd 01Ind 02Ind 10Ind 05Ind 03Ind 17Ind 06Ind 04Ind 32Ind 07Ind 08Ind 48Ind 13Ind 09Ind 14Ind 11Ind 15Ind 12Ind 16Ind 18Ind 21Ind 19Ind 22Ind 20Ind 27Ind 23Ind 28Ind 24Ind 31Ind 25Ind 34Ind 26Ind 36Ind 29Ind 37Ind 30Ind 38Ind 33Ind 40Ind 35 127

PAGE 141

APPENDIX G RECRUITMENT LETTER FOR NODE IDENTIFICATION STUDY Dear Dr. The University of Florida and the Central Florida Health Care Coalition are conducting a study that aims to evaluate the underlying system factors involved in errors that originate at different nodes of the medication use system in primary care. The goal is to identify errors that originate at specific nodes of the medication use system and then to establish system factors that appear to be common and unique to the nodes. (This is Brian Sauers Ph.D. dissertation research.) This study includes the following three steps: (1) assign the preventable adverse drug event (pADE) scenarios to the nodes of the medication use system where the process failure originated, (2) translate the pADE scenarios to billing codes and query the coalitions database using the pADE indicators, and (3) conduct cause and effect diagramming (with a group of providers affiliated with the coalition) to uncover latent system conditions that may be causing or contributing to errors occurring in the medication use system. We have written to you because of your expertise in medication error research. As you know, this field has relatively few real opinion leaders, so your assistance and subsequent participation would be invaluable. We are soliciting your help in two ways. First, please nominate other experts in the field of medication safety so that we may invite them to serve on our Delphi panel. We are aware of researchers currently publishing in this field, but we need your advice about which health care providers and researchers are trusted experts in medication error research. We are seeking a panel of 10-20 members. We will send a letter similar to this to each person you nominate. This would require only a few minutes of your time to write names and contact information on a form and return it to us. Second, we would like you to participate on the Delphi panel itself. This panel will match up 15 pADE scenarios with the nodes of the medication use system where the process failure likely originated. The first round of the Delphi process should take approximately one hour to complete, and the subsequent two to three rounds are not expected to exceed 30 minutes. Appended is a participation and recommendation form. Please indicate whether you are willing to participate by circling yes or no, and please provide names and contact information for individuals that you believe have expertise in medication errors. Brian will telephone you in a few days to follow up and answer any questions. You participation will be greatly appreciated. Respectfully, Brian C. Sauer Charles D. Hepler 128

PAGE 142

129 1. Will you participate in the Delphi panel that will assign the pADE scenarios to nodes of the medication use system? (yes or no) Please sign name 2. If you agree to participate would you like to complete the survey electronically or would you rather receive a paper copy? Please circle one: (electronic copy or paper copy) 3. Please list any health care professionals or researchers, and if possible, a form of contact. a. ________________________________________________________________ b. ________________________________________________________________ c. ________________________________________________________________ d. ________________________________________________________________ e. ________________________________________________________________ f. ________________________________________________________________ g. ________________________________________________________________

PAGE 143

APPENDIX H NODE IDENTIFICATION SURVEY Thank you for agreeing to participate on the Process Identification Panel for the Central Florida Health Care Coalition and the University of Florida. Your expertise will be valuable in identifying the nodes in the primary care medication use system where preventable adverse drug events may have originated. The information from this analysis will be used to better understand the relationships between the processes of care and system factors. These include patient involvement, microsystems, organizations, and the environment. The attached survey contains detailed instructions to help you identify which nodes of the medication use system in primary care may be involved in the generation of specific drug therapy problems which, if unresolved, may lead to preventable adverse drug events. Based on pretesting, it should take approximately one hour to complete. Your input is extremely important. This project will not be successful without your gracious input. Please return the Process Identification survey, along with comments to Brian C. Sauer, at sauer@cop3.health.ufl.edu, by November 25 th Please e-mail or call me at (352) 273-6296 if you have any questions. Thank you again for participating. Respectfully, Brian C. Sauer Charles D. Hepler PhD Candidate Distinguished Professor 130

PAGE 144

131 You will be given a list of scenarios that represent cases of potential preventable adverse drug events (pADE) from the primary care setting. These pADE scenarios, which were developed and content validated by two Delphi panels of physicians, include errors of omission, therapeutic failures and drug induced adverse outcomes. The appendix provides more information for pADE operationalization. Survey Instructions The objective of this exercise is to identify the nodes of the primary care medication use system at which specific pADEs may have originated. In other words, we are seeking the node of the medication use system where the problem most probably was initiated. (Cause and effect diagramming, at a later time, will consider events downstream from the initiating node.) Listed below are 15 pADE scenarios. Each includes an adverse outcome and a pattern of care. We would like you to identify the node in the MUS where you believe the inappropriate care most likely originated. Please adopt a population perspective: That is, although many specific issues could have arisen in the care of an individual patient, we are asking you to consider the most common issues in a large number of patients. To do this, please first familiarize yourself with the definitions of the nodes of the medication use system below. Then please read each pADE scenario and assign values between 0 and 10 (0 highly unlikely and 10 highly likely) to represent your belief that the process failure originated in a particular node of the medication use system. The sum of the 4 nodes needs to equal 10. Please distribute the 10 points across the nodes to represent your belief that a particular drug-therapy problem originated in a specific node. The distinction between appropriate care for individuals and populations is very important here. We would like you to assign higher points to the node that is most likely to be where the process

PAGE 145

132 problem originated for the majority of patients with the specific pADE described in the scenario. Thinking at a population level can be difficult, one way to simplify this it to think of the last 10 patients you saw who had the problem described in the pADE scenario and allocate points to the nodes to represent those experiences. We would also like you to briefly state your reasoning and offer comments for each pADE scenario. Please see the hypothetical example below. A model of the Medication Use System and definitions for its nodes are presented below: Prescribing node (P)-Assess patient, determine need for medication, if need exists then select medication, and order medication. Dispensing node (D) Review and assess prescribers orders, look for problems with drug therapy, prepare medications, distribute medications for patients access, educate patient on proper use of medication. (Self) administration node (A) Administer medications according to prescribers plan, and report any problems with therapy. Monitoring node (M) Assess patient response to medication, detect problems with drug therapy, and act on the information gathered by reporting and documenting outcomes and changing therapy if necessary.

PAGE 146

133 Hypothetical Example This outcome has occurred after the pattern of care below: ER visit/hospitalization due to a major/minor hemorrhagic event This is the pattern of care: 1. Prescribed use of Warfarin 2. Prescription NSAID use (e.g., diclofenac, ibuprofen, ketoprofen, etc.) 3. INR not done within 10 days Node P D A M Assigned Points 8 0 0 2 P+D+A+M = 10 Decision Logic : Note: your responses are not expected to be as thorough as the example; this was done for instructional purposes. The process failure likely originated in the prescribing node of the MUS because it is well established that concomitant use of NSAIDs and warfarin should be avoided. If anti-pyretic effects are desired, then consider acetaminophen. If anti-inflammatory effects are necessary, then cyclooxygenase-2 (COX-2) inhibitor therapy may be safer. The dispensing node could have been involved in the process failure if the pharmacist knew or should have known that the patient was prescribed and using prescription NSAIDS and Warfarin concomitantly. Nevertheless, the problem would have been originated at the prescribing node; the error was reinforced at the dispensing node. Note: this example is using prescription NSAIDs, if OTC NSAIDs were included then the dispensing node would clearly be involved in the initiation of this problem. The administration node is involved to the degree that you would expect a patient to be aware of the medications they are taking and potential interactions and contraindications.

PAGE 147

134 Nevertheless, patients can not initiate prescription therapy nor are they capable of monitoring their INR levels and for this reason the administration node can not be where the problem originated. It also appears the problem could have originated at the monitoring node. If NSAIDS were considered necessary, then INRs should be monitored every week when NSAIDS and warfarin are co-administered. Signs and symptoms of an active bleed should have been monitored and if present appropriate actions should have been taken. Comments : Even though this problem could have originated in the prescribing and monitoring nodes of the medication use system, I placed more weight on the prescribing node because NSAIDs should generally be avoided in patients taking warfarin. Process Node Identification Survey Please assign points from 0 to 10 (0 highly unlikely and 10 highly likely) across the nodes of the medication use system. The points you allocate will represent the strength of your belief about where the process failure originated. 1. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to congestive heart failure and/or fluid overload This is the pattern of care: 1. History/diagnosis of high blood pressure (over 140/90) and/or congestive heart failure 2. Use of prescription NSAID for at least 3 months Node P D A M Assigned Points Decision Logic: Comments:

PAGE 148

135 2. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to congestive heart failure This is the pattern of care: 1. Diagnosis/history of congestive heart failure 2. Not on an ACE inhibitor (e.g., captopril, enalapril, etc.) Node P D A M Assigned Points Decision Logic: Comments: 3. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to congestive heart failure and/or heart block This is the pattern of care: 1. History/diagnosis of congestive heart failure with heart block or advanced bradycardia 2. Use of digoxin Node P D A M Assigned Points Decision Logic: Comments: 4. This outcome has occurred after the pattern of care below: ER visit/hospitalization for Gastritis and/or upper GI bleed and/or upper GI perforation and/or GI ulcers and anemia This is the pattern of care: 1. History/diagnosis of ulcers and/or GI bleeding 2. Use of prescription NSAIDs for at least 1 month Node P D A M Assigned Points Decision Logic: Comments: 5. This outcome has occurred after the pattern of care below: ER visit/hospitalization for hypokalemia

PAGE 149

136 This is the pattern of care: 1. Use of a non potassium-sparing diuretic (e.g., hydrochlorothiazide, etc.) 2. No concurrent use of potassium supplement 3. Electrolytes not checked at least every 2 months initially, then every 6 months Node P D A M Assigned Points Decision Logic: Comments: 6. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to seizure activity This is the pattern of care: 1. Use of anticonvulsant requiring drug level monitoring (e.g., phenytoin, carbamazepine, valproic acid) 2. Drug level not done upon initiation of therapy and at least every 6 months thereafter Node P D A M Assigned Points Decision Logic: Comments: 7. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to gastritis and/or upper GI bleed and/or upper GI perforation and/or GI ulcers and anemia This is the pattern of care: 1. History/diagnosis of ulcers and/or GI bleeding 2. Use of prescription NSAID for at least 1 month Node P D A M Assigned Points Decision Logic: Comments: 8. This outcome has occurred after the pattern of care below: ER visit/hospitalization for Lithium toxicity This is the pattern of care: 1. Use of lithium 2. Lithium level not done monthly until stable, then every 3 months

PAGE 150

137 Node P D A M Assigned Points Decision Logic: Comments: 9. This outcome has occurred after the pattern of care below: ER Visit/ hospitalization due to tachycardia This is the pattern of care: 1. History of hypertension 2. Use of sympathomimetic decongestants Node P D A M Assigned Points Decision Logic: Comments: Assuming OTC 10. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to hyperkalemia This is the pattern of care: 1. Use of ACE inhibitor 2. Electrolytes/CBC not done at least every 6 months Node P D A M Assigned Points Decision Logic: Comments: 11. This outcome has occurred after the pattern of care below: ER visit/hospitalization for acute renal failure and/or renal insufficiency This is the pattern of care: 1. Use of allopurinol 2. BUN/serum creatinine not done at least every 6 months Node P D A M Assigned Points Decision Logic: Comments:

PAGE 151

138 12. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to major and/or minor hemorrhagic event This is the pattern of care: 1. Use of warfarin 2. INR not done before therapy starts and at least every month thereafter Node P D A M Assigned Points Decision Logic: Comments: 13. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to asthma This is the pattern of care: 1. Diagnosis of moderate to severe asthma 2. Use of a bronchodilator 3. No use of maintenance corticosteroid (e.g., beclomethasone, etc.) Node P D A M Assigned Points Decision Logic: Comments: 14. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to a major/minor hemorrhagic event This is the pattern of care: 1. Warfarin use 2. Antibiotic use (e.g., Bactrim, etc.) 3. PT not done within 5 days Node P D A M Assigned Points Decision Logic: Comments: Not enough info in example. 15. This outcome has occurred after the pattern of care below: ER visit/hospitalization due to acute renal failure and/or renal insufficiency

PAGE 152

139 This is the pattern of care: 1. Use of an ACE inhibitor 2. BUN/serum creatinine not done at initiation of therapy and at least every 3-6 months thereafter Node P D A M Assigned Points Decision Logic: Comments: Demographic information Name __________ Title_ Degrees__ ______ Age ____ Gender_______ Are you a licensed clinician?_ (Are you a practicing clinician?_ ___ What is your medical specialty?_____________________________________ Do you practice in an academic medical center?___________________ Are you involved in research on medication errs, patient safety, pharmacovigilance, etc..? ___________ Do you believe medication errors and adverse drug events are a serious public health issue? _________ What do you believe the incidence of pADEs is in ambulatory care? You have finished. Please return survey to Brian Sauer,at sauer@cop3.health.ufl.edu Thank you.

PAGE 153

140 Appendix to Survey The PDRM scenarios have been content validated by two separate Delphi panels of Geriatric Medicine experts, meaning the expert panel reached consensus that these scenarios represented situations in care that would be considered pADE. The following definitions were used for the validation process: 1) Preventable adverse drug event was defined as a significant clinical outcome in which drug therapy has not produced a reasonable intended result by (a) producing a noxious, unintended and undesired drug effect, (b) by failing to produce the intended effect within a reasonable time or (c) by omitting a necessary therapy. 2) Drug therapy problem was defined as a situation in care that is inconsistent with the treatment objectives. 3) Preventability: Four criteria were used to define the characteristics of preventable adverse drug event. The drug therapy problem had to be recognizable and the likelihood of adverse drug event must have been foreseeable In addition, the proximate cause(s) or medication error must be identifiable and those causes must be controllable

PAGE 154

APPENDIX I MEDICATION USE PERFORMANCE INDICATOR EVALUATION TEAM BRIEFING Thank you for agreeing to participate on the Medication Use Performance Indicator (MU-PI) Evaluation Team. The quality of medication use is an important issue for the providers, and members of the Central Florida Health Care Coalition. The goal of our meeting on May 4 th is to discuss the results of our MU-PI study with the intent to discover underlying system factors that may contribute to the process failures and morbidity. We will formally evaluate the following MU-PIs during our meeting. Pattern of Care (process) Outcome that occurred Diagnosis asthma 1. Use of a bronchodilator (persistence) 2. No use of maintenance therapy (e.g., corticosteroid, leukotrienes, etc.) ER visit/hospitalization due to asthma Use of warfarin 3. INR not done at least every month ER visit/hospitalization due to major and/or minor hemorrhagic event We will evaluate the two MU-PIs separately, and you will be asked to do the following: 1. Brainstorm possible system related issues that may have contributed to the prevalence of each pattern of care and outcome listed above. 2. Clarify and organize the proposed system causes. 3. Prioritize the proposed system cause and use top 5 to 10 for tree-diagramming. 4. Identify relationships among the proposed causes in relation to the different levels of the health care system through tree diagramming (described pages 3-5). 141

PAGE 155

142 5. Establish commonality with other MU_PIs. After reading through the background section below please prepare for the May 4 th meeting by thinking about the system factors that might explain: A) the number of asthmatics not using maintenance therapy B) the number of people using warfarin who appear to be missing monthly INRs. Background The MU-PI study used forty medication use indicators to screen the coalitions claims data for patterns of care that are inconsistent with guidelines and associated adverse outcomes. The 40 MU-PIs used in this study were based on published evidence and content validated by two separate Delphi panels of medical doctors. Population demographic and summary statistics for the 40 MU-PI are provided below. Table 1. Population Age Distributions mean age(SD)A g e Freq[%]<156,622 [19.07]1,084[18.51]1,263[17.61]4,956[14.3]15 4516,519 [47.58]2,143[36.6]3,399[47.39]16,461[47.48]45 6510,743 [30.94]2,144[36.62]2,270[31.65]11,421[32.94]65<=835 [2.41]484[8.27]240[3.35]1830[5.28]GenderFemale22,861 [65.85]3243[71.84]4,659[64.96]23,499[67.78]Male11,858 [34.15]1271[28.16]2,513[35.04]11,169[32.22]Missing0000All ClaimsHospital AdmissionsED visitsPharmacy Claims33.97 (18.23)34.48 (22.12)34.71 (18.54)37.03 (18.34)

PAGE 156

143 Table 2. Process and PDRM Positives # Members Positive# of PositivesPercentProcess 6,12911,49016.54PDRM_ED4146PDRM_HA2641PDRM_ED_HA1819Total_ PDRM831060.23 There were 11,490 process positives in 6,129 members. Table 2 also shows the results for those who screened positive for both the process and the outcome. The term preventable drug related morbidity (PDRM) is used to describe those who screen positive for both the process and the outcome. There were 106 PDRM positives in 83 members. 46 led to emergency department visit (ED), 41 led to a hospital admission (HA) and 19 went from the ED to the HA. Table 4. lists the MU-PIs that will be included in the evaluation. For a complete list of the 40 MU-PI used in this study and individual indicator results please call or e-mail Brian Sauer (352) 273-6296 sauer@cop3.health.ufl.edu and I will e-mail them to you. Table 4. Indicators for Evaluation IndMnemonicHxDrug_ADrug_BRiskProcessP_HAP_EDPDRM1Asthma--Bron-NoMaintAsthma215337043878496626292Hx/CHF-NoACE-ICHF6454059398105123War-INR(1)Hemr34330436929ACEI-BUN(6) ARF405924092371325 1. The Mnemonic is an abbreviated version of the MU-PI scenario listed below 2. Hx = the number of members who had an ICD-9 code for the particular history of disease or diagnosis of interest 3. Drug A and Drug B = the number of members who had a prescription claim for the drug of interest.

PAGE 157

144 4. Risk = the number of members who met the criteria for analysis. For example; if the process component includes a drug and monitoring interval of 6 months, then the members at risk would be those who were using the medication of interest for at least 6 months. 5. Process = the number of members who had the pattern of care represented in the MU-PI 6. PDRM = the number of members who had both the process component and the adverse outcome within the specified time frame. Ind Process of Care Outcome 1 Diagnosis of moderate to severe asthma Use of a bronchodilator No use of maintenance therapy (e.g., corticosteroid, leukotrienes, etc.) ER visit/hospitalization due to asthma 2 Diagnosis/history of congestive heart failure AND Not on an ACE inhibitor or ARB ER visit/hospitalization due to congestive heart failure 3 Use of warfarin AND INR not done at least every month ER visit/hospitalization due to major and/or minor hemorrhagic event 4 Use of ACE inhibitor and BUN/serum creatinine done every 6 months ER visit/hospitalization due acute renal failure Levels of Health Care System Berwick (2002), published a users manual for the IOM report, Crossing the Quality Chasm. The purpose was to provide a conceptual framework for better understanding and evaluation the affect of health care systems on patient care. In it he addresses the issue of embedded systems by presenting a framework for the different levels within health care. He separates the system into the following: the experience of the patients and communities (Level A), the functioning of small units of care delivery called microsystems (Level B), the functioning of organizations that house or otherwise support microsystems (Level C) and the environment of policy, payment, regulation, accreditation and other factors (Level D). We will use this

PAGE 158

145 framework to organize the proposed caused from the brainstorming step and to explicitly describe how the relationships among the system levels affect the quality of medication use. Figure 1. Levels of Health Care System The Patient Level (Level A) represents the patients experience and perception of the care they received. It also represents the patients active participation in their health and health care. The Medication Use System (MUS) includes recognition of drug therapy indication, prescribing, dispensing, administering, and patient monitoring. The MUS spans multiple professions and microsystems, which include, but are not limited to, physician offices, pharmacy, and laboratory.

PAGE 159

146 The Microsystems (Level B) are the small productive units that actually give the care that the patient experiences. Clinical microsystems are basically the small organized groups of providers and staff caring for a defined population of patients. The Health Care Organizations (Level C) house and support microsystems -they provide the necessary resources for the microsystems to deliver care. Common organizations include hospitals, large provider groups, nursing homes, and pharmacy benefit managers, all of which are typically embedded within some form of managed care organizations. The Health Care Environment (Level D) includes multiple entities that influence the activities of the organizations and microsystems. Important environmental systems include financing, regulation, accreditation, policy, litigation, professional education and social policy

PAGE 160

APPENDIX J MU-PI RESULTS B Hem GD D S L He Hy BldyAUR ActRespFai AUR CHFHx H AntHy Am CO HT Hy BldyARF Di G CH ARF He MIHx/MIHem ARF Depress F F CHFHxCHFHx As F See Appendix D to link indicator numbers to the indicator scenarios IndicatorMnemonicHxDrug_ADrug_BRiskProcessP_HAP_EDPDRMDelphi1DepHx/Dep-Benzo1,8982,6893217592TheoToxTheo-DrugLvl(6)19179750003 PHx/BP-Li-DrugLvl(3)9765990004rhep-NoLab7770005IHx/GI-NSAID2,5265,89931600046epHx/Dep-Barb1,898662510007epHx/Dep-Symp1,8982,9711362138eizAntconv-DrugLvl(6)37715914311269iToxLi-DrugLvl(3)652321000810mrWar/NSAID-INR3437,2831923400011pothLi-TSH(6)65181600012sTic-CBCP(2)74100013Hx/bladderAtony-Imip10597000014lCOPD-Benzo6412,68910231315Hx/BPH-Antic6781,8112800016/HBP-NSAID6,0017,2831,427033117ypoKKwd-NoK-Elctro(2)1,810837814643010518iconvToxAnticonv-DrugLvl(6)37715914500019pothThy-T4TSH(12)2,6151,51091514520glyToxAmgly-SCr-DrugLvl*(7dys)41413900021PDHx/COPD-BB5602,8327100022NHx/HTN-Decon6,03600000923prKACEI-ElctroCBC(6)4,0592,4092,2090221024sCbz-ElctroCBC(6)103403300025Li-BUN(3)65232100026gToxDif-BUN/DrugLvl**(6)30930913600027IHx/GI-Ocort2,5264,07527201128FHx/CHF_HB-Dig6453097911612329ACEI-BUN(6)4,0592,4092,2823251530mrWar-INR(1)3432563691231--ASA-BB**1812,6688951532rWar-Antibiot-NoLab(5dys)3439,785320001433Allop-BUN/Scr(6)2111201081121134Hx/Depress-BB1,8981,0753600035allAlphaBlkr-NoBP(2)49035530400036alllongactingBenzo2,68917911137/CHF-NoACEI6454,05939810512238/CHF-antiarrhythmic645811821239thmaHx-Bron-NoSteroid2,1533,7043,878190626261340allTriclyantidep8365111111,1935769103 The Mnemonic is an abbreviated version of the MU-PI scenario 147

PAGE 161

148 Hx = the number of members who had an ICD-9 code for the particular history of disease or diagnosis of interest Drug A and Drug B = the number of members who had a prescription claim for the drugs of interest. Risk = the number of members who met the criteria for analysis. For example; if the process component includes a drug and six month monitoring interval, then the members at risk would be those who were using the medication of interest for at least six months. Process = the number of members who had the pattern of care represented in the MU-PI PDRM = the number of members who had both the process component and the outcome component of the MU-PI within the specified time frame. Fifteen indicators were sent to a Delphi process for node identification, they were numbered consecutively from one to fifteen. This column provides a cross reference to the MU-PI number

PAGE 162

APPENDIX K SURVEDY TO ESTABLISH COMMONALITY AND UNIQUENESS OF CAUSE SEQUENCES FROM TREE DIAGRAMS Survey to determine if the relationships identified among the system levels from the warfarin indicator are also contributing factors to other indicators. 1 Use of an ACE inhibitor AND BUN/serum creatinine not done at least every 6 monthsER visit/hospitalization due to acute renal failure and/or renal insufficiencyDiagnosis asthma 1. Use of a bronchodilator 2. No use of maintenance therapy (e.g., corticosteroid, leukotrienes, etc.)2ER visit/hospitalization due to asthma Branch 1. (Ind 1) Yes________ No________ Branch 1. (Ind 2) Yes________ No________ Branch Max (Ind 1) Yes________ No________ Branch Max. (Ind 2) Yes________ No________ Survey to determine if the relationships identified among the system levels from the Asthma indicator are also contributing factors to other indicators. 2Use of warfarin AND INR not done at least every monthER visit/hospitalization due to major and/or minor hemorrhagic event1ER visit/hospitalization due to congestive heart failure Diagnosis/history of congestive heart failure AND Not on an ACE inhibitor or ARB Branch 1. (Ind 1) Yes________ No________ Branch 1. (Ind 2) Yes________ No________ Branch Max (Ind 1) Yes________ No________ Branch Max. (Ind 2) Yes________ No________ 149

PAGE 163

APPENDIX L NODE IDENTIFICATION BOX PLOTS Each point indicates minimum, 25, 50 (median), 75, percentile and maximum. The Red Plus is the mean. mean median 25 percentile 75 percentile min max + Indicator 1 (16) Indicator 2 (37) 150 Indicator 6 (8) Indicator 5 (17) Indicator 4 (5) Indicator 3 (28)

PAGE 164

151 Indicator 9 (22) Indicator 8 (9) 2 nd round Indicator 9 (22) Indicator 10 (23) Indicator 11 (33) Indicator 12 (30) Indicator 13 ( 39 ) 2 nd round Indicator 12 (30) Indicator 14 (42) 2 nd round Indicator 14 (42)

PAGE 165

152 Indicator 15 ( 29 )

PAGE 166

LIST OF REFERENCES 1. Winterstein AG, Sauer BC, Hepler CD, Poole C. Preventable drug-related hospital admissions. Ann Pharmacother. 2002; 36(7-8):1238-48. 2. Institute of Medicine. Committee on Quality of Health Care in America. To err is human--building a safer health system. Washington, D.C.: National Academy Press, 1999. 3. Johnson JA, Bootman JL. Drug-related morbidity and mortality and the economic impact of pharmaceutical care. Am J Health Syst Pharm 1997; 54(5):554-558. 4. Cullen DJ, Bates DW, Leape LL. Prevention of adverse drug events: a decade of progress in patient safety. J Clin Anesth 2000; 12(8):600-614. 5. Bero LA, Lipton HL, Bird JA. Characterization of geriatric drug-related hospital readmissions. Med Care 1991; 29:989-1003. 6. Bigby J, Dunn J, Goldman L, Barclay AJ, Jen P, Seth Landfeld C. Assessing the preventability of emergency hospital admissions. A method for evaluating the quality of medical care in a primary care facility. Am J Med 1987;83:1031-1036. 7. Courtman BJ, Stallings SB. Characterization of drug-related problems in elderly patients on admission to a medical ward. Can J Hosp Pharm 1995; 48(3):161-166. 8. Darchy B, Le Miere E, Figueredo B Bavoux E, Omart Y. Iatrogenic diseases as a reason for admission to the intensive care unit. Arch Intern Med 1999; 159:71-78. 9. Dartnell JGA, Anderson RP, Chohan V, Galbraith KJ, Lyon MEH, Nestor PJ, Moulds RFW. Hospitalisation for adverse events related to drug therapy: incidence, avoidability, and costs. Med J Austr 1996; 164:659-662. 10. Hallas J, Gram LF, Grodum E, Damsbo N, Brosen K, Haghfelt T, Harvald B, Beck-Nielsen J, Worm J, Jensen KB. Drug related admissions to medical wards: a population based survey. Br J Clin Pharmacol 1992; 33:61-68. 11. Lakshmanan MC, Hershey CO, Breslau D. Hospital admissions caused by iatrogenic disease. Arch Intern Med 1986; 146:1931-1934. 12. Nikolaus T, Specht-Leible N, Kruse W, Oster P, Schlierf G. [The early rehospitalization of elderly patients. Causes and prevention]. Dtsch Med Wochenschr 1992; 117(11):403-407. 153

PAGE 167

154 13. Faris RJ. Explicit definitions to identify preventable drug-related morbidity in an elderly population and use as an indicator to evaluate quality in medications use system. Doctoral Dissertation, University of Florida, 2001. 14. Wilson PF, Dell LD, Anderson GF. Root Cause Analysis: A Tool for Total Quality Management. Milwaukee: Quality Press, 1993. 15. Hepler CD, Segal R. Preventing Medication Errors and Improving Drug Therapy Outcomes through Systems Management. Boca Raton: CRC Press Inc., 2003. 16. Berwick DM. A user's manual for the IOM's "Quality Chasm" report. Health Aff 2002; 21(3):80-90. 17. Reason J. Human error: models and management. West J Med 2000; 172(6):393-396. 18. Hepler CD, Strand LM. Opportunities and responsibilities in pharmaceutical care. Am J Hosp Pharm 1990; 47(3):533-543. 19. Bates DW, Leape LL, Petrycki S. Incidence and preventability of adverse drug events in hospitalized adults. J Gen Intern Med 1993; 8(6):289-294. 20. Bates DW, O'Neil AC, Boyle D, Teich J, Chertow GM, Komaroff AL, Brennan TA. Potential identifiability and preventability of adverse events using information systems. J Am Med Inform Assoc 1994; 1(5):404-411. 21. Bates DW, Spell N, Cullen DJ, Burdick E, Laird N, Petersen LA, Small SD, Sweitzer BJ, Leape LL. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA 1997; 277(4):307-311. 22. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'luf N, Boyle D, Leape L. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 6(4):313-321. 23. Leape LL, Kabcenell AI, Gandhi TK, Carver P, Nolan TW, Berwick DM. Reducing adverse drug events: lessons from a breakthrough series collaborative. Jt Comm J Qual Improv 2000; 26(6):321-331. 24. Grainger-Rousseau TJ, Miralles MA, Hepler CD, Segal R, Doty RE, Ben Joseph R. Therapeutic outcomes monitoring: application of pharmaceutical care guidelines to community pharmacy. J Am Pharm Assoc 1997; NS37(6):647-661. 25. Hepler CD, Grainger-Rousseau TJ. Pharmaceutical care versus traditional drug treatment. Is there a difference? Drugs 1995; 49(1):1-10. 26. Nelson EC, Batalden PB, Mohr JJ, Plume SK. Building a quality future. Front Health Serv Manage 1998; 15(1):3-32.

PAGE 168

155 27. Donabedian A. The quality of medical care. Science 1978; 200(4344):856-864. 28. Mohr JJ, Batalden PB. Improving safety on the front lines: the role of clinical microsystems. Qual Saf Health Care 2002; 11(1):45-50. 29. Wasson JH, Jette AM, Johnson DJ, Mohr JJ, Nelson EC. A replicable and customizable approach to improve ambulatory care and research. J Ambul Care Manage 1997; 20(1):17-27. 30. Torrens PR. Historic Evolution and Overview of Health Services in the United States. Introduction to Health Services. Albany: Delmar Publisher, 1999. 31. Leape LL, Bates DW, Cullen DJ, Cooper J, Demonaco HJ, Gallivan T, Hallisey R, Ives J, Laird N, Laffel G. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA 1995; 274(1):35-43. 32. Plsek PE, Greenhalgh T. Complexity science: The challenge of complexity in health care. BMJ 2001; 323(7313):625-628. 33. Iezzoni L. Risk Adjustment for Measuring Health Care Outcomes. Ann Arbor: Health Administration Press, 1994. 34. Waterstraat FL, Barlow J, Newman F. Diagnostic coding quality and its impact on healthcare reimbursement: research prospectives. J Am Med Rec Assoc 1990; 61(9):52-59. 35. Quam L, Ellis LB, Venus P, Clouse J, Taylor CG, Leatherman S. Using claims data for epidemiologic research. The concordance of claims-based criteria with the medical record and patient survey for identifying a hypertensive population. Med Care 1993; 31(6):498-507. 36. Coalition for Quality in Medication Use. Study of Clinically Relevant Indicators of Pharmacologic Therapy: Final Report. http://www.scriptproject.org. 2002. Accessed 07/2004. 37. Ashton CM, Kuykendall DH, Johnson ML, Wray NP. An empirical assessment of the validity of explicit and implicit process-of-care criteria for quality assessment. Med Care 1999; 37(8):798-808. 38. Brennan T.A. The Institute of Medicine Report on Medical Errors: Could it do harm? N Engl J Med 2000; 342(15):1123-1125. 39. Jha AK, Kuperman GJ, Rittenberg E, Teich JM, Bates DW. Identifying hospital admissions due to adverse drug events using a computer-based monitor. Pharmacoepidemiol Drug Saf 2001; 10(2):113-119. 40. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA 1991; 266(20):2847-2851.

PAGE 169

156 41. Honigman B, Lee J, Rothschild J, Light P, Pulling RM, Yu T, Bates DW. Using computerized data to identify adverse drug events in outpatients. J Am Med Inform Assoc 2001; 8(3):254-266. 42. MacKinnon NJ. Risk Assessment of Preventable Drug-Related Morbidity in Older Persons. Doctoral Dissertation, University of Florida, 1999. 43. Helmer, O. Analysis of the Future: The Delphi Method. Santa Monica, CA, The RAND Corporation, 1967. 44. Dalkey, N. C. The Delphi Method: An Experimental Study of Group Opinion (RM-5957-PR). Santa Monica, CA, The RAND Corporation, 1969. 45. Dalkey, N. C. The Delphi Method, III: Use of Self Rating to Imporve Group Estimates (RM-6115-PR). Santa Monica, CA, The RAND Corporation, 1969. 46. Anderson B, Fargerhaug T. Root Cause Analysis: Simplified Tools and Techniques. Milwaukee: ASQ Quality Press, 2000. 47. Mclaughlin CP, Kaluzny AD. Continuous Quality Improvement in Health Care: Theory, Implementation, and Application. Gaitherburg: Aspen, 1999. 48. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, Newhouse JP, Weiler PC, Hiatt HH. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324:370-376. 49. Cunningham G, Dodd TRP, Grant DJ, McMurdo MET, Richards RME. Drug-related problems in elderly patients admitted to Tayside hospitals, methods for prevention and subsequent reassessment. Age and Ageing 1997; 26:375-382. 50. Lindley CM, Tully MP, Paramsothy V, Tallis RC. Inappropriate medication is a major cause of adverse drug reactions in elderly patients. Age Ageing 1992; 21(4):294-300. 51. Nelson KM, Talbert RL. Drug-related hospital admissions. Pharmacotherapy 1996; 16(4):701-707. 52. Ng D, Gosh DG, Harris J, Whitehead C. Unplanned medication-related admissions to an acute care general teaching hospital. Aust J Hosp Pharm 1999; 29:84-87. 53. Raschetti R, Menniti-Ippolito F, Morgutti M, Belisari A, Rossignoli A. Adverse drug events in hospitalized patients. JAMA 1997; 277(17):1351-1352. 54. Trunet P, Le Gall JR, Lhoste F, Rapin M. Admissions to intensive care units for iatrogenic diseases. Leg Med 1982; 73-84.

PAGE 170

157 55. Trunet P, Borda IT, Rouget AV, Rapin M, Lhoste F. The role of drug-induced illness in admissions to an intensive care unit. Intensive Care Med 1986; 12(1):43-46. 56. Hall MJ, Popovic JR. 1998 summary: national hospital discharge survey. Advance Data, NCHS 2000; 316:1-17. 57. Scott J, Pope M. Nonadherence with mood stabilizers: prevalence and predictors. J Clin Psychiatry 2002; 63(5):384-390. 58. Biernacki P, Waldorf D. Snowball sampling. Social Methods Res 1981; 10:141-163. 59. Mackinnon NJ, Helper CD. Indicators of preventable drug-related morbidity in older adults 2. Use within a managed care organization. J Manag Care Pharm 2003; 9(2):134-141. 60. Gandhi TK, Weingart SN, Borus J, Seger AC, Peterson J, Burdick E, Seger DL, Shu DL, Federico F, Leape LL. Adverse drug events in ambulatory care. N Engl J Med 2003; 348(16):1556-1564. 61. Field TS, Gurwitz JH, Avorn J, McCormick D, Jain S, Eckler M, Benser M, Bates DW. Risk factors for adverse drug events among nursing home residents. Arch Intern Med 2001; 161(13):1629-1634. 62. Jha AK, Kuperman GJ, Teich JM, Leape L, Shea B, Rittenberg E, Burdick E, Seger DL, Vander Vliet M, Bates DW. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998; 5(3):305-314. 63. Carbonin P, Pahor M, Bernabei R, Sgadari A. Is age an independent risk factor of adverse drug reactions in hospitalized medical patients? J Am Geriatr Soc 1991; 39(11):1093-1099. 64. Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care 2003; 41(1 Suppl):I30-I38. 65. Holmes-Rovner M, Llewellyn-Thomas H, Entwistle V, Coulter A, O'Connor A, Rovner DR. Patient choice modules for summaries of clinical effectiveness: a proposal. BMJ 2001; 322(7287):664-667. 66. O'Connor AM, Stacey D, Entwistle V, Llewellyn-Thomas H, Rovner D, Holmes-Rovner M, Tait V, Tetroe J, Fiset V, Barry M, Jones J. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2003;(2):CD001431.

PAGE 171

158 67. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, Rubin HR. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282(15):1458-1465. 68. Hepler CD. Looking toward a market for pharmaceutical care. J Am Pharm Assoc Wash 1997; NS37(6):625-626.

PAGE 172

BIOGRAPHICAL SKETCH Brian Sauer is interested in the assessment and improvement of medications use. His core research goals are to use quantitative and qualitative methods to better understand the nature of drug-related morbidity. Specifically, he plans to contribute to the development of automated performance indicators of preventable drug-related morbidity, and develop methods to better understand how structure and processes (system design) in health care microsystems and organizations affect the quality of medication use. 159