UFDC Home  myUFDC Home  Help 



Full Text  
EFFECT OF ACADEMIC DETAILING ON COX2 UTILIZATION RATES By STEPHEN DOUGLAS GRAHAM A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2005 Copyright 2005 by Stephen D. Graham To Evie ACKNOWLEDGMENTS I would like to thank my wife, Andrea, and sons, Nicolas and Andrew, for their love and support. I thank my dissertation chair, Dr. Abraham Hartzema, and committee members, Drs. Ingrid Sketris, Almut Winterstein, Richard Segal, and Babette Brumback for their guidance through the dissertation process. I would like to extend special thanks to Ms. Dawn Frail at the Nova Scotia Department of Health and again to Dr. Ingrid Sketris at Dalhousie University for providing me with overwhelming support and encouragement to succeed and to return to Canada. Finally, I would like to thank the graduate students for giving me many happy memories of Florida. TABLE OF CONTENTS page A C K N O W L E D G M E N T S ................................................................................................. iv LIST OF TABLES ......... ........ ................................... .......... .... .......... .. ix L IST O F F IG U R E S .... ...... ................................................ .. .. ..... .............. xii ABSTRACT ........ .............. ............. ...... ...................... xiv CHAPTER 1 IN TR OD U CTION ............................................... .. ......................... .. B background ........................................................................ ............... Problem Statem ent .......................................................... .. ..... ...... .... Research Questions and H ypotheses ........................................ ........................ 4 R research Q u estion 1 ................................................... .. ........ .......... .. .. .... R research Question 1 H ypothesis....................................... ......... ............... 5 Research Question 2 .................. .............................. ......... .. .......... R research Question 2 hypothesis....................................... ......................... 6 R research Q question 3 ................................................... .. ........ ............ .. .. . R research Question 3 hypotheses ........................................ ....................... 6 Research Question 4 .................. .............................. ......... .. .......... R research Question 4 hypotheses ........................................ ....................... 7 Significance of R research .......................................................... ..............7 2 LITERA TURE REVIEW .......................................................... ..............9 Review Articles Addressing Effects of Academic Detailing ......................................9 Academic Detailing Studies Reporting No Statistically Significant Effect ..............12 P ro p e n sity S c o re s.................................................................................................. 17 3 M E T H O D S ........................................................................................................... 2 2 Step One: Extraction and Validation of Data ..........................................................22 S o u rc e s o f D ata .............................................................................................. 2 2 G P Inclusion C riteria........... ........................................ ...... .......................25 P patient Inclusion C riteria.......................................................... ............... 26 Step Two: Adjustment for Confounding Using Three Distinct Propensity Score M e th o d s .................................. .............................................. 2 6 Quintile Propensity Score M ethod ........................................... ............... 29 Regression on the Propensity Score M ethod......................................................29 "Greedy Matching" Method ............. .................... .......................... 29 Propensity Score M ethod Selection................................ ............... ............... 29 Step Three: Primary Outcome Analysis; Intervention Effect on COX2 Utilization R a te s ........................................... ................ ............ ...... .......................... 3 2 Step Four: Secondary Outcome Analyses; The Utilization of Other Health Care Resources Associated with NSAID Induced GI Side Effects..............................35 4 R E S U L T S .......................................................................... 3 7 Step One: Extraction and Validation of Data ................................. ...... ............ ....37 Step Two: Establishment of Balanced Control and Experimental Groups Using Three Propensity Score M ethods ........................................ ........................ 37 PrePropensity Score A analysis ........................................ ........................ 37 Quintile PS M ethod Analysis ................................................... ............... 39 Regression on the Propensity Score Method Analysis.................. .......... 41 "Greedy Matching" Method Analysis............................................. 41 Selection of a Preferred Propensity Score Method..................................... 43 Exploratory Analysis of the Propensity Score Methods Effect on Adjusting for Bias on Unm measured Variables ...................................... ............... 44 Step 3: Prim ary Outcom e A nalysis....................................... .......................... 49 M odel D evelopm ent ..................... ............... ....................... ............... 49 Between Group Results ........... .................... ...... ............... 51 W within Group (Longitudinal) Results....................................... ............... 53 Step 4: Secondary Outcom e Analyses............................................... .................. 55 M isoprostol Utilization Rates ......................................... ....................... ....... 55 M odel develop ent....... .................................... ...... ........ ............. 55 B etw een group results .............................................. ......... ............... 55 Within group (longitudinal) results ............................................................57 PPI U tilization R ates .................................... ................... ..... .... 59 M odel develop ent...................... ..... .............................. 59 B etw een group results ............................................... ........ ............... 60 Within group (longitudinal) results ............................................................61 H 2A U tilization R ates .............................................................. .....................63 M odel develop ent...................... ..... .............................. 63 B etw een group results ............................................... ........ ............... 63 Within group (longitudinal) results ............................................................66 G P O office V isit R ates ............................................... ............................... 67 M odel develop ent...................... ..... .............................. 67 B etw een group results .............................................. ......... ............... 67 Within group (longitudinal) results ............................................................70 Rheumatologist and GI Specialist Visit Rates................... .......................... 72 M odel develop ent...................... ..... .............................. 72 B etw een group results ............................................................................ 72 Within group (longitudinal) results ............................................................74 Hospitalization Rates Due to GI Complications ...........................................76 M odel develop ent...................... ..... .............................. 76 B etw een group results ............................................... ........ ............... 77 Within group (longitudinal) results ............................................................80 D eath Rates Due to GI Com plications ..................................... .................81 M odel develop ent.................... .. .................... ...... ................ .....8 1 Between and within group results ............................................ ..........82 5 D ISCU SSION .............. .... .. ..... .............. ............................ 83 The Academic Detailing Program in Nova Scotia ............................................. 83 Qualifications of the D etailers...................................................... ................. 83 Changes Which Occurred Over the Period of the Intervention (History Effects) ............. ............. ........ ...............................................83 Policy Options Available to Decision Makers .......................................... 84 Distribution of educational m aterial............................................... 85 Educational m meetings ............................................................................85 Audit and feedback .......... ............................ ........ .... ................85 Reminders and reminder systems............................. .......................86 D rug benefit changes................ .. ..... ............................ ......... 86 Primary Outcome: Effect on COX2 Utilization Rates ...........................................86 S tatistic al R e su lts........................................................................................... 8 7 Practical Significance .......................... .............. ... ...... .. .... ...........87 Com prison w ith Literature ....................................................... ............... 88 Secondary Outcom es .......................................................... .... ...... .. ...... .. 88 Effect on GastroProtective Agents Utilization Rates .........................................88 M iso p ro sto l ................................................................................ 8 8 P P Is ......................................................................................................... 8 9 H 2A s ................................................................ .. .................. 89 Effect on Utilization of Medical Services ....................................................89 GP office visits ..... ........... ........ ...... ...........90 Specialist office visits................................................ ................... 90 Hospitalization rates due to GI side effects...............................................91 Death due to GI complications....................... ...............92 Propensity Score Analysis M ethods ..................................................................... 92 "Greedy Matching" Method ............................ ............... ............... 92 Q uintile M ethod...................................................... ............. 93 R egression on the PS M ethod......................................... .......... ............... 93 P S Exploratory A nalysis............................................ .............................. 93 L im stations ......... ..... ................................................................. ....... 94 D ata L im stations .......................................... ................... ........ 94 D esign Lim stations ...................................... ................... ..... .... 96 C o n clu sio n s..................................................... ................ 9 8 APPENDIX A AN APPRAISAL OF THE NOVA SCOTIA OA AD INTERVENTION.............101 Conduct Interview s w ith Physicians .............................. ..................... 102 Focus Intervention on Specific Physicians ........... ....................... ....................103 D efin e C lear O bjectiv es............................................. ......................................... 104 E establish Credibility .......................... ......... .. ...... ............... 106 Stim ulate Physician Interaction .................................................... ............... ... 108 U se Concise Graphic Educational M materials ...........................................................109 H highlight and R enforce E ssentials ................................. ...................................... 110 Positive Reinforcement with Followup.......................... .......................... 11 S u m m a ry ........................................................................ ............... 1 12 B OA AD DESKTOP REM INDER ................................... ....................114 C THE THEORETICAL FOUNDATION FOR ACADEMIC DETAILING.............116 L IST O F R E F E R E N C E S ......... ................. ................................................................. 122 BIOGRAPHICAL SKETCH ...... ........ ................... ............................ 127 LIST OF TABLES Table pge 21 Summary of Included Studies for Thomson O'Brien and Grimshaw....................11 31 PS M odel Variable Descriptions and Abbreviations ................................................27 41 Descriptive Statistics for Continuous Variables in the PS Model............................37 42 Descriptive Statistics for Categorical Variables in the PS Model............................38 43 PrePS Univariate Analysis for Included Variables ......................................... 39 44 Physician D distribution by Quintile............................................... ........ ....... 40 45 Quintile Method Regression Analysis Results ....................................................40 46 Distribution of Influenza AD Participants by Propensity Score Quintile ................41 47 Regression on PS Method Analysis Results. ........................................................42 48 "Greedy Matching" Method Analysis Results. .................. ....................... 43 49 Quintile Method Results for Excluded Variable Models......................................45 410 Regression on PS Results for Excluded Variable Models. .....................................47 411 "Greedy Matching" Results for Excluded Variable Models ..................................47 412 Correlation Matrix Between VOC and PS Covariates...................... ..............48 413 Primary Outcome Model Results (Periods = 3,4,5,6). ...........................................52 414 Primary Outcome Model Results (Periods = 1,2). ................................................52 415 Least Square Means for Change in COX2 Rates by Group............................... 53 416 Unadjusted Means for Change in COX2 Rates by Group. ....................................53 417 Primary Outcome Model Results (AD = yes). ................................. ...............54 418 Primary Outcome Model Results (AD = no). .................................. .................55 419 Secondary Misoprostol Outcome Model Results (Periods = 3,4,5,6)......................56 420 Secondary Misoprostol Outcome Model Results (Periods = 1,2) ..........................56 421 Least Square Means for Change in Misoprostol Rate by Group...........................57 422 Unadjusted Means and Standard Deviations for Change in Misoprostol Rate by G ro u p ...................... .. .. ......... .. .. ...................................................5 8 423 Secondary Misoprostol Outcome Model Results (AD = yes)..............................58 424 Secondary Misoprostol Outcome Model Results (AD = no). ................................59 425 Secondary PPI Outcome Model Results (Periods = 3,4,5,6). .................................60 426 Secondary PPI Outcome Model Results (Periods = 1,2). ......................................61 427 Least Square Means for Change in PPI Rates by Group................................61 428 Unadjusted Means for Change in PPI Rate by Group.........................................62 429 Secondary PPI Outcome Model Results (AD = yes). ............................................63 430 Secondary PPI Outcome Model Results (AD = no) .................. ... .............63 431 Secondary H2A Outcome Model Results (Periods = 3,4,5,6). .............................64 432 Secondary H2A Outcome Model Results (Periods = 1,2). .....................................65 433 Least Square Means for Change in H2A Rate by Group. ......................................65 434 Unadjusted Means for Change in H2A Rate by Group ............................................66 435 Secondary H2A Outcome Model Results (AD = yes). .........................................67 436 Secondary H2A Outcome Model Results (AD = no).........................................67 437 Secondary GP Office Visit Model Results (Periods = 3,4,5,6) .............. ...............68 438 Secondary GP Office Visit Outcome Model Results (Periods = 1,2)......................69 439 Least Square Means for Change in GP Office Visit Rate by Group......................69 440 Unadjusted Means and Standard Deviations for Change in GP Office Visit Rate b y G ro u p ....................................................................... 7 0 441 Secondary GP Office Visit Outcome Model Results (AD = yes). .........................71 442 Secondary GP Office Visit Outcome Model Results (AD = no). ..........................72 443 Secondary Specialist Office Visit Model Results (Periods = 3,4,5,6). ..................73 444 Secondary Specialist Office Visit Outcome Model Results (Periods = 1,2)............74 445 Least Square Means for Change in Specialist Office Visit Rate by Group ............74 446 Unadjusted Means and Standard Deviations for Change in Specialist Office Visit Rate by Group.............. .. .. ...................... .. ......75 447 Secondary Specialist Office Visit Outcome Model Results (AD = yes) ................76 448 Secondary Specialist Office Visit Outcome Model Results (AD = no) ..................76 449 Secondary Hospital Length of Stay Model Results (Periods = 3,4,5,6)...................77 450 Secondary Hospital Length of Stay Outcome Model Results (Periods = 1,2)........78 451 Least Square Means for Change in Hospital Length of Stay Rates by Group........79 452 Unadjusted Means and Standard Deviations for Change in Hospital Length of Stay R ates by G roup............. .... .................................................... .. .... ........79 453 Secondary Hospital Length of Stay Outcome Model Results (AD = yes)...............80 454 Secondary Hospital Length of Stay Outcome Model Results (AD = no) ..............81 LIST OF FIGURES Figure page 21 Distribution of Propensity Score Article Objectives: 1987 to July 20, 2005...........19 31 Propensity Score Logistic Regression M odel .................................. ............... 28 32 Experim ental D esign Tim eline.......................................... ........................... 32 33 Primary Outcome Model for Between Group Effect ............................................34 41 Frequency of Influenza AD Participants by Propensity Score..............................42 42 Comparison of PS methods Ability to Reduce Bias on VOC................................45 43 Summary of PS Models Effects on Reducing Bias on the VOC ...........................48 44 Scatterplots of Propensity Score Versus Unbalanced Variables............................49 45 Line Graph Comparing Correlations and Percent Bias Reduction ........................50 46 Prim ary O utcom e M odel ......... ................. ..........................................................51 47 Least Square Means for Change in COX2 Rates by Group................ ......... 53 48 Unadjusted Means for Change in COX2 Rates by Group ....................................54 49 Secondary Outcome Model for Misoprostol Utilization.............................55 410 Least Square Means for Change in Misoprostol Rates by Group..........................57 411 Unadjusted Means for Change in Misoprostol Rates by Group..............................58 412 Secondary PPI Outcom e M odel ........................................ .......................... 59 413 Least Square Means for Change in PPI Rates by Group............... ...................61 414 Unadjusted Means for Change in PPI Rates by Group. ........................................62 415 Secondary Outcome Model for H2A Utilization ........................................... 64 416 Least Square Means for Change in H2A Rates by Group ..................................65 417 Unadjusted Means for Change in H2A Rates by Group. .......................................66 418 Secondary Outcome Model for GP Office Visits................... .......................... 68 419 Least Square Means for Change in GP Office Visit Rates by Group ....................70 420 Unadjusted Means for Change in GP Office Visit Rates by Group......................71 421 Secondary Outcome Model for Specialist Office Visits.......................................72 422 Least Square Means for Change in Specialist Office Visit Rates by Group............75 423 Unadjusted Means for Change in Specialist Office Visit Rates by Group .............75 424 Secondary Outcome Model for Hospital Length of Stay ...................................77 425 Least Square Means for Change in Hospital Length of Stay Rates by Group........79 426 Unadjusted Means for Change in Hospital Length of Stay Rates by Group............80 427 Secondary Outcome Model for Deaths Due to GI Complications........................81 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy EFFECT OF ACADEMIC DETAILING ON COX2 UTILIZATION RATES By Stephen Douglas Graham December, 2005 Chair: Abraham Hartzema Major Department: Pharmacy Health Care Administration Background: The prevalence of osteoarthritis (OA) is estimated at 50 to 80 % of the elderly population and therapy aims to relieve symptoms since there is no cure. Nova Scotia general practitioners (GPs) identified a need for an academic detailing (AD) intervention aimed at optimizing the management of OA. Objectives: The primary objective was to measure the effect of an OA AD intervention to reduce the utilization rate of COX2 inhibitors in the elderly population. Secondary objectives were to examine the intervention effect on the utilization rates of gastroprotective agents and medical services. Methods: We conducted a retrospective cohort study employing administrative data to examine the effects of the intervention. Differences in utilization rates were evaluated using generalized estimating equation (GEE) analysis for longitudinal data. Selection bias was anticipated since the intervention was voluntary, and randomization not possible. Three methods of propensity score (PS) analysis (quintile stratification, regression on the PS, and "greedy matching") were evaluated for the ability to adjust for bias on PS model covariates. Findings: We identified a significant difference in the change in COX2 utilization rates between groups for the three month period following the intervention (p = 0.0395, 95% CI (0.0365, 1.4815)) and a significant decrease in the intervention group's within group utilization rate between the pre and post intervention periods (z = 2.34, p = 0.0191). The GP office visit rate was the only secondary outcome where the intervention group was significantly higher (p = 0.0275, 95% CI (0.7926, 0.0464)). The difference occurred in the time period from three to six months post intervention. Conclusions: The OA AD intervention was associated with a significant decrease in COX2 utilization rates in the three month period immediately following the intervention. The effect of decreased utilization continued for the rest of the post intervention periods but was not statistically significant. The only secondary outcome to show a significant between groups effect was the GP office visit rate which was higher for the intervention group in the second three month post intervention time period. CHAPTER 1 INTRODUCTION Background In June 2002, the Division of Continuing Medical Education (CME), Dalhousie University Faculty of Medicine, began their second academic detailing (AD) intervention with provincial physicians aimed at optimizing the care of osteoarthritis (OA) within the seniors population (persons greater than 65 years of age). The AD program is an ongoing initiative funded by the Nova Scotia Department of Health and managed by the Drug Evaluation Alliance of Nova Scotia (DEANS). As the AD program is a continuing effort and represents a significant cost to DEANS it is necessary to evaluate the effectiveness of the intervention. The OA topic was chosen as an AD intervention based on the extent to which OA affects the elderly population and on the feedback that Dalhousie CME received from general practitioners (GPs) in a survey filled out following the previous influenza AD intervention which indicated the GPs' desire to have an OA AD intervention developed. The Dalhousie CME Division then presented the OA topic to a GP focus group where the need for education pertaining to available OA therapies was determined. OA is a progressive disease that affects the joint cartilage and eventually leads to joint failure.3 The prevalence of OA in the population is extremely high. It is estimated that 50 to 80% of the elderly population experience symptomatic OA.4 Estimates specific to the province of Ontario, propose that almost all persons over the age of 65 exhibit signs of OA on radiographic evidence and of these 33% are symptomatic.5 OA is equally prevalent in men and women, with women showing more manifestation in the knees and hands and men more prevalent in the hip. Arthritis has been associated with half of all disability in the elderly population.4 There is no known cure for OA3 and available palliative treatments are associated with substantial toxicity and side effects.4 Treatment is therefore primarily aimed at reducing pain, improving joint mobility, and limiting functional disability. Patient education regarding medications used in the treatment of OA (primarily for the control of pain) and appropriate exercise regimens is also important.3' 4 The OA AD intervention has set four learning objectives. Each physician visit will include at least the following: (1) a discussion of the goals of therapy, (2) recommendations for nonpharmacological treatments when appropriate (e.g., physio therapy and exercise), (3) advice for patients about the safety and efficacy of acetaminophen, and (4) a discussion of the role of traditional nonsteroidal anti inflammatory drugs (NSAIDs).6 The primary interest of this research dealt with the fourth message specifically, the analysis of the effectiveness of the OA AD intervention as it pertains to the pharmacotherapy of OA and in particular the usage of COX2 inhibitors. The Nova Scotia OA AD was developed in 2002 and the intervention called for the use of acetaminophen as a first line therapy for mild to moderate OA. The intervention suggested that if acetaminophen did not control pain symptoms, then the use of traditional NSAIDs in as low a dose as possible and for as short duration as possible was indicated. NSAIDs were considered appropriate therapy for moderate to severe OA.6 The role of COX2 inhibitors in the management of OA was assessed by the OA AD group as controversial. The Ontario Treatment Guidelines for OA recommend that, based on evidence of similar efficacy and early evidence of somewhat lower rates of serious GI events, selective COX2 inhibiting NSAIDs can be considered for patients at high risk of serious GI events.3 This recommendation, however, is one that is from well designed, randomized controlled trials or metaanalyses with inconsistent results or demonstrating equivocal benefit.3 The Nova Scotia program states that, the precise role of COX2 inhibitors in the treatment of OA remains to be determined.6 The summary statements in the OA AD intervention6 relay two points that are relevant to this analysis. Firstly, COX2 inhibitors are as effective but not more effective than traditional NSAIDs for symptomatic treatment of OA and secondly, the CLASS7 and VIGOR8 trials were inconclusive in the analysis of the gastroprotective effects of COX2 inhibitors. When faced with the substantial effect of OA on the population,6 the uncertain role of COX2 inhibitors in the treatment of OA, the increased cost of COX2 inhibitors over the traditional NSAIDs (appendix c), and the utilization rate of COX2 inhibitors in the Nova Scotia pharmacare population of approximately 6% in 2001,9 the DEANS Management Committee undertook to develop the AD intervention on OA Management. Problem Statement The effect of AD on clinical and economic outcomes is of great interest to the Nova Scotia government's policy makers as funding for interventions to improve the health care system is scarce. This research addresses the question of whether the AD program on OA is effective in lowering the utilization rate of COX2 inhibitors. At the same time the study measures the effects that the program has on the utilization rates of other healthcare resources such as hospital or physician visits that occur as a result of GI side effects associated with drug therapy with traditional NSAIDs and COX2 inhibitors. The efficacy of traditional NSAIDs and COX2 inhibitors in relieving pain is similar but the GI side effects profile for traditional NSAIDs is higher.10 It is expected that the intervention could increase the utilization rates of gastroprotective agents (particularly misoprostol and proton pump inhibitors (PPIs)) but it is not expected to increase other health care utilization rates and will therefore not have negative impacts on the outcomes of care. The methodological challenge for the evaluation of the OA AD intervention is the need to significantly adjust for selection bias that is likely present since GPs can choose to participate and those that do participate might be different from those that do not participate. Statistical adjustment through regression on the propensity score (PS) methods have been shown to be effective in reducing between group biases on many confounding variables.11, 12 The use of PSs in studies that examine the unit of analysis other than the patient is uncommon in the medical literature. In this study the unit of measure was the GP. No other studies with the GP as the unit of measure were found in the medical literature so the evaluation of different PS method's ability to adjust for bias between GP groups was warranted. Research Questions and Hypotheses The term statistically significant is defined as results where the type I error (alpha) is less than 0.05. The results are statistically significant if the analysis yields pvalues less than 0.05. Hypotheses relating to research questions one to three are examining the effect of the OA AD intervention in the Nova Scotia residents who are greater than 65 years old and have a GP who has participated in the intervention as compared with GPs in the province who did not participate in the intervention. The first research question examined the expectation that GPs will consider the information provided in the OA AD intervention and choose not to prescribe COX2 inhibitors for their elderly patients. Research Question 1 Do the patients of GPs who have undertaken the OA AD intervention have significantly lower COX2 inhibitor utilization rates after the GP has undergone the AD intervention as compared to a GP control cohort? (Are there significant between group differences?) Research Question 1 Hypothesis The null hypothesis is that the OA AD intervention will have no effect on the utilization rate of COX2 inhibitors. The alternative hypothesis is that the OA AD intervention will have the effect of decreasing the utilization rate of COX2 inhibitors. The second research question examined the sustainability of the intervention (if research question 1 hypothesis is found to be significant) since a shortcoming of the OA AD intervention (appendix a) is the lack of a followup visit to GPs who participated in the intervention.13 Research Question 2 Does the decreased utilization rate of COX2 inhibitors for patients of GPs who have taken the AD intervention remain significant for a period of oneyear post intervention? (Is the intervention effect sustainable?) Research Question 2 hypothesis The null hypothesis is that the OA AD intervention will not have a sustained effect on the decreased utilization rate of COX2 inhibitors. The alternative hypothesis is that the OA AD intervention will have a sustained effect on decreasing the utilization rate of COX2 inhibitors. The third research question examined whether patients of GPs in the intervention group experienced a change in the rate of medical services utilization due to a change in GI adverse events associated with traditional NSAID therapy (if there was a significant finding to research hypothesis 1). The hypothesis is divided into two categories: those that are related to pharmacotherapy and those that involve other medical services. Research Question 3 Do patients of GPs who have undertaken the OA AD program have medical utilization rates associated with their OA that are significantly different from patients of GPs who have not participated in the intervention? Research Question 3 hypotheses The null hypothesis is that the OA AD intervention will have no effect on the utilization rate of (1) PPIs, (2) H2As, (3) misoprostol (4) GP office visits, (5) specialist office visits, and (6) death rates. The alternative hypothesis is that the OA AD intervention will have the effect of changing the utilization rate of(1) PPIs, (2) H2As, (3) misoprostol (4) GP office visits, (5) specialist office visits, and (6) death rate. The fourth research question examined whether one PS adjustment method was more successful adjusting for bias between groups based on measured bias reduction for covariates that were not balanced after group assignment and the resulting sample size after PS methods were applied. Research Question 4 Is there a superior PS method for the reduction of selection bias between the intervention and control groups? Research Question 4 hypotheses The null hypothesis is that there will be no difference in the three PS method's (quintile stratification, regression on the PS, and "greedy matching") ability to adjust for bias on unbalanced covariates. The alternative hypothesis is that one PS method will adjust for bias on unbalanced covariates to a greater extent than the other two. Significance of Research This research is of significance to several groups within the healthcare system. The three groups that benefit directly from the research are patients, physicians, and health policy decisionmakers. The results also add to the academic research in the area of effective behavioral change methodology and it adds to the methodology and understanding surrounding the use of PSs. The largest impact of this research is in the area of health policy decision making. The decision to proceed with one course of action is often at the expense of others. This study will inform decision makers regarding the effectiveness of the OA AD intervention and allow them to make a more informed decision to continue with the AD detailing program to educate physicians on other health related topics or disease states. This research adds to the validity of the research that has been accumulated in the area ofAD. This is significant as it was concluded by Davis et al. in a systematic literature review of AD that while AD is effective it is seldom used by providers of continuing medical education.14 The uniqueness of this research lies in its analysis of a population based continuing AD program and not one that has been developed for the purposes of a single study. This research advances PS methodology. It compares three PS methods in a real world and population based intervention. The results should contribute to the choice of PS methods employed by future researchers. The study also analyzes each of the propensity score method's ability to balance the control and intervention groups on unmeasured administrative variables. The ability of the propensity score methodology to balance groups on measured variables has been widely reported; however the ability of the methodology to balance unmeasured variables is assumed 15, 16 and studies attempting to measure the ability of the PS method to balance physician groups on a number of unmeasured administrative variables were not found in the literature. CHAPTER 2 LITERATURE REVIEW Literature reviews were conducted on two areas of interest: articles dealing with studies relating to AD interventions which have not shown statistical significance and articles relating to the use of PS methods. The AD studies which reported no statistically significant effects of AD interventions are of interest because they possibly give examples of shortcomings of methodology that may be of use in this study. The PS articles that are of interest to our study are those which involved studies that identified some unit, other than the patient, as the unit of analysis in the PS development and articles that dealt with PS methods. The positive effect of AD on prescribing behavior has been summarized in a number of review articles on AD or educational outreach.14, 17,18 This body of evidence shows that AD moderately improves physician behavior and patient outcomes. Three review articles are summarized. Review Articles Addressing Effects of Academic Detailing Davis et al.14 reviewed 99 studies which met their inclusion criteria from a total of more than 6000 articles. The 99 studies included 160 separate continuing education interventions, including academic detailing. Sixtytwo percent of the interventions showed improvement in at least one major outcome with effect sizes ranging from small to moderate (quantified effect sizes not provided). There were fourteen AD interventions in the category of prescribing and 75% of these showed positive effects. AD was reported as an effective change agent for prescribing. The authors concluded that AD is an effective strategy for continuing medical education (CME) however, it is not widely used by CME providers. Thomson O'Brien et al.18 conducted a systematic review of the effect of educational outreach on professional practice and health care outcomes. Eighteen studies were included in the review with thirteen of the studies targeting prescribing practices. Nine of the thirteen studies employed multifaceted interventions (educational outreach combined with reminders, audit and feedback, marketing, or patientmediated interventions). Seven of the nine studies using multifaceted interventions showed statistically significant effects with relative effects ranging from 1 to 45% improvement (table 21). The authors noted that potential bias exists in thirteen of the eighteen studies due to lack of randomization and six of the studies contained potentially important baseline differences and adjustment for these differences was not carried out in the statistical analysis. It was also noted that only one of the eighteen studies considered patient outcomes. The authors concluded that the effects of educational outreach are small to moderate but potentially of practical importance. Grimshaw et al.17 conducted a systematic review of the effectiveness and costs of different guideline development, dissemination, and implementation strategies. 235 studies representing 309 comparisons were included in the review. The sections of the review that are germane to our study are the multifaceted comparisons involving academic outreach with continuous measures for process or outcome variables. Ten comparisons were reviewed which contained measures on continuous variables. Six of the comparisons involved process measures (five cluster randomized control trials and one controlled before and after trial) and all reported improvements in performance with a median effect of 15.0% (range 1.7% to 24.0%) relative improvement. None of the studies included enough information to calculate standardized mean difference, and two studies were not statistically significant. Four of the comparisons involved outcome measures (three cluster randomized control trials and one controlled before and after trial). The median effect of the cluster randomized control trials was 0% (range 1.4 to 2.7%) and the standardized mean difference was calculated as 0 for one trail. The controlled before and after trial reported a relative improvement of 13.9% with a standardized mean difference of 2.38. The authors summarized the multifaceted interventions, including academic outreach, to be at best moderately effective (table 21). Table 21. Summary of Included Studies for Thomson O'Brien and Grimshaw. Author (year) Reviewed by Interventions (plus AD) Relative Effect (%) McConnell Thomson O'Brien Audit and Feedback (AF), 45.8 (1982) Educational Material (EMat) Stergachis Thomson O'Brien AF, Patient Mediated (PM), 35.7 (1987)* Conferences Meador (1997) Grimshaw EMat, Educational Meeting 24.0 (EMeet) RossDegnan Thomson O'Brien EMat, Social Marketing 21.0 (1996) (SM), PM Peterson (1996) Grimshaw EMat 20.0 Avorn (1983) Thomson O'Brien EMat, SM 15.2 Avorn (1992) Thomson O'Brien, EMat, SM, Conferences 15.0 Grimshaw Ray (1993) Grimshaw EMat, EMeet 13.9 de Burgh Thomson O'Brien EMat, PM 13.0 (1995)* Diwan (1995) Grimshaw EMat 11.3 Steele (1989) Thomson O'Brien Reminders 11.2 Santoso (1996) Thomson O'Brien EMat, SM 8.7 Schmidt (1998) Grimshaw Organizational Change 5.5 Elliott (1997) Grimshaw EMat, Opinion Leaders 2.7 Feder (1995) Grimshaw AF 0.0 Moore (1997) Grimshaw EMat, Reminders, PM 1.4 nonsignificant study results Academic Detailing Studies Reporting No Statistically Significant Effect Five articles were reviewed in which the authors reported nonsignificant results for AD interventions with pharmacotherapeutic outcomes. The review of results that were not positive is important because it will possibly indicate to investigators methodological similarities that may have been employed in previous unsuccessful studies. If identified the methodological shortcomings could be avoided. Lin et al.19 studied the effects of physician training on the management of depression. The study was a before and after design with an equivalent control group. The physician sample was made up of 109 primary care physician volunteers and they were associated with fifteen primary clinics. Randomization of groups was at the clinic level resulting in 56 physicians in the intervention group and 53 physicians in the control group. The intervention was outlined including the four key messages and the use of opinion leaders in intervention delivery.20 Case managers were used for followup visits with the physicians. The intervention involved other components such as small group discussions, roleplay and psychiatric consults. The authors reported that the physicians in the intervention arm of the trial did not differ significantly from the control group in adequacy of pharmacotherapy (p=0.53). While insignificant, the results showed a decrease of 7.5% in the percent of patients in the intervention group who received adequate pharmacotherapy with no change in the control group. The decrease in the intervention group is opposite to the desired outcome of the intervention. The study also failed to show significant differences in the number of antidepressant prescriptions per 100 patients (p=0.10). The percent of patients receiving new prescriptions in the intervention group decreased by 10.4% and increased in the control group by 4.8%. These results are opposite to the desired outcome of the intervention. The authors reported that the study's main failure was its lack of power to detect a significant change between groups. The sample size used was sufficient to detect a 40% to 50% difference in adequate pharmacotherapy and a 15% to 30% difference in new antidepressant prescriptions. The fact that the effect of the intervention was the opposite of the hypothesis was not explained by the authors. Brown et al.21 studied the effect of AD and continuous quality improvement (CQI) interventions on the treatment of patients for depression. The study was a randomized controlled trial. The primary care clinician groups were randomized by first matching clinicians according to specialty (internal medicine or family practice), sex, training (physician or allied health clinician), and number of patients in a highrisk depressive cohort. The resulting sample size was 160 with 79 in the intervention arm and 81 in the control arm. The AD intervention involved focus groups for the collection of baseline knowledge of primary care providers (physicians, physicians' assistants, and nurse practitioners) in preparing the intervention. The intervention was based on guidelines from the Agency for Health Care Policy and Research and used the same material as the Goldberg study.22 Three main messages were summarized on letter sized illustrated handouts. Four visits were used to deliver the message and the detailers were pharmacists from the clinicians' own medical office. The study showed mixed results. It was successful in increasing the percent of patients receiving antidepressant treatment (7.5% increase, p=0.046 in depressed arm and 0.7% increase, p=0.025 in the non depressed population) however, it was not successful in increasing the total days of antidepressant therapy (16.7 days effect, p=0.189 in the depressed arm and 1.3 days effect, p=0.606 in the nondepressed population). The study did not exhibit significant differences in nonpharmacotherapeutic outcomes (improvement of symptoms and measures of functional status). The authors report that the mixed findings could be due to the complexity of the implementation of a clinical guideline and the evidence base for the guidelines may not be generalizable to the study population. They propose that AD may be appropriate for behavioral change but is not sufficient for the implementation of clinical guidelines. This conclusion is important for our study since the primary outcome is change in prescribing behavior. Goldberg et al.22 studied the effect of AD and CQI interventions on compliance with guidelines for hypertension and depression. The study was a randomized before and after design with two experimental groups (AD only and AD combined with CQI) and an equivalent control group. The physicians were part of fifteen clinics and group randomization was carried out at the clinic level. The resulting sample size was 78 with 18, 37 and 23 physicians in the AD only, AD combined with CQI and usual care groups respectively. The AD intervention was based on national guidelines for hypertension and depression from the Agency for Health Care Policy and Research. Five recommendations were developed including two which specifically addressed pharmacotherapy. The AD intervention was delivered by opinion leader physicians and followup visits were conducted by staff pharmacists. The intervention was supported by handouts and pocket cards for quick reference. The study found significant effect in only one of the pharmacotherapeutic outcomes which was a decrease in the prescribing of 1st generation antidepressants to previously diagnosed depressed patients (relative effect  4.7%, p=0.04). The other outcomes prescribing of hypertension medications, antidepressants to previously undiagnosed patients, 2nd generation antidepressants to previously diagnosed depressed patients, and SSRIs to previously diagnosed depressed patients exhibited insignificant change with relative effect sizes and pvalues of 1.3%, p=0.06; 2.4%, p=0.68; 2.1%, p=0.43; and 3.3%, p=0.11 respectively. One possible explanation for the failure of the study to show significant effect for all but one of the pharmacotherapeutic outcomes can be attributed to the presentation of too much information. A successful AD intervention should include only a limited number of messages regarding a disease state.13 The presentation of an intervention covering two distinctly different disease states clearly violates this principle. Zwar et al.23 studied the effect of AD on prescribing rates of benzodiazepines for all indications. The study was a before and after design with an equivalent control group. There were 157 physicians who participated in the study. They were randomized into the benzodiazepine AD group (n=79) and the control group who received AD on another topic (n=78). The AD intervention was based on guidelines developed by the Royal Australian College of General Practitioners and it was delivered by physicians trained in AD techniques. The intervention was not accompanied by any other methods (i.e. handouts, etc.). The study found significant effect in overall prescribing of benzodiazepines (26.7%, p=0.042) however, there was no significant between group relative effect (1.2%, p=0.99). The authors attributed the lack of significant results to the effects of a preintervention practice survey that was given to all physicians in the study and a lack of power to detect a difference between groups due to the decision to aggregate data into eight subgroups thereby reducing the sample size dramatically. Tomson et al.24 studied the effects of AD on physicians' practice in the management of asthma and on patient knowledge. The study was not randomized and the sample consisted of 63 GPs in two regions, one region was assigned as the treatment group (n=44) and the other region was assigned as the control group (n=19). The intervention was developed using existing physician knowledge as the baseline and the input of respirologists. It was delivered by a clinical pharmacologist and a pharmacist and contained three main messages. The facetoface visits with physicians were augmented with written materials. The study found that there was not a significant difference between the treatment and control groups in prescribing ratios of betaagonists and inhaled corticosteroids (no pvalue reported). One explanation for the insignificant results could be attributed, at least in part, to insufficient power (due to the small sample size) to detect a meaningful change. The authors identified a possible selection bias in the physicians volunteering for the intervention as they may have been largely physician interested in asthma therapy to begin with. It is important to note that a review of the negative findings of studies in the literature cannot be considered to be complete since many studies, and their fatal flaws, are not published if they are not considered to be methodologically sound or clinically important (publication bias). However, from the review of literature that did not report significant results there are four areas of inadequacy that the studies appear to have in common; * the authors reported that there were insufficient sample sizes to yield enough power to show a meaningful change in the studies conducted by Lin19, Gorins25, Zwar23 and Tomson24, however in all but one of the studies19 the effect of the intervention was consistent with the study hypothesis. It is important to note that lack of power is only one explanation for the lack of study significance, * there were intervention development problems in that the interventions were too complex21 22 * the interventions may have been compromised through the use of less than credible academic detailers19' 25, and * the use of pretests or preintervention surveys decreased the intervention effect due to a presensitization of the subjects to the intervention.23 25 The results from the above studies are applicable to our study for the following reasons. The lack of power reported by a number of the studies is only one explanation. Other explanations could include a large variation in measurement on the dependent variable or a lack of control for the variables that are associated with the outcome variable. For example, in our study we could have a large sample but if the number of elderly patients in the GP's panel is not controlled for then the variation could be inflated and a nonsignificant result could occur. In our nonrandomized study design it is important to adjust for variables which are associated with the outcome but it must be acknowledged that there will be variables which are important confounders and are not measured so residual confounding (bias) will exist. There may be a need to adjust for patient variables as well. For example, if the GP's patient panel is markedly ill then this will confound the results. A measure of patient wellness would help to address this problem. The lack of a followup visit to GPs in our study may play an important role in the outcome. Propensity Scores A literature search was conducted using PubMed for all years up to and including July 20, 2005. The search terms used were "propensity score" and "propensity scores". The search yielded 341 articles. The abstracts for all 341 articles were reviewed and the distribution of articles by article objective and year is illustrated in figure 21. The distribution shows an initial surge of articles dealing with PS methods in the late 1990's with articles containing objectives other than medical (e.g., economic) and only a few articles with stated medical objectives. Since 2000 there has been a surge in published articles using PSs particularly in the field of cardiology. The increase in use of PSs has been mirrored by an increase in published articles dealing with PS methods. There were four articles which used a unit of analysis for PS other than the patient. Two of the articles used human couples as the unit of analysis one article developed PS on hospitals and one article used communities as the unit of analysis for the PS. There were no articles found which used the physician as the unit of analysis in the PS analysis. There were 50 articles that described PS methods and 24 of these were selected for further review. Criteria for selection included PS studies using GEE for outcome models, studies comparing small experimental groups, studies describing PS and sample sizes or studies which described PS methods in detail. The information gained from these articles plus reference material from previous course work, library searches, and colleagues formed the basis for the PS method as it has been applied in this study. In our study the PS represents the probability of a physician volunteering for the OA AD intervention given a number of personal and practice characteristics. For studies using quasiexperimental designs it is important to include methods to compensate for the lack of randomization to experimental groups. In our study we have made multiple measures of outcome variables both before and after the intervention and we have included a control group for comparison. The control group is not equivalent to the intervention group so adjustment on PSs was used to reduce the effect of the between group bias. Three methods for applying PS in observational studies are predominant in the literature.26 The three methods are; sub classification on the PS11 12,27, regression on the PS12 and matching on the propensity score using Mahalanobis metric matching12' 27 or "greedy matching" techniques.28 All three of the methods; stratification, regression on the PS, and matching have been applied successfully in observational studies and therefore all three will be considered for application in this research. Distribution of PS Article Objectives S40 Jl ?0 20 # of Articles 10 5 0 E nonmedical o methods * medical * cardiology Figure 21. Distribution of Propensity Score Article Objectives: 1987 to July 20, 2005 An overarching limitation of all three PS methods is that the PS can only adjust for bias in observed covariates27 and the extent to which the bias is abated in unobserved covariates depends on the correlation of the unobserved covariate with one that is observed.11 Shadish stated that if the PS method was successful in abating bias in the measured covariates then the assumption can be made that the methodology would be successful in decreasing the bias in unmeasured covariates as well.16 2 %%t 1 2001 Year 2000 1999 1998 8797 Lo S \0 Objective A recent study has tested the ability of PS based on covariates extracted from administrative data to reduce the bias in unmeasured clinical variables. In this study the clinical data was extracted from patients' charts after PS methods were applied. The experimental groups set by the PS method were tested for significant difference on the clinical variables and it was found that the clinical variables were not balanced between the groups.29 Other studies have explored the number of events per variable that are needed for logistic regression analysis to outperform the PS method. Cepeda et al. reported that in their simulation model if there are six or fewer events per independent variable (covariates in the PS model) then the PS estimates are less biased then the regression estimates.3 It is important to note that even if the number of subjects exceeds six the use of PS methods is warranted since it is a variable which predicts the exposure of interest without including the outcome11 and that the use of PS methods is intended to complement modelbased procedures not replace them.31 There are two measures of PS model fit that are reported in studies. The cstatistic is the area under the receiver operating characteristics (ROC) curve and is a measure of the discriminative ability of the PS model.32' 33 The range of the statistic is from 0.5 to 1.0. If a model has a cstatistic of 0.80 this can be interpreted as the model accurately assigning random pairs of subjects to their experimental groups based on PS alone 80% of the time. The cstatistic is intended to be an indicator in the model building process but it is not a measure of the PS model's ability to adjust for bias15 and it has not been found to be associated with the ability of a PS model to reduce residual confounding.32 The goodness of fit is another statistic that is commonly used in regression analysis. Like 21 the cstatistic these tests were not found to be useful in predicting the ability of the PS model to reduce residual confounding.32 As a result, these measures were not used in our study to decide which PS method to use for the outcomes analyses. The cstatistic was however, used to explain the effects on the model's discriminatory ability when variables were intentionally removed from the PS Regression model. CHAPTER 3 METHODS This study is a retrospective cohort, before and after longitudinal design with a nonequivalent control group using the Nova Scotia Medical Services Insurance and the Canadian Institutes of Health Information datasets for analysis. The nonequivalent control group design requires the use of procedures to abate selection bias in the treatment group.12 The methodology for the study can be broken down into four distinct sections, which are as follows; * the extraction and validation of data from the administrative databases, * the establishment of balanced control and experimental groups using three distinct PS methods, * the primary outcome analysis of the intervention effect on the utilization rate of COX2 inhibitors and, * the secondary outcome analysis of the intervention effects on the utilization of PPIs, misoprostol, and H2As. Step One: Extraction and Validation of Data Sources of Data All of the data used in this study was collected in preexisting administrative databases. There were no occurrences of missing data since the variables included in the analysis were extracted from long standing registrar data which is complete for all fields listed in the registry1 (GP demographics), complete census information2 (geographic data) or the data was reported in terms of rates with the GP inclusion criteria ensuring that each GP panel contained at least twenty patients so the rates for the outcomes measures were always defined (i.e. rate denominators were not equal to zero). Administrative data must be used with caution as it is not 100% reliable. Chapter five outlines the limitations of the administration used in our study. GP demographic data for all GPs in the province was obtained from the Nova Scotia College of Physicians and Surgeons Physician Registry (2002).1 The Dalhousie CME Division provided data which contained demographic information of the GPs who were detailed and the dates when the detailing visits were carried out. These two sources of data were merged and the resulting file was submitted for encryption using the same encryption methods as the provincial administrative data. The resulting encrypted GP demographic profiles were augmented with data from the Nova Scotia Medical Services Insurance (MSI) physician registry (2002) to include dates indicating when the GPs opted in and opted out of the provincial pharmacare billing scheme. GP practice information such as population of the community and average income of the county in which the practice is located was added to the demographic profile of each GP.2 Patient data was extracted from the Nova Scotia Pharmacare Seniors Dataset (20022004) and the hospital discharge data found in the Canadian Institute of Health Information (CIHI) hospital discharge dataset (20022003). Patient level GP visit data was used to determine to which GP's patient panel a patient belonged (see patient inclusion criteria). Once the patients were assigned to GP panels the patient prescription claims data and hospital length of stay data were aggregated at the GP level. Drug utilization variables were created at the GP level with the unit of measure equal to DDDs per elderly patient per 90 day study period. Change in utilization rate variables were created for each GP by subtracting each period utilization rate (period = 1 to 6) from the baseline (period two) utilization rate. Period two was chosen as the baseline utilization rate since it was the preintervention measure most proximal to the GP index date. Descriptive statistics for the GP demographic variables were calculated to confirm that the variables did not contain any missing data and to confirm that the variables fell with acceptable ranges (i.e. no GPs 200 years old, not all male GPs). The descriptive statistics are reported in tables 41 and 42. Prescription claims, GP visits and vital statistics were checked to ensure that there were not instances of missing data. The prescription claims and GP visit data were complete on all fields necessary for our study. Only hospital admissions and deaths due to GI events were included in the hospital length of stay and death measures. A detailed description of the inclusion criteria for data is contained in chapter four. The underlying and primary causes of death were used to determine death rates and cause of death and data was reported for all included patients who died over the study period. The first four diagnoses codes for hospital admission were used to determine if GI complications were associated with admission. In all cases there was at least a primary diagnoses on admission. While the data for our study was complete it was administrative data and there are shortcomings associated with it. The limitations of administrative data are described in chapter five. Data from several administrative databases was linked to create the datafile necessary for the PS analysis and for the outcomes analysis. The data linkage was carried out using the encrypted physician identifiers and the encrypted patient identifiers. The encryption of the patient and physician identifiers was carried out according to standards set by the Canadian Institutes of Health Information (CIHI).34 GP Inclusion Criteria The academic detailing intervention was targeted at GPs and, therefore, the experimental unit is the GP and the patient data for the GP's practice is the unit of measure. Each GP's practice is measured as an aggregate of the individual patient's data from his or her practice. The aggregation of patient data is described in greater detail later in this chapter. The date on which the GP received the OA AD intervention was defined as the index date. For GPs in the control group the indexdate was randomly assigned from the time period over which the AD intervention took place. There are four criteria that a GP had to meet to be included in the study. They are as follows; * The GP had to be registered as a GP with the Nova Scotia College of Physicians and Surgeons for the entire study period. * The GP had to be included on the billing registry with the Nova Scotia Medical Services Insurance (MSI) (the provincial government payment agency for seniors' medical and pharmacy claims) for the entire length of the study. This registry is the source of the medical and pharmacy claims data that will be used for the outcomes analysis. * The GP had to have an elderly patient panel equal to or greater than twenty patients. The rational for the cut score of twenty was based on the premise that a 5% decrease in COX2 utilization (i.e. COX2 utilization rate change from 6.0% to 5.7%) will equate to annual savings to the elderly population of approximately $100,000. Therefore, if the GP had an elderly patient panel equal to twenty he or she was required to change prescribing behavior for one patient over the study period to realize a 5% change. * The GP had to have at least one prescription claim for a COX2 inhibitor recorded in the preintervention period (6 months preceding the GP's index date). Patient Inclusion Criteria The patient is the unit of measure for this study. Patients had to meet two criteria for inclusion in the study. The criteria are: * The patient had to be included on a GP's patient panel. For inclusion on a GPs panel the patient must have seen a specific GP for more that 50% of his or her total GP visits for the fiscal year ending March 31, 2002. For example, if a patient had a total of forty GP visits in the period from April 1, 2001 to March 31,2002 and twentyfour (60%) of the visits were billed by one GP the patient was included on the GPs patient panel. Once the patient was assigned to a particular GP they remained with that GP throughout the study. * The patient had to be 66 years of age or older as of the GP's AD index date. This ensures that the patient was at least 65 years old and eligible for the MSI pharmacare coverage for the entire study period and it provides a period of time of at least six months for the patient to become accustomed to the new MSI pharmacare coverage. Step Two: Adjustment for Confounding Using Three Distinct Propensity Score Methods The definition of the PS is the conditional probability of treatment given the individual's covariates. In this case it would be the conditional probability of taking the OA AD intervention given the GP's personal and practice characteristics. The PS is obtained by fitting the data using a logistic regression model.5 Once the PSs were calculated for each GP three PS methods were applied to the PS data and the optimal method in term of bias reduction and resultant sample size was determined. The three PS methods used in this study were; the stratification into quintiles, regression on the propensity score12, and "greedy matching" or onetoone matching for group assignment.28 The variables in the regression model describe the GPs' personal characteristics (age, sex, birthplace, etc.) and practice characteristics (size of patient panel, population of community in which the practice is located, etc.). All variables in the data that fit within these two descriptive categories were included in the regression model. This approach is consistent with the literature which calls for the inclusion of all variables which have some relevance to the outcome variable.16 A description of the included variables and the abbreviations used in our study are included in table 31. Table 31. PS Model Variable Descriptions and Abbreviations Variable Description # Levels Abbreviation GP participation in the OA 2 (Y/N) OA ADintervention GP participation in previous 2 (Y/N) flu AD influenza AD service GP's sex 2 (M/F) sex GP's birthplace 3 (Nova Scotia, birth place Canada, Other) GP's location of initial licensure 5 (Nova Scotia, license Canada East, Canada Center, Canada West, Other) GP's COX2 utilization rate at continuous BL rate baseline (DDDs / patient) GP's age (years) continuous GP age population size of community in continuous population which GP's practice is located average income of county in which continuous aver income GP's practice is located ($cdn) number of patients in the GP's continuous total # pt practice percent of GP's patients diagnosed continuous % OA dx with OA (ICD9 CM = 715) percent of GP's patients > 65 years continuous % elderly old average hospital length of stay for continuous los rate elderly patients in the GP's practice (days/patient) A logistic regression model was used to accommodate the dichotomous nature of the outcome variable, OA. The same regression model was applied using PROC REG (SAS 8.2)35 for all three methods to determine GP PSs. Models described in this study have categorical variables listed as single entities which are consistent with the SAS coding techniques. The model analysis creates (t1) dummy variables (where t = the number of levels) for each categorical variable. The PS regression model is shown in figure 31. Y = a + P1X1 + 02X2 + 03X3 + P4X4 + 5X5 + P6X6 + 7X7 + 8sXs + 89X9 + P1oXio + +11X11 + P12X12 Where; Y GP participation in the intervention (0 = no, 1 = yes), Xi GP participation in previous influenza AD service (0 = no, 1 = yes), X2 GP's sex (Male, Female)', X3 GP's birthplace (Nova Scotia, Canada, Other)1, X4 GP's location of initial licensure (Nova Scotia, Canada East, Canada Center, Canada West, Other)', X5 the GP's COX2 utilization rate at baseline (DDDs / patient). X6 GP's age (years)1, X7 population size of community in which GP's practice is located2, Xs average income of county in which GP's practice is located ($cdn)2, X9 number of patients in the GP's practice, Xi1 percent of GP's patients diagnosed with OA (ICD9 CM = 715), Xii percent of GP's patients > 65 years old, Xi2 average hospital length of stay for elderly patients in the GP's practice (days / patient). Figure 31. Propensity Score Logistic Regression Model Variables were kept in the model regardless of their significance. Variables that are not statistically significant still contribute to the model and the population based nature of the data ensures a large enough sample size to support the model with twelve predictor variables. The final model predicts the probability that each GP would receive the intervention based on his or her individual variables. This probability is the GP's PS. Once the PSs were calculated they were applied according to the three methods stated earlier. Quintile Propensity Score Method For the quintile method; the GPs in the treatment and intervention groups were stratified, based on their participation in the OA AD intervention, and then ordered according to the GP's PS. The treatment and control groups were stratified into five levels, or quintiles. Each quintile contains 20% of the GPs (table 44). Regression on the Propensity Score Method For the regression on the PS method; the PS was used in the outcome model. "Greedy Matching" Method For the "greedy matching" method; the GPs in the treatment and intervention groups were stratified, based on their participation in the OA AD intervention, and then ordered according to the GP's PS. A matching procedure was applied28 that involved matching the groups on PS beginning with matches accurate to five decimal places and concluding with matches to one decimal place. The number of included GP only allowed for a onetoone match between groups. Once matched the GP was removed from the sample pool. Those GPs that were not matched were deleted. The "greedy matching" method resulted in group sizes of 104 each (N = 208 total). Propensity Score Method Selection The regression on the PS method was selected for use in the outcomes analysis. The regression on the PS method was selected based on the following criteria; the adjustment for selection bias on the covariates measured before and after the PS procedure is carried out, and the resultant sample size. The adjustment for selection bias after application of PSs was determined for each PS method using the following methods. For continuous variables the percent decrease in bias was calculated using the formula:11 100 x [ 1 (bias post) / (bias pre) ], where bias post was the difference between PS adjusted group means and bias pre was the difference between unadjusted group means (group means before PS analysis). Variable means before PS analysis are reported in table 43 as the unadjusted means of the groups. Variable means after PS adjustment for the regression on PS and quintile methods were the least square means reported using PROC GENMOD (SAS 8.2)35 after adjustment for propensity score (or quintile depending on the method). For the "greedy matching" method unadjusted means were used for both the pre and post means calculations. Results are reported in tables 4 5, 47, and 48. For categorical variables the percent decrease in bias was calculated using the following formula:12 100 x [ 1 (1 OR post) / (1 OR pre)] where OR post is the odds ratio of the groups (adjusted for PS) and OR pre is the odds ratio of the groups before PS adjustment. For both the pre and post odds ratio measures PROC GENMOD (SAS 8.2)35 was used. The odds ratios were calculated using the same procedure for all three PS methods. Results are reported in tables 45, 47, and 48. A further test of the effect of the different PS methods involved the purposeful removal of independent variables from the regression model and the subsequent test for PS adjustment on the "unmeasured" variable. The logistic regression model was run twelve times. Each time one of the independent variables was removed from the model and the percent bias reduction on the now "unmeasured" variable was calculated for each of the three PS methods. The same equations for continuous and categorical variables were used to calculate percent bias reduction on the variable that had been removed. The results are reported in tables 49 to 411. The measurement of adjustment for bias on "unmeasured" variables was not considered in the selection of the PS method. It has been included in this study as a means of contributing to the PS methodology. Work has been done on the PS model's ability to adjust for bias on unmeasured clinical variables29 and the PS model's ability to adjust for bias on unmeasured variables in a large computer generated dataset.32 Our study is unique, however, since it examines the PS model's ability to adjust for bias on demographic variables contained in a relatively small, real world dataset. There were five PS model covariates that showed significant between group differences after the initial OA group assignment. The variables were percent of patients diagnosed with OA (% OA dx), the average income of the county in which the GP's practice is located (aver income) the average hospital length of stay per patient (los rate), the population size of the community in which the GP's practice was located and participation in a previous influenza AD intervention (flu AD). The PS adjustment on the flu AD variable was not successful for any of the three PS methods so it was included in the outcomes models as a covariate. The other four variables were of interest in the analysis of effect of PS method's ability to adjust for bias on unmeasured administrative variables. The correlations between the variable and the PS were calculated and graphed against the percent reduction in bias for each PS method. Correlations between the variables and the included PS model covariates were calculated and tabulated. The relationship between the reduction in bias in unmeasured variables and each PS methods was studied. The results are contained in chapter four. Step Three: Primary Outcome Analysis; Intervention Effect on COX2 Utilization Rates Once the method of PS analysis was selected and the GP intervention and control groups had been determined, the analysis of the primary outcome effect was carried out as described below. To enable the analysis of the changes in COX2 utilization rates over time the COX2 utilization rates were determined for each GP in the study for six consecutive ninetyday time periods. Two time periods were preintervention and four time periods were postintervention.(Figure 32) The indexdate is reported as the date that the GP received the AD intervention and the indexdates for the control group were assigned by randomly selecting dates from the range of time that the AD intervention spanned. The COX2 utilization change rate will be calculated by subtracting the GP's baseline utilization rate (period 2 utilization rate) from the utilization rates in each study period. Intervention Group O O X O O O O Control Group O O O O O O Time from Index Timefrom 180 to 91 90 to1 Index to 90 91 to 180 181 to 270 271 to 360 intervention (days) date Figure 32. Experimental Design Timeline Before the utilization rates could be calculated the inclusion and exclusion criteria for claims in a given time period were defined. An example of the operationalization of the decision rules for the inclusion or exclusion of claims within a time period is presented using a fictitious ninety day time period (January 1st until March 31st) and describing how different scenarios were adjudicated. If a prescription claim is submitted for a twomonth supply on January 2nd it is clear that the period of time for the entire claim falls within the given time period and the claim is included. If a prescription claim is submitted for the same twomonth supply on March 28th it is clear that the entire claim period does not fall within the period ending March 31st. In this case the claim would still be counted, in its entirety, in the claim period that it was submitted. The reason for inclusion of the claim in the initial time period is that it was in this time period that the GP's prescribing behavior took place and the intention was to have the patient take the medication as prescribed. Refills were considered to be an extension of the original claim until such a time as the refill claim was submitted more than thirty days after the intended fill date for the refill. If the refill was more than thirty days late the rest of the claim was not counted in any time period. The COX2 utilization rates for each GP was determined through the use of the World Health Organization's (WHO) Anatomic and Therapeutic Classification System/ Defined Daily Dose (ATC/DDD) methodology36 and was reported for each GP as the average number of DDDs per included patient per ninetyday intervention time period. The reporting of DDDs is often given as per thousand patients, however, since most GPs in the study will not have one thousand patients that meet the criteria this could be misleading. DDDs are drug consumption data that are independent of price and formulation. Once set, the WHO is reluctant to change DDD measures and as such the DDD is stable over time. This makes the DDD measure more reliable for drug consumption studies but it is not appropriate for clinical analysis. The DDD, therefore, "enables the researcher to assess trends in drug consumption and to perform comparisons between population groups."36 An analysis of the intervention effect on the primary outcome, change in COX2 utilization rates, was carried out. The primary outcome model initially included the dependent variable (change in COX2 rates for the four postintervention periods), the independent variables indicating the between group effect (OA) and longitudinal effects (period), the PS variable, the variable flu AD (as indicated from the PS analysis), as well as baseline COX2 rate (BL rate), and number of elderly patients in the GP's panel (# elderly pt). The model is depicted in figure 33. Each of the variables was retained in the model regardless of its significance. The covariates were all included as adjustments for confounding which if not controlled would be questioned in peer review. The included variables for the primary outcome model with their associated coefficients and significance levels are reported in table 413 in the results section. Y = Po + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + P6(X6) Where; Y = change in COX2 utilization rate (periods 3 to 6 (postintervention)), Xi = GP participation in the intervention (0 = no, 1 = yes), X2 = experimental time period (period = 3,4,5,6), X3 = PS, X4 = GP participation in the influenza AD service (0 = no, 1 = yes), X5 = GP baseline COX2 rate (DDD / patient / period = 2), X6 = number of GP's patients > 65 years old, Figure 33. Primary Outcome Model for Between Group Effect The model determined the statistical significance of the between group effect (between group effect) as well as the longitudinal effect (within subjects effect) of the intervention. The model was analyzed using PROC GENMOD (SAS 8.2)35 and significance is reported at the alpha= 0.05 level. The two ninetyday preintervention utilization rates for the experimental groups were analyzed to determine if there were any significant between group differences in the change in COX2 utilization occurring before the study commenced. The pre intervention analysis was carried out using the same model as the primary outcome model described in figure 33 however, only the first two time period measurements (period = 1,2) for each GP were entered into the model. This examined whether or not significant differences for change in utilization rates were present between the groups before the intervention was applied. A longitudinal model was tested using the primary outcome model in figure 33 with a preintervention / postintervention variable added which described whether the change in COX2 utilization rate was pre or postintervention. The measurements for periods one and two were coded as prepost =1 and the measurements for periods three to six were coded as prepost = 2. The longitudinal effects model was run twice with only one of the intervention groups included each time. The prepost variable indicated whether a significant within group intervention effect occurred. The results are reported in tables 418 and 419. Step Four: Secondary Outcome Analyses; The Utilization of Other Health Care Resources Associated with NSAID Induced GI Side Effects The primary outcome model exhibited significant between group differences and therefore, all of the secondary outcome analyses were carried out using the same GP groups as in the primary outcome analysis. The models for the secondary outcomes were developed using the same variables as the primary outcome model. Each secondary outcome model had the change in COX2 utilization rate substituted with the appropriate secondary outcome rate. The secondary outcomes that were analyzed are the intervention 36 effect on changes in rates from baseline for; PPI utilization, misoprostol utilization, H2A utilization, GP office visits, specialist office visits, and death rates due to GI complications. Rates for secondary outcomes (described individually with each outcome analysis) and the results for the secondary outcomes are described and reported in chapter four. CHAPTER 4 RESULTS Step One: Extraction and Validation of Data PROC MEANS35 was used to perform the calculations for the continuous variables. The variable mean, standard deviation, median, minimum and maximum are reported in table 41. Table 41. Descriptive Statistics for Continuous Variables in the PS Model Group N Variable Mean Std Dev Median Minimum Maximum AD= 0 265 % OA dx 0.0913 0.1071 0.0638 0.0000 0.7391 GP age 47.30 9.78 47.00 27.00 79.00 % elderly 0.1910 0.1160 0.1676 0.0253 0.9000 total # pt 1054.18 438.55 1021.00 30.00 2575.00 aver income 27688.68 4683.37 27500.00 22500.00 32500.00 BLrate 3.6013 2.7048 3.0675 0.0769 16.8750 los rate 0.0453132 0.1449 0.0000 0.0000 1.2647 population 183342.23 162415.32 109330.00 991.00 359183.00 AD= 1 231 % OA dx 0.0719 0.0598 0.0608 0.0000 0.2601 GP age 45.74 9.18 45.00 27.00 77.00 % elderly 0.1772 0.0788 0.1681 0.0251 0.5564516 total # pt 1037.69 418.99 1009.00 171.00 2481.00 aver income 25833.33 4488.31 22500.00 22500.00 32500.00 BL rate 3.9758 2.8707 3.4456 0.0487 14.2500 los rate 0.0882 0.1767 0.0000 0.0000 1.2592 population 122705.25 155567.12 22430.00 550.00 359183.00 PROC FREQ35 was used to perform the calculations for the categorical variables. The proportion of each variable level is reported in table 42. Step Two: Establishment of Balanced Control and Experimental Groups Using Three Propensity Score Methods PrePropensity Score Analysis Twelve variables were identified in the administrative data as describing personal and practice characteristics of GPs. GP age, sex, participation in a previous influenza AD intervention, place of initial licensure, baseline COX2 prescribing rate, birthplace and percent of patients diagnosed with OA describe personal characteristics. The percent of elderly patients, total number of patients, average income of county where the practice is located, population size of the community where the practice is located and average hospital length of stay for patients describe the GP's practice characteristics. Table 42. Descriptive Statistics for Categorical Variables in the PS Model. Variable Level Proportion AD = 0 (n=265) AD = 1 (n=231) Sex Female 0.3057 0.2987 Male 0.6943 0.7013 Flu AD Yes 0.1585 0.7273 No 0.8415 0.2727 License Canada 0.1208 0.1732 Nova Scotia 0.6717 0.6623 Other 0.2075 0.1645 Birth place Nova Scotia 0.4830 0.4545 Canada East 0.1245 0.1255 Canada Centre 0.0679 0.0736 Canada West 0.0264 0.0390 Other 0.2981 0.3074 Table 43 contains the prePS variable values which include; means and standard deviations for continuous variables, Fstatistics (square ofttest for continuous variables and F for coefficient estimate from PROC GENMOD35 for categorical variables), e values, coefficient estimates for the main effect of the intervention for the categorical variables, and the odds ratio for the main effect. These values were used in subsequent tables to calculate percent bias reduction for each PS technique. The prePS analysis indicates that there are five variables that are not balanced. These variables are of the greatest concern since the goal of PS methods is to balance the groups on measured covariates. 1216 The five variables that show significant differences at the alpha = 0.05 level will be collectively referred to as the variables of concern (VOC) and they are; the percent of elderly patients with a diagnosis of OA (% OA dx), the average income of the county in which the physician's practice is located (aver income), the average hospital length of stay rate for elderly patients per physician (los rate), the population of the community in which the physician's practice is located (population), and physician participation in a previous influenza AD service (flu AD). Table 43. PrePS Univariate Analysis for Included Variables. PrePropensity Score Values (ttest and proc genmod) AD = 0 (n = 265) AD = 1 (n = 231) OR std std p Variable mean dev mean dev F value B (exp B) % OA dx* 0.0913 0.1071 0.0719 0.0598 9.9191 0.0116 GP age 47.3 9.8 45.7 9.2 9.2191 0.0678 % elderly 0.1910 0.1160 0.1772 0.0788 8.9591 0.1182 total # pt 1054 439 1038 419 7.8191 0.6700 aver income* 27689 4683 25833 4488 11.8791 0.0001 BL rate 3.60 2.70 3.98 2.87 5.8991 0.1356 los rate* 0.0453 0.1449 0.0882 0.1767 4.4191 0.0031 population* 183342 162415 122705 155567 11.6191 0.0001 sex 0.0300 0.8663 0.0330 0.9675 flu AD* 140.1500 0.0001 2.6503 0.0706 license 3.3600 0.0669 0.3460 1.4134 birth place 0.1100 0.7415 0.0553 0.9462 variables that show significant differences at the alpha = 0.05 level Quintile PS Method Analysis The distribution of GPs within the quintiles is reported in table 44. Quintile one represents the GPs with the lowest PSs (lowest propensity to volunteer for the intervention) and quintile five represents the GPs with the highest PSs (highest propensity to volunteer for the intervention). The table is consistent with the expected PS distribution with fewer subjects in the high propensity quintile for the control group and fewer subjects in the low propensity quintile for the intervention group. The results for the quintile method were generated using PROC GENMOD35 and are reported in table 45. The main effect column represents the main effect of the AD variable and the interaction effect column represents the effect of the AD by quintile interaction. The quintile method resulted in no statistically significant difference between groups on all five VOC while maintaining balance on the rest of the covariates. Table 44. Physician Distribution by Quintile Quintile # Intervention Control # of GPs 1 86 13 99 2 77 22 99 3 65 35 100 4 24 75 99 5 13 86 99 TOTAL 265 231 496 Table 45. Quintile Method Regression Analysis Results. Quintile Method Interaction Ismean Main Effect Effect AD=0 AD= 1 OR % bias Variable (n= 265) (n= 231) F p F p B (exp B) reduction % OA dx* 0.0827 0.0783 0.20 0.6562 1.74 0.1403 77.32 GP age 47.2 47.1 0.01 0.9313 1.01 0.4029 93.75 % elderly 0.1918 0.1808 0.93 0.3351 2.04 0.0880 20.29 total #pt 1045 1031 0.09 0.7673 0.22 0.9256 12.50 aver income* 26757 26693 0.02 0.8900 1.34 0.2526 96.55 BL rate 3.58 3.68 0.10 0.7479 0.90 0.4616 72.89 los rate* 0.0491 0.0677 1.05 0.3058 0.76 0.5529 56.64 population* 149548 148905 0.00 0.9670 1.27 0.2797 98.94 sex 0.00 0.9818 0.00 0.9801 0.0130 0.9871 60.21 flu AD* 0.35 0.5537 xx xx 0.3723 0.6891 66.55 license 3.26 0.0709 3.28 0.0700 0.9702 0.3790 50.22 birth place 0.01 0.9372 0.01 0.9295 0.0381 0.9626 30.51 Average** 82.36 variables that were not significant at the alpha = 0.05 level in the prePS model ** average % bias reduction for variables with significant differences in the prePS model (excluding flu AD) + estimates not available (see table 46 for explanation) The interaction effect (AD*quintile) was not significant for four of the five VOC however, the flu AD variable exhibited an almost complete separation of data points (table 46) and as such the interaction effect was not estimated. The distribution of the flu AD variable on the PS was problematic for all three PS methods (figure 41). Therefore, the reported average percent bias reduction on the VOC does not include the flu AD variable. The average percent bias reduction for the quintile method is 82%. It is evident at this point that the flu AD variable will have to be included in the outcome models regardless of the PS method chosen. Table 46. Distribution of Influenza AD Participants by Propensity Score Quintile Flu AD Participation Quintile Total 1 2 3 4 5 No 99 99 88 0 0 286 Yes 0 0 12 99 99 210 Regression on the Propensity Score Method Analysis The results for the regression on the PS method were generated using PROC GENMOD35 and are reported in table 47. This method was successful in balancing three of the five VOC while maintaining balance on the rest of the covariates. The average percent bias reduction on the VOC (flu AD excluded) is 99%. The variable, population, retained a significance level less than 0.05 and it also exhibited a significant interaction effect (population*AD) at the alpha = 0.05 level. The variable aver income showed a nonsignificant main effect with a p value > 0.05 however, the interaction effect (aver income*AD) is less than the 0.05 level. The separation of data points for the flu AD variable on the PS was again evident. Figure 41 shows the distribution of flu AD on PS (stratified at 0.05 intervals). This separation precluded the model from estimating main and interaction effects for flu AD. "Greedy Matching" Method Analysis The results for the "greedy matching" method were generated using PROC GENMOD35 and are reported in table 48. This method was successful in balancing four 42 of the five VOC while maintaining balance on the remainder of the covariates. The average percent bias reduction on the VOC (flu AD excluded) is 75%. Table 47. Regression on PS Method Analysis Results. Regression on Propensity Score Method Interaction Ismean Main Effect Effect AD = 0 AD = 1 OR % bias Variable (n=265) (n=231) F P F p B (exp B) reduction % OA dx* 0.0770 0.0772 2.19 0.1394 3.07 0.0802 98.97 GP age 47.0 46.9 0.90 0.3441 1.30 0.2548 97.50 % elderly 0.1819 0.1819 0.45 0.5030 0.63 0.4291 100.00 total #pt 1036 1037 0.33 0.5637 0.47 0.4924 93.75 aver income* 26531 26530 2.83 0.0934 3.92 0.0482 99.95 BL rate 3.68 3.69 0.61 0.4338 0.89 0.3462 97.63 los rate* 0.0613 0.0620 0.32 0.5697 0.48 0.4872 98.37 population* 142500 142661 4.23 0.0402 5.90 0.0155 99.73 sex 0.22 0.6392 0.31 0.5746 0.2073 1.2304 609.62 flu AD* xx+ xx+ xx+ xx 728.77 license 2.48 0.1152 3.48 0.0622 0.6744 0.5095 18.66 birth place 0.03 0.8588 0.04 0.8377 0.0674 0.9348 21.15 Average** 99.25 variables that showed significant differences at the alpha = 0.05 level in the prePS model ** average % bias reduction for variables that were not significant in the prePS model (excluding flu AD) +estimates not available (see figure 41 for explanation) Graph of Propensity Score vs. Frequency of Flu AD Participants c 70 *. 60 S50 40 *fluAD= no o 30 *flu AD= yes 20 LL 10 O 0 Propensity Score Figure 41. Frequency of Influenza AD Participants by Propensity Score. 43 The flu AD variable estimates were not obtained for the same reasons described in the regression on the PS method section. With the exception of the flu AD variable, the "greedy method" balanced all variables and associated interaction terms. Table 48. "Greedy Matching" Method Analysis Results. "Greedy Matching" Method Interaction Ismean Main Effect Effect AD= 0 AD= 1 OR % bias Variable (n=104) (n=104) F p F p B (exp B) reduction % OA dx* 0.0722 0.0788 0.18 0.6712 0.01 0.9330 65.98 GP age 46.4 47.0 2.10 0.1492 1.99 0.1598 62.50 % elderly 0.1768 0.1876 0.39 0.5328 0.05 0.8299 21.74 Total # pt 1074 1008 1.25 0.2650 0.37 0.5410 312.50 aver income* 26535 26300 0.23 0.6301 0.12 0.7308 87.34 BL rate 3.75 3.78 0.85 0.3573 1.25 0.2651 92.37 los rate* 0.0456 0.0535 1.04 0.3097 0.89 0.3471 81.59 population* 154497 132123 2.06 0.1527 1.14 0.2862 63.10 Sex 0.53 0.4654 1.26 0.2619 0.4266 1.5320 1538.99 flu AD* xx+ xx+ xx+ xx 446.3283 License 1.35 0.2459 1.04 0.3077 0.6730 0.5102 18.49 birthplace 1.79 0.1809 1.31 0.2520 0.6828 0.5052 819.72 Average** 74.50 variables that showed significant differences at the alpha = 0.05 level in the prePS model ** average % bias reduction for variables that were not significant in the prePS model (excluding flu AD) +estimates not available The "greedy matching" method resulted in a decrease in total sample size from 496 (sample size of the two previous methods) to 208. This represents a decrease in sample size of 58%. The eliminated GPs had PSs that were predominantly in the highest or lowest ranges of the distribution. The elimination of these GPs could affect the generalizability of the study since only the GPs who are in the midrange of the PS distribution would be left in the study. Selection of a Preferred Propensity Score Method The selection of a preferred PS method was carried out by measuring each of the three methods against the following two criteria; * the resulting sample size after application of the PS method, and * the PS method's ability to adjust for bias on the VOC. A major disadvantage of the "greedy matching" method is the reduction in sample size resulting from the discarding of subjects that are not matched. In this case the sample size is reduced by 58% which possibly results in a loss of power to detect significance in the main effects of the outcome models and a loss of generalizability of the findings. Since the "greedy matching" method does not show advantages over the regression on the PS method in terms of adjusting for bias on the covariates it is considered less desirable than the regression on the PS method and will not be selected as the PS method for inclusion in the outcome models. The regression on PS method was responsible for the greatest adjustment for bias between groups on all of the VOC (figure 42). The average reduction in bias for the regression on the PS method was 99% versus 82% for the quintile method. With this dataset the regression on the PS method is preferred and it is the method that will be applied to the outcome analyses. It is important to note that the failure to adjust for bias on the flu AD variable still exists and as such the flu AD variable will be included in the outcome models. Exploratory Analysis of the Propensity Score Methods Effect on Adjusting for Bias on Unmeasured Variables The purpose of this exploratory analysis is to determine whether any one PS method is better at reducing bias on variables that are not included in the PS model and are, therefore, considered unmeasured. The cstatistic is a measure of the model's ability to discriminate between groups. The cstatistic for the full model is 0.832 which can be interpreted as follows; if one randomly select one subject from each AD group the model will accurately predict the group from which the subjects originated 83.2% of the time. With the exception of the models dealing with the exclusion of the flu AD variable, the cstatistic remains stable for all of the PS models. The range is from 0.830 to 0.835 (table 49). Percent Reduction in Bias on Unbalanced Variables (VOC) S 100.00 .= 80.00 " 60.00 "Z 40.00  20.00 l 0.00 % OA dx Aver Income los rate Population Average Variable Quintile u Regr on PS c Greedy Match Figure 42. Comparison of PS methods Ability to Reduce Bias on VOC. Table 49. Quintile Method Results for Excluded Variable Models. Quintile Method Ismean AD= 0 AD= 1 (n=265) (n=231) 0.0935 0.0708 46.8 47.5 0.1960 0.1754 1085 1006 26777 3.6 0.0471 153438 26652 3.7 0.0730 148662 Main Effect P 0.0207 0.5426 0.079 0.1111 0.7862 0.7418 0.1615 0.7746 0.9217 0.0001 0.3673 0.7528 Interaction Effect P 0.5298 0.5112 0.1374 0.0231 0.2361 0.5274 0.5214 0.1805 0.6197 0.2957 0.0896 0.8820 OR % bias B (exp B) reduction 17.01 56.25 49.28 393.75 0.0593 2.0411 0.5001 0.1524 0.9424 0.1299 0.6065 0.8586 93.27 73.68 39.63 92.12 77.37 6.38 4.81 162.75 42.88 * variables that showed significant differences at the alpha 0.05 level in the prePS model ** average % bias reduction for variables that showed significant differences in the prePS model Excluded Variable % OA dx* GP age % elderly total # pt aver income* BL rate los rate* population* sex flu AD* license birth place Average** c 0.833 0.833 0.831 0.833 0.834 0.834 0.833 0.832 0.835 0.662 0.830 0.835 There are three cstatistics that are worth noting. The first is the cstatistic that is generated for the model when the flu AD variable is removed. It has been noted that there exists an almost complete separation of data for the flu AD variable on the PS so when the flu AD variable is excluded from the model the ability of the model to discriminate decreases from 0.834 to 0.662. The other two are the cstatistics associated with sex and birth place. These two variables have the distinction of being the most closely balanced variables in the prePS analysis (table 43) with pvalues of 0.8663 and 0.7415 respectively. The PS model cstatistics when these variables are excluded is equal to 0.835 in both cases. This value is greater than the cstatistic for the full model thereby indicating that the inclusion of these variables in the PS model decreases the model's discriminative ability. The complete results from the reduced PS models are reported in tables 49 through 411. The analysis of the reduced models effect's on balancing the VOC is summarized in figure 43. Figure 43 shows that no one PS method systematically reduces bias on unmeasured variables to a greater extent than the others. Regression on PS does, on average, reduce bias on the VOC to the greatest degree. The summary of PS models effects (figure 43) shows that bias between groups on unmeasured variables can be reduced by PS methods. The correlation matrix between the VOC and the PS covariates was calculated and reported in table 412. Table 412 shows limited correlation (less than 0.30) between the VOC and the PS covariates in all cases except one. The one exception is the correlation between population (population of community where the GP practice is located) and aver income (average income for county where GP practice located) which was 0.91. The correlation between population 47 and aver income is associated with the higher reduction in bias for those variables when they are not included in the PS model. Table 410. Regression on PS Results for Excluded Variable Models. Regression on Propensity Score Method Interaction Ismean Main Effect Effect Excluded AD = 0 AD= 1 OR % bias Variable c (n=265) (n=231) F p F p B (exp B) reduction % OA dx* 0.833 0.0901 0.0736 0.74 0.3909 0.00 0.9540 GP age 0.833 46.5 47.2 1.14 0.2854 0.72 0.3974 % elderly 0.831 0.1918 0.1777 0.27 0.6065 0.04 0.8445 total #pt 0.833 1061 1018 0.69 0.4080 0.15 0.6946 14.95 56.25 2.17 168.75 aver income* 0.834 26567 26510 2.83 0.0929 3.65 0.0566 96.93 BL rate 0.834 3.7 3.7 0.57 0.4515 0.96 0.3286 100.00 Los rate* 0.833 0.0513 0.0684 0.19 0.6641 1.26 0.2618 60.14 population* 0.832 146919 139289 4.70 0.0307 5.12 0.0240 87.42 sex 0.835 0.06 0.7993 0.68 0.4108 0.1155 1.1224 277.17 Flu AD* 0.662 3.16 0.0756 2.12 0.1456 1.4258 0.2403 18.26 license 0.830 1.41 0.2348 4.59 0.0322 0.5203 0.5943 1.87 birth place 0.835 0.09 0.7695 0.02 0.8911 0.1117 0.8943 96.45 Average** 55.54 variables that showed significant differences at the alpha 0.05 level in the prePS model ** average % bias reduction for variables that showed significant differences in the prePS model Table 411. "Greedy Matching" Results for Excluded Variable Models. "Greedy Matching" Method Interaction Ismean Main Effect Effect Excluded ni OR % bias Variable (i=0,1) AD= 0 AD= 1 F p F p B (exp B) reduction % OA dx* GP age % elderly total # pt aver income* BL rate Los rate* population* sex Flu AD* license birth place Average** 106 0.0969 0.0707 4.36 0.0380 0.00 0.9677 101 46.2 47.5 1.40 0.2381 0.59 0.4450 105 0.1944 0.1801 0.41 0.5205 0.01 0.9365 103 1049 1005 0.61 0.4371 0.17 0.6676 105 26875 26123 1.02 0.3136 0.22 0.6419 103 3.7 3.7 0.77 0.3806 1.01 0.3169 105 0.0352 0.0614 1.09 0.2967 0.06 0.8145 104 154497 132123 2.06 0.1527 1.14 0.2862 104 0.00 0.9674 0.24 0.6260 0.0119 1.0120 190 2.00 0.1570 1.34 0.2471 0.7168 0.4883 104 5.04 0.0248 9.09 0.0026 0.6618 0.5159 105 0.00 0.9787 0.28 0.5943 0.0069 1.0069 35.05 15.69 3.62 175.00 59.48 97.11 38.93 63.10 63.12 44.94 17.10 87.22 34.28 * variables that showed significant differences at the alpha 0.05 level in the prePS model ** average % bias reduction for variables that showed significant differences in the prePS model Percent Reduction in Bias on Unbalanced and Unmeasured Variables 100.00 80.00 C o 60.00 S40.00o 20.00 0.00  40.00 Variable SQuintile 0 Regron PS l Greedy Match Figure 43. Summary of PS Models Effects on Reducing Bias on the VOC Table 412. Correlation Matrix Between VOC and PS Covariates. Correlation Between VOC and All PS Covariates Covariate VOC % OA aver los rate dx population income flu AD BL rate 0.06 0.02 0.22 0.26 0.00 los rate 1.00 0.01 0.05 0.05 0.03 % OA dx 0.01 1.00 0.01 0.01 0.01 total # pt 0.12 0.10 0.00 0.02 0.02 % elderly 0.02 0.26 0.09 0.09 0.03 sex 0.08 0.04 0.13 0.13 0.03 flu AD 0.03 0.01 0.14 0.17 1.00 population 0.05 0.01 1.00 0.91 0.14 GP age 0.05 0.07 0.06 0.06 0.09 aver income 0.05 0.01 0.91 1.00 0.17 The effect of the correlation between the PS and the VOC and the reduction in bias was tested. The correlation between the PS and the VOC was calculated and scatter plots were compiled to display the results graphically in figure 44. The absolute values of the correlations ranged from 0.182 to 0.329. The absolute value of the correlations was plotted against the percent bias reductions on the VOC for each of the three PS methods (figure 45). The results from figure 45 show an overall effect of increasing percent bias reduction with increasing absolute correlation between the PS and the excluded variable. Scatter Plot: Propensity Score vs. Percent of Patients with OA Diagnosis (Rho = 0.182) .* ** . ..* S01 0 2 03 04 05 06 07 08 09 Propensity Score Scatter Plot: Propensity Score vs. Community Population (Rho= 0.311) 02 04 06 08 Propenity Score Scatter Plot: Propensity Score vs. Length of Stay Rate (Rho = 0.220) * S ** * so $ *. *% .* e..* .esme Propensity Score Figure 44. Scatterplots of Propensity Score Versus Unbalanced Variables Step 3: Primary Outcome Analysis Model Development The analysis of the primary outcome, the effect of the OA AD intervention on the COX2 utilization rates was carried out using a repeated measures model on longitudinal *** ..* br ~bfl. . data (PROC GENMOD35). There were six experimental time periods over which the outcomes measures were assessed (figure 32). Correlation (PS vs. Excluded Var.) vs. Percent Bias Reduction (by PS Method) 100.00 80.00 S60.00 01 S40.00 0 20.00 S0.00 0 0 5 o1 /' 1 0 21 0 23 025 0 2 029 0 31 033 0 5 20.00 40.00 Rho . Quintile i Regr on PS Greedy Match Figure 45. Line Graph Comparing Correlations and Percent Bias Reduction The primary outcome measure, the change in COX2 prescribing from baseline, was calculated for each physician by aggregating all of the COX2 prescription claims for all of the elderly patients in the physician's panel and dividing by the number of elderly patients in the panel. The resulting rate, number of COX2 DDDs per patient per physician was subtracted from the baseline prescribing rate to yield a measure of change in COX2 prescribing. The primary outcome model included the variables intervention participation (AD), the PS (pr), the time period in which the measurement took place (period), participation in a previous influenza AD service (flu AD), the baseline COX2 prescribing rate (BL rate), and the number of elderly patients in the GP's panel (# elderly). The model is depicted in figure 46. The variables were included for the following reasons. The PS variable represents the outcome from the PS analysis, the period variable controls for the longitudinal changes, the flu AD variable was not successfully balanced by the PS method, and the baseline COX2 rate and number of elderly patients control for the GP's preintervention prescribing behavior and practice size respectively. Y = Po + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + (X6) Where; Y = change in COX2 utilization rate (periods 3 to 6 (postintervention)), Xi = physician participation in the intervention (0 = no, 1 = yes), X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 = no, 1 = yes), X5 = physician baseline COX2 rate (DDD / patient, (period = 2)), X6 = number of patients in the GP's practice >65 years old Figure 46. Primary Outcome Model Between Group Results The significance level of each variable from the primary outcome model (figure 4 6) is listed in table 413. The values of the coefficient estimates in GEE are not interpreted in the same manner as GLM models37 and as such the values of the coefficient estimates are not reported in the results tables. A more indepth discussion of the interpretation of GEE results is included in the discussions in chapter five. The between groups effect of the intervention is interpreted from the value of the z statistic for the AD variable. The z value of 0.85 and associated pvalue of 0.3976 indicates that the main intervention effect over the entire postintervention period is not statistically significant. The model in figure 46 was also used to determine between group differences in the preintervention time periods (period = 1, 2). The z statistics and associated pvalues of each variable are listed in table 414. The preintervention results are interpreted in the same manner as the postintervention results. The z statistic and pvalue for the AD variable are 0.88 and 0.3775 respectively. The pvalue indicates that the groups are not significantly different on the outcome measure in the preintervention periods at the alpha = 0.05 level. Table 413. Primary Outcome Model Results (Periods = 3,4,5,6). Primary Outcome Model Results for Postintervention Periods (COX2 Prescribing Rates) Effect Z pvalue AD (AD = no) 0.85 0.3976 PS 0.84 0.4023 period 2.69 0.0072 flu AD (flu AD = no) 1.21 0.2255 BL rate 10.68 <0.0001 # elderly 1.64 0.1017 Table 414. Primary Outcome Model Results (Periods = 1,2). Primary Outcome Model Results for Preintervention Periods (COX2 Prescribing Rates) Effect Z pvalue AD (AD = no) 0.88 0.3775 PS 0.31 0.7588 period 0.38 0.7018 flu AD (flu AD = no) 0.23 0.8170 BL rate 14.45 <0.0001 # elderly 0.48 0.6313 Table 415 depicts the least square means for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The least square mean values are also presented in a graph in figure 47. Table 416 depicts the unadjusted means and standard deviations for the two groups (AD = yes and AD = no) for each of the six experimental time periods. A positive value indicates that the prescribing rate has increased from the baseline rate by the amount indicated and a negative value indicates a decrease in the prescribing rate from baseline. The unadjusted mean values are also presented in a graph in figure 48. Table 415. Least Square Means for Change in COX2 Rates by Group (DDDs/patient). AD group period 1 2 3 4 5 6 0 0.0516 0 0.2275 0.1778 0.2471 0.4178 1 0.2136 0 0.5315 0.0018 0.261 0.1457 Primary Outcome: Change in COX2 Rates (adjusted) 0.5 0 o 0.5 1 Time Period Intervention = No Intervention = Yes Figure 47. Least Square Means for Change in COX2 Rates by Group Table 416. Unadjusted Means for Change in COX2 Rates by Group (DDDs/patient). Period AD group 0 1 Mean Std Dev Mean Std Dev 1 0.1587 3.4287 0.3026 3.0709 2 0.0000 0.0000 0.0000 0.0000 3 0.3396 3.9059 0.5321 3.4410 4 0.0236 3.6700 0.1257 3.7358 5 0.5266 3.7285 0.0008 3.9072 6 0.6454 3.8566 0.1281 3.9248 Within Group (Longitudinal) Results The within group models were the same as the between group model in figure 46 except that the AD group variable is replaced by a prepost variable which measures significant within group differences between change in COX2 rates preintervention and postintervention. The model is run two times; once including only the intervention group and once including only the control group. The z statistic value and associated significance level (pvalue) of each variable are listed in table 417 for the intervention group and table 418 for the control group. Primary Outcome: Change in COX2 Rates (unadjusted) 0.8000 0.6000 ) 0.4000 a 0.0000 C o 0.2000 3 5 6 0.4000 0.6000 Time Period Intervention = No  Intervention = Yes Figure 48. Unadjusted Means for Change in COX2 Rates by Group The within group effect of the intervention is interpreted from the values of the z statistic and significance level of the prepost variable. For the intervention group, the z and pvalues of 2.34 and 0.0191 respectively indicates that the within group effect is significant at the alpha = 0.05 level. For the control group, the z statistic and pvalue of  0.22 and 0.8273 respectively indicates that the within group effect is not significant at the alpha = 0.05 level. Table 417. Primary Outcome Model Results (AD = yes). Primary Outcome Results for the Intervention Group (COX2 Prescribing Rates) Effect Z pvalue PS 0.04 0.9708 period 2.82 0.0049 prepost 2.34 0.0191 flu AD (flu AD = no) 0.49 0.6217 BL rate 9.74 <0.0001 # elderly 0.63 0.5271 Step 4: Secondary Outcome Analyses Misoprostol Utilization Rates Model development The analysis of the secondary outcome, the effect of the OA AD intervention on the misoprostol utilization rate was carried out using the same methods as the primary outcome analysis with the data for misoprostol utilization substituted for the COX2 utilization data (figure 49). Table 418. Primary Outcome Model Results (AD = no). Primary Outcome Results for the Control Group (COX2 Prescribing Rates) Effect Z pvalue PS period prepost flu AD (flu AD BL rate # elderly 1.32 1.31 0.22 0.95 8.68 1.10 0.1881 0.1910 0.8273 0.3412 <0.0001 0.2727 Y = Po + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + P6(X6) Where; Y = change in misoprostol utilization rate (periods 3 to 6 (postintervention)), X1 = physician participation in the intervention (0 = no, 1 = yes), X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 = no, 1 = yes), X5 = physician baseline misoprostol rate (DDD / patient, (period = 2)), X6 = number of patients in the GP's practice >65 years old Figure 49. Secondary Outcome Model for Misoprostol Utilization Between group results The z statistic and the significance level (pvalue) of each variable from the secondary misoprostol outcome model (figure 49) are listed in table 419. The between group effect of the intervention is interpreted from the z statistics and associated pvalue of the AD variable. The z statistic and pvalue of 0.87 and 0.3866 respectively indicate that the effect is not significant at the alpha = 0.05 level. Table 419. Secondary Misoprostol Outcome Model Results (Periods = 3,4,5,6). Secondary Outcome Model Results for Postintervention Periods 3 to 6 (Change in Misoprostol Prescribing Rates) Effect Z pvalue AD (ad = 0) 0.87 0.3866 PS 0.61 0.5412 period 0.96 0.3359 flu AD (flu AD = 0) 0.53 0.5943 BL rate 6.31 <0.0001 # elderly 0.24 0.8091 The model in figure 49 was used to determine intervention effects on each post intervention time period. None of the post intervention (analyzed individually) showed significant between group differences at the alpha = 0.05 level. The model in figure 49 was used to determine between group differences in the preintervention time periods (period = 1, 2). The z statistic and associated significance level of each variable is listed in table 420. The results are interpreted in the same manner as the postintervention results. The z statistic and pvalue for the AD variable are 0.22 and 0.8269 respectively. The pvalue indicates that the groups are not significantly different in the preintervention periods at the alpha = 0.05 level. Table 420. Secondary Misoprostol Outcome Model Results (Periods = 1,2). Secondary Outcome Model Results for Postintervention Periods 1 and 2 (Change in Misoprostol Prescribing Rates) Effect Z pvalue AD (ad = 0) 0.22 0.8269 PS 1.20 0.2308 period 0.28 0.7758 flu AD (flu AD = 0) 1.18 0.2396 BL rate 4.55 <0.0001 # elderly 0.58 0.5612 Table 421 depicts the least square means for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The least square mean values are also presented in a graph in figure 410. Table 421. Least Square Means for Change in Misoprostol Rate by Group (DDDs/patient). Secondary Outcome (Misoprostol) Least Square Means by AD Group AD group Period 1 2 3 4 5 6 0 0.0191 0.0000 0.0172 0.0516 0.0390 0.0388 1 0.0127 0.0000 0.0383 0.0743 0.0604 0.0652 Secondary Outcome: Change in Misoprostol Rates (adjusted) 0.1 0.1 5, fI 0.0  M3oe< 4  0.0  ~o 0.0  0.0 2 4 5 6 0.0 Time Period SIntervention = No Intervention = Yes Figure 410. Least Square Means for Change in Misoprostol Rates by Group. Table 422 depicts the unadjusted means and standard deviations for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The unadjusted mean values are also presented in a graph in figure 411. Within group (longitudinal) results The within group model was the same as the between group model in figure 49 except that the AD group variable is replaced by a prepost variable which measures within group differences between change in misoprostol rates preintervention and post intervention. The model is run two times; once including only the intervention group and once including only the control group. The z statistic and the associated significance level (pvalue) of each variable are listed in table 423 for the intervention group and table 424 for the control group. Table 422. Unadjusted Means and Standard Deviations for Change in Misoprostol Rate by Group (DDDs/patient). Period AD group Mean 0.0153 0.0000 0.0032 0.0586 0.0484 0.0235 Std Dev 0.2877 0.0000 0.2693 0.3289 0.3750 0.3782 Mean 0.0096 0.0000 0.0437 0.0570 0.0497 0.0850 Secondary Outcome: Change in Misoprostol Rates (unadjusted) 0.10 0.08 0.06 0.04 0.02 0.00 0.02 0.04 Std Dev 0.2969 0.0000 0.2581 0.3120 0.3472 0.4005 Time Period  Intervention = no  Intervention = yes Figure 411. Unadjusted Means for Change in Misoprostol Rates by Group. Table 423. Secondary Misoprostol Outcome Model Results (AD = yes). Secondary Outcome Model Results for All Periods (Intervention Group) (Change in Misoprostol Prescribing Rates) Effect PS period prepost flu AD (flu AD BL rate # elderly Z 0.58 1.65 0.25 0.30 1.23 0.55 pvalue 0.5594 0.0990 0.8075 0.7612 0.2195 0.5802 4 5 6 Table 424. Secondary Misoprostol Outcome Model Results (AD = no). Secondary Outcome Model Results for All Periods (Control Group) (Change in Misoprostol Prescribing Rates) Effect Z pvalue PS 0.01 0.9921 period 0.75 0.4523 prepost 1.00 0.3176 flu AD (flu AD = 0) 0.27 0.7888 BL rate 4.69 <0.0001 # elderly 1.59 0.1109 The within group effect of the intervention is interpreted from the values of the z statistic and associated pvalue of the prepost variable. For the intervention and control groups, z statistics and the pvalues of 0.25, 0.8075 and 1.00, 0.3176 respectively indicates that the within group effect is not statistically significant for both groups at the alpha = 0.05 level. PPI Utilization Rates Model development The analysis of the secondary outcome, the effect of the OA AD intervention on the PPI utilization rates was carried out using the same methods as the primary outcome analysis with the data for PPI utilization substituted for the COX2 utilization data (figure 412). Y = Po + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + P6(X6) Where; Y = change in PPI utilization rate (periods 3 to 6 (postintervention)), X1 = physician participation in the intervention (0 = no, 1 = yes), X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 = no, 1 = yes), X5 = physician baseline PPI rate (DDD / patient, (period = 2)), X6 = number of patients in the GP's practice >65 years old Figure 412. Secondary PPI Outcome Model Between group results The z statistic and the significance level (pvalue) of each variable from the secondary PPI outcome model (figure 412) are listed in table 425. Table 425. Secondary PPI Outcome Model Results (Periods = 3,4,5,6). Secondary Outcome Model Results for Postintervention Periods 3 to 6 (Change in PPI Prescribing Rates) Effect Z pvalue AD (ad = 0) 0.27 0.7906 PS 1.09 0.2755 period 1.43 0.1519 flu AD (flu AD = 0) 1.45 0.1478 BL rate 2.92 0.0035 # elderly 1.74 0.0813 The between group effect of the intervention is interpreted from the z statistics and associated pvalue of the AD variable. The z statistic and pvalue of 0.27 and 0.7906 respectively indicate that the effect is not significant at the alpha = 0.05 level. The model (figure 412) was used to determine intervention effects on each post intervention time period. None of the post intervention (analyzed individually) showed significant between group differences at the alpha = 0.05 level. The model in figure 412 was used to determine between group differences in the preintervention time periods (period = 1, 2). The z statistic and associated significance level of each variable is listed in table 426. The results are interpreted in the same manner as the postintervention results. The z statistic and pvalue for the AD variable are 0.13 and 0.8989 respectively. The pvalue indicates that the groups are not significantly different in the preintervention periods at the alpha = 0.05 level. Table 427 depicts the least square means for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The least square mean values are also presented in a graph in figure 413. Table 426. Secondary PPI Outcome Model Results (Periods = 1,2). Secondary Outcome Model Results for Postintervention Periods 1 and 2 (Change in PPI Prescribing Rates) Effect Z pvalue AD (ad = 0) 0.13 0.8989 PS 1.10 0.2726 period 0.14 0.8911 flu AD (flu AD = 0) 1.08 0.2818 BL rate 5.61 <0.0001 # elderly 0.04 0.9700 Table 427. Least Square Means for Change in PPI Rates by Group (DDDs/patient). Secondary Outcome (PPI) Least Square Means by AD Group AD group period 1 2 3 4 5 6 0 0.0388 0 0.3194 0.5271 0.507 0.4842 1 0.0553 0 0.3675 0.5513 0.555 0.4982 Secondary Outcome: Change in PPI Rates (adjusted) 0.6 0.4 0.2 0.2 1 2 3 4 5 6 Time Period Intervention = No  Intervention =Yes Figure 413. Least Square Means for Change in PPI Rates by Group. Table 428 depicts the unadjusted means and standard deviations for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The unadjusted mean values are also presented in a graph in figure 414. Within group (longitudinal) results The within group model was the same as the between group model in figure 412 except that the AD group variable is replaced by a prepost variable which measures within group differences between change in PPI rates preintervention and post intervention. The model is run two times; once including only the intervention group and once including only the control group. The z statistic and the associated significance level (pvalue) of each variable are listed in table 429 for the intervention group and table 430 for the control group. Table 428. Unadjusted Means for Change in PPI Rate by Group (DDDs/patient). Period OA group 0 1 Mean Std Dev Mean Std Dev 1 0.0203 1.4843 0.0041 1.4211 2 0.0000 0.0000 0.0000 0.0000 3 0.3932 1.3733 0.3372 1.5783 4 0.6307 1.6214 0.5810 1.6686 5 0.5844 1.6238 0.6182 1.7077 6 0.5575 1.7553 0.5490 1.6955 Secondary Outcome: Change in PPI Rates (unadjusted) 0.8 0.6  w 0 0.4 o . 0.2 o 0.0  0.2 41 5 4 Time Period SIntervention = no  Intervention = yes Figure 414. Unadjusted Means for Change in PPI Rates by Group. The within group effect of the intervention is interpreted from the values of the z statistic and associated pvalue of the prepost variable. For the intervention and control groups, the z statistics (pvalues) of 2.59 (0.0097) and 4.22 (<0.0001) respectively indicates that the within group effect is statistically significant for both groups at the alpha = 0.05 level and both changes are in the direction of increased utilization. Table 429. Secondary PPI Outcome Model Results (AD = yes). Secondary Outcome Model Results for All Periods (Intervention Group) (Change in PPI Prescribing Rates) Effect Z pvalue PS 1.41 0.1596 period 0.69 0.4873 prepost 2.59 0.0097 flu AD (flu AD = 0) 1.37 0.1717 BL rate 3.63 0.0003 # elderly 0.40 0.6879 Table 430. Secondary PPI Outcome Model Results (AD = no). Secondary Outcome Model Results for All Periods (Control Group) (Change in PPI Prescribing Rates) Effect Z pvalue PS 0.67 0.5012 period 0.02 0.9877 prepost 4.22 <0.0001 flu AD (flu AD = 0) 1.16 0.2450 BL rate 3.32 0.0009 # elderly 1.61 0.1065 H2A Utilization Rates Model development The analysis of the secondary outcome, the effect of the OA AD intervention on the H2A utilization rates was carried out using the same methods as the primary outcome analysis with the data for H2A utilization substituted for the COX2 utilization data (figure 415). Between group results The z statistic and the significance level (pvalue) of each variable from the secondary H2A outcome model (figure 415) are listed in table 431. The between group effect of the intervention is interpreted from the z statistics and associated pvalue of the AD variable. The z statistic and pvalue of 0.05 and 0.9619 respectively indicate that the effect is not significant at the alpha = 0.05 level. Y = Po + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + P6(X6) Where; Y = change in H2A utilization rate (periods 3 to 6 (postint X1 = physician participation in the intervention (0 = no, 1 = X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 X5 = physician baseline H2A rate (DDD / patient, (period = X6 = number of patients in the GP's practice >65 years old ervention)), yes), = no,1 = ye! 2)), Figure 415. Secondary Outcome Model for H2A Utilization Table 431. Secondary H2A Outcome Model Results (Periods = 3,4,5,6). Secondary Outcome Model Results for Postintervention Periods 3 to 6 (Change in H2A Prescribing Rates) Effect Z pvalue AD (ad = 0) 0.05 0.9619 PS 1.18 0.2381 period 7.29 <0.0001 flu AD (flu AD = 0) 1.12 0.2642 BL rate 7.31 <0.0001 # elderly 1.77 0.0766 The model (figure 415) was used to determine intervention effects on each post intervention time period. None of the post intervention (analyzed individually) showed significant between group differences at the alpha = 0.05 level. The model in figure 415 was used to determine between group differences in the preintervention time periods (period = 1, 2). The z statistic and associated significance level of each variable is listed in table 432. The results are interpreted in the same manner as the postintervention results. The z statistic and pvalue for the AD variable are 1.09 and 0.2764 respectively. The pvalue indicates that the groups are not significantly different in the preintervention periods at the alpha = 0.05 level. s), Table 433 depicts the least square means for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The least square mean values are also presented in a graph in figure 416. Table 432. Secondary H2A Outcome Model Results (Periods = 1,2). Secondary Outcome Model Results for Postintervention Periods 1 and 2 (Change in H2A Prescribing Rates) Effect Z pvalue AD (ad = 0) 1.09 0.2764 PS 0.98 0.3293 Period 2.74 0.0062 flu AD (flu AD = 0) 0.64 0.5230 BL rate 5.75 <0.0001 # elderly 1.49 0.1368 Table 433. Least Square Means for Change in H2A Rate by Group (DDDs/patient). Secondary Outcome (H2A) Least Square Means by AD Group AD group period 1 2 3 4 5 6 0 0.3007 0 0.1833 0.0114 0.1433 0.5532 1 0.0881 0 0.0157 0.1413 0.0867 0.6818 Secondary Outcome: Change in H2A Rates (adjusted) 0.5 0I S 1 2 3 4 6 0.5  1 Time Period Intervention = no  Intervention =yes Figure 416. Least Square Means for Change in H2A Rates by Group. Table 434 depicts the unadjusted means and standard deviations for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The unadjusted mean values are also presented in a graph in figure 417. Table 434. Unadjusted Means for Change in H2A Rate by Group (DDDs/patient). Period AD group 0 1 Mean Std Dev Mean Std Dev 1 0.2440 1.7956 0.2117 1.9819 2 0.0000 0.0000 0.0000 0.0000 3 0.1655 1.9023 0.0783 2.0181 4 0.0607 1.8949 0.2191 2.2941 5 0.2484 2.4465 0.2929 2.4570 6 0.5101 2.5878 0.5380 2.5854 Secondary Outcome: Change in H2A Rates (unadjusted) 0 CD r I0 oc o .s oii Time Period  Intervention = no  Intervention = yes Figure 417. Unadjusted Means for Change in H2A Rates by Group. Within group (longitudinal) results The within group model was the same as the between group model in figure 415 except that the AD group variable is replaced by a prepost variable which measures within group differences between change in H2A rates preintervention and post intervention. The model is run two times; once including only the intervention group and once including only the control group. The z statistic and the associated significance level (pvalue) of each variable are listed in table 435 for the intervention group and table 436 for the control group. The within group effect of the intervention is interpreted from the values of the z statistic and associated pvalue of the prepost variable. For the intervention and control groups, the z statistics (pvalues) of 5.56 (<0.0001) and 4.06 (<0.0001) respectively indicates that the within group effect is statistically significant for both groups at the alpha = 0.05 level and both changes are in the direction of decreased utilization. Table 435. Secondary H2A Outcome Model Results (AD = yes). Secondary Outcome Model Results for All Periods (Intervention Group) (Change in H2A Prescribing Rates) Effect Z pvalue PS 1.70 0.0897 period 6.59 <0.0001 prepost 5.56 <0.0001 flu AD (flu AD = 0) 1.53 0.1262 BL rate 7.48 <0.0001 # elderly 2.80 0.0051 Table 436. Secondary H2A Outcome Model Results (AD = no). Secondary Outcome Model Results for All Periods (Control Group) (Change in H2A Prescribing Rates) Effect Z pvalue PS 0.79 0.4282 period 4.06 <0.0001 prepost 2.33 0.0201 flu AD (flu AD = 0) 0.78 0.4366 BL rate 3.43 0.0006 # elderly 1.64 0.1003 GP Office Visit Rates Model development The analysis of the secondary outcome, the effect of the OA AD intervention on GP office visit rates was carried out using the same methods as the primary outcome analysis with the data for GP office visit rates substituted for the COX2 utilization data (figure 418). Between group results The z statistic and the significance level (pvalue) of each variable from the secondary GP office visit outcome model (figure 418) are listed in table 437. Y = Po + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + P6(X6) Where; Y = change in GP visit rates (periods 3 to 6 (postintervention)), Xi = physician participation in the intervention (0 = no, 1 = yes), X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 = no, 1 = yes), X5 = physician baseline GP visit rate rate (visits / patient, (period = 2)), X6 = number of patients in the GP's practice >65 years old Figure 418. Secondary Outcome Model for GP Office Visits Table 437. Secondary GP Office Visit Model Results (Periods = 3,4,5,6). Secondary Outcome Model Results for Postintervention Periods 3 to 6 (Change in GP Office Visit Rates) Effect Z pvalue AD (ad = 0) 1.06 0.2888 PS 0.74 0.4587 period 9.26 <0.0001 Flu AD (flu AD = 0) 0.02 0.9815 BL rate 1.97 0.0487 # elderly 1.26 0.2077 The between group effect of the intervention is interpreted from the z statistics and associated pvalue of the OA AD variable. The z statistic and pvalue of 1.06 and 0.2888 respectively indicate that the effect is not significant at the alpha = 0.05 level. The model (figure 418) was used to determine intervention effects on each post intervention time period. Only the period from 91 to 180 days (period four) following the intervention showed significant difference between groups at the alpha = 0.05 level. The zstatistic and pvalue associated with the intervention effect are 2.20 and 0.0275 respectively (95% CI 0.7926, 0.0464). In this case, where the analysis only includes one time period, the interpretation of the coefficient estimate is similar to traditional GLM methods. That is, the coefficient estimate of 0.4195 (AD = no) is interpreted as the nonintervention group having measures of average change rate 0.4195 fewer visits/patient/GP than the intervention group (equal values for the groups is hypothesized). The model in figure 418 was used to determine between group differences in the preintervention time periods (period = 1, 2). The z statistic and associated significance level of each variable is listed in table 438. The results are interpreted in the same manner as the postintervention results. The z statistic and pvalue for the OA AD variable are 0.37 and 0.7097 respectively. The pvalue indicates that the groups are not significantly different in the preintervention periods at the alpha = 0.05 level. Table 438. Secondary GP Office Visit Outcome Model Results (Periods = 1,2). Secondary Outcome Model Results for Postintervention Periods 1 and 2 (Change in GP Office Visit Rates) Effect Z pvalue AD (ad = 0) 0.37 0.7097 PS 1.16 0.2457 Period 0.08 0.9390 Flu AD (flu AD = 0) 1.19 0.2341 BL rate 7.17 <0.0001 # elderly 2.13 0.0332 Table 439 depicts the least square means for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The least square mean values are also presented in a graph in figure 419. Table 439. Least Square Means for Change in GP Office Visit Rate by Group (visits/patient). Secondary Outcome (GP Visits) Least Square Means by AD Group AD group Period 1 2 3 4 5 6 0 0.0182 0 0.4652 0.3882 0.3346 0.4341 1 0.0563 0 0.3813 0.79 0.0201 0.0069 Table 440 depicts the unadjusted means and standard deviations for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The unadjusted mean values are also presented in a graph in figure 420. Secondary Outcome: Change in GP Office Visit Rates o. 1 .2 0.5 o 0.5 Time Period  Intervention = no  Intervention = yes Figure 419. Least Square Means for Change in GP Office Visit Rates by Group. Table 440. Unadjusted Means and Standard Deviations for Change in GP Office Visit Rate by Group (visits/patient). Period AD group 0 1 Mean Std Dev Mean Std Dev 1 0.0057 0.5235 0.0026 0.5586 2 0.0000 0.0000 0.0000 0.0000 3 0.5282 0.9720 0.3191 0.9419 4 0.2597 0.8025 0.6388 1.3687 5 0.2269 0.7578 0.0570 1.4364 6 0.2742 0.7616 0.0290 1.7479 Within group (longitudinal) results The within group model was the same as the between group model in figure 418 except that the AD group variable is replaced by a prepost variable which measures within group differences between change in GP office visit rates preintervention and postintervention. The model is run two times; once including only the intervention group and once including only the control group. The z statistic and the associated significance level (pvalue) of each variable are listed in table 441 for the intervention group and table 442 for the control group. Secondary Outcome: Change in GP Office Visit Rates (unadjusted) 0.8 0.6  S0.4 ,M. 0.2 S0.0 _0.2 0.4 Time Period Intervention = no  Intervention = yes Figure 420. Unadjusted Means for Change in GP Office Visit Rates by Group. Table 441. Secondary GP Office Visit Outcome Model Results (AD = yes). Secondary Outcome Model Results for All Periods (Intervention Group) (Change in GP Office Visit Rates) Effect Z pvalue PS 1.56 0.1199 Period 10.95 <0.0001 Prepost 17.54 <0.0001 flu AD (flu AD = 0) 2.41 0.0159 BL rate 0.10 0.9187 # elderly 0.17 0.8680 The within group effect of the intervention is interpreted from the values of the z statistic and associated pvalue of the prepost variable. For the intervention and control groups, z statistics (pvalues) of 17.54 (<0.0001) and 20.21 (<0.0001) respectively indicates that the within group effect is statistically significant for both groups at the alpha = 0.05 level. The significant results for the longitudinal prepost effect is similar between the control and intervention groups as indicated in figure 420 and also indicated in the negative values of the z statistics for both groups. Table 442. Secondary GP Office Visit Outcome Model Results (AD = no). Secondary Outcome Model Results for All Periods (Control Group) (Change in GP Office Visit Rates) Effect Z pvalue PS 2.60 0.0093 Period 19.91 <0.0001 Prepost 20.21 <0.0001 flu AD (flu AD = 0) 2.62 0.0089 BL rate 0.99 0.3212 # elderly 2.73 0.0064 Rheumatologist and GI Specialist Visit Rates Model development The analysis of the secondary outcome, the effect of the OA AD intervention on rheumatologist and GI specialist office visit rates was carried out using the same methods as the primary outcome analysis with the data for rheumatologist and GI specialist office visit rates substituted for the COX2 utilization data (figure 421). Between group results The z statistic and the significance level (pvalue) of each variable from the secondary specialist office visit outcome model (figure 421) are listed in table 443. Y = lo + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + (X6) Where; Y = change in specialist visit rates (periods 3 to 6 (postintervention)), X1 = physician participation in the intervention (0 = no, 1 = yes), X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 = no, 1 = yes), X5 = physician baseline specialist visit rate (visits / patient, (period = 2)), X6 = number of patients in the GP's practice >65 years old Figure 421. Secondary Outcome Model for Specialist Office Visits The between group effect of the intervention is interpreted from the z statistics and associated pvalue of the AD variable. The z statistic and pvalue of 1.44 and 0.1498 respectively indicate that the effect is not significant at the alpha = 0.05 level. Table 443. Secondary Specialist Office Visit Model Results (Periods = 3,4,5,6). Secondary Outcome Model Results for Postintervention Periods 3 to 6 (Change in Specialist Office Visit Rates) Effect Z pvalue AD (ad = 0) 1.44 0.1498 PS 5.98 <0.0001 period 0.04 0.9700 flu AD (flu AD = 0) 5.43 <0.0001 BL rate 23.22 <0.0001 # elderly 3.01 0.0026 The model (figure 421) was used to determine intervention effects on each post intervention time period. Only the period from 181 to 270 days (period five) following the intervention showed significant difference between groups at the alpha = 0.05 level. The zstatistic and pvalue associated with the intervention effect are 2.10 and 0.0356 respectively (95% CI (0.0001, 0.0022)). In this case, where the analysis only includes one time period, the interpretation of the coefficient estimate is similar to traditional GLM methods. That is, the coefficient estimate of 0.0012 (AD = no) is interpreted as the nonintervention group having measures of average change rate 0.0012 greater visits/patient/GP than the intervention group. The model in figure 421 was used to determine between group differences in the preintervention time periods (period = 1, 2). The z statistic and associated significance level of each variable is listed in table 444. The results are interpreted in the same manner as the postintervention results. The z statistic and pvalue for the AD variable are 1.29 and 0.1976 respectively. The pvalue indicates that the groups are not significantly different in the preintervention periods at the alpha = 0.05 level. Table 444. Secondary Specialist Office Visit Outcome Model Results (Periods = 1,2). Secondary Outcome Model Results for Postintervention Periods 1 and 2 (Change in Specialist Office Visit Rates) Effect Z pvalue AD (ad = 0) 1.29 0.1976 PS 4.71 <0.0001 period 0.90 0.3670 flu AD (flu AD = 0) 3.86 0.0001 BL rate 9.21 <0.0001 # elderly 3.51 0.0004 Table 445 depicts the least square means for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The least square mean values are also presented in a graph in figure 422. Table 446 depicts the unadjusted means and standard deviations for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The unadjusted mean values are also presented in a graph in figure 423. Secondary Outcome (Specialist Visits) Least Square Means by AD Group AD group period 1 2 3 4 5 6 0 0.0005 0 0.0007 0.0005 0.0011 0.0001 1 0.0016 0 0.0002 0.0003 0.0001 0.0003 Table 445. Least Square Means for Change in Specialist Office Visit Rate by Group (visits/patient). Within group (longitudinal) results The within group model was the same as the between group model in figure 421 except that the AD group variable is replaced by a prepost variable which measures within group differences between change in specialist office visit rates preintervention and postintervention. The model is run two times; once including only the intervention group and once including only the control group. The z statistic and the associated significance level (pvalue) of each variable are listed in table 447 for the intervention group and table 448 for the control group. Secondary Outcome: Change in Specialist Office Visit Rates (adjusted) 0.002 0.0015 0.001 0.0005 0 0.0005 Time Period *Intervention = no  Intervention = yes Figure 422. Least Square Means for Change in Specialist Office Visit Rates by Group. Period AD group Mean 0.0008 0.0000 0.0005 0.0008 0.0007 0.0002 Std Dev 0.0089 0.0000 0.0077 0.0086 0.0093 0.0083 Mean 0.0002 0.0000 0.0011 0.0012 0.0013 0.0006 Std Dev 0.0087 0.0000 0.0089 0.0078 0.0079 0.0082 Table 446. Unadjusted Means and Standard Deviations for Change in Specialist Office Visit Rate by Group (visits/patient). Secondary Outcome: Change in Specialist Office Visit Rates (unadjusted) 0.0010 0.0005 0.0000 0.0005 0.0010 0.0015 Time Period  Intervention = no  Intervention = yes Figure 423. Unadjusted Means for Change in Specialist Office Visit Rates by Group. V Table 447. Secondary Specialist Office Visit Outcome Model Results (AD = yes). Secondary Outcome Model Results for All Periods (Intervention Group) (Change in Specialist Office Visit Rates) Effect Z pvalue PS 6.45 <0.0001 period 0.87 0.3857 prepost 1.94 0.0519 flu AD (flu AD = 0) 6.29 <0.0001 BL rate 17.54 <0.0001 # elderly 4.56 <0.0001 Table 448. Secondary Specialist Office Visit Outcome Model Results (AD = no). Secondary Outcome Model Results for All Periods (Control Group) (Change in Specialist Office Visit Rates) Effect Z pvalue PS 4.24 <0.0001 period 0.87 0.3870 prepost 0.70 0.4811 flu AD (flu AD = 0) 3.25 0.0012 BL rate 14.67 <0.0001 # elderly 2.01 0.0444 The within group effect of the intervention is interpreted from the values of the z statistic and associated pvalue of the prepost variable. For the intervention and control groups, z statistics (pvalues) of 1.94 (0.0519) and 0.70 (0.4811) respectively indicates that the within group effect is not statistically significant for both groups at the alpha = 0.05 level. The results for the longitudinal prepost effect are similar between the control and intervention groups as indicated in figure 423. Hospitalization Rates Due to GI Complications Model development The analysis of the secondary outcome, the effect of the OA AD intervention on hospitalization rates was carried out using the same methods as the primary outcome analysis with the data for hospital length of stay rates substituted for the COX2 utilization data (figure 424). Y = Po + Pl(Xl) + P2(X2) + P3(X3) + 4(X4) + 5(X5) + P6(X6) Where; Y = change in hospital utilization rate (periods 3 to 6 (postintervention)), Xi = physician participation in the intervention (0 = no, 1 = yes), X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 = no, 1 = yes), X5 = physician baseline hospital LOS rate (LOS / patient, (period = 2)), X6 = number of patients in the GP's practice >65 years old Figure 424. Secondary Outcome Model for Hospital Length of Stay Between group results The z statistic and the significance level (pvalue) of each variable from the secondary hospitalization length of stay outcome model (figure 424) are listed in table 4 49. Table 449. Secondary Hospital Length of Stay Model Results (Periods = 3,4,5,6). Secondary Outcome Model Results for Postintervention Periods 3 to 6 (Change in Hospital Length of Stay) Effect Z pvalue AD (AD = 0) 0.33 0.7389 PS 1.48 0.1396 Period 1.15 0.2500 flu AD (flu AD = 0) 1.13 0.2568 Flu AD*quintile (flu AD = 0) 0.94 0.3468 BL rate 15.58 <0.0001 los rate 2.36 0.0183 The between group effect of the intervention is interpreted from the z statistics and associated pvalue of the AD variable. The z statistic and pvalue of 0.33 and 0.7389 respectively indicate that the effect is not significant at the alpha = 0.05 level. The model (figure 424) was used to determine intervention effects on each post intervention time period. Only the period from 181 to 270 days (period five) following the intervention showed significant difference between groups at the alpha = 0.05 level. The zstatistic and pvalue associated with the intervention effect are 2.49 and 0.0128 respectively (95% CI (1.1093, 3.4627)). In this case, where the analysis only includes one time period, the interpretation of the coefficient estimate is similar to traditional GLM methods. That is, the coefficient estimate of 2.2860 (AD = no) is interpreted as the nonintervention group having measures of average change rate 2.2860 greater visits/patient/GP than the intervention group. The model in figure 418 was used to determine between group differences in the preintervention time periods (period = 1, 2). The z statistic and associated significance level of each variable is listed in table 450. The results are interpreted in the same manner as the postintervention results. The z statistic and pvalue for the AD variable are 1.58 and 0.1152 respectively. The pvalue indicates that the groups are not significantly different in the preintervention periods at the alpha = 0.05 level. Table 450. Secondary Hospital Length of Stay Outcome Model Results (Periods = 1,2). Secondary Outcome Model Results for Postintervention Periods 1 and 2 (Change in Hospital Length of Stay) Effect z pvalue AD (AD = 0) 1.58 0.1152 quintile 0.56 0.5751 period 0.67 0.5014 flu AD (flu AD = 0) 0.10 0.9217 flu AD*quintile (flu AD = 0) 0.72 0.4735 BL rate 10.32 <0.0001 los rate 2.84 0.0044 Table 451 depicts the least square means for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The least square mean values are also presented in a graph in figure 425. Table 452 depicts the unadjusted means and standard deviations for the two groups (AD = yes and AD = no) for each of the six experimental time periods. The unadjusted mean values are also presented in a graph in figure 426. Table 451. Least Square Means for Change in Hospital Length of Stay Rates by Group (days/patient). Secondary Outcome (Hospital LOS) Least Square Means by AD Group AD group period 1 0.6168 0.3396 3 4 5 6 1.0337 0.8165 2.0962 1.7457 0.2072 1.0211 0.1898 3.6511 Secondary Outcome: Change in Hospital LOS Rates (adjusted) Time Period  Intervention = no Ienterntion = yes Figure 425. Least Square Means for Change in Hospital Length of Stay Rates by Group. Table 452. Unadjusted Means and Standard Deviations for Change in Hospital Length of Stay Rates by Group (days/patient). Period AD group Mean 0.7409 0.0000 1.3957 1.0678 1.5773 0.5662 Std Dev 8.6598 0.0000 9.9291 8.4797 10.2326 8.8979 Mean 0.2482 0.0000 0.6193 1.0061 0.5130 5.5624 Std Dev 9.9641 0.0000 9.7833 14.5320 8.3747 51.6778 Within group (longitudinal) results The within group model was the same as the between group model in figure 424 except that the AD group variable is replaced by a prepost variable which measures within group differences between change in specialist office visit rates preintervention and postintervention. The model is run two times; once including only the intervention group and once including only the control group. The z statistic and the associated significance level (pvalue) of each variable are listed in table 453 for the intervention group and table 454 for the control group. Secondary Outcome: Change in Hospital LOS Rates (unadjusted) 6.0 5.0 S4.0 S3.0 P 2.0 1.0 0.0 1.0 1 2 3 4 6 Time Period Intervention = no Intervention = yes Figure 426. Unadjusted Means for Change in Hospital Length of Stay Rates by Group. Table 453. Secondary Hospital Length of Stay Outcome Model Results (AD = yes). Secondary Outcome Model Results for All Periods (Intervention Group) (Change in Hospital Length of Stay) Effect Z pvalue PS 0.76 0.4489 period 1.44 0.1491 prepost 0.96 0.3388 flu AD (flu AD = 0) 0.00 0.9976 BL rate 13.11 <0.0001 # elderly 1.56 0.1193 Table 454. Secondary Hospital Length of Stay Outcome Model Results (AD = no). Secondary Outcome Model Results for All Periods (Control Group) (Change in Hospital Length of Stay) Effect Z pvalue PS 3.13 0.0018 period 1.21 0.2256 prepost 2.08 0.0375 flu AD (flu AD = 0) 2.99 0.0028 BL rate 14.40 <0.0001 # elderly 1.87 0.0614 The within group effect of the intervention is interpreted from the values of the z statistic and associated pvalue of the prepost variable. For the intervention and control groups, z statistics (pvalues) of 0.96 (0.3388) and 2.08 (0.0375) respectively indicates that the within group effect is not statistically significant for the intervention group and is statistically significant for the control group at the alpha = 0.05 level. Death Rates Due to GI Complications Model development The analysis of the secondary outcome, the effect of the OA AD intervention on death rates due to GI complications was carried out using the same methods as the primary outcome analysis with the data for hospital length of stay rates substituted for the COX2 utilization data (figure 427). Ln Y= Po + 0P(X) + p2(X2) + P3(X3)+ 04(X4) + 5(X) + p6(X6) Where; Y = death rates (periods 3 to 6 (postintervention)), X1 = physician participation in the intervention (0 = no, 1 = yes), X2 = PS (range from 0 to 1), X3 = experimental time period (period = 3,4,5,6), X4 = physician participation in the influenza AD service (0 = no, 1 = yes), X5 = physician baseline hospital LOS rate (LOS / patient, (period = 2)), X6 = number of patients in the GP's practice >65 years old Figure 427. Secondary Outcome Model for Deaths Due to GI Complications Special consideration had to be given to the distribution of the data since the number of deaths per GP per study period was quite low. There were 1984 data points analyzed (496 GPs with six measures each) and in all cases except four the number of deaths per physician was equal to zero or one. The four other cases all contained two deaths (three of the four occurred in the control group). A dichotomous variable representing death/nodeath for each period measurement was developed and since the majority of the period measurements represented nodeath (142 with death and 2834 without death) a negative binomial distribution was used in the analysis model. The total number of deaths per group per period was less than five in a number of cases. For this reason, the number of study periods was reduced to three by combining periods one and two, three and four, and five and six. Between and within group results None of the between or within group analyses of death rates showed significance at the alpha = 0.05 level. The z statistics (pvalues) associated with the preintervention and postintervention between group analyses were 0.63 (0.5317) and 0.81 (0.4203) respectively and the z statistics (pvalues) associated with the within group analyses for the intervention and control groups were 0.36 (0.7189) and 0.03 (0.9742) respectively. CHAPTER 5 DISCUSSION The Academic Detailing Program in Nova Scotia An analysis of the effect of the OA AD intervention on prescribing behavior should be taken in context of the qualifications of the detailers, the dynamic changes over the course of the intervention and the policy options available to the decision makers. A description of these three topics should add to the determination of generalizability of the intervention to other jurisdictions. Qualifications of the Detailers The OA AD intervention employed three detailers; two pharmacists and one registered nurse. One pharmacist worked within the province's capitol district and the other pharmacist and the registered nurse divided the rural area of the province in two. The nurse detailed GPs in the region that she was native to and as such was very familiar with local customs and practices. All three of the detailers were trained in techniques associated with successful AD programs. These techniques are described in greater detail in appendix A. The intervention was designed to take approximately twenty minutes to present with opportunity for the GP to interact with the detailer over the course of the presentation. Changes Which Occurred Over the Period of the Intervention (History Effects) The OA AD intervention was delivered from April, 2002 to November, 2002. The analysis timeframe for our study spanned from October, 2001 (six months before the intervention commenced) to November, 2003 (one year after the intervention concluded). Between October, 2001 and May, 2003 two warnings regarding the safety of COX 2 inhibitors were issued by Health Canada.38 The first warning in April, 2002 concerned the results of the VIGOR trial8 and warned of increased cardiac risk associated with rofecoxib use and the second warning in May, 2002 concerned the results of the CLASS trial7 and warned of the GI risk associated with celecoxib use particularly in combination with low dose ASA therapy. Analysis of the VIGOR and CLASS trials was included in the OA AD intervention (appendix A). In December, 2004 rofecoxib was withdrawn from the market.38 The withdrawal occurred after the post intervention study period. The Nova Scotia pharmacare plan issued a policy change with respect to the benefit status of a combination product containing diclofenac and misoprostol.39 The product was changed to open benefit status in September, 2002. Announcement of the change was disseminated equally to all GPs in the province. The benefit status of rabeprazole was changed to open benefit in June, 200339 after the end of the post intervention analysis period. Policy Options Available to Decision Makers Our study examined the effect of the fourth message of the OA AD intervention which addressed pharmacotherapy of OA. The other three messages contained in the intervention were intended to change physician behavior in terms of prescribing non pharmacologic treatment for OA and research into the effectiveness of these messages is warranted. The OA AD intervention lacked a followup visit which is a limitation of the intervention design.13 Five options available to policy decision makers which could address this shortcoming without the costs associated with a oneonone followup visit are; the distribution of educational material, educational meetings, audit and feedback, reminders, and changes in benefit schedules.17' 40 While some of the instruments have not shown significant effects on their own the combination with AD can be effective.14,17 Distribution of educational material The distribution of educational material involves the dissemination of media (written or video) to the GPs with information reinforcing the messages of the OA AD intervention. It is the decision of the GP to review the message or not. It is relatively low cost and has been shown to have a modest but shortlived effect.17 The message contained in this medium should be limited to the intervention messages in such a way that does not require "active" learning or interaction with an educator. Educational meetings Educational meetings involve meeting in groups to review the messages from the intervention. This instrument can be more complex in nature than the distribution of written material but they are still limited by the inability of the participant to interact with the instructor on a onetoone basis. Used as a single intervention this instrument has shown little17 to no effect14 on improving pharmaceutical use. Audit and feedback Audit and feedback is an instrument that involves the analysis of the performance of the provider and/or the provider's peers over a period of time. The instrument is costly to implement as it involves a significant amount of data analysis to produce the audits. Audit and feedback can address some complex issues through the use of the analysis and comparison with peers. Studies using audit and feedback as a single intervention have shown a modest effect.17 