This item is only available as the following downloads:
! ! ! ! ! """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" #$%&'(! )%*+,-./0*12!3$4'&$ %12!/+4!%*'!)5$4'+6'!%*/%!7+%*.,0,&,-1!7448(!9:00&';'+%$+-! %*'!3$4'&$%1!<.,6'88!$+!/!=&$+$6/& !#.$/&!,>!9:00,.%'4!);0&,1;'+% 7:%*,.8(! =/.,&1+!9;$%* ? @,..$82!A$&B'.%,!C,0'D2!C$8/!E%%,;/+'&&$2!C/+6'!A,'%D!/+4! F$;B'.&1!G$H,+ ? C/I8,+ !"#$%&'()*+,./.'.01(23&-+"-'1 !JKL J M( NON P N QN R!JSNOT #*$8!$8!%*'!/:%*,. 8 U !0,8% ? 0.$+%T!<&'/8'!6$%'!%*'!>$+/&!5'.8$,+!,>!%*'!/.%$6&'2!/5/$&/B&'!/%! *%%0(VV4HT4,$T,.-V NSTNNNNV;/WTNJSXY T """""""""""""""""""""""""""""""""" """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" ! ! ! ! ! !"#$%&'()*+,-./.'.01(23&-+"-'1( $8!6,&&/B,./%$+-!I$%*!%*'!Z+$5'.8$%1!,>!3&,.$4/!C$B./.$'8!%,!4$-$%$D'2! 0.'8'.5' 2 !/+4!'H%'+4!/66'88!%, !%*'![,:.+/&U8!6,+%'+% T! #*' !G$-$%/&!C$B./.1!='+%'.!LGC=M!/%!%*'! Z+$5'.8$%1!,>!3&,.$4/!$8!/!+,+0.,>$%!6'+%'.!.'80,+8$B&'!>,.!%*'!6,&&'6%$,+!/+4!0.'8'.5/%$,+!,>!4$-$%/&! .'8,:.6'8!>,.!'4:6/%$,+T!3,.!;,.'!$+>,.;/%$,+!/B,:%!GC=2!0&'/8'!6,+%/6%! GC=\:> &$BT:>&T'4: T
1 Carolyn Smith Morris Department of Anthropology Southern M ethodist University, Dallas, TX Gilberto Lopez Social and Behavioral Sciences Department Harvard School of Public Health, Cambridge, MA Lisa Ottomanelli HSR&D/RR&D Center of Innovation on Disabili ty and Rehabilitation Research VISN 8 (CIDRR8), James A, Haley Veterans Hospital, Department of Mental Health and Rehabilitation Counseling University of South Florida, Tampa, FL Lance Goetz Department of Veterans Affairs, Physician, Spinal Cord Injury an d Disorders, Hunter Holmes McGuire VA Medical Center and Department of Physical Medicine and Rehabilitation Virginia Commonwealth University, Richmond, VA Kimberly Dixon Lawson Dallas VA Medical Center, Dallas, TX Ethnography, Fidelity, and the Evidence that Anthropology Adds: Supplementing the Fidelity Process in a Clinical Trial of Supported Employment ______________________________________________________________________________ This discussion considers the role and findings of ethnographic research w ithin a clinical trial of supported employment for veterans with spinal cord injury. Contributing to qualitative evaluation research and to debates over anthropological evidence vis vis clinical trials, we demonstrate how enactors of a randomized control led trial can simultaneously attend to both the trial's evidentiary and procedural requirements and to the lived experiences and needs of patients and clinicians. Three major findings are described: (1) contextual information essential to fidelity efforts within the trial; (2) the role of human interrelationships and idiosyncratic networks in the trial's success; and (3) a mapping of the power and authority structures relevant to the staff's ability to perform the protocol. We emphasize s trengths of anthrop ological ethnography in clinical trials that include the provision of complementary, qualitative data, the capture of otherwise unmeasured parts of the trial, and the realization of important information for the translation of the clinical findings into ne w settings. [ randomized controlled trial, spinal cord injury, ethnography ] ______________________________________________________________________________ Mythological, ritualized and culturally embedded aspects of all healing systems, biomedicine include d, can in themselves possess great healing potential. Orthodox medicine can be blind to such aspects of its own
2 praxis in its claim to scientific legitimacy, and these elements are not studied in the [randomized controlled trial]. Barry ( 2006: 2649 ) We try to give [the staff] updates on the progress of the study as well as share successes and talk about the challenges and how we wan t to meet those challenges for the next plan In the last meeting we set it up so that the front line people, the site coordin ators and the VR c ounselors were presenting information from their perspective on what recruitment strategies were working at their site. And the VR counselor presented success stories, which was very energizing, motivating, and led to a lot of discussion of things. C linician explaining team culture and information sharing within the RCT The randomized controlled trial (RCT) i s considered Western medicine's most rigorous scientific test of a clinical treatment. T he RCT tests an experimental intervention against a control treatment, usually focused on a single specific outcome. 1 Randomization is achieved through strict adherence to statistical methods standardized sampling procedures and defining the dependent variables and controlling for independent variables in the sampling process (Matthews 2000 ) Control is achieved through strategies like stan dardized diagnostic methods, fixed dose medication administration, use of detailed treatment manuals, and rigorous clinician training and certification procedures But for all its highly structured and faithfully attended protocols the RCT is but one hamm er in a very expensive toolbox. As part of the discussions surrounding evidence based medicine, RCT s have been critiqued for the limited degree to which such highly controlled results can be generalized to the larger population of people in need of treatme nt (Lambert 2006 ; Mykhalovskiy and Weir 2004 ) And RCT s often do not "provide the information necessary for clinicians to tailor treatment strategies to the social and cultural needs of their individual patients, who liv e in a wide variety of circumstances" (Hohmann and Shear 2002: 201) We present data that reveal how t he distinctive benefits of an RCT even the mo st regulated, clinic based efficacy trials can be expanded through strategic application of anthropological constructs and ethnographic method s A nthropologists have contributed much to research in clinical settings (Hunter et al. 2008) and to clinical trials (Cabassa et al. 2008; Fisher 2006a, 2006b) but their gaze has rarely been directed unambiguously at the RCT itself, its distinct language, its native perspectives, and its priori ties. Leadership in this work comes from a fairly small group of anthropological scholars (e.g., Adams 2005; Barry 2006; Landsman 2006) This article reports an ethnograph ic study of a clinical trial that treated the c linical processes and context ra ther t han patient experiences as the main focus of research. W e join the critique of the "rhetorics of evidence in biomedicine" (Barry 2006) to reveal how enactors here of a n RCT can simultaneously attend to both the trial's evidenti ary and procedural requirements but also to the lived experiences and needs of patients and clinicians. We therefore illuminate the specific dimensions of ethnography that provide new information to trial assessments and reco nsider and expand the relevent evidence in an evidence based medicine. And w e challenge a common assumption that the RCT is necessarily a sterile and context free outcome of rigorous scientific method.
3 The RCT discussed here wa s not the iconic drug trial in which individual variation i s all but erased from the line of questioning. Instead, this RCT tested a new method of person centered rehabilitation for spinal cord injured v eterans It is a contextually sensitive treatment directed at achieving the outcome of post care employment : a m odel that envisions a broader view of life quality and rehabilitation than traditional medical approaches. Combining qualitative data with this expanded approach to rehabilitation from spinal cord injury, this trial offers a model for more widely applied r esearch in clinical intervention trials, redefining current standards for not only rehabilitation outcomes but also evidence. We identify several advantages of qualitative and ethnographic assessment within clinical trials. First we examine the process b ehind the trial's ultimate results : that is, a broader and more detailed view of the trial in its context so that the specifics of negotiated and "contingent" performance (Adams et al. 2005) re emer ge and the non focal variables are not erased. Second we catalog ue the human interrelationships and idiosyncratic networks that prove important to the trial's outcome. This type of work is not only context driven and locally specific, but requires the eth nographer to discover the social dynamics and processes that undergird but are often invisible to the traditional clinical trial. Third, we map vital power and authority structures that were determinate of key services and features of the trial. By fulfill ing t hese three critical engagements the ethnographic data make the ultimate translation of clinical trial results into real life social hierarchies and the limited resources of patient life worlds more likely. Background and Definitions: Two Histories o f Exclusion W e begin with a review of key background definitions and intersections of on the one hand, RCT s strict rules of evidence and scientific inquiry and, on the other, the contested notions of rehabilitation and its goal of returning patients t o a societally sanctioned and defined standard of function or health. That the forms of scientific and medical research are under constant revision is obvious. Even as the RCT remains valorized as the gold standard in research, ethnographic methodologies h ave been embraced by clinical trial researchers in certain ways Nevertheless a wide gulf remains between the methodologies and rules for an RCT and the inherent variability that anthropological ethnography presumes. By using ethnography to illustrate how this rehabilitation RCT challenges old notions of post injury quality of life, we hope to expose and address two convergent histories of exclusion : one regarding domains accessible only by ethnograph y and the second rehabilitation care What, then, is an RCT? Its proponents argue that an RCT is "a true experiment whereby study participants are assigned by chance, following a pretest, to at least two conditions: An experimental treatment or intervention, and a control intervention sued for purposes of compa rison on outcomes" (Solomon et al. 2009: 5) Evidence from randomized controlled clinical trials is widely accepted as the only sound basis for assessing the efficacy of new medical treatments (Matthews 2000) An RCT typically involve s large sample sizes, has complicated and long documentation requirements, 2 and place s a priorit y on the standardization of treatment delivered. Citing the demanding nature of these characteristics, researchers regularly exclude certain categories of data uniquely accessible through ethnographic methods such as nuances of communication, information sharing, and hierarchy that reflect undocumented power structures profoundly relevant to intervention outcomes ; the narrated experiences of participants, clinical staff, or investigators; and subjective impressions and opinions not captured in written
4 inst ruments In particular, p articipant observation, which requires flexibility as well as a refusal of a priori etic categories, fits very poorly in this context" (Sobo 2009: 70 71) In medical anthropology, sample sizes are usually small, methods are both embedded and unfoldin g, and findings are not easily replicated or validated. Anthropologists value context and culture over things like control and randomization so it is difficult for them to fully recognize the Western/industrial world cultural assumptions of trial protocol s Adams et al. expose several cross cultural challenges to "the very idea of a clini cal trial" (2005: 269). More recently, however, standard ethnographic techniques such as semi structured intervi ewing, participant (and non participatory) observation, and other qualitative methods have been used in clinical studies with growing frequency (Fisher and Kalbaugh 2011; Joseph and Dohan 2009; Swanson and Ward 1995 ) E thnography in clinical settings is typically narrow er than in traditional anthropology; that is, long term immersion and participant observation are impossible or rejected in the cl i nic and a rich data set available from social context is thrown away or never seen (Abe l and Sankar 1995; Sobo 2009 ) But these and more recent studies reveal tha t social process and contextual factors can be incorporated into study designs for very specific treatments (e.g., Hohmann and Shear 2 002; Ware et al. 1999) Our ethnographic r esearch design was based on an ecological, or context rich, perspective (Perkins et al. 2005) using individual and group interviews plus participant observation in both the clinic and in v eterans' lives (though only data on staff narratives are discussed here). 3 Anthropologists were able to participate in all clin ical interactions for the trial as well as staff meetings and regular office activities related to the project. Our participant status was also aided by the treatment of the lead anthropologist as a member of the trial's principal investigative team. Other anthropologists do engage in RCT work, but their subjects are typically the same patients that the trial is designed to treat. In our case, this broad acces s was granted so that the trial itself the broader context in which clinical practices were designed, implemented, and evaluated, and the structures that sustained and limited the focal subject of the study could be evaluated. 4 Our trial therefore presents a promising break in the history of exclusion from evidence that ethnography has experienced. The second history of exclusion intersecting the clinical trial is that of rehabilitation. In medicine, rehabilitation is a broad concept, referring to a variety of care that aims to restore patients to health or normal life ( Oxford Online Dictionary; emphasis added ). Of course, the notion of health or normal life is contingent and contentious Within the confines of an RCT, the definition of health has traditiona ll y been exceedingly narrow, and normalcy was defined almost exclusively by reference to unitary standards and norms inferred through central tendency statistics rather than by its social standards Here is the crux of our problem and the reason an RCT of a rehabilitation intervention i s a good model for redressing this exclusionary bias Rehabilitation as a medical specialty expanded greatly after World War II, in response to the demand to care for returning soldiers with amputations, spinal cord injuries (SCIs) and other physical limitations (Gritzer and Arluke 1989) By the 1950s and 1960s, advocates for the rights of persons with disabilities shifted the focus to de institutionalization and independent li ving supports and o pportunities (Fleischer and Zames 2011) H elping injured soldiers and other disabled persons return to work was an important focus of pioneers such as Howard A. Rusk, M ary Switzer, and others (H. Rusk 1977; H. A. Rusk 1977; Verville 2009) Over time, the importance of employment to the individual and society grew, and government al entities, particularly the individual states, assumed the largest roles in vocational rehabi litation (VR).
5 A national disability rights movement gathered momentum after the signing of the Rehabilitation Act in 1973, and further amendments were added in 1978. Persons with disabilities partnered with rehabilitation professionals in many instances to promote health care access for persons with disabilities and, more centrally, to fight segregation and discrimination against persons with disabilities. In sho rt the social history of rehabilitation has emerged as a praxis that is highly sensitive to context and to social and behavioral activities of value for individuals undergoing rehabilitation, including work. A notable diversity of worldviews exists among VR professionals Advocates of VR argue that disabled persons live in a "catch 22" situation related to work: L ack of money for housing or personal care may force a disabled person into a Medicaid funded nursing home which makes employment seeking difficu lt. Paid employment, they argue, would allow the disabled person the resources to live independently ( Hershey N .d.). But others critique this employment focus as inappropriate for all patient groups and biased in favor of market centric rather than person centered values ( see e.g., www.disabled veterans.org). Complications confront the ideals of rehabilitation that does not share a single definition of what health or a normal life may mean. Of major concern to t he principal investiga tors of this RCT was th e narrow view of rehabilitation for persons with SCI focused on unemployment after SCI and its direct connection to lower quality of life and li fe satisfaction (Chapin and Kewman 2001; De Vivo et al. 1999 ) Yet c om mon approaches to rehabilitation excise employment support from the rest of rehabilitation medicine and are then of such limited intensity or depth as to be of little use for patients with complex needs. The effect of these structural decisions wa s in our view, to (re) produce further reduction in quality of life, completeness of health, and avenues to normalcy. Aware of these concerns, in designing the SCI Vocational Integration Program (SCI VIP) described below, the clinical researchers planned to not onl y take a committed a nd engaged view toward helping v eterans with SCI who wanted to return to work, but also to redefine the scope of possibilities that rehabilitation could include. And they would do this while adhering to rigorous standards of measurement fidelity to an evidence based model, and accuracy of reporting befitting the nomenclature RCT This particular intervention is alr eady laudably context sensitive in that the patient's life context and evolvi ng needs are taken as a whole. That they accomp lish ed this within a n RCT format is noteworthy and may offer model directions for future anthropological work. Research Design and Methods T he SCI VIP tested a treatment model that emphasized social interaction s both between clinicians and between clinic ians and patients T he model is termed supported emplo yment and it calls for both a political and a spatial integration of the VR counselor a role traditiona lly outside the clinical team directly into the space of clinic al negotiations (Becker and Drake 200 3 ; Bond 200 8a; Bond 2008b ) This is different from conventional VR in an important way. While c onventional VR involves intensive assessment, case management, and sometimes therapy, these services typically occur outside the Veterans Healthcare A dministration SCI centers with either the Veterans Benefits Administration V ocational Rehabilitation and Employment office or with state contracted VR providers Supported e mployment, on the other hand, integrates the VR counselor more fully into the clinical team and provides more holistic and continuous support to the client (Becker et al. 2006; Bond et al. 2008 a ; Bond et al. 2008 b )
6 The se priorities are outlined in the core principles of the supported em ployment model of care ( s ee Table 1). Table 1. Principles of Individual Placement & Support (IPS)/Supported Employment (adapted from Becker and Drake 2003:22) 1 Rehabilitation is considered an integral component of treatment, rather than a separate service; 2 The goal of IPS is competitive employment in integrated work settings, rather than prevocational, sheltered, or segregated work experiences; 3 People with spinal cord injury can obtain and succeed in competitive jobs directly, without preemployment training; 4 Vocational assessment is continuous and based in competitive work exper iences, rather than in artificial or sheltered settings; 5 Follow along supports continue for a time that fits the individual, rather than terminating at a set point after starting a job; 6 Job finding, disclosure, and job supports are based on clients' preferences and choices, rather than on providers' judgments; 7 Services are provided in the community, rather than in treatment or rehabilitation settings; 8 A multidisciplinary team approach, rather than parallel interventions in separate agencies or s ystems, promotes the integration of vocational, clinical, and support services. Participants were v eterans with SCI between the ages of 18 and 65 years who received medical and/or rehabilitation health ca re services at the site. Only v eterans who were no t employed or employed but not at a substantial income level were eligible. Veterans were randomly assigned to either the SCI VIP or the standard care condition. Veterans in the standard care condition were free to use whatever vocational services were ava ilable at their Veterans Affairs Medical Center including referrals to outside agencies (e.g., state VR). Detailed protocols were prepared for each activity, including treatment, reporting of activities to the principal investigators ( PIs ) communication within clinical teams, and the like (Ottomanelli 2012) Ethnographic Methods A nthropological methods sought to capture a greater diversity and depth of data on program outcomes and to describe both staff and v eteran's experiences of the trial. T he PI's f oresigh t in measuring social and inter personal process es plus the fact that t he supported employment model they were testing was already a context sensitive intervention, differentiated this RCT from most others. Standard ethnographic methods involved sem i structured and open ended interviews with staff and participants, observations within provider participant encounters, and participant observation in clinical and service activities within the SCI unit. Observations of 28 Veterans Affairs Medical Center staff and/or SCI VIP staff members were conducted Formal audiotaped interviews were conducted with a ll six of the staff Interview q uestionnaires focused on the program schedule and staff roles and re sponsibilities; perceptions of success and failure in t he research project; details of supervision, trai ning, and employee evaluations; significant experiences in the research program; and topics emerging from staff meetings or current cases. Interview narratives were transcribed verbatim, then coded and analy zed using Atlas.ti software to identify recurrent themes, categories, and expressions.
7 Coding was performed to identify k ey and recurrent concepts Two independent coders marked transcripts, followed by discussion and reconciliation of the two lists. Discr epancies were resolved through consensus producing a two level hierarchy of 22 final codes each containing at least five references in at least three different informant interviews. Research Findings Three major ethnographic findings from coded, narra tive data were as follows: (1) i t was important yet difficult to implement the model precisely as designed; (2) a shared culture among the trial's staff had to be cultivated for the model to be successfully implemented ; and (3) a locally authoritative cham pion for the treatment model was essential to implementation. Fidelity Driven Research and Integrated Care The term fidelity refers to whether a n intervention was conducted consistently and completely according to the procedures specified in the proje ct protocol. Clinical trials that adopt the fidelity approach rigorously monitor, coach, and supervise the services performed to ensure ideal adherence to the intervention model (Bond et al. 2008 a ; Bond et al. 2008 b ) V alues associated with the fidelity approac h are the replicability of data and the reliability with which researchers can attest that the model, as designed, was indeed what occurred during research. This charac teristic of the SCI VIP trial a measured dedication to the principles behind the intervention is important because fidelity to the treatment model is one of the requisite features of scientific rigor in an RCT 5 The Intensive Effort of Home Visits To demonstrate fidelity to all of the principl es of the supported emp l oyment model involves rapid engagement zero exclusion and ongoing support (see Table 1) making the VR counselor 's job exceptionally intensive. In the traditional model, clients came to the VR counselor; it is rarely the other way around. In the supported emplo yment model, VR involves numerous home visits and intensive support. One VR counselor for example, had a client who lived at the maximum allowed distance for the trial's catchment area (100 miles). The drives to see him were long, and the patient's reluctance to visit with the VR counselor (even at his own home) was exacerbated by his drug use. In the traditional VR model, this man may never have received support for employment, because those supports were only availa ble in a professional office. But t he trial's VR counselor wouldn't relent : I had a guy in Fort Worth, who had an obvious drug problem and didn't want anyone in his house. He hated company. He admitted to me he signed on [to the SCI VIP] just to get the fifty dol lars [research stipend] every month. So I continued to go out there, you know. Then he started to get angry, because he didn't really want me to visit but I continued to go out anyway. Just kept going, kept going, kept trying to talk to this guy. And he and I started to get along pretty well. Through persistence and a flexible approach to goal setting, the VR counselor established rapport and was able to help this client landed two job interviews, a much better result than traditional methods of VR migh t have achieved. Below, the VR counselor reflects on the
8 difficulty of achieving the same outcome (i.e., paid employment) with clients who start in such different states of readiness : That's difficult, trying to find people at that right place. And spinal cord cases are delicate, and some people take a long time. You know, I've had cases that didn't get a job until that twelfth month. I have people who have never found a job. You know, to think it's going to be just a blanket fix, you know, it's not going to happen. Identifying good targe ts and then providing job leads or setting up interviews for participants was not enough to reach a positive employment outcome. The VR counselor had to discover participants' interests, maximize their capabilities through mechanical, technical, and clinical supports, and then discover possible sources for work where the participant might apply. She was simultaneously a support counselor for the patient and administrative assistant for setting up (and sometimes driving the v eteran to) job interviews. But even these efforts were not enough, considering the tremendous perso nal transformations that these v eterans were experiencing (Smith Morr is et al. 2013) No small aspect of the VR counsel or's task was to help v eterans develop new ideas about productivity and work in their post injury capacities. One VR counselor reported having to do a significant amount of motivational work with her clients : You have to find that caveat, that one carrot that they will grasp on to and say, Wow! Because, I don't want to oversimplify things, but when you present them with the right option, they will do it. The supported emplo yment model demands more of the VR counselor than is typical for non supported em ploy ment settings. B ut b y offering more to the client in terms of support the v eteran's sustained employment becomes more likely. Clinical Integration of the VR Counselor It is s till unclear was how the VR counselor accomplished so much more than her n on supported emp loyment counterparts. A portion of her success was attributable to a smaller caseload. However, as the ethnographers learned through duplicating the VR counselor's driving and home/employer visit schedule, the time savings of the smaller ca seload wer e quickly and completely overshadowed by travel time to homes and c ommunities within a 100 mile radius of the office. More important to the VR counselor's success was clinical integr ation, the first of the eight principles listed in Table 1. As a n integral member of the existing interdisciplinary SCI treatment team, the VR counselor gained access to the treatment team's resources, including clinical professionals and knowledge central to SCI rehabilitation. Persons with SCI have more frequent medi cal complications, such as urinary tract infections, skin issues, or depression. The VR counselor could communicate directly with the veteran's physician, facilitate medical care when necessary, and offer input to treatment planning that connected the heal ing process to a return to work. T he VR counselor also conducts job finding and utilizes techniques that may include job carving, job coaching, and follow along with employers that must be informed by a v eteran's medical status These are all listed in the principles of supported empl oyment and so were the focus of the RCT itself What Ethnography Adds What, then, did ethnography uncover that the RCT outcomes did not already capture? The medical ethnography was designed without reference to any specific e lements of the model or even to the model itself, but rather as an open ended exploration of
9 salient issues and concerns of the informants. The structures and protocols of the program were important to understand. But it was the topic of integrated care th at rose most frequently and naturally within staff discourse. As staff talked about t he supported em ployment principles, and the challenges of meeting these guidelines, integration of care was mentioned far more often than any other as critical to their su ccess. Here is how a non researcher clinician spoke about the inclusion of the VR counselor in the Inter Disciplinary Team (IDT) meetings: [In the IDT m eetings], we talk about cases things that have happened with their medical [condition], things that hav e happened with people and how that has interfered or helped. Or maybe good things, like so and so had a medical issue that came up and [the VR counselor] rapidly referred him to me and we took care of it and avoided a problem [The VR counselor] came t o our IDT meeting [to tell us about this problem] and those are things that help. This clinician is describing the novel feature of the SCI VIP program; namely, the VR counselor's regular access t o the inpatient clinical team the same team of clinicians wh o cared for each client when she or he first received their SCI. Research staff members had this to say: When the [inpatient] staff meeting occurs, we try to give an update on [SCI VIP] enrollment We try to kind of keep it in the forefront of [the clin ical staff's] minds. We've been trying to kind of translate the success stories back to the team, so that they can see kind of the fruits of their labor in terms of what actually is happening; and try to facilitate some of that mutual learning process be tween the vocational clinician and the SCI clinician that didn't previously have any exposure to what voc rehab meant. So, you know, the vocational clinician maybe doesn't know what an OT [occupational therapist] does and so we've been sort of trying to cu ltivate this cross, the integration. Cultivating communication and mutual support between VR and the other disciplines on the clinical team is a key part of integration of care, and is named by multiple types of staff as key to success of the supported em ployment model. By educating the clinical team on the principles and methods of the supported emplo yment model, the trial expanded the normal purview of that team well beyond the clinic and into the communities and work settings of their patients. Likewise clinical integration of the VR counselor promoted her greater knowledge of and sensitivity to the medical needs of v eterans, yielding greater success in her own work The staff n arratives above reveal that primarily fidelity to the first principle of sup ported emplo yment (i.e., integration) was often more important to the overall success of the program than any other And although fidelity monitoring might capture the number and type of services provided or contacts made by the VR counselor it could not capture or characterize what made the relationships work or the interactions successful Thus, ethnographic findings complement the clinical trial outcomes for a more robust description and model of care. Teaching and Celebration The process of bi annual fidelity monitoring reviews could be psychologically painful for those whose work was undergoing scrutiny. Identification of errors or gaps in records is meticulous
10 and preparations for these site visits can be intense and worrisome. However, in their larg er context, these re views became a teachable moment and an opportunity for reviewers/coaches to offer support in this difficult work while promoting the program throughout the site. The six month [fidelity] eval., that's very useful. It's like a day and a half, and it's done at all the sites That includes, you know, opening sessions, and then meeting with different people, and then a closing session They get information from the progress notes, but they also get information from the team. Like, say t hey talk to the team about [the VR counselor], how much is she coming to the [inpatient] team meetings, h ow well is integration working. The biannual reviews were part of the clinical research protocol, but the political support and in depth supervision a nd training that these reviews promoted were not. The fostering of a shared ethos a round supported emplo yment and its notions of integration and exceptional effort by VR counselors was a major undertaking. These bi annual reviews, combined with many other s maller events and meetings at each individual site, provided the mechanisms through which a shared culture could be built among the diverse staff involved in the RCT. Annual conferences of research staff from all sites included (again) training about th e supported e mployment model for the staff and enjoyable trips out of the office, but also an enthusiastic session of awards and recognition for many of the project VR counselors and site co ordinators. The positive and enduring effects of these celebrations could be heard in staff narratives many months later, as in this doctor's comments: We have had annual meetings, and at the annual meeting we try to give people updates on the progress of the study as well as share successes and talk about the challenges a nd how we want to meet those challenges for the next plan [We] also focus on a little bit of skill building because we do bring in our coach [on] the evidenced based principles. In the last meeting, which I think people enjoyed the most and by enjoy, they sa y they got the most out of it we set it up so that the front line people, the site coord inators and the VR counselors were presenting information from their perspective on what recruitment strategies were working at their site And the VR counselo r presented success stories, which was very energizing, motivating, and led to a lot of discussion of things. The centrality of this culture building to the overall success of the trial should not be underestimated, although it is not recorded as part of the trial's outcomes. Even so, to replicate this trial s success, future programmers will need to take seriously the need for a system of shared values and mechanisms of support for advancing these new ideas Here again is the VR counselor: You know, so yo u have to be all over the place. Be out there. With this program being so brand new and the whole notion of SCI and work, I think my first year was spent just doing a lot of marketing face to face, getting to know people, getting your name out there, getti ng the program's name out there, you know? Because people had to get used to the idea of people in wheel chairs going back to work, doing work outside of the computer, and just work outside of going to some kind of workshop. I mean real world
11 work. So peop le have to kind of grasp that concept. I can say this though, now people are more friendly to the idea. I don't meet as much opposition about it. Arguably, the mechanisms and processes through which RCT s taff developed a shared focus the culture behind the RCT is pivotal to the success of any RCT Yet these are precisely the aspects of care that are made invisible in outcome based studies. Trial protocol demands that the model of care itself, not the variable context in which the model is implemented, is th e factor of interest. Yet no matter how carefully we measure and count the interventions, time spent, and minutes invested, it somehow fails to convey the character of "the concept" and whether people are "friendly to the idea Local Champions Finally, the genuine and wholehearted advocacy for th e supported employment model b y specific clinically influen tial staff could not be ignored and was a major theme in staff narratives. Our SCI VIP site (recall we only operated at one of several nationwide sites of the clinical trial) had several advocates, each of whom might have been considered a local champion of its cause. A local cause champion is a person of influence within a setting, who can convince or lead others to join the cause. For the SCI VIP, the m ost important champion would have been a physician on the SCI clinical team. For reasons of confidentiality, we will not reveal the professional role identity of these speakers, but a diversity of voices on the topic concur that a champion must be both kno wledgeable and influential within the hierarchy of a given setting. You get that local cheerleader in leadership. A cheerleader is not somebody who just loves the program. A cheerleader would be someone who talks it up all the time They talk up the pro gram at staff meetings. They tell everybody, you know, show some caring. You motivate employees, you know, to be involved. You motivate them to do their best for the v eterans. When the p rogram is running effectively, v eterans tend to be referred to the pr ogram by their clinicians, and not solely recruited to the program by project staff. Here, a non SCI VIP staff person states: Well, all of us have people that are enrolled in this. All of the clinicians have people that are enrolled. But yeah, so each of u s, all of us have referred people. I feel like that's been a part of my role, is to make sure that things are clinically relevant and that the medical problems that might interfere with people's participation [in the SCI VIP] are looked at and the appropri ate people get into the study or don't get into the study, from a medical perspective. Clinicians work with v eterans regularly but may never have addressed work related questions or treatment needs. When the topic of work is made relevant early in treatme nt, the likelihood of a return to work is not only increased but moved forward in time (Ottomanelli 2012) Two v eterans describe their introduction to the idea this way:
12 They have pamphlets and, at the outpatient desk, and they the doctor asked me to participate in it. So after about two or three years of them asking me I finally decided to go ahead and try to participate in it and see what it was like. I believe it was [my outpatient doctor] mentioned it to me while I was in for an appointment and told me about and I was interested in going back to work anyway, so it was like perfect timing. Another clinician, reflecting on the experience at another site for the SCI VIP, summarized: In [another site], their performance kind of waxed and waned in terms of recruitment. And I think, I'm not sure how much the PI had the time to be a champion. And initially, I think most of the referrals were coming through the PI himself, when you can't get enough referrals through one person. It has to be through the team. So it was actually after that meeting in '08, where we did the success stories, where [he] made some comment about, "Oh, okay, I finally get it. We're not just kind of picking and choosing the people who we think are going to d o well at voc rehab. We're offering it to everybody." So yeah, it's kind of like, it s important for the clinical champion not to necessarily be a screener as it is like, "Let's open this gate really well, let everybody through and see how far they can g o!" Champions are the charismatic and influential leaders that draw both who attention to the program so that the scientific and evidence based ideas develop some local enthusiasm and meaning. Further, they are authoritative members of the clinical team wh ose support for the otherwise burdensome change to normal operations is essential to short and long term integration of employment services in the clinical setting. The fact that the se champions speak as existing insiders to clinical medicine gives the VR counselor not only a social and political support but a vehicle by which to enter the otherwise closed clinical realm. However, these champions must also be willing to use the administrative influence they have to support the myriad needs for implementing such a complex and novel new strategy. In short, no matter how great the quantifiable or scientific success of a program, it must be shared and spread through human relationships and power networks. Discussion The Benefits and Challenges of Ethnography in Clinical Trials One needs charts and soundings to navigate fraught channels, but only the seasoned river pilot has the lived in wisdom to find the way safely through channels. Charon and Wyer ( 2008) Ethnography gets behind the curtain of protocol and models, to describe what makes the RCT work. Ethnographic data on the SCI VIP pointed to three main factors in staff culture and practice. These were, first, the personal forbearance of VR counselors and the client by client adjustments necessary to achie ve the high standards of the model's core principles. Second, there was in this trial, as is likely in many RCT s, an enormous investment in creating a shared vision of the trial's goals, a sense of community across the staff in multiple sites, and intentio nal efforts
13 to foster throughout all levels of staff a high level of capacity, skill, and motivation. And third, locally influential clinicians had to invest in the treatment model and champion its cause in the surrounding political and professional contex ts, for the trial to succeed. These themes represent a substantial body of evidence relevant to the trial's success, yet it is routinely excluded from the clinical literature. How is it that the gold s tandard for scientific research would regularly ignore so much relevant information? Critiques of the RCT, and of the evidence based medicine movement have been steady over the past two decades. 6 Adams assesses the changing context of Tibetan practice under an increasingly biomedical marketplace (Adams 2002, 2005; Adams et al. 2005) The state enjoys the privilege of defin ing what is and what is not valid and safe medical practice s criminalizing certain traditional Tibetan practices for not conforming to dominant stan dards and definitions of care. That the RCT fails to consider patient views and narratives, ignores modalities that are less easily manipulated into standardized metrics (e.g., for physical therapy see Landsman [ 2006 ]; for homeopathy see Barry [ 2006 ] ), or produces overly formulaic guidelines that do not translate well into individual patient care are among the major critiques of RCT (and evidence based medicine ) by social scientists (Lambert 2006) These scholars lament the over valuation of the RCT as a tool for measur ing what works and the large and growing "audit culture" within biomedicine where notions of evidence, efficacy, and cost effectiveness predominate (Barry 2006) 7 Anthropological researc h suffers some of the same politico scientific ma rginalization that Tibetan and h omeopathic traditions have undergone. Our forms of evidence are typically excluded from dominant biomedical discourse and are releg ated a standard far lower than gold in the h ierarchy of scientific achievement (Landsman 2006; Mulligan 2010) Yet, to challenge biomedical and scientific suppression of knowledge is to take up a double edged sword. Although we push the socio cultural dime nsions of health into a conversation that see ms endlessly downward looking microscopically and pharmaceutically focused we are also professionally and humanistically compelled to seek the most effective healing options for our informants in their lived con texts. To succeed in arguments over what constitutes meaningful evidence, medical anthropologists must offer both theoretical and methodological remedies to this problem of effectiveness. Toward this end, we consider and engage with the RCT more closely. 8 First, the RCT can take multiple forms (Schulz et al. 2010; Stolberg et al. 2004; Wells 1999) Efficacy t rials examine whether treatments (e.g., talismans or pills) improve certain outcomes under controlled and iso lating conditions. These types of studies are important for understanding biological, chemical, and physiological events, but are very limited in value for understanding homeopathy or faith healing. Effectiveness t rials, on the other hand, evaluate treatme nts on health outcomes in conditions closer to normal care, including community settings. Effectiveness studies attempt to determine the feasibility of bringing a treatment to a population in need, and therefore sample representative patients. Effectivenes s studies can also evaluate longer term clinical and morbidity outcomes than efficacy trials. Broader still are long term ethnographic studies, which have the greatest ability to map the complexity and inter relatedness of variables in human life but which also seek out causal and explanatory patterns either intra or inter culturally, if not also over time. Neither efficacy nor effectiveness trial s deals as well or as fully with context as anthropological studies. We are sensitive to the admonitions of othe r scholars that RCT s even effectiveness trials, can fail to holistically address the reality of clinical practice and of patients'
14 lives. This stance can, thereby, put itself at odds with patient centered care. We suggest that anthropological me thods can improve both effectiveness and even efficacy trials in important ways. The RCT we have presented was a hybrid of efficacy and effectiveness research that produced clinically significant results. However, it simultaneously produced several compelling ethno graphic findings that offer insight into the forces shaping delivery of these programs in community settings. This RCT was distinctive in two ways: 1. The PIs envisioned the need for qualitative and anthropological perspectives early and then built these int o the trial, addressing some of the gaps that should concern social scient ists about what is touted as a gold standard of scientific research. 2. The treatment model we tested was a community based intervention based on personalized methods and integration o f the VR counselor into the larger clinical team. Given that the VR counselor was simultaneously providing community based services while also serving a s an integrated member of the clinical team, this model shows further hybridity of clinic and community based roles S o, c ontrary to the stage model required by the U.S. Food and Drug Administration for trials of a drug ( U.S. FDA 2012 ) that requires careful determination of drug effects well before a drug can be te sted on humans clinical trials of service models should be hybridized at an earlier phase as this supported e mployment trial was The highly decontextualized and sterile form of RCT must not be considered the gold standard for these subjects and strategies Certain typical trial protocols were more useful than may have been expected at the onset of the research Fidelity monitoring, in particular the quantitative test executed periodically during an RCT to ensure records and treatment conform to the model's principles helped reveal the relativ e importance of integration to the rest of the model's principles and to the trial's overall success in this site As discussed above, the principle of integration helped ensure a high degree of VR cou n selor flexibility and creativity in her work. And ther e was broad agreement among staff that integration was the most important aspect of the model. This strength is largely attributable to the supported empl oyment model being tested, but some credit should also be given to the fidelity monitoring process for capturing its relative importance among all of the model's principles. Ethnography of these protocols identified key structural supports that made care more successful, easier, or more efficient For example, the relative importance of integrated c are to the overall success of the intervention emerged through grounded analysis of this narrative them e and through two independent codings of narratives. That other principles did not emerge in the same way does not suggest they were unimportant to the trial, b ut does suggest that the first principle of integrated c are is a fundamental or first concern for a successful intervention. The combination of careful qualititative analysis, in tandem with rigorous fidelity monitoring (of a highly context sensitive model ), helps make a good model for future social scientific research of and employing the RCT format. T he two other major ethnographic themes were less collaborative with the RCT process, but offered proof of the narrow view that most RCTs take to outc ome evid ence The e ducational and celebration events aimed at ensuring our trial was successful were not an explicit part of the intervention model nor part of the evidence recognized by the gold standard CONSORT guidelines for reporting on RCTs (see N ote 3 and Sc hulz et al. 2010). These celebratory and
15 training/ supervisory events produced i nformation (well known to the staff) that would normally be considered background to the trial or part of the procedures manuals of a clinical intervention, rather than part of its published findings. Ethnographic analysis of these events revealed that these events had a major impact on staff's ability and resources to enact the protoc o ls Local project champions lik ewise were not an explicit part of the trial's protocol, yet the y ensured locally authoritative support for the VR counselor's integration as well as needed training and professional development along the way These events create the needed common culture and mobilize social capital for the trial. In sum, ethnographic data fill gaps of knowledge that traditional RCT s have created. These research methodologies, when embedded within clinical trials, help re define and decolonize notions of evidence. Through these hybrid RCTs, evidence is expanded to include not just CONSO RT guidelines and statistically significant outcomes, but better information about "essential methodological factors" (Devereau x et al. 2002 : 381 ) that would allow "clinicians to tailor treatment strategies to the social and cultural needs of their patients" (Hohmann and Shear 2002 : 201 ) Conclusion Ther e were limitations and challenges to this anthropological ethnography. It was limited both in its duration it did not span the entire course of the trial and in its depth only part time ethnographic study was conducted, and in only one of the three clinica l trial sites. In the future, it would be important to address these weaknesses through long term investigation of the intervention at all sites, ideally from its conceptualization through it s conclusion. However, the access ethnographic staff had to both clinical and home sites was quite good. Further, because of this trial's design, which included home and community settings, there was a diversity of settings for which ethnographic methods are well suited. The findings described here are being used to imp rove the design and reach of a subsequent iteration of this trial, specifically for long term placement of and follow up with participants in the supported emplo yment program. Ethnographic data offer critical evidence about outcomes and about performance of treatment models that expand the impact of an RCT T he complementarity of ethnographic methods to clinical trials, particularly to fidelity driven research models is an opportunity to impact what evidence counts A supplemental team of anthropological researchers can offer substantially different and relevant data on both the model being tested and on any RCT process and context. This model helps pave a new direction in the field of qualitative evaluation research, but also encouraged more clinical and trial based applications of medical anthropology. Notes Acknowledgments This research was supported by the Office of Research and Development, Rehabilitation Research and Development Service, Department of Veterans Affairs (VA RR&D grant no. B3773R). Co ntents of this article do not represent the views of the Department of Veteran s Affairs or the United States g overnment. Clinical Trial Registration No.: NCT00117806. Thanks also to the insightful comments of anonymous reviewers and Editor Mark Luborsky on earlier versions of this manuscript.
16 1. A distinction is made between efficacy trials, typically conducted in tightly controlled clinical or laboratory settings, and effectiveness trials, for which some of the controls are relaxed to achieve samples that ar e more representative of the community in which the intervention is likely to be delivered (Solomon et al. 2009; Stolberg et al. 2004) This distinction is blurred for this discussion of our RCT because it had elements of both types of trial 2. The CONSORT Guidelines include a 25 item checklist of information t o include when reporting an RCT such as: methods for randomization, allocation, and blinding; details on participant flow, recruitment, outcomes and estimation; and discussion of limitati ons, genera lizability, and interpretations (Schulz et al. 2010) 3. Savage summarizes and defines ethnography as it is typically applied in clinical settings as the "study [of] phenomena in their everyday context in an attempt to secure an in depth understa nding" ( Savage 200 6:284 ) Our methodology might further be described as a "naturalist ethnography" ( Skeggs 2001 ) to present our data "in a way that conveys a sense of being there', or indicates the nature of the relationship/s between the research and the researched" (Savage 2006:385). F inally, it is also an applied ethnography, since its focus on the clinical trial is a "narrow field of inquiry" and targets more specific research questions than the traditional descriptive study might address (Savage 2000: 388). 4. As previously stated, this article discusses only one portion of the ethnography, namely the study of the trial's design and implementation. Intensive ethnography was also conducted among patients to e xpand the trial's findings for v eteran patients and their outcomes, though those results are reported elsewhere. It is important to note that there would have been interactions between the availability of these data to staff during the trial and their subsequent motivation, and sense of accomplishment ( or failure). This point i s taken up in the ethnographic fin dings section. 5. Fidelity to t he supported employment m odel was measured in a number of ways. Bi annually, the program records of SCI VIP as well as participant medical records are reviewed to assess the degree to which the program (documented in these notes) adheres to the principles of evidence b ased supported empl oyment described by Bond ( 2008a, 2008b ). During site visits, consultant trainers gather data on program fidelity through semi structured i nterviews with consumers, employers, clinicians, program managers, health care team members, administrative management and leadership; chart reviews; and observation of team meetings. A fidelity scale f or supported emplo yment that has been widely used and is consistently linked with employment outcomes among programs that serve persons with mental illness was also used in SCI VIP (Becker et al. 2001; Bond et al. 2008 b ) 6. A very useful review of this literature begins with Mykhalovskiy and Weir's 2004 publication, followed by a special issue of Social Science & Medicine in 2006 on social science perspectives on evidence based health care. Adams' s work in this field (specifically for Tibet (2002, 2005a, 2005b), is parti cularly compelling, as is Timmerman s and Almeling's broader review and discussion of standardization and objectification in modern medicine (2009 ). 7. Barry suggests three main problems with the audi t culture's reliance on RCTs: (1 ) a preponderance of gaps i n the evidence base, owing to the very narrow g aze of each controlled trial; (2 ) the failure of RCT evidence to address the reality of clinical practice; and ( 3 ) conflicts of evidence oriented medicine "with other accepted healthcare policies, such as pati ent centred medicine, characterized by increasing involvement of patients in decision making about healthcare" ( Barry 2006:2649 2650). She does not, however, address the distinction between efficacy trials and effectiveness trials (see N ote 1 ), however her argument is best aimed at the
17 more highly controlled efficacy trials. In fact, effectiveness trials have come into practice to address some of the concerns that Barry raises. 8. We acknowledge that this stance, and our contribution to the debate over eviden ce based medicine (EBM) may prove part of an "assimilationist response" of biomedicine to the criticisms of EBM. Lambert's 2006 discussion of these criticisms, and biomedicine's strategic responses to them, is exceptionally good. But if, following the mod el our research offers, more RCT scientists work to incorporate ethnographic data and holistic perspectives into the very models they are testing, then the sanctity of research evidence may well be broken and expanded. References Cited Abel, E. K. and A. Sankar 1995 Int roduction: The U ses and E valuation of Qualitative Re search. Research on Aging 17:3 7. Adams, V. 2002 Randomized Controlled Crime: Postcolonial Scienc es in Alternative Medicine R esearch. Social Studies of Science 32:659 690. 2005 Saving Tibet? An Inquiry into Modernity, Lies, Truths, and Beliefs. Medical Anthropology 24:71 110. doi: 10.1080/01459740590905651 Adams, V., S. Miller, S. Craig Nyima, Sonam, Droyoung, Lhakpen, and M. Varner 2005 The Challenge of C ross cultural Clinical Trials Research: Case Repor t from the Tibetan Autonomous Region, People s Republic of China. Medical Anthropology Quarterly 19:267 289. Barry, C. A. 2006 The Role of E vidence in Alternative Medicine: Contrasting Biomed ical and Anthropological Appro aches. Social Science & Medicine:2646 2657. Becker D. R. and R E. Drake. 2003 A Working Life for P eople with Severe Mental Illness. Oxford: Oxford University Press Becker, D. R., J. Smith, B. Tanzman, R. E. Drake, and T. Tremblay 2001 Fidelity of Supported Employment Progra ms and Employment Outcom es. Psychiatr ic Services 52:834 836. Becker, D. R., H. Xie, G. McHugo, J. Halliday, and R. Martinez 2006 What Predicts Supported Employment Program Outcomes? Community Mental Health J ournal 42:303 313. doi: 10.1007/s10597 006 9037 8 Bond, G. R., R. E. Drake, and D. R. Becker 2008 a An U pdate on Randomized Controlled Tri als of E vidence based Supported Em ployment. Psychiatric Rehabilitation Journal 31:280 290. doi: 10.2975/31.4.2008. 280.290 Bond, G. R., G. McHugo, D. R. Becker, C. A. Rapp, and R. Whitley 2008 b Fidelity of Supported Employment: Lessons Lear ned from the National. Psychiatric Rehabilitation Journal 31:300 305. doi: 10.2975/31.4.2008.300.305 Cabassa, L. J., M. C. Hans en, L. A. Palinkas, and K. Ell 2008 Azucar y nervios: Explanatory M odels and Treatment Expe riences of Hispanics with Diabetes and D epression. Social Science & Medicine 66:2413 2424. Chapin, M. and D. Kewman 2001 Factors Affecting Employment Following Spinal Cord Injury: A Qualitative St udy.
18 Rehabilitation Psychology 46:400 416. Charon, R. and P. Wyer 2008 The A rt of M edicine: Narrative E vid ence based M edicine. The Lancet 371:296 297. Devereaux, P. J., B. J. Manns, W. A. Ghali H. Quan, and G. H. Gu yatt 2002 The R eporting of Methodological Factors in Randomized Controlled Trials the A ssociation with a Journal Policy to Promote Adherence to the Consolidated Standards of Reporting Trials (CONSORT) C hecklist. Controlled Clinical Trials 23:380 388. D e Vivo, M. J., J. Stuart Krause, and D. P. Lammertse 1999 Recent T rends in M ortality and Causes of D eath among P ersons with Spinal Cord Inj ury. Archives of Physical Medicine and Rehabilitation 80:1411 1419. doi: 10.1016/S0003 9993(99)90252 6 Fisher, J. A. 2006a Co ordinating Ethical Clinical Trials: The Rol e of Research Coord inators in the Contract Research Indust ry. Sociology of Health & Illness 28:678 694. 2006b Procedural M isconceptions and Informed Cons ent: Insights from Empirical Rese arch on the Clinical Trials Ind ustry. Kennedy Institute of Ethics Journal 16:251 268. Fisher, J. A. and C. A. Kalbaugh 2011 Challenging A ssumptions about Minority Participa tion in U S Clinical Rese arch. American Journal of Public Health 101: 2217 2222. Fleische r, D. and F. Zames 2011 The Disability Rights Movement: From Charity to C onfrontation: Temple University Press. Gritzer, G. and A. Arluke 1989 The M aking of Rehabilitation: A Political Ec onomy of Medical Speci alization, 1890 1980. Berkeley : Universty of California Press. Hershey, L. N. d. Debating Advocacy Prior ities: Free the O ppressed, or Promote Employment? http://www.laurahershey.com/?tag+disability rights movement (accessed February 3, 2010). Hohman n, A. A. and M. K. Shear 2002 Community based Intervention Resea rch: Coping with the "N oise of Real Li fe in Study Des ign. American Journal of Psychiatry 159:201 207. Hunter, C. L., K. Spence, and A. Scheinberg 2008 Untangling the W eb of Critical Inci dents : Ethnography in a Paediatric Se tting. A nthropology & Medicine 15:91 103. Joseph, G. and D. Dohan 2009 Recruiting M inorities W here They Receive Care: Institutional Barri ers to Cancer Clinical Trials Recruit ment in a Safety net H ospital. Contemporary Clinical Trials 30:552 559. Lambert, H. 2 006 Accounting for EBM: Notions o f Evidence in M edicine. Social Science & Medicine 62:2633 2645. Landsman, G. H. 2006 What Evidence, Whose Evidence?: Physical Ther apy in New York State s C linical Practice Gui deline and in the L ives of M others of Disable d Chi ldren. Social Science & Medicine 62:2670 2680.
19 Matthews, J. N. S. 2000 Introduction to Randomized Controlled Clinical Trials. London: Arnold Publishing. Mulligan, J. 2010 It Gets Better If You Do ? Measuring Quality C are in Puerto Rico. Medical An thropology 29:303 329. Mykhalovskiy, E. and L. Weir 2004 The Problem of E vidence based M edicine: Directions for Social Scie nce. Social Science & Medicine 59:1059 1069 Ottomanelli, L., L. L. Goetz A. Suris C. McGeough P. L. Sinnott R. Tocsano S. D. Barnett, D. J. Cipher, L. M. Lind, T. M. Dixon, S. A. Imes, A. J. Kerrigan, and F. P. Thomas 2012 Effectiveness of Supported Employment for Veterans with Spinal Cord Injuries: Result s from a Randomized Multisite S tudy. Arch ives of Physical Medicine and R ehabilitation 93 :740 747. Perkins, D. V., D. L. Born J. A. Raines, and S. W. Galka 2005 Program E valuation from an Ecological Perspective: Supported Employment Serv ices for P ersons with Serious Psychiatric Disabi lities. Psychiatric Rehabilitation Journ al 28:217 224. Rusk, H. 1977 A World to Care For. New York: Random House. Rusk, H. A. 1977 Rehabilitation Medicine. St. Louis: C. V. Mosby. Savage, J. 2000 Participative O bservation; Standing in the S hoes of O thers? Qualitative Health Research 10 : 324 339 Savage, J. 2006 Ethnographic Evidence: The Value of Applied Ethnography in H ea l thcare. Journal of Research in Nursing 11 : 383 393. Schulz, K. F., D. G. Altman D. Moher, and F. T. C. Group 2010 CONSORT 2010 Statement: Updated G uidelines for Reporting Pa rallel Group Randomized Trial s. Annals of Internal Medicine 152:1 7. Skeggs, B. 2001 Feminist E thnography. In Handbook of Ethnography P. Atkinson, A. Cof fey, S. Delamong, J. Lofland, and L. Lofland, eds. P p. 426 442. New York: Peter Lang. Smith Morris, C., G. Lopez L. Ottomanelli, and L. Goetz 2013 Narrating a Return to W ork after Spinal Cord Inju ry. In Reframing D isability and Quality of L ife: A Global Per spective. N. Warren and L. Manderson eds. Pp. 145 162. Netherlands: Springer. Sobo, E. J. 200 9 Culture and M eaning in Health Services Research: A Practical Field Gui de. Walnut Creek, C A : Left Coast Press, Inc. Solomon, P., M. M. Cavanaugh, and J. Draine 2009 Randomized Controlled Tria ls: Design and Implementation for C ommunity based Psychosocia l Inter ventions. New York: Oxford University Press. Stolberg, H. O., G. Norman, and I. Trop
20 2004 Fundamentals of Clinical Resea rch for Radiologists: Randomized Controlled Tria ls. American Journal of Radiology 183 :1539 1544 Swanson, S. M. and A. J. War d 1995 Recruiting M inorities into Clinical Tria ls: Toward a P articipant friendly S ystem. Journal of the National Cancer Institute 87:1747 1759. Timmermans, S ., and R Almeling 2009 Objectification, S tandardization, and C ommodification in Health Care: A Conceptual Readjustment. Social Science & Medicine 69 : 21 27 U.S. Food and Drug Administration 2012 The FDA s Drug Review Process: Ensuring Drugs Are Safe and Ef fective. http://www.fda.gov/drugs/resourcesforyou/consumers/ucm143534.htm (accessed May 1, 2012) Vervi lle, R. 2009 War, P olitics and Philanthropy: The Histo ry of Rehabilitation Medi cine. Lanham, MD: The United Press of America. Ware, N. C., T. Tugenberg, B. Dickey, and C. A. McHorney 1999 An Ethnographic Stu dy of the Meaning of C ontinuity of care in Men tal Health S ervices. Psychiatric Services 50:395 400. Wells, K. B. 1999 Treatment R esearch at the Crossroads: The Scientific In terface of Clinical Tr ials and Effectiveness Re search. The American Journal of Psychiatry 156:5 10.