<%BANNER%>

Effectiveness of an Online Graduate Preparation Program

Permanent Link: http://ufdc.ufl.edu/UFE0024631/00001

Material Information

Title: Effectiveness of an Online Graduate Preparation Program
Physical Description: 1 online resource (68 p.)
Language: english
Creator: Brucato, Brittany
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: graduate, preparation, psychology, school
Psychology -- Dissertations, Academic -- UF
Genre: Counseling Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The present study provides a controlled study of the effectiveness of an online graduate preparation program, The Virtual Advisor. The effects of the Virtual Advisor were compared to a website control and a wait-list control group in relation to a range of graduate school preparation variables. Students who utilized The Virtual Advisor were expected to show (1) an increased level of knowledge and information about graduate preparation; (2) higher levels of confidence in relation to their ability to make career decisions; and (3) higher levels of achievement of their vocational identity. Results showed evidence confirming the first and second hypotheses, that using the Virtual Advisor significantly increases students confidence in their preparedness and knowledge about graduate school, and their belief in their ability to make career related decisions. The results did not show support for the hypothesis regarding vocational identity. Finally, the nature of these findings in relation to the literature on graduate preparation is discussed and limitations and future directions are noted.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Brittany Brucato.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Neimeyer, Greg J.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2012-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0024631:00001

Permanent Link: http://ufdc.ufl.edu/UFE0024631/00001

Material Information

Title: Effectiveness of an Online Graduate Preparation Program
Physical Description: 1 online resource (68 p.)
Language: english
Creator: Brucato, Brittany
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: graduate, preparation, psychology, school
Psychology -- Dissertations, Academic -- UF
Genre: Counseling Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The present study provides a controlled study of the effectiveness of an online graduate preparation program, The Virtual Advisor. The effects of the Virtual Advisor were compared to a website control and a wait-list control group in relation to a range of graduate school preparation variables. Students who utilized The Virtual Advisor were expected to show (1) an increased level of knowledge and information about graduate preparation; (2) higher levels of confidence in relation to their ability to make career decisions; and (3) higher levels of achievement of their vocational identity. Results showed evidence confirming the first and second hypotheses, that using the Virtual Advisor significantly increases students confidence in their preparedness and knowledge about graduate school, and their belief in their ability to make career related decisions. The results did not show support for the hypothesis regarding vocational identity. Finally, the nature of these findings in relation to the literature on graduate preparation is discussed and limitations and future directions are noted.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Brittany Brucato.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Neimeyer, Greg J.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2012-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0024631:00001


This item has the following downloads:


Full Text

PAGE 1

1 EFFECTIVENESS OF AN ONLINE GRADUATE PREPARATION PROGRAM By BRITTANY BRUCATO A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2010

PAGE 2

2 2010 Brittany Brucato

PAGE 3

3 To my Parents

PAGE 4

4 TABLE OF CONTENTS page LIST OF TABLES ................................................................................................................................ 6 ABSTRACT .......................................................................................................................................... 7 CHAPTER 1 INTRODUCTION ......................................................................................................................... 8 2 LITERATURE REVIEW ........................................................................................................... 11 Graduate Admissions .................................................................................................................. 11 Student Awareness about Graduate Admissions ....................................................................... 13 Student Preparedness .................................................................................................................. 15 Getting into Graduate School Meetings ................................................................................. 17 Graduate Preparation Books and Web Resources ............................................................. 17 Graduate Preparation and Career Courses ......................................................................... 19 Graduate Preparation Programs .................................................................................................. 25 Specific Aims .............................................................................................................................. 26 Hypotheses ................................................................................................................................... 29 3 METHODS .................................................................................................................................. 31 Participants .................................................................................................................................. 31 Procedure ..................................................................................................................................... 31 (1) Treatment Group: ........................................................................................................... 31 (2) Website Wait -List Control Group: ............................................................................... 32 (3) Natural Wait -List Control Group: ................................................................................. 32 Measures ...................................................................................................................................... 33 Grad Prep Quiz (Neimeyer, 2003) ...................................................................................... 33 Psychology Majors Career Information Survey (Thomas & McDaniel, 2004) ............... 34 Psychology Majors Career Information Quiz (Thomas & McDaniel, 2004) .................. 34 Career Decision Self -efficacy Scale Short Form (Betz, Klein & Taylor, 1996) ............ 35 Career Exploration and Decidedness Inventory Revised (Thomas, McDaniel & Wagner 2006) ................................................................................................................... 35 Virtual Advisor Module Quizzes (Neimeyer & Stevenson, 2008) ................................... 36 Demographic and Self Evaluation Instrument ................................................................... 36 4 RESULTS .................................................................................................................................... 38 Demographics .............................................................................................................................. 38 Descriptiv e Statistics and Preliminary Analyses ....................................................................... 40 Manipulation Checks .................................................................................................................. 43

PAGE 5

5 Multivariate Test ......................................................................................................................... 43 Follow -Up Univariate Tests ....................................................................................................... 44 Behavioral Reports ...................................................................................................................... 48 5 DISCUSSION .............................................................................................................................. 51 Summary of the Results .............................................................................................................. 51 Behavioral Reports ...................................................................................................................... 53 Limitations ................................................................................................................................... 56 Implications and Future Directions ............................................................................................ 57 Conclusion ................................................................................................................................... 58 APPENDIX A BAR CHART FOR GRAD PREP QUIZ SELF EFFIC ACY AT PRE AND POST TEST ............................................................................................................................................ 60 B BAR CHART FOR PSYCHOLOGY MAJORS CAREER SURVEY AT PRE AND POST TEST ................................................................................................................................. 61 C BAR CHART FOR PSYCHOLOGY M AJORS CAREER QUIZ (GRAD PREP KNOWLEDGE) AT PRE AND POST TEST ........................................................................... 62 D BAR CHART FOR CAREER DECISION SELF EFFICACY AT PRE AND POST TEST ............................................................................................................................................ 63 LIST OF REFERENCES ................................................................................................................... 64 BIOGRAPHICAL SKETCH ............................................................................................................. 68

PAGE 6

6 LIST OF TABLES Table page 3 1 Meas ures ................................................................................................................................. 33 4 1 Frequency by number of modules completed ...................................................................... 39 4 2 Summary Statistics of Measures ........................................................................................... 41 4 3 Intercorrelations Among Variables ....................................................................................... 42 4 4 Means and Standard Deviations for Grad Prep Quiz Self -efficacy Subscale .................... 45 4 5 Means and Standard Deviations for Psychology Majors Career Information Survey ....... 46 4 6 Means and Standard Deviations for Psychology Majors Career Information Quiz .......... 47 4 7 Means and Standard Deviations for Career Decision Self -efficacy Scale ......................... 48 4 8 Means and Standard Deviations for Career Explorat ion and Decidedness Inventory ....... 48 4 9 Percentages by Group Endorsing Behavioral Report Items ................................................ 50

PAGE 7

7 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy EFFECTIVENESS OF AN ONLINE GRADUATE PREPARATION PROGRAM By Brittany Brucato August 2010 Chair: Greg Neimeyer Major: Counseling Psychology The present study provides a controlled study of the effectiveness of an online graduate preparation program, The Virtual Advisor. The effects of the Virtual Advisor were compared to a website control and a wait list control group in relation t o a range of graduate school preparation variables. Students who utilized The Virtual Advisor were expected to show (1) an increased level of knowledge and information about graduate preparation; (2) higher levels of confidence in relation to their ability to make career decisions; and (3) higher levels of achievement of their vocational identity. Results showed evidence confirming the first and second hypotheses, that using the Virtual Advisor significantly increases students confidence in their preparedn ess and knowledge about graduate school, and their belief in their ability to make career related decisions. The results did not show support for the hypothesis regarding vocational identity. Finally, the nature of these findings in relation to the literat ure on graduate preparation is discussed and limitations and future directions are noted.

PAGE 8

8 CHAPTER 1 INTRODUCTION This purpose of the current study is to evaluate the validity and effectiveness of an online graduate preparation program on undergraduate ps ychology majors. The need for such research is clear when looking at the demand placed on current advising structures within the psychology major. As a discipline, psychology has seen record increases in undergraduate majors, and correspondingly in gradua te applications and degrees since the 1980s. There has been a 59% increase in the number of bachelors degrees awarded in psychology since 1985, which is an average increase of 3,800 more graduates per year (McDonald, 1997). This trend has continued with a total of 85,614 bachelors degrees awarded in psychology in 2005 (National Center for Education Statistics, 2006). McDonald notes that what is most problematic about this trend is that there has not been a corresponding increase in psychology faculty (onl y 9.8% change since 1985) to deal with the increasing demands of enrollment in psychology courses. Student to faculty ratios have been estimated up to 50:1 and continue growing (Shepperd, 1991). Moreover, this exponential increase in psychology undergradua tes has produced a parallel demand for graduate school advisement, with 23,936 graduate degrees conferred in 2005, compared to only 13,338 in 1985 (National Center for Education Statistics, 2006). This surge in graduate training has created a considerable demand on undergraduate programs to offer guidance and preparation for graduate study in psychology. Due to the increasing competition for graduate admissions in psychology, students can benefit greatly from any involvement that helps facilitate the skil ls and knowledge to effectively deal with aspects of the application process (Buskist, 1999). While it is optimal to have faculty facilitating this mentoring process, the increasing numbers of psychology undergrads, and lack of concurrent increases in facu lty, is placing considerable demand on existing mechanisms of

PAGE 9

9 advisement; the resources simply do not exist to give individualized attention to students (Satterfield & Abramson, 1998; Shepperd, 1993). In fact, this strain on resources has become so severe that some have suggested specific criteria to limit the number of psychology majors (Shepperd, 1993). Satterfield & Abramson (1998) note that many faculty tend to have very minimal interaction with undergraduate students because their time is being spread so thinly across teaching, research, publishing and their own graduate students. Accordingly, several studies have found that, when asked, students often reported a desire for more information and advising in career and graduate planning and preparation (Johanson and Fried, 2002; Ogletree, 1999; Sheehan, 1994; Sheehan and Granrud, 1995; Ware 1986). A review of the literature shows that overall there is a paucity of research on graduate preparation. Historically, ways for meeting this demand have not bee n thoroughly developed or studied. Although there is fair amount of conceptual material available in the form of books and informational meetings, strikingly little empirical research exists to support the effectiveness of such resources. There has been re search in related areas; for example a number of studies have looked at various issues in relation to graduate admissions standards, procedures and criteria (Bonafazi, Crespy, & Rieker, 1997; Cashin & Landrum, 1991; Johanson & Fired, 2002; KeithSpiegel, T abachnik, & Spiegel, 1994; Landrum, Jeglum & Cashin, 1994; Norcross, Haynch & Terranova, 1996) but only a few have looked specifically at the mechanisms or outcomes associated with graduate preparation. To deal with the increasing demand for graduate pre paration, several strategies have been used, including: (1) informational meetings, (2) books and (3) courses/seminars designed to teach about careers in psychology and/or graduate preparation. What follows is a comprehensive review of literature on gradua te admissions, preparation, and support for the effectiveness of

PAGE 10

10 these current methods The purpose of the present study is to examine the effectiveness of a novel mechanism for preparing undergraduates for admission to graduate programs in psychology. S pecifically this study will be evaluating the validity and efficacy of an interactive online graduate preparation program.

PAGE 11

11 CHAPTER 2 LITERATURE REVIEW This chapter provides a review of the literature on graduate preparation. First, current research findi ngs on graduate admissions criteria will be discussed. What follows is a review of the literature on how much students know about graduate admissions and how prepared they are for the graduate admission process. Finally, the literature on current strategie s for graduate preparation will be explored. The review will conclude with the specific aims and predictions put forth in this study. Graduate Admissions Research has shown that there are many criteria that are important in the graduate admissions proce ss. The main selection criteria used by graduate admissions committees are usually cited as GPAs, GRE scores and letters of recommendation (Keith Spiegel, 1991). Landrum, Jeglum and Cashin (1994) surveyed 55 doctoral programs in psychology and their resu lts reaffirm that GPA, GRE and letters of recommendation are among the most heavily weighted criteria. In addition, the personal statement and research experience were also found to be highly important. Of lesser, but still moderate importance, were speci fied courses, clinical experience, and interviews. In reality these criteria only represent a portion of what these committees are looking for and Keith -Spiegel, Tabachnick and Spiegel (1994) developed a survey to explore how graduate programs (clinical a nd nonclinical) would rank the relative importance of various second order criteria. Those criteria that were ranked the highest included research experience, a publication or professional presentation, degree of fit between applicant and program, clarity and focus of personal statement, and writing skills among many others. The authors support the conclusions of Landrum et al (1994) noting that due to the increasing volume of applicants, admissions

PAGE 12

12 committees are relying more heavily on these second -orde r criteria as it becomes harder to discriminate amongst so many applicants based solely on GPA and GRE scores. Similarly, in a survey of 2,023 graduate psychology programs (terminal masters and doctoral) Norcross, Haynch and Terranova (1996) reported on the required and average GRE scores and GPAs, as well as how programs ranked the importance of certain non -objective criteria. Programs ranked the following items (in order of importance): (1) personal statement, (2) letters of recommendation (3) research experience (4) interview (5) work experience (6) clinically related volunteer work and (7) extracurricular activities. Previous studies support these findings (Eddy, Lloyd & Lubin, 1987; Smith, 1985). Doctoral programs rated the importance of research he avier than terminal masters programs. Finally, the authors conclude by noting the many robust differences among doctoral versus masters programs, including more costly applications, earlier deadlines, higher requirements for GPA and GRE scores, more weight given to nonobjective criteria especially research experience, and more financial support. More specifically, Mayne, Norcross and Sayette (1994) noted that amongst clinical psychology doctoral programs there are differences in admissions criteria dependin g on the level of research versus practice emphasis. All clinical programs surveyed had similar expectations for undergraduate coursework, but differences emerged with research -oriented programs having on average higher GRE scores and GPAs and offering more financial support. Finally, Bonafazi et al. (1997) did a study indicating that Clinical Ph.D. programs view applying with a masters degree more negatively than PsyDs, counseling, and school psychology programs. Additionally, clinical Ph.D. programs ra ted research experience higher with less emphasis on work or life experience in comparison to the PsyD programs. Counseling and school psychology programs rated both as important.

PAGE 13

13 Student Awareness about Graduate Admissions The main concern associated wi th graduate admissions is whether students are aware of the relative importance placed on various admissions criteria. In attempting to explore this question, Cashin and Landrum (1991) surveyed 36 psychology undergraduates examining what criteria they perc eived as most important in graduate admissions. Findings showed that while 97% endorsed GPA as important, little more than half of the participants recognized the importance of letters of recommendation (58%) and GRE scores (56%), and even fewer identified the importance of research experience (31%), the personal statement (11%) and fit with the graduate program (11%). More recently, Brihl (2001) supported these findings with sample of 90 undergraduate psychology majors, 75% of which reported that they p lanned to go to graduate school. Similarly, the results indicated that undergrads were aware of most of the major criteria such as minimum GPA and GRE scores, but were still not as aware of the importance placed on earlier mentioned second order criteria s uch as letters of recommendation, personal statement, research experience and interviews. Moreover, students tended to underestimate average GPA and GRE scores of accepted applicants yet overestimated starting salaries of recent graduates. This work is fur ther supported by Neimeyer et al. (2004) whose findings indicated that undergraduate psychology majors displayed only modest amounts of knowledge on the graduate preparation process. The majority of the sample (248 undergraduates) was unaware of average GP A and GRE scores of successful graduate applicants even when asked in multiple choice format. It should be noted that in this study seniors showed a higher level of grad prep knowledge than underclassmen which suggests that students may become more knowledgeable and prepared as they approach graduation.

PAGE 14

14 Keith -Spiegel et al. (1994) note that some students may be distressed when they learn of certain second order criteria and realize that it is too late in the process to fill in those gaps. At the same time, it is critical that they realize that they can still focus their energy on other second order criteria whose importance is often underestimated. Even late starters can assure that the statement of purpose is exceptionally well crafted and select graduat e schools carefully, keeping a keen eye to goodness of fit (Keith Spiegel et al., 1994, p. 81). Taking these findings into account, it is understandable that both current and former students have reported a desire for more assistance and information in th e preparation of post graduate plans (Johanson and Fried, 2002; Ogletree, 1999; Sheehan, 1994; Sheehan and Granrud, 1995; Ware 1986). Ware (1986) surveyed students at a small university and found that they rated courses in career development and field plac ement for psychology as the most valuable sources of career information. Those were followed by psychology advisors, instructors and departmental presentations on careers. In response to an open -ended question about what additional support students desire d, the most commonly cited themes were more information about graduate school and career opportunities and more help in career planning. More specifically, in an assessment evaluating student satisfaction with an undergraduate psychology program, Sheehan and Ganrud (1995) found that both students and alumni reported the lowest satisfaction in areas such as faculty assistance with postgraduate career planning and opportunities to be involved in research projects. In fact, when asked for suggestions on ways of improving the program, both groups recommended offering students more advising on careers and graduate school options in psychology. Likewise, in an assessment of another undergraduate psychology program, Ogeltree (1999) found similar concerns when surveying recent graduates;

PAGE 15

15 overall students rated assistance in future planning and career advising as the areas that needed the most improvement. Student Preparedness In order to assess what is needed in the area of graduate preparation it is first importa nt to gain a clear of understanding of what students are currently doing to prepare for graduate study. Ogeltree (1999) assessed what career information resources recent graduates reported using and what they perceived as being most helpful. The most commonly reported sources were the psychology department faculty (29%), followed by the department chair (25%), a guide for psychology majors office booklet (25%), psychology advisors (22%), books (18%), programs through Psi Chi (13%), and careers in psychology speaker series (13%). All of these resources had average helpfulness ratings ranging from 2.06 to 2.44 on a scale of 1 to 3 (little help to great help). Additional resources utilized such as career services (20%), career advising sessions (13%), an d the Psychology career advising center (4%) all had lower helpfulness ratings averaging a score of 1.75. Expanding upon this Lee and Neimeyer (in press) surveyed 590 undergraduate psychology majors from across the nation to examine the range of graduat e preparation resources being utilized. Findings indicated the main resources for obtaining graduate school guidance were through visiting a departmental advisor and talking with a professor on a few occasions which were both endorsed by 69% of the partici pants. In addition, 50% reported having an ongoing relationship with a faculty mentor, 42% read graduate preparation books, 25% went to a career resource center, 25% attended a getting into graduate school seminar, and 5% reported taking a graduate preparation course. Within this sample, seniors not only displayed higher levels of knowledge about graduate preparation, but also reported engaging in more graduate preparation activities. Additionally, those students approaching graduation reported higher leve ls

PAGE 16

16 of confidence in relation to the graduate preparation process. These results replicate earlier findings by Neimeyer et al. (2004). This study also evaluated the types of academic and extracurricular involvements in which students were participating to better prepare them for graduate school. Sixty eight percent reported doing at least one semester of research, 52% were members of Psi Chi, 45% had written or planned to write a senior thesis, 29% participated in some kind of mental health volunteer work, 17% had presented at a conference, and 5% had a publication. Additionally, there were significant differences in the types of activities engaged in based on type of degree being pursued by the participant (Masters, PsyD or PhD). For example, while all thr ee engaged in a high rate of research and volunteer work, those pursuing Ph.D.s were more likely to have done a senior thesis, to have a publication, and to have presented at a conference. In conclusion, it is clear that graduate admissions are a nuanced process that students may find difficult to navigate without the proper advising and/or resources. An American Psychological Association (1983) survey of department chairs highlighted the need for improvements in career advising for psychology students. O ut of 100 department chairs interviewed by phone, 40 % endorsed this as a problem facing their department and 26% felt that APA should provide more information on careers and graduate school opportunities for psychology majors. Accordingly, recent studies continue to support this conclusion with repeated findings that students would like more advisement on careers and graduate preparation. Results from certain studies have suggested that part of the problem is that students underutilize and/or lack awarenes s of all of the career resources that are available to them (Lunnenborg, 1986; Ogeltree, 1999), raising questions concerning the goodness of -fit between the mechanisms available for graduate school advisement (e.g., meetings with advisors) and common

PAGE 17

17 mecha nisms or contexts for contemporary learning on the part of aspiring graduate students (e.g. the Internet). A number of strategies have evolved to try to assist students with their graduate student preparation and application process, and these include a r ange of conventional and more contemporary mechanisms. Getting into Graduate School Meetings Traditionally, one of the most commonplace methods for disseminating information about graduate school has been the getting into graduate school meeting whic h is typically some kind of lecture or panel discussion hosted by a local Psi Chi or psychology club chapter, faculty, or graduate students (Buskist, 1999). Satterfield and Abramson (1998) discuss how one of the main benefits of a psychology club is to hos t workshops on the graduate preparation process including information on how to write a vita and personal statement and advice on how to choose the appropriate type of grad programs. In a study of study of 590 psychology majors, 25% endorsed having attended a getting into graduate school seminar (Lee & Neimeyer, in press). While such meetings can clearly be helpful, they also have many limitations such as limited time, typically lasting 12 hours, inadequate individualized attention, and little or no foll ow up (Buskist, 1999). Moreover, there has been no documented evidence to substantiate whether this type of format is comprehensive enough to fulfill the growing demands of all that is involved in the graduate admissions process. Graduate Preparation Bo oks and Web Resources Another common way for students to gain information on graduate study is through graduate preparation books. Titles include Getting In: A Step -by -Step Plan for Gaining Admission to Graduate School in Psychology (APA, 2007), The Comple te Guide to Graduate School Admission: Psychology and Related Fields (Keith Speigel & Wiederman, 2000) and Preparing for G raduate Study in P sychology: 101 Questions & Answers ( Buskist & Burke,

PAGE 18

18 2006) among many others. Common topics covered in these guides are various careers in psychology as well as different areas of specialization, assessing the fit of a graduate degree in psychology, and looking at the different types of degrees and training models. Additionally, another large focus is deciding which pr ograms to apply to, information about the various criteria that are used in the admissions process, information on filling out the applications and tips on writing your vitae, essays, asking for letters of recommendation and the interview process. Additio nally some books such as The Insiders Guide to Graduate Study in Clinical and Counseling Psychology (Norcross, Sayette & Mayne, 2008) include all this information in addition to the most current data on more than 300 accredited programs (i.e. training mode l, descriptions of specializations or tracks, listing of specific acceptance and attrition rates). Perhaps one of the most commonly used books, APAs Getting In, has recently become available for online access as an e -book. In addition, a multitude of websites and web resources have become available which also cover the gamut of career related and graduate preparation information for psychology majors. While books and web resources are certainly are helpful, the question remains, are they sufficient? On e study found that 41% of 590 participants endorsed reading a book on preparing for school (Lee & Neimeyer, in press). Yet there is still little research on whether books or internet websites are adequately addressing students needs in relation to the gra duate preparation process. For this reason, some institutions have created seminars or courses which address graduate preparation. Buskist (1999) notes that while there is considerable written information on graduate preparation, this does not negate the a dded benefits that a seminar can offer to students.

PAGE 19

19 Graduate Preparation and Career Courses Buckalew and Lewis (1982) were among the first to publish on a seminar for senior psychology majors that they describe as going beyond the traditional psychology curriculum and addressing life preparation. Topics of this course included career opportunities in psychology as well as alternative careers, graduate preparation, personal and fiscal development, management, resumes, correspondence, ethics and selected topics. While this is one of the first documented courses that addressed graduate preparation, it was one area of focus out of an otherwise varied number of topics. This course did not represent a comprehensive format for graduate school preparation but was a precursor to other courses which built upon this format but specified their content to graduate preparation. Many of these seminars are designed as career development courses specifically for psychology majors that offer a component of graduate school preparation and information on graduate school options. In these cases though, it is one topic of many and is usually not the focus. One such course developed for upperclassmen by Ware (1988) included the objectives of increasing students knowledge about the self, job and graduate school opportunities and to teach job -search skills. The section which explores graduate school options addressed psychology in addition to related professions such as medicine, dentistry and law. Course activities included inviting speakers from various fields to speak on their area level of satisfaction, and graduate preparation. Additionally, this course presented a videotape called Pursuing Graduate Study in Psychology as well as a booklet with additional information o n the process. Several studies were conducted to assess the effectiveness of this course. The results showed that students had higher ratings on self knowledge, career maturity and vocational identity indicating that overall the course helped to increase knowledge about ones self. Additionally there were increases in decisiveness and knowledge about the job world and job search skills and a reduction in career

PAGE 20

20 related anxiety. Findings were general and did not specifically address the effectiveness of the graduate preparation component. Davis (1988) created a course designed specifically to introduce student to the profession of psychology which was open to junior and senior psychology majors. A one credit course which included weekly presentations and di scussions on topics including: (1) Resume Preparation (2) Becoming Involved: Professional Organizations and Research (3) Graduate Training? Where? What kind? (4) Employment? Where? What level? Strategies? And (5) Future of the Field. Additionally course requirements included completing a resume and a statement of goals and purposes and several short answer exams aimed at integrating the material. Dodson, Chastain and Landrum (1996) describe a seminar for upper division psychology majors designed to teach students about both career and graduate school options in psychology. The authors note that this endeavor grew out of the awareness that many students did not seem aware of the various career opportunities available to them. Furthermore, they seemed even less knowledgeable about the various elements involved in applying to graduate school. They developed a pass -fail three credit course which involved weekly journal entries with personal reactions to class activities. No exams or formal papers we re required and no textbook was used. The journal entries allowed the instructors to receive feedback on presentations and activities and to monitor areas of confusion for students. The authors conducted a pre and post course survey on the forty two enrol led students to examine how students responses changed after completing the course. Results showed that the students tended to have changes in relation to the level of degree they planned to pursue (e.g. those who initially reported desiring a doctoral de gree were now planning on pursuing a masters degree) as well as how they planned to finance graduate

PAGE 21

21 school. Overall on a scale of 1 10 the students reported overall satisfaction with the course as a 9.5 and instructor effectiveness of 9.65. Thus the inst ructors concluded that the course was an effective way to inform students about career options and graduate study in psychology. A similar course developed by Buskist (1999) was designed solely to prepare undergrads for graduate school in psychology. Thi s was a two credit course intended to help students become more informed about the entire graduate school application process as well as to begin creating their application materials. Activities included two graded written assignments, a letter of intent a nd a vita, on which were they were given feedback and could turn back in as many times as necessary to receive an A. There were also non -graded components including figuring out which area of specialty within psychology most fit with their goals, creating a list of their top 10 grad programs, completing a mock interview, and becoming familiarized with the GRE. The first few classes were centered on general information about grad school and questions. Later in the course Buskist has several guest speakers come and address various questions about graduate admissions in their specified field. In a mailed follow up survey, 11 out of 28 respondents had applied to graduate school of which 10 were accepted (6 to masters programs and 4 to doctoral programs). The a verage number of programs applied to was 5 with a range of 1 to 11. These 11 students rated the course as a 4.4 out of 5 (1= not very helpful; 5 = very helpful). According to Buskist (1999) the advantage of this seminar is that it provides students with a structured, systematic approach to learning about graduate school and the application process as well as a supportive environment...(p.287). The most comprehensive study on this type of course to date was conducted by Thomas and McDaniel (2004), who present empirical support for the effectiveness of a required career planning course for psychology majors. This one credit graded course included lectures,

PAGE 22

22 discussions, small group activities and guest speakers and was a requirement for all psychology maj ors. This course was designed to meet three objectives, (1) to increase knowledge about career opportunities for psychology majors (including information on careers, graduate preparation and on other related helping disciplines such as social work, etc.); (2) to increase students confidence in making career decisions (through exploring career related interests, skills and values); and (3) to help students achieve their vocational identities. The sample consisted of 165 students, 85 of whom took the careers course and the other 80 (sampled from the prior years required research methods course before the career course was implemented) who served as a control group. The authors tested the students knowledge about psychology careers with a measure they develop ed for the study call the Psychology Majors Career Information Survey (PCMIS); results indicated that those who took the career planning course significantly increased their self -perceived knowledge in relation to career options. Additionally findings indi cated that those same students also had increased levels of confidence in their career decision-making abilities (as measured by the Career Decision Self -efficacy Scale-Short Form) and movement toward achieving their vocational identity (as measured by the one item Brief Evaluation of Vocational Identity BEVIS). The authors noted several limitations of the study, including the fact that the PCMIS measure perceived rather than actual knowledge; also the BEVIS only included one item which combined two aspects of vocational identity achievement, exploration and contemplation, rather than separating them. Building upon this work in a second study, Thomas and McDaniel (2004) sought to replicate these results and address some of the aforementioned limitations by creating more comprehensive measures. This study had a sample of 72 students enrolled in the careers course and included all of study 1s measures in addition to two new measures the authors created. The

PAGE 23

23 first measure was the Psychology Majors Career Info rmation Quiz (PMCIQ) which was a true/false measure designed to tap into actual knowledge about psychology careers. The second was the Career Exploration and Decidedness Inventory (CEDI) which had two subscales that separately measured career exploration ( exploring career options) and decidedness (making a career choice). The results replicated study 1 and also showed support for increases in actual knowledge about psychology careers as well as changes in both career exploration and decidedness. The author s note that their course succeeded in meeting its goals and highlight the importance of the finding that the careers course was able to increase students confidence in their ability to make appropriate career related decisions. Finally, the authors conclu de by asserting that career courses are an effective way for psychology departments to help students make a successful transition from college into the world of work and professional careers (p.26). While the aforementioned courses are not necessarily a common component of academic curriculum, a growing number of Universities have been offering careers in psychology courses, graduate preparation courses or both. A recent request to the Teaching of Psychology listserv, yielded a number of responses from fa culty who are teaching this type of course. Some of the courses focus more on identifying career opportunities and choices. Others combined both components of career planning and graduate preparation. Common components of the syllabi from these courses inc luded: Differentiating between types of degrees and programs in psychology, assessing career goals, creating a curriculum vitae, how to secure strong letters of recommendation, writing a personal statement and researching and identifying graduate programs.

PAGE 24

24 Many have noted that when these courses are designed for upper division students it can often be too late for them to implement what they have learned before they graduate (Dodson et al, 1996; Murray, 1999). In response to such criticisms Dillinger and La ndrum (2002) have suggested an introduction to psychology course which they refer to as IPM or Introduction to the Psychology Major, that would be 1 credit pass -fail course required for incoming freshmen. This course would cover various topics including career options, whether psychology is a good fit, benefits of research, and ethics; ideally it would allow students a place to make a preliminary assessment of what they think they would like to do after graduation. The authors did a pre and post tes t of 190 students enrolled in the IPM and found significant differences in students levels of awareness about the field of psychology and the various opportunities available to them. Therefore, they conclude that the IPM is an effective way to introduce s tudents early on to the profession of psychology. As beneficial as these courses may prove to be, they are only offered at a limited number of academic institutions. In a survey of 357 psychology departments only 34.2% reported offering some type of introduction to psychology course (Landrum, Shoemaker & Davis, 2003). Additionally, in a study of 590 psychology majors, only 4.9% reported having taken a graduate preparation course (Lee & Neimeyer, in press). Ultimately, both types of courses have their adva ntages. Courses taken later offer students a chance to have matured and gained more knowledge and focus about their goals. Courses offered earlier in freshman or sophomore year can help students to be more informed early on about potential career choices a nd what relevant extracurricular activities (i.e. research, volunteer work etc.) are necessary for them to achieve future goals ( Lloyd, 2002).

PAGE 25

25 Graduate Preparation Programs While a few studies have shown preliminary evidence that courses on preparing fo r graduate school are effective, overall there is still a paucity of research in this area. Furthermore, many schools do not offer such a course which invariably leaves some students at a disadvantage. Neimeyer et al (2004) argue that the current resourc es are not adequately meeting the growing demand and that many graduate students are left poorly informed in relation to graduate preparation. Only one study to date has examined a different type of graduate preparation intervention. Neimeyer et al. (2004) examined the effectiveness of an audio -taped graduate preparation program which included six 1 hour tapes titled Successful Strategies for Getting Into Graduate School in Psychology; Ten Critical Things You Need to Know Before Applying to Graduate Stud y in Psychology; Distinguishing Clinical and Counseling Psychology; All About Personal Statements, Resumes, and Letters of Recommendation; Effective Interviewing for Graduate Schools in Psychology; and Expert Advice: Questions and Answers. Eightee n undergraduates were randomly assigned to either a treatment or wait -list control group. All participants first met and completed the Grad Prep Quiz, then the treatment group listened to the audiotape series over a two week period and both groups then re took the Grad Prep Quiz. The Grad Prep Quiz measures students degree of knowledge and level of preparedness in relation to graduate study in psychology (Neimeyer, 2003). Results showed that as predicted, participants in the treatment group who listened t o the full grad preparation program had significantly increased their Grad Prep Quiz scores at follow up while the control groups scores did not significantly change. Additionally, consistent with the findings of Dodson et al. (1996), 38% of participants i n the treatment group changed their graduate degree plans compared to only 11% in the control group. These findings support the potential usefulness of a low -cost, portable, and

PAGE 26

26 flexible intervention (i.e., an audiotape program) as an effective mechanism for supplementing conventional approaches for providing graduate school preparation (p. 31). It must be noted that this study has several limitations including first and foremost a very small sample size. Additionally, the results offer only proximal indicators of the actual level of graduate school readiness or effectiveness in the application process. Likewise, the data was self report with a lack of any concomitant behavioral indicators of success. The hope of the current study is to build upon thi s work with a new and improved version of a graduate preparation program as well as to address some of the aforementioned limitations. Specific Aims The Program being evaluated in this study is The Virtual Advisor: Successful Strategies for Getting into Graduate School in Psychology (Neimeyer & Stevenson, 2008) which is broken down into five different modules each of which includes video and/or PowerPoint audio presentations (totaling approximately 9 hours), in addition to written resources, interactive e xercises, self assessments, and weblinks. Below in bulleted format are the titles of each module, followed by a listing of the presentations within that module: Module 1: Exploring Careers in Psychology o (1) Introduction to the Virtual Advisor o (2) Areas of Psychology o (3) Levels of training: Masters vs. Doctoral programs o (4) Counseling & Clinical Psychology: How special are these specialties? o (5) Your financial future: Making dollars and sense. Module 2: Preparing for Careers in Psychology o (1) Preparing i nside the classroom

PAGE 27

27 o (2) Preparing outside the classroom and o (3) Finding the best program. Module 3: Applying to Graduate Schools in Psychology o (1) Objective components of the graduate application such as GPA and GRE scores o (2) Subjective components s uch as the personal statement, curriculum vitae, and letters of recommendation. Module 4: Interviewing for Graduate Schools in Psychology o (1) Interviewing isnt everything, its the only thing o (2) The 10 point performance check o (3) Graduate school inter view o (4) Evaluating the interview. Module 5: Putting it All Together (serves as a conclusion and integration of the material) o (1) Deciding, notifying and celebrating o (2) Transitioning to graduate school o (3) Overall summary Additionally, each module concludes with a section titled Extra Advisement which includes various links to web resources, self assessments, sample personal statements, vitas and letters of recommendation along with many other additional resources. The advantage of such a program i s that it available to all students at any time during their education. Moreover the program offers flexibility in that it is portable and it can be completed at each individuals convenience. This study proposes that such a program not only

PAGE 28

28 can suppleme nt current resources but can also help fill in the gap for students whose institutions offer fewer graduate preparation resources. In sum, it is clear that the number of applicants to graduate programs in psychology continues to grow and the need for graduate preparation is increasing rapidly. Yet, the empirical study of the mechanisms of graduate preparation is developing at a gradual pace in an unprogrammatic way. Essentially, the literature on graduate preparation is in its infancy. There is a scarcity of systematic empirical work and a lack of standardized measurements or controlled research. What is available is limited by reliance primarily on anecdotal accounts, with little if any empirical data and, when present, these data are based on pre -post (o r sometimes posttest only) nonrandomized samples of convenience. The confounds associated with studies of this type are legion, including selection biases, demand characteristics, the absence of control groups, and the use of questionnaires without known psychometrics, among others. As with the graduate preparation courses described earlier, The Virtual Advisor is designed to provide assistance in the graduate preparation process as well as to enhance students clarity regarding their career goals and asp irations. Based on the work previously done by Thomas and McDaniel (2004), this study purports similarly that systematic efforts to measure the effectiveness of The Virtual Advisor would likely have an impact on three areas. The first and most obvious area of impact would be confidence in and knowledge about the graduate preparation process; this would be measured by the Grad Prep Quiz and the Psychology Majors Career Information Survey and Quiz. Additionally, because the virtual advisor has two modules th at tap directly into exploring and preparing for careers in psychology, it is also expected that completion of this program would have an impact on students confidence in their career decision making abilities. This would be measured by the Career Decision Self -efficacy Scale-

PAGE 29

29 Short -Form. Career decision self -efficacy is defined as an individuals degree of belief that he or she can successfully complete tasks necessary to making career decisions (Betz & Luzzo, 1996, p.415). Finally, we would expect that the wealth of career related and graduate preparation information in The Virtual Advisor would likely have some impact on overall vocational identity. Vocational identity is described as being comprised of career exploration (exploring career options) and career decidedness (making a career choice) so that increases in both of these areas indicate achievement of ones vocational identity (Thomas, McDaniel & Wagner, 2005). Achievement of vocational identity will be measured with the Career Decidedness and E xploration Inventory -Revised (CEDI R). Vocational identity is distinguished from career decision self -efficacy in that it is directly tied to actions, for example questions begin with I have. In contrast, career decision self efficacy is not tied to a ction, rather to a persons level of confidence or belief in his or her ability to successfully perform a career related action; accordingly, each question begins with how much confidence do you have that you could. The decidedness subscale of the CEDI R is positively correlated ( r = .61) with the Career Decision Self -efficacy Scale, because those who are career decided are likely to have more confidence in their ability to make career decisions. Although, just because a person has confidence in their a bility to make career decisions does not necessarily mean that they are career decided. Additionally, the career exploration subscale of the CEDI R is also positively correlated with the Career Decision Self Efficacy Scale ( r = .36) because students who ha ve done more career exploration are likely to be more confident in their ability to make a career decision. Hypotheses Hypothesis 1: The current study hypothesizes that those students who utilize The Virtual Advisor will have an increased level of graduat e preparation self -efficacy and knowledge.

PAGE 30

30 Hypothesis 2 : The current study hypothesizes that students who utilize The Virtual Advisor will report higher levels of confidence in relation to their ability to make career decisions. Hypothesis 3: The current study hypothesizes that students who utilize The Virtual Advisor will report higher levels of achievement of their vocational identity.

PAGE 31

31 CHAPTER 3 METHODS Participants Participants were 308 college students from across the nation (representing 34 differe nt states) and came from online sampling. Participants were solicited through an email that was sent to all Psi Chi chapter Presidents in the United States. The email encouraged the Psi Chi Presidents to forward it to the students in their chapter as well as psychology majors at their college. Procedure An internet survey was conducted that took participants approximately 20 minutes to complete. The soliciting email contained a link that directed participants to a website where they read and electronically signed the informed consent. Participants were then directed to a link where they completed a demographics section and the pre test questionnaires. At the end of the survey students submitted their answers electronically to an online database. Upon compl etion they were randomly assigned to one of three groups: (1) the treatment group, (2) the website wait -list control group, or (3) the natural wait -list control group. (1 ) Treatment Group: Upon completion of the pre -test survey, participants in the treatme nt group were given an access code that allowed them immediate access to The Virtual Advisor for a period of three weeks. Participants were given the following instructions: It is up to you how you break up the time over the next three weeks and how much time you spend reviewing each module. But you should plan on spending 2 3 hours per week (for the following three weeks) reviewing The Virtual Advisor materials. At the end of this study you will be asked to report the amount of time you spent on each of the five modules in the program. In addition, they were asked to

PAGE 32

32 characterize anything else that they did to find out about graduate preparation and application over the course of the time they were completing the study. (2 ) Website Wait-List Control Gr oup: Participants in this group were informed that they would be given access to The Virtual Advisor in three weeks time, upon completion of the post -test questionnaires. Additionally, during the interim, these participants were instructed to visit and exp lore a specified website titled Career Related Information. This site includes 4 major topic areas which are titled: (1) career related resources, (2) graduate school preparation resources, (3) organizations and the major subfields in psychology, and (4) resources for psychology undergraduates. Each topic is linked to a page with multiple different web resources and related links for specified information on that subject. Participants were given the following instructions It is up to you how you break up the time over the next three weeks and how much time you spend reviewing the websites. But you should plan on spending 2 3 hours per week (for the following three weeks) reviewing the websites. At the end of this study you will be asked to report the a mount of time you spent reviewing the various web resources. In addition, they were asked to characterize anything else that they did to find out about graduate preparation and application over the course of the time they were completing the study. (3) Na tural Wait -List Control Group: Participants in this group were informed that they would be given access to The Virtual Advisor in three weeks time, upon completion of the post test questionnaires. Participants in the wait -list control group were not given any instructions for what to do during the interim between pre and post test. However, in the post test, the natural wait list control participants were asked to characterize everything that they had done to find out about graduate preparation and applicat ion over the course of the time the study was running.

PAGE 33

33 After three weeks from the date that the pretest questionnaires were completed, all participants were solicited by email to complete the post test questionnaires following the same procedure. Upon co mpletion of the post test, participants in both control groups were given access to The Virtual Advisor for three weeks. Measures Measures of graduate preparation knowledge and self -efficacy, career decision self efficacy and vocational identity were used in this study. Table 1 depicts each measure included in this study and the relevant hypotheses. All measures included are administered at both the pre test and post test. Table 3 1: Measures Measure Purpose Grad Prep Quiz: efficacy subscale Hypothesis 1 : measures self efficacy in graduate preparation knowledge Grad Prep Quiz: knowledge subscale Hypothesis 1: measures actual graduate preparation knowledge Psychology Majors Career Information Survey Hypothesis 1: measures self efficacy in graduate preparation knowledge Psychology Majors Career Information Quiz Hypothesis 1: measures actual graduate preparation knowledge Career Decision Self efficacy Scale Short Form Hypothesis 2: measures career decision self efficacy Career Decidedness & Exploratio n Inventory Hypothesis 3: measures vocational identity Virtual Advisor Modules Quiz Manipulation Check: measures how well participants comprehend the material presented in the Virtual Advisor. Grad Prep Quiz (Neimeyer, 2003) The GPD is a 25 item self report measure which is used to assess students degree of knowledge and preparedness in relation to graduate study in psychology. This measure includes a subscale with 10 questions which are designed to measure self -efficacy and knowledge. The self -effica cy questions are measured on a scale of 1 (not confident) to 5 (highly confident) ; an

PAGE 34

34 example is I know how to assess my strengths and to find the best graduate program for my particular interests." Additionally there is a subscale with 15 knowledge quest ions which are given in multiple choice format; an example is "Which of the following best describes the difference(s) between a PhD and a PsyD ? a) the PhD is a research degree and a PsyD is a practice degree b) a PhD program can be APA approved whereas a PsyD cannot, c) a PhD can be licensed to practice psychology but a PsyD cannot, d) all of the above. Accordingly the GPD produces two separate subscale scores: (1) the self -efficacy score (ranging from 0 to 50) which indicates degree of confidence in gr aduate preparation; (2) the knowledge score (ranging from 0 to 15) which measures level of knowledge about and preparedness for graduate school. I nternal reliability of this scale has been demonstrated with an alpha coefficient of .89. Additionally, vali dity of this measure has been demonstrated by showing that it accurately discriminates between undergraduate and graduate students (Neimeyer et al., 2004). Psychology Majors Career Information Survey (Thomas & McDaniel, 2004) The PCMIS includes six items w hich measure students perceived knowledge in relation to career possibilities for psychology majors. Questions are rated on a five point likert scale (1 = strongly disagree; 5 = strongly agree.) A sample item is I can identify several different fields of study that would allow me to do counseling/therapy and I understand what each of these involves. Internal reliability of this scale is demonstrated with an alpha coefficient of .87. Psychology Majors Career Information Quiz (Thomas & McDaniel, 2004) The PMCIQ was developed to measure psychology students actual career related knowledge by tapping into common misconceptions held by psychology majors in relation to careers. The measure includes 15 items which are rated on a likert scale ranging from 1 (con fident it is false) to 5 (confident it is true). The goal of using the likert scale was to be able to

PAGE 35

35 discern between students who were guessing the answers from those who were confident in their answers. Internal consistency for this scale was shown with an alpha coefficient of .55. Career Decision Self -efficacy Scale Short Form (Betz, Klein & Taylor, 1996) The CDSE -SF was developed to measure an individuals degree of belief that he or she can successfully complete tasks necessary to making career decis ions (Betz & Luzzo, 1996, p.415). The measure consists of five subscales (five questions each) including (1) accurate self appraisal (2) gathering occupational information (3) goal selection (4) making plans for the future; and (5) problem solving. Each of the 25 items begins with How much confidence do you have that you could for an example Find information about graduate or professional schools. Each item is rated on a likert scale ranging from 1 (no confidence at all) to 5 (complete confidence). Bet z et al., 1995 demonstrated internal consistency reliability for this measure with alpha coefficients for the subscales ranging from .73 for Self -Appraisal to .83 for Goal Selection and .94 for the total score. Additionally, Betz & Taylor have established comprehensive support for the construct, content, and concurrent validity of the CDSE SF. Career Exploration and Decidedness Inventory -Revised (Thomas, McDaniel & Wagner 2006) The CEDI R was designed to measure two essential components of career developme nt described as exploration (exploring career options) and decidedness (making a career choice). The measure includes 22 items divided into three subscales: (1) decidedness (9 items), (2) career exploration (7 items) (3) self-exploration (6 items). Each it em is measured on a likert scale ranging from 1 (not at all like me) to 5 (very much like me). A sample item from the career exploration subscale is I have struggled to make a decision about what career is right for me. The CEDI has shown internal relia bility with alpha coefficients of .96 for the decidedness subscale, .85 for the career exploration subscale and .78 for the self -exploration subscale.

PAGE 36

36 Additionally, concurrent validity of the CEDI R has been demonstrated with correlations ranging from .50 to .61 with the Career Exploration survey and the Career Decision Self Efficacy Scale. Additionally it has shown an inverse relationship with the indecision subscale of the Career Decision Scale (r = .78) and the Career Decision -Making Difficulties Questi onnaire (r = -.76). Virtual Advisor Module Quizzes (Neimeyer & Stevenson, 2008) Each module in the virtual advisor program has a test your knowledge quiz which is designed to check for comprehension of the material provided. There are five quizzes with 5 multiple choice questions each (scores ranging from 0 to 5). A sample item is Which of the following represent(s) important ways that you can start preparing for graduate school outside the classroom? Demographic and Self -Evaluation Instrument This measure includes general demographic items such as age, gender, ethnicity, year in school, major university attended and overall GPA, as well as questions intended to gain information about participants interest in, and experiences related to, graduate scho ol preparation. Some of the questions examine graduate school plans such as: whether they were planning to go to graduate school in psychology, the degree they planned to pursue (e.g. M.A. or M.S., Ph. D, Psy. D) and their intended area of study (e.g. cli nical, social, etc.). Other items assess the mechanisms utilized by students in their graduate school preparation, such as whether or not they had taken a graduate preparation course, seminar, or preparation meeting, what kind of services they had utilized in preparing for graduate school in psychology (e.g. talked to a professor). This instrument also included measures of behavior exploring what activities undergraduates might be involved in to improve their chances of admission. These items included wheth er they had researched graduate programs, begun writing a personal statement, begun writing a C. V., decided on who their letter writers would be, asked their letter writers,

PAGE 37

37 begun studying for the GRE, taken the GRE, talked with departmental advisors, tal ked with their professors about graduate school, visited the Career Center, had an ongoing mentor, became involved in Psychology club/Psi Chi, utilized a grad school prep book, visited a grad prep website, conducted research, volunteered in the community, wrote or planned to write a senior thesis and presented or made plans to present at a conference.

PAGE 38

38 CHAPTER 4 RESULTS The results section begins with demographic characteristics of the study participants. The next section presents descriptive information a nd relationships among the primary variables of interest, together with the preliminarily analyses. Finally, a mixed factorial MANOVA was conducted to examine the relationship of all variables of interest with the various groups over time. This was subsequ ently broken down into a series of follow up univariate repeated measures ANOVAs performed on each dependent variable to examine the location of the significant effects. Demographics At the pre -test there were 180 participants in the treatment group, 183 participants in the website control group, and 172 participants in the wait list control group. At the post test, 86 participants remained in the treatment group (47.8% retention), 113 in the website control group (61.7% retention), and 130 in the wait -li st control group (75.6% retention). Manipulation checks performed on the Virtual Advisor condition indicated that 21 of the participants reported spending one hour or less on The Virtual Advisor. This subgroup of participants was eliminated from subsequent analyses because on average they reported completing less than one full module of the Virtual Advisors five modules ( M =.81). For the final analyses, participants in the Virtual Advisor condition were retained only if they completed a minimum of two hours of The Virtual Advisor, leaving 65 treatment group participants who reported spending an average of 6.15 hours reviewing the program ( SD = 4.3) and completing, on average, 3 of the 5 program modules. See Table 2 for a frequency distribution of the numbe r of modules completed for all treatment group participants.

PAGE 39

3 9 Table 4 1 : Frequency by number of modules completed Number of Modules Completed Number of Participants 0 7 1 7 2 11 3 17 4 3 5 20 The final sample consisted of 308 participants (Treatme nt = 65; Website Control =113; Wait List Control = 130). Among those who designated their demographic information, 87.5% were female and 12.5% were male with a mean age of 22 years (age range = 18 to 39). A Chi square analysis was conducted to see if gende r was randomly distributed across groups and was found to be significant [ X2(2) = 6.164, p = .046]. This indicates that there was a slightly higher percentage of males in the control group than in the other two groups (Treatment = 10.8%; Website Control = 7.3%; Wait list Control = 17.8%). Given this difference in gender distribution across groups, a MANOVA was conducted to examine if there were gender differences on the primary dependent variables at the pretest and it was not found to be significant ( F [6, 296] = .952, p = .458, 2 = .019, pillais trace = .019). Despite the moderate disparity in gender distribution, this confirms that this difference does not confound the current studys results. Concerning ethnicity, t he majority of the participants in this study were Caucasian 78.6% (N =239), followed by Hispanic, 8.9% (N = 27), African American, 5.9% (N = 18), other, 3.6% (N = 11), and Asian American and Pacific Islander, 3% (N =9). The majority of participants were seniors, 57.4% (N = 175), followed by juniors, 30.8% (N= 94) sophomores, 9.2% (N=28), and freshmen 2.6% (N=8). A Chi squared analysis was conducted to see if participants year in school was randomly distributed across groups and it was not found to be

PAGE 40

40 significant [ X2(6) = 8.523, p = .202], indicating that year in school was randomly distributed across the groups. The average GPA of participants was 3.49 ( SD = .40). Among this sample, 37.9% of participants were in the process of applying to graduate school during the study. Of the remaining 62.1% of participants, 87% of those were planning to apply in the future. At the post -test, a substantial percentage of the participants reported that they planned to pursue a PhD, 49.3% (N = 142), followed by Masters Degree, 35.1% (N = 101), Psy.D, 12.8% (N=37), and undecided or other, 2.8% (N = 8). Descriptive Statistics and Preliminary Analyses Table 3 includes the summary statistics for all of the measures. This includes the scales measuring graduate preparation self efficacy, graduate preparation knowledge, career decision self -efficacy, and vocational identity at both pre -test and post -test. Grad prep self -efficacy is measured with the self -efficacy questions from the Grad Prep Quiz and the Psychology Majors Career Information Survey. The grad prep knowledge variable is me asured with the multiple choice knowledge questions from the Grad Prep Quiz and the Psychology Majors Career Information Quiz. Career Decision self -efficacy is measured by the Career Decision Self efficacy Scale Short Form. Vocational Identity is measured by the Career Exploration and Decidedness Inventory -Revised. All scores were normally distributed. All measures displayed skewness and kurtosis values in the acceptable range for appropriate distribution. All scales, except for the Grad Prep Quiz knowledg e subscale, had moderate to high reliability coefficients. Due to its inadequate reliability coefficient, The Grad Prep Quiz knowledge subscale was eliminated from the primary analyses in the study. Table 4 depicts the Pearson Product Moment correlation c oefficients for all variables of interest.

PAGE 41

41 Table 4 2 : Summary Statistics of M easures ___________________________________________ ___________________________________ Measures M SD Skewness Kurtosis N of Items Alpha 1. Grad Prep Quiz Eff: Pre -test 23.24 6.78 .26 .49 10 .82 Post test 30.86 8.15 .18 .41 10 .87 2. Psych Career Survey: Pre test 17.01 5.07 .14 -.61 6 .86 Post -test 21.01 5.22 .61 .25 6 .90 2. Grad Prep Quiz Know: Pre test 6.42 1.91 -.03 .07 15 .13 Post test 6.55 2.08 .19 .41 15 .28 3. Psych Career Quiz: Pre test 50.39 4.88 .06 .38 14 .44 Post -test 50.26 6.30 .72 1.89 14 .63 4. Career Decision Eff: Pre test 97.03 15. 04 .41 -.44 25 .93 Post test 100.59 16.37 1.22 3.24 25 .95 5. Career Decide & Ex p: Pre test 80.87 15.72 .47 -.20 22 .92 Post -test 84.23 15.64 -.48 1.1 ____ 22______ .92_ Note. Grad Prep Quiz Eff = Grad Prep Quiz self-efficacy subscale; Psych Caree r Survey = Psychology Majors Career Information Survey; Grad Prep Quiz Know = Grad Prep Quiz knowledge subscale; Psych Career Quiz = Psychology Majors Career Information Quiz; Career Decision Eff = The Career Decision Self -efficacy Scale-Short Form; Career Decide & Exp: The Career Exploration & Decidedness Inventory -Revised.

PAGE 42

42 Table 4 3 : Intercorrelations Among Variables ____________________________________________________________________________ Variable 1 2 3 4 5 6 7 8 9 10 11 12 13 1. GPET1 -.46** .66 .38** .27** .08 .23 .07 .42** .29** .42** .33** .03 2. GPET2 -. 43** .77** .16** .15* .24** .19** .07 .42** .29** .42** .02 3. PCST1 -.57** .27** .10 .25** .15** .55** .43 .49** .43** .05 4. PCST2 -.27** 22** .32** .26** .51** .61** .47** .72** .10 5. GPKT1 -.35** .24** .21** .17** .19** .12* .18** .03 6. GPKT2 -.28** .2 6** .19** .18** .11 .17** .10 7. PCQT1 -.50** .23** .18** .23** .25** .07 8. PCQT2 -.19** .24** .18** .17** -.05 9. CEDT1 -.73** .65** .55** -.05 10. CEDT2 -.52** .74** .01 11. CSET1 -. 62** .03 12. CSET2 --.01 13. TIMSP -__________________________________________________________________ Note. GPE = Grad Prep Quiz self -efficacy subscale; PCS = Psychology Majors Career Information Survey; GPK = Grad Prep Quiz knowledge subscale; PCQ = Psychology Majors Career Information Quiz; CED = The Career Exploration & Decidedness Inventory Revised; CSE = The Career Decision Self -efficacy Scale-Short Form; TIMSP = Time participants s pent reviewing either the Virtual Advisor or the assigned webresources; For all T1 = Pre test, T2 = Post -test. p >.05 ** p < .01

PAGE 43

43 Manipulation Checks As a manipulation check, a repeated measures ANOVA was conducted on the Virtual Advisor Modu le quizzes. This measure was comprised of questions that came from the quizzes within The Virtual Advisor program and therefore are derived specifically from the material covered in the five modules of the program. Significant main effects for were found f or time (F [1, 305] = 20.629 p < .001, 2 = .063) and group ( F [2, 305] = 9.072 p < .001, 2 = .056). These were qualified by a significant interaction for time by group (F[2, 305] = 39.114, p < .001, 2 = .092). To adjust for familywise error, Bonferroni adjusted post -hocs were performed and revealed that all three groups were equal on their scores on The Virtual Advisor module measure at the pre test. At post -test, the treatment group differed significantly from the website control group and the wait list c ontrol group. Examination of the means revealed that at the posttest participants in the Virtual Advisor condition scored significantly higher on the Virtual Advisor module quiz than those in both the website control group and the wait list control group (Treatment M =9.22, Website Control M =7.44, Wait -List Control M =7.72; p <.001). This confirms that the treatment group answered more questions correctly on this measure which demonstrates their understanding of the material provided in The Virtual Advisor. Multivariate Test In order to test the primary hypotheses of the current study, a mixed factorial MANOVA was conducted. The overall repeated measures multivariate analysis of variance found significant main effects for time (pre test vs. post test) ( F [1 305] = 169.165, p < .001, 2 = .357, pillais trace = .357) and measure (the dependent variables) ( F [3, 303] = 2319.686, p < .001, 2 = .958, pillais trace = .958). These main effects were qualified by significant interactions of time by group ( F [2,305] =26.023, p < .001, 2 = .146, pillais trace = .146 ), time by measure ( F [3, 303]

PAGE 44

44 = 131.224, p < .001, 2 = .565, pillais trace = .565), and measure by group ( F [6,608] = 4.487, p < .001, 2 = .042, pillais trace = .085 ). These interactions were further qualified by a significant three way interaction of time by measure by intervention group ( F [6,608] = 11.28, p < .001, 2 = .100, pillais trace = .200 ). Follow -Up Univariate Tests For the purposes of this study, we are only interested in examining the time (pre test vs. post test) by group interaction for each dependent variable. Therefore, follow up univariate ANOVAs we re conducted separately for each dependent variable, to examine the location of the significant effects. See appendices for bar charts of each variable by group at pre -test and post test. Hypothesis 1: The current study hypothesized that those students who utilized The Virtual Advisor would have an increased level of graduate preparation self -efficacy and knowledge Graduate preparati on self -efficacy measures students level of self -efficacy and belief in their knowledge and preparedness for applying to graduate school. Grad prep self -efficacy is measured in two ways; with the efficacy subscale of the Grad Prep Quiz and the Psychology Majors Career Information Survey. The grad prep knowledge variable is assessing students actual levels of knowledge on graduate school related information. Grad prep knowledge is measured with the Psychology Majors Career Information Quiz. A repeated measures ANOVA was conducted on the Grad Prep Quiz self -efficacy measure and significant main effects for were found for time ( F [1, 305] = 444.484 p < .001, 2 = .593) and group ( F [2, 305] = 13.736 p < .001, 2 = .083). These were qualified by a significant interaction for time by group (F[2, 305] = 48.408, p < .001, 2 = .241). To adjust for familywise error, Bonferroni adjusted post -hocs were performed and revealed that all three groups were

PAGE 45

45 equal on grad prep self -effic acy at the pre test (see Table 5 ). A t post test, the treatment group differed significantly from the website control group and the waitlist control group. Examination of the means revealed that at the posttest participants in the Virtual Advisor condition scored significantly higher on gra d prep self -efficacy than both the website control group and the wait list control group participants (Treatment M =36.20, Website Control M =32.50, Wait List Control M =26.76; p <.001). Additionally, the website control group scored significantly higher than the waitlist control group ( p <.001). Finally, one tailed paired T tests were conducted and confirmed that there was a significant change from pre -test to post -test for the treatment group (t(64) = 11.992, p < .001) and the website control group (t(112) = 12.45, p <.001). Table 4 4 : Means and Standard Deviations for Grad Prep Quiz Self -efficacy Subscale Treatment Website Control Wait List Control Pre test M = 22.79 SD = 6.34 M = 23.50 SD = 7.44 M = 23.25 SD = 6.43 Post test M = 36.20 SD = 7.75 M = 32.50 SD =7.62 M = 26.76 SD = 6.66 Another repeated measures ANOVA was conducted on the Psychology Majors Career Information Survey (another measure of grad prep self -efficacy) and significant main effects were found for time ( F [1, 305] = 308.491 p < .001, 2 = .503) and group ( F [2, 305] = 9.667 p < .001, 2 = .060). These were qualified by a significant interaction for time by group (F[2, 305] = 341.543, p < .001, 2 = .195). To adjust for familywise error, Bonferroni adjusted post -hocs were performed and revealed that all three groups were equal on the Psychology Majors Career Information Sur vey at the pre -test (see Table 6 ). At post test, the treatment group and the website control group both differed significantly from the wait list control group. Examination of the

PAGE 46

46 means revealed that at the posttest participants in the Virtual Advisor condition and the website control condition, both scored significantly higher on this measure than the wait -list control group participants (Treatment M =23.79, Website C ontrol M = 22.21, Control M =18.57; p <.001). The treatment group and website control group were not found to be significantly different. Finally, one tailed paired T tests were conducted and confirmed that there was a significant change from pre -test to post test for the treatment group (t(64) = 11.524, p < .001) and the website control group (t(112) = 10.793, p <.001). Table 4 5 : Means and Standard Deviations for Psychology Majors Career Information Survey Treatment Website Control Wait List Control Pre test M = 16.99 SD = 4.74 M = 17.13 SD = 5.28 M = 16.93 SD = 5.09 Post test M = 23.79 SD = 4.71 M = 22.21 SD = 4.42 M =18.57 SD = 5.08 A repeated measures ANOVA was conducted on the Psychology Majors Career Information Quiz (a mea sure of grad prep knowledge) and the main effects for time ( F [1, 305] = .157 p = .692, 2 = .001) and group ( F [2, 305] = 1.200 p = .303, 2 = .008) were not found to be significant. There was a significant effect for the interaction of time by group (F[2, 305] = 3.619, p < .05, 2 = .023). To adjust for familywise error, Bonferroni adjusted post -hocs were performed and revealed that all three groups were equal on the Psychology Majors Career Information Survey at the pre -test (see Table 7 ). At the post test, post -hocs were inconclusive in finding the location of the significant effects. For this reason, a one tailed paired T test was conducted on the treatment group to see if participants scores had changed from pre test to post test and was found to be significant (t(64) = 1.852, p < .05). This indicates that the treatment group did in fa ct show a significant increase in their scores on the Psychology Majors

PAGE 47

47 Career Quiz as a function of completing The Virtual Advisor. No significant differences were noted in the website control group or wait list control group. Table 4 6 : Means and Stand ard Deviations for Psychology Majors Career Information Quiz Treatment Website Control Wait List Control Pre test M = 49.58 SD = 4.50 M = 51.05 SD = 5.22 M = 50.22 SD = 4.71 Post test M = 51.11 SD = 6.04 M = 50.62 SD = 7.39 M =49.52 S D = 5.28 Hypothesis 2: Students who utilized The Virtual Advisor would report higher levels of confidence in relation to their ability to make career decisions To address the second hypothesis, another repeated measures ANOVA was conducted on the career decision self -efficacy variable and a significant main effect was found for time ( F [1, 305] = 27.561, p <.001, 2 = .083). The main effect for group was not found to be significant (F [2, 305] = .886 p = .413, 2 = .006). These effects were qualified by a significant interaction for time by group (F[2, 305] = 9.634, p < .001, 2 = .059). To adjust for familywise err or, Bonferroni adjusted post hocs were performed and revealed that all three groups were equal on career decision self -efficacy at the pre test (see Table 8 ). At the post test, the treatment group and website control group both differed significantly from the wait -list control group. Examination of the means revealed that at the posttest participants in the Virtual Advisor condition and the website control condition both scored significantly higher on career decision self -efficacy than the wait list contro l group participants (Treatment M =103.43, Website Control M =102.61, Wait List Control M =97.42; p <.05). The treatment group and website control group were not significantly different on career decision self -efficacy. Finally, one tailed paired T tests wer e conducted and confirmed that there was a significant change from pre -test to post -test for

PAGE 48

48 the treatment group (t(64) = 3.476, p < .001) and the website control group (t(112) = 5.714, p <.001). Table 4 7 : Means and Standard Deviations for Career Decis ion Self -efficacy Scale Treatment Website Control Wait List Control Pre test M = 96.90 SD =15.04 M = 96.25 SD =15.49 M = 97.77 SD =14.72 Post test M = 103.42 S D =18.02 M = 102.61 SD =15.24 M = 97.42 SD =16.06 Hypothesis 3: Students who uti lized The Virtual Advisor would report higher levels of achievement of their vocational identity. To address the final hypothesis, another repeated measures ANOVA was conducted on the Career Exploration and Decidedness Inventory Revised (the measure of vo cational identity) and a significant main effect was found for time ( F [1, 305] = 30.374, p <.001, 2 = .091). The main effect for group ( F [2, 305] = .2.649 p = .072, 2 = .017) and the interaction for time by group were not found to be significant (F[2, 3 05] = 2.094, p = .125, 2 = .014). Table 9 lists the means and standard deviations at pre -test and post -test by group. Table 4 8 : Means and Standard Deviations for Career Exploration and Decidedness Inventory Treatment Website Control Wait List Control Pre test M = 83.19 SD =14.86 M = 80.77 SD =15.77 M = 79.80 SD =16.08 Post test M = 88.48 S D =16.06 M = 84.76 SD =15.44 M = 81.75 SD =15.22 Behavioral Reports The behavioral report items assessed the graduate school preparation behaviors that participants had engaged in during the interim since completing the pre test (at least three weeks prior). Chi squared analyses were performed on each of the behavioral report variables to assess

PAGE 49

49 whether there were differences in reports of grad prep rela ted behaviors at the post test across groups. As depicted in Table 10 the majority of the behavioral report items were not found to be significant. The few behavioral report items that were found to be significant were whether participants had begun read ing a book on preparing for graduate school in psychology ( X2(2) = 9.296, p < .01), had visited a grad prep website ( X2(2) = 21.804, p <.001) and had begun volunteering in the community ( X2(2) = 7.292, p < .05). The treatment group was the group most likel y to have begun reading a grad school preparation book (Treatment = 51.6%; Website control = 38.9%; Wait -List Control = 29.2%). The website control group participants were the most likely to have begun volunteering (Treatment = 12.5%; Website control = 16. 2%; Wait -List Control = 5.5%) and to have visited a grad prep website (Treatment = 53.1%; Website control = 61.9%; Control = 32.6%). The finding in relation to visiting a grad prep website should be interpreted with caution as it is confounded by the fact that the website control group was specifically instructed to visit a grad prep website as part of the study.

PAGE 50

50 Table 4 9 : Percentages by Group Endorsing Behavioral Report Items Behavioral Report Items Treatment Website Control Wait list Contro l Began researching grad programs 75.4% 77% 69.2% Began writing Personal statement 24.6% 21.4% 16.2% Began writing C.V. 37.5% 38.9% 26.9% Decided on letter writers 60.0% 67.0% 58.1% Asking letter writers 33.8% 33.6% 24.8% Began studying for the GRE 4 3.1% 40.7% 34.9% Taken the GRE 16.1% 24.8% 18.5% Visited a department academic advisor 60.0% 57.5% 48.1% Read a grad prep book** 51.6% 38.9% 29.2% Talked to a professor about grad school 64.1% 64.6% 54.7% Visited career center 19.0% 25.9% 15.4% Culti vated a relationship with a faculty 70.8% 69.0% 63.1% Visited a grad prep website*** 53.1% 61.9% 32.6% Began volunteering* 12.5% 16.2% 5.5% New member of Psi Chi/Psych club 18.0% 14.7% 10.2% Decided to write a senior thesis 21.5% 18.6% 19.6% Made plan s to present at a conference 26.2% 17.3% 18.0% p < .05. ** p < .01 *** p < .001

PAGE 51

51 CHAPTER 5 DISCUSSION The discussion section begins with a summary of the results. Additionally, an interpretation of the findings in light of the literature on graduate preparation is presented. Next the limitations of the current study will be discussed. Finally, implications and future directions for this line of research will be provided. Summary of the Results The current study is the first randomized clinical trial with a national, non -convenience sample in the graduate preparation literature. The purpose of this study was to examine the validity and effectiveness of The Virtual Advisor, an online graduate preparation program; more specifically, to look at its effec t on graduate preparation self -efficacy and knowledge, career decision self -efficacy and vocational identity. The strongest finding was that the treatment groups graduate preparation self -efficacy was increased more robustly than both control groups as a function of completing The Virtual Advisor. Interestingly, the website control groups grad prep self -efficacy was also significantly increased over the wait list control group. This indicates that The Virtual Advisor was effective in increasing participa nts confidence and belief in their preparedness and knowledge about graduate school in psychology. Furthermore, being assigned to look over graduate preparation websites was also effective in increasing students self -efficacy in relation to graduate preparation. Therefore, the overall findings seems to suggest that engaging in either process of preparation increases ones self -efficacy in the graduate preparation process, with the Virtual Advisor nonetheless registering the greatest impact in this regard. The findings that addressed actual graduate preparation knowledge were mixed. The Grad Prep Quiz knowledge subscale analysis had to be excluded due to low scale reliability and the Psychology Majors Career Information Quiz analysis suggested a moderate increase in

PAGE 52

52 graduate preparation knowledge for those in the treatment group. Furthermore, although significant, the practical meaningfulness of this increase remains indeterminate. For this reason, conclusions based on this finding will be limited. The la ck of robust grad prep knowledge effects may be tied to the fact that the Psychology Majors Career Information Quiz is a broad instrument designed to assess learning across a full, semester long, careers in psychology course. Similarly, in this study, trea tment group participants spent on average only 6 hours reviewing the program, completing on average only 3 modules. Therefore, it is reasonable that one would find stronger effects over the range of an entire careers course, as participants in this study spent substantially less time reviewing grad prep material than would be expected in a full semester course. Additional findings suggest that in addition to grad prep self -efficacy and modest knowledge gains, the treatment group and website control group also reported increased career decision self -efficacy. Career decision self -efficacy is regarded as an individuals level of confidence or belief in his or her ability to complete the tasks necessary for making career decisions. This supports the conclusi on that doing graduate preparation related activities, regardless of the format, is likely to improve students self -efficacy. Specifically, doing grad prep activities significantly increased participants belief in their ability to successfully complete th e tasks involved in making career decisions. Finally, the hypothesis that doing grad prep activities would increase participants vocational identity was not supported. There are several explanations for the lack of effects in relation to vocational iden tity. The most relevant cause is likely tied to the fact that vocational identity is often conceptualized as a developmental process, and identity tends to unfold longitudinally. When Thomas and McDaniel (2004) examined this construct in relation to their careers for psychology majors course, they did find significant changes in vocational identity

PAGE 53

53 (both exploration and decidedness). In this case, post test measurement was after the culmination of an entire semester and therefore spanned a significantly l onger time period than the current study. Therefore, it is possible that a graduate preparation program such as The Virtual Advisor could have more significant effects on vocational identity over a longer period of time. In addition, Thomas and McDaniels (2004) study included a convenience sample of those who chose to take the careers in psychology course. Examining those who chose to take a careers course may have biased the sample and selected for people more uncertain/undecided regarding their career di rections at the outset of the course. Future research could better examine this relationship in the context of a longitudinal design In general it has been noted that there is a lack of systematic empirical research to support the validity and utility of the various graduate preparation resources available to students. The current study supports the findings of Thomas & McDaniel (2004) who showed that a careers for psychology majors course did effectively increase students confidence in their ability to make appropriate career related decisions. This study also supports and builds upon the work of Neimeyer et al. (2004) who found that a preliminary audiotape version of a graduate preparation program significantly increased students scores on the Grad Pre p Quiz. The current study expands this work by taking a newer and more improved version of this graduate preparation program and testing it in a randomized clinical trial with a large national sample. Findings suggest that utilizing The Virtual Advisor s ignificantly increases students confidence in their preparedness and knowledge about graduate school and their belief in their ability to make career related decisions. Behavioral Reports The majority of behavioral report items, I have begun writing my personal statement, taken the GRE, I have asked my letter writers, were not found to be significant at the post test.

PAGE 54

54 This is somewhat expected because these variables were constrained in many ways. First and foremost, many of the behavioral items are more or less relevant to students depending on what year a student is in school (i.e. freshman vs. senior). Therefore, many of these variables are more contingent on the participants level of development rather than on the group they were assigned to. In one study, Lee and Neimeyer (in press) found that seniors not only displayed higher levels of knowledge about graduate preparation, but also reported engaging in more graduate preparation activities. In support of this conclusion, a Chi -square analysis was con ducted to assess whether students had begun writing their personal statement at the post -test depending on their year in school, and it was found to be significant (X2(3) = 30.720, p <.001) As to be expected, the seniors were most likely have begun writi ng a personal statement (Seniors= 31.0%; Juniors = 5.3%, Sophomores = 7.2%; Freshman = 0%). For this reason, post hoc analyses were conducted on all of the behavioral report items separately for seniors and also for juniors by group. When examining just s eniors by group, the only behavioral report item that became significant was whether the student had begun writing a curriculum vita (C.V.) ( X2(2) = 6.021, p <.05) For this analysis, the treatment group seniors were most likely to have started writing a C.V. (Treatment=60%; Website Control = 53.6%; Wait list Control = 37.3%). Similarly, when examining juniors by group, the only behavioral report item that became significant was whether the student had become a member of Psi Chi/Psychology club ( X2(2) = 6.2 38, p <.05). For this analysis, the treatment group juniors were most likely to have joined Psi Chi since beginning the study (Treatment= 30.4%; Website Control = 6.1%, Wait list Control = 14.3).This makes sense as one would assume that students who are se niors are much more likely to already be members of Psi Chi/Psychology club, thus constraining the amount of new membership that is possible.

PAGE 55

55 Furthermore, these variables were measured during a time period ranging from approximately 3 weeks to 2 months, a nd several of the behavioral report items cannot be feasibly accomplished in that limited time period regardless of student motivation (i.e. made plans to write a senior thesis, made plans to present at a conference). Finally, it should be noted that a sub stantial percentage of the participants who volunteer to be in this type of study are likely to have already engaged in many of the relevant graduate preparation activities for their level of development. It is likely that there is self -selection of the m ost motivated students into this type of study; therefore, it is possible that only limited behavioral changes could be expected during a brief time period. This is supported by the findings of Lee and Neimeyer (in press) who showed that in a sample of 5 90 undergraduate psychology majors from across the nation, significant percentages of the participants were already engaging in graduate preparation activities. Specifically, some of the most endorsed activities included: visiting a departmental advisor and talking with a professor on a few occasions which were both reported by 69% of the participants; 50% endorsed having an ongoing relationship with a faculty mentor; 42% read graduate preparation books; 25% attended a getting into graduate school seminar. Additionally, 52% were members of Psi Chi, 45% had written or planned to write a senior thesis and 29% participated in some kind of mental health volunteer work. Consistent with these assumptions, the few items that were found to be significant were less constrained by the aforementioned concerns and could be achieved in a short period of time, such as volunteering in the community, reading a grad prep book and visiting a grad prep website. Participants in the treatment group were most likely to have rea d a grad prep book. Participants in the website control group were most likely to have begun volunteering in the community. As one would expect, both of those groups were more likely than the wait -list

PAGE 56

56 control group to have visited a grad prep website, as it was specifically assigned to them as part of the study. In order to better address possible behavioral effects of the Virtual Advisor, future work might take into consideration the specific target behaviors that would be expected to change as a result of completing a graduate preparation program. Limitations Overall this study adds to the literature on preparation for graduate school in psychology, although there are limitations. One important limitation of this study is the use of self -report measure s and a lack of true behavioral indicators in interpreting the results. Self report measures are susceptible to social desirability and misrepresentation. That is, how do we know that students are actually utilizing the Virtual Advisor or whether they a re accurately reflecting the amount of time that they spent reviewing the program? Another thing to consider is the evaluative nature of graduate preparation; this could potentially cause participants to present as more prepared, efficacious, or involved i n graduate preparation activities. Additionally, the methodology of this study involved online sampling and, while there are several benefits to this technique, there are also limitations. For one thing, there is no control over the setting in which pa rticipants completed the survey. There is also no control over how long the treatment group and website control group actually spent reviewing their assigned materials. These limitations are counterbalanced by the ability to recruit non -convenience nationa l sample of students and to be able to easily randomize participants to groups. Furthermore, the anonymity associated with online surveys may make participants feel more comfortable disclosing personal information (Gosling et al., 2004). Finally, research has failed to find differences in participant responses when comparing paper -pencil methods to online data collection methods (Gosling et al., 2004).

PAGE 57

57 The current study is distinctive in that it involves a randomized national sample. At the same time, it has the potential for selection bias which has implications for the generalizability of findings. It is possible that this study had a biased sample of the most motivated students as well as those with enough free discretionary time to participate in a res earch survey. This could skew the results somewhat and create a ceiling effect for amount of improvement that can be achieved during a limited time period. Finally, Recruitment strategies for this study involved soliciting psychology club and Psi Chi members which limited our sample to predominantly psychology majors. While psychology majors represent the majority of those applying to graduate school in this field, there are also applicants to graduate schools in psychology from a variety of different disc iplines, thus our sample was biased to psychology majors specifically. Implications and Future Directions Within the context of these limitations, the goal of this study has been to examine the validity and effectiveness of The Virtual Advisor for graduat e preparation and to support further work in this area. The current study represents a significant contribution to an area of study with a paucity of systematic empirical research. In this same regard it is the first controlled, non convenience randomized national sample for a study on graduate preparation. This study supports the validity and effectiveness of The Virtual Advisor as a tool for increasing students self -efficacy and confidence in grad school preparedness, grad school knowledge, and ability to complete the tasks necessary for making career decisions. The Virtual Advisor is a unique grad prep resource that fits in nicely with the digital age and the current shift toward online resources; students are increasingly accustomed to utilizing info rmation through this medium. Moreover, online programs such as this one have the advantage of accommodating to the busy schedules that students often have by allowing for flexibility and portability. Furthermore, students can complete the program at any t ime in their

PAGE 58

58 education. Ideally it would be advisable for students who think early on that they might be interested in grad school to use this program so that they can be better prepared. In addition, online resources such as The Virtual Advisor are impor tant because they help fill in the gap for students whose schools dont offer as many grad prep resources. In this way, an online seminar can help level the playing field as research has shown a large discrepancy across schools in the amount of grad school prep resources available to students. Finally, The Virtual Advisor is a supplement to any current resources that are available to students. In opting for breadth and scope in our sample we compromised greater control over whether participants completed the program fully, how much time they spent, level of depth and the setting. Accordingly, wide variability was reported in how long participants spent on the program. For this reason, it is possible that the current study is a somewhat conservative test o f the hypotheses and future research could more effectively test the complete program. Future research may also benefit from examining graduate preparation in the context of more behavioral indicators of success and not relying solely on self report. It would have been helpful to have a closer inspection of the behaviors that students reported engaging in. Following up on students admissions offers was beyond the scope of this study, but is a natural next step in this line of research. Follow up studies which incorporated the perspective of students once they are admitted to graduate school would also be helpful in building the literature on graduate preparation. Conclusion Future work in this area of research may benefit from the findings of this stud y. Overall, significant effects were found for the treatment group in relation to increased graduate preparation self -efficacy and career decision self -efficacy, along with modest grad prep knowledge gains. While it is clear that exposure to graduate prepa ration websites for those in the

PAGE 59

59 website control condition also reflected some changes in the aforementioned areas, the greatest effect was shown for those who were in the Virtual Advisor condition.

PAGE 60

60 APPENDIX A BAR CHART FOR GRAD P REP QUIZ SELF EFFICACY A T PRE AND POST TEST 0 5 10 15 20 25 30 35 40 Treatment Website Control Pre-Test Post-Test

PAGE 61

61 APPENDIX B BAR CHART FOR PSYCHO LOGY MAJORS CAREER S URVEY AT PRE AND POS T TEST 0 5 10 15 20 25 Treatment Website Control Pre-Test Post-Test

PAGE 62

62 APPENDIX C BAR CHART FOR PSYCHO LOGY MAJORS CAREER Q UIZ (GRAD PREP KNOWL EDGE) AT PRE AND POST T EST 48.5 49 49.5 50 50.5 51 51.5 Treatment Website Control Pre-Test Post-Test

PAGE 63

63 APPENDIX D BAR CHART FOR CAREER DECISION SELF EFFICACY AT PRE AND POST TEST 92 94 96 98 100 102 104 Treatment Website Control Pre-Test Post-Test

PAGE 64

64 LIST OF REFERENCES American Psychological Association, Educational Affairs Office. (1983). R esults: Phase 1surve y of undergraduate department chairs Washington, DC: Author. American Psychological Association. (2007). Getting in: A step -by -step plan for gaining admission to graduate school in psychology. Washington, DC: Author. Betz, N. E., Klein, K.L. & Taylo r, K. M. (1996). Evaluation of a short form of the Career Decision -Making Self -Efficacy Scale. Journal of Career Assessment 4, 47 -57. Betz, N. E. & Luzzo, D. A. (1996). Career assessment and the Career Decision -Making Self Efficacy Scale. Journal of Career Assessment, 4, 413428. Bonafazi, D. Z., Crespy, S. D., & Rieker, P. (1997). Value of a masters degree for gaining admission to doctoral programs in psychology. Teaching of Psychology, 24, 176182. Brihl, D. S. (2001). Life after college: Psych ology students perceptions of salary, business hiring criteria, and graduate admission criteria. North American Journal of Psychology, 3, 321330. Buckalew, L. W., & Lewis, H. H. (1982). Curriculum needs: Life preparation for undergraduate psychology majors. Psychological Reports, 51, 77 78. Buskist, W. (1999). Teaching an undergraduate course in preparing for graduate study in psychology. Teaching of Psychology, 26, 286288. Buskist, W., & Burke, C. (2007). Preparing for g raduate study in psych ology : 101 Questions and answers Malden, MA: Blackwell Publishing Ltd. Cashin. K. R., & Landrum, R. E. (1991). Undergraduate student perceptions of graduate admissions criteria in psychology. Psychological Reports, 69, 11071110. Davis, S. F. (1988). The professional psychologist: A course designed to introduce students to the profession of psychology. In P. J. Woods (Ed.), Is psychology for them? A guide to undergraduate advising (pp. 64 74). Dillinger, R. J., & Landrum, E. R. (2002). An infor mation Course for the beginning psychology major. Teaching of Psychology, 29, 230232. Dodson, J. P., Chastain, G., & Landrum, R. E. (1996). Psychology seminar: Careers and graduate study in psychology. Teaching of Psychology, 23, 238240. Eddy, B., L loyd, P. J., & Lubin, B. (1987). Enhancing the application to doctoral professional programs: Suggestions from a national survey. Teaching of Psychology, 14, 160163.

PAGE 65

65 Gosling, S. D. Vazire, S ., Srivastava, S. & John, O. P. (2004) Should we trust web-b ased studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist 59, 93 103. Johanson, J. C., & Fried, C. B. (2002). Job training versus graduate school preparation: Are separate educational tracks warr anted? Teaching of Psychology, 29, 241243. Keith -Spiegel, P. (1991). The complete guide to graduate school admission: Psychology and related fields. Hillsdale, NJ: Lawrence Erlbaum. Keith -Spiegel, P., Tabachnick, B.G., & Spiegel, G. B. (1994). When demand exceeds supply: Second -order criteria used by graduate school selection. Teaching of Psychology, 21, 7981. Keith -Spiegel, P., & Wiederman, M. W. (2000). The complete guide to graduate school admission: Psychology, counseling and related professions Hillsdale NJ: Lawrence Erlbaum Associates, Inc. Landrum, R. E., Jeglum, E. B., & Cashin, J. R. (1994). The decision -making processes of graduate admission committees in psychology. Journal of Social Behavior and Personality, 9, 239248. Lee, G. A. & Neimeyer, G. J. (in press). Preparing for graduate study in psychology: An exploratory study. Teaching of Psychology Lunnenborg, P. W., & Baker, E. C. (1986). Advising undergraduates in psychology: Exploring the neglected dimension. Teaching of Psychology, 13, 181 185. Mayne, T. J., Norcross, J. C., & Sayette, M. A. (1994). Financial assistance in clinical psychology programs: Diversity across the practice research continuum. American Psychologist, 48, 806811. McDonald, D. G. (1997). Psyc hologys surge in undergraduate majors. Teaching of Psychology, 24, 2226. Murray, B. (1999, February). College students getting brand new wake up call. APA monitor, 30 p. 42. Landrum, E. R., Sh oemaker, C. S., & Davis, S. F. (2003). Important topics in an introduction to the psychology major course. Teaching of Psychology 30, 48 51. Lloyd, M. A.. (2002). Courses on careers for psychology majors. In W. Buskist, V. Hevern, & G. W. Hill, IV, (E ds.). Essays from e -xcellence in teaching 20002001 chap. 6).

PAGE 66

66 National Center for Education Statistics. (2006). Digest of education statistics Chap. 3. Retrieved July 1, 2008, from http://nces.ed.gov/pubs2002/digest2001/tables/dt297.asp Neimeyer, G. J. (2003). The Grad Prep Quiz Unpublished manuscript, University of Florida at Gainesville, Fl. Neimeyer, G. J., Lee, G. A., Saferstein, J. &Pickett, Y. (2004). Effects of a Graduat e Preparation Program on Undergraduate Psychology Majors. Teaching of Psychology, 31, 247251. Neimeyer, G. J., & Stevenson, D. (2008). The virtual advisor: Successful strategies for getting into graduate school in psychology. Thousand Oaks, CA : Sage Publ ications. Retrieved July 1, 2008, from http://www.per -ce.net/va/index.php Norcross, J. C., Haynch, J. M., & Terranova, R. D. (1996). Graduate study in psychology: 19921993. American Psychologist, 51, 631643. Norcross, J. C., Sayette, M. A., & Mayne, T. J. (2008). The insiders guide to graduate study in clinical and counseling psychology: 2008/2009 Edition. New York: Guilford. Olgetree, S. M. (1999). Improving career advising: Responding to alumni surveys. Journal of Website Psychology, 26, 4245. Satterfield, C. D., & Abramson, C. I. (1998). The undergraduate psychology club: Possibilities and suggested activities. Teaching of Psychology, 25, 36 38. Sheehan, E. P. (1994). A multimethod asses sment of the psychology major. Teaching of Psychology, 21, 7478. Sheehan, E. P., & Granrud, C. E. (1995). Assessment of student outcomes: Evaluating an undergraduate psychology program. Journal of Instructional Psychology, 22, 366372. Shepperd, J. A. (1991). [Major trends in psychology departments]. Unpublished raw data, Holy Cross College, Worcester, MA. Shepperd, J. A. (1993). Developing a prediction model to reduce a growing number of psychology majors. Teaching of Psychology, 20, 97 101. Smit h, R. A. (1985). Advising beginning psychology majors for graduate school. Teaching of Psychology, 12, 194198. Thomas, J. H., & McDaniel, C. R. (2004). Effectiveness of a required course in career planning of psychology majors. Teaching of Psychology 31, 2227. Thomas, J. H., McDaniel, C. R., & Wagner, B. M. (2005, April). Improving the validity of the career exploration and decidedness inventory. Poster presented at the Southeastern Psychological Association, Nashville, TN.

PAGE 67

67 Thomas, J. H., McDan iel, C. R., & Wagner, B. M. (2006, April). Contributions of decidedness, career exploration, and self -exploration to career development. Poster presented at the Southeastern Psychological Association, Atlanta, GA. Ware, M. E. (1986). Assessing student s career needs at a small private university. Teaching of Psychology, 13, 185188. Ware, M. E. (1988). Teaching and evaluating a career development course for psychology majors. In P. J. Woods (Ed.), Is psychology for them? A guide to undergraduate adv ising (pp. 64 74).

PAGE 68

68 BIOGRAPHICAL SKETCH Brittany Brucato was born in 1 983 in Cleveland Ohio. She moved to Miami Florida, when she was five years old, where she attended Palmetto Senior High School. She graduated from the University of Florida in 2005, w here she received her B.S. in psychology. She then continued at the University of Florida, completing a masters degree in psychology in 2007. She received her Ph.D. from the University of Florida in the summer of 2010.