Assessment
Assessment
WOU's Office of Study Abroad and International Exchanges is committed to continuous improvement. This ongoing assessment project is conducted by Michele V. Price, Director and Dr. Victor Savicki, Professor of Psychology, Emeritus. This project has been approved by WOU's Institutional Review Board.
Program Design-Assessment Integration
Over a seven year period the Office of Study Abroad and International Exchanges at Western Oregon University has engaged in an ongoing assessment and research effort aimed at increasing its effectiveness in enhancing the study abroad experience for its students. As a result of repeated assessments and more focused research endeavors, the required study abroad capstone project, student orientation, and re-entry interventions have been revised and strengthened based on systematic data collection, analysis, and interpretation. This integration of assessment/research with program components has led to a richer, more effective program for students that is evidence based.
The following timeline highlights specific assessment/research projects and their subsequent impact on study abroad program improvements.
2006 Selection of assessment instruments and integration of assessment and evaluation into the application and selection process for study abroad students. Pre and Post survey questions were incorporated into an online format available to students via web browsers at a time and place of their choosing. The pre- and post survey became a required component of the study abroad capstone project. A journal assignment, completed while abroad, also was added. (Evolution of a Study Abroad Capstone Project)
2007 First data analysis findings indicate marginal changes in hypothesized directions on a majority of measures, yet a need to focus on deeper reflection in on-site and re-entry requirements. Therefore, the structure for the final reflection paper or capstone assignment was revised, and a new grading rubric was developed to guide students in a more focused way to successfully complete the assignment. (Re-entry Postdiction of Pre-Departure Readiness)
2008 Analysis of capstone papers using content analysis software indicated high degrees of insight, and inclusiveness, focus on positive emotions, and a linkage between self-reported study abroad goal attainment and critical thinking and systemic thinking. To further the reflective process, changes were made in orientation to help focus student attention on issues that may be topics for culturally-based reflection. (Re-entry Integration, Pre-departure Readiness: Development and Research on Student Writing)
2009 Content analysis based on the Developmental Model of Intercultural Sensitivity indicated 57% of code-able units as ethnocentric and 43% ethno-relative. An unexpected result was the discovery of patterns of how students avoid effective reflection by complete externalization or internalization rather than integration of the two. To encourage students to overcome these avoidance strategies, a mandatory re-entry session was added to the capstone project. (Student Readiness and Intercultural Sensitivity)
2010 Orientation, on-site, and re-entry activities continued to be revised, including the elimination of the reflective paper and the addition of a re-entry resume development assignment, focused on transferable skills based on study abroad experiences as a mechanism to enhance reflection The journal assignment, previously modeled on the “Strategies for Keeping a Journal” chapter of the University of Minnesota’s text, Maximizing Study Abroad, also was revised. The new assignment focused on asking the right questions pre, while abroad, and the post was designed to encourage students to view their study abroad experience as an integrated whole and to be able to articulate why the experience was important. Chisolm’s text, Charting a Hero’s Journey, was used in developing the questions.
2011 Over the five years of this project, it has become more and more evident that students need guidance to unpack their experience abroad. The effectiveness of the redesigned capstone project in helping students to cultivate awareness hone observational skills and articulate expression will be assessed in the 2011/12 academic year. (Asking the Right Questions)
Reflection Provoking Questions Assignment
Prior to departure
While Abroad
After Returning
These questions are part of a new journal assignment, the most significant change in the capstone project. They are designed to help students reflect back. Pre-departure questions plant seeds, which students have to cultivate by reflecting back while abroad and then again after returning.
Results of Current Research Project
Students who had completed both pre and post, online study abroad student assessment questionnaires (N= 136) were divided into two groups (median split) based on their report of how well they attained their personal goals during their study abroad sojourn. Overall goal attainment was based on the sum of the 13 item Study Abroad Goals Scale (Kitsantas, 2004) which included sub-scales on Cross-cultural competence, Subject interest and competence, and Social gathering goals.
Conclusions
Linkages between program design and assessment, evaluation, and research projects have enhanced and will continue to enhance our students’ study abroad experience. Changes in program design have been evidence-based.
The effectiveness of the redesigned capstone project in helping students to cultivate awareness, hone observational
skills, and articulate expression will be assessed in the coming year with particular attention:
2011/12 These graphs indicate changes overtime on several key variables that have been tracked by the WOU Study Abroad Office since 2006. Only changes that were statistically significant are reported. Graphs accompany each of the reported assessment results for ease of visualizing the outcomes.
WOU's Office of Study Abroad and International Exchanges is committed to continuous improvement. This ongoing assessment project is conducted by Michele V. Price, Director and Dr. Victor Savicki, Professor of Psychology, Emeritus. This project has been approved by WOU's Institutional Review Board.
Program Design-Assessment Integration
Over a seven year period the Office of Study Abroad and International Exchanges at Western Oregon University has engaged in an ongoing assessment and research effort aimed at increasing its effectiveness in enhancing the study abroad experience for its students. As a result of repeated assessments and more focused research endeavors, the required study abroad capstone project, student orientation, and re-entry interventions have been revised and strengthened based on systematic data collection, analysis, and interpretation. This integration of assessment/research with program components has led to a richer, more effective program for students that is evidence based.
The following timeline highlights specific assessment/research projects and their subsequent impact on study abroad program improvements.
2006 Selection of assessment instruments and integration of assessment and evaluation into the application and selection process for study abroad students. Pre and Post survey questions were incorporated into an online format available to students via web browsers at a time and place of their choosing. The pre- and post survey became a required component of the study abroad capstone project. A journal assignment, completed while abroad, also was added. (Evolution of a Study Abroad Capstone Project)
2007 First data analysis findings indicate marginal changes in hypothesized directions on a majority of measures, yet a need to focus on deeper reflection in on-site and re-entry requirements. Therefore, the structure for the final reflection paper or capstone assignment was revised, and a new grading rubric was developed to guide students in a more focused way to successfully complete the assignment. (Re-entry Postdiction of Pre-Departure Readiness)
2008 Analysis of capstone papers using content analysis software indicated high degrees of insight, and inclusiveness, focus on positive emotions, and a linkage between self-reported study abroad goal attainment and critical thinking and systemic thinking. To further the reflective process, changes were made in orientation to help focus student attention on issues that may be topics for culturally-based reflection. (Re-entry Integration, Pre-departure Readiness: Development and Research on Student Writing)
2009 Content analysis based on the Developmental Model of Intercultural Sensitivity indicated 57% of code-able units as ethnocentric and 43% ethno-relative. An unexpected result was the discovery of patterns of how students avoid effective reflection by complete externalization or internalization rather than integration of the two. To encourage students to overcome these avoidance strategies, a mandatory re-entry session was added to the capstone project. (Student Readiness and Intercultural Sensitivity)
2010 Orientation, on-site, and re-entry activities continued to be revised, including the elimination of the reflective paper and the addition of a re-entry resume development assignment, focused on transferable skills based on study abroad experiences as a mechanism to enhance reflection The journal assignment, previously modeled on the “Strategies for Keeping a Journal” chapter of the University of Minnesota’s text, Maximizing Study Abroad, also was revised. The new assignment focused on asking the right questions pre, while abroad, and the post was designed to encourage students to view their study abroad experience as an integrated whole and to be able to articulate why the experience was important. Chisolm’s text, Charting a Hero’s Journey, was used in developing the questions.
2011 Over the five years of this project, it has become more and more evident that students need guidance to unpack their experience abroad. The effectiveness of the redesigned capstone project in helping students to cultivate awareness hone observational skills and articulate expression will be assessed in the 2011/12 academic year. (Asking the Right Questions)
Reflection Provoking Questions Assignment
Prior to departure
- How and why did you choose the particular location of your study abroad? Who or what influenced your choice?
- What expectations and what preconceived notions do you have of your host culture? How do you see yourself interacting with the host culture in light of your expectations and pre-conceived notions?
- What are your expectations and what are your preconceived notions about your academic program? What do you hope to learn, and how do you envision the instructors and learning environments?
- How do you feel about leaving home and campus? What are you glad to be leaving behind? What do you think you’ll miss?
- What ideologies are you taking with you? Identifying these beliefs and their source may help you when you are challenged abroad by those with a different point of view.
While Abroad
- What has the separation from friends, family, school, and a familiar way of life revealed to you about yourself?
- Describe the scene that greeted you upon arrival in the airport and recount the behavior you observed. What bewildered, delighted, interested, amused, or frightened you? Why?
- Describe how your expectations and pre-conceived notions of the host culture are being met or not. How have your first impressions of the country and its people changed since your arrival?
- Systems of education and forms of teaching vary from country to country. What have you discovered about the system of education in your host nation compared to the system of education you experience in the U.S.? Are your preconceived notions hindering your progress or aiding you in navigating new academic rules?
- Have your relationships with individuals or your understanding of the culture become more complicated as your stay lengthens? How long do you think it takes to begin to understand and be part of the complexities of your host culture?
After Returning
- Were the expectations you had of your host culture met? Why or why not? Was the vision of yourself in interaction with the host culture realized? Why or why not? What about your preconceived notions? Which have you changed and which beliefs or attitudes have you retained?
- Were the expectations you had of your academic program abroad met? Why or why not? What about your preconceived notions? Which have you changed and which beliefs or attitudes have you retained?
- How do you feel about being home and back on campus? Are you glad to be home? What do you miss the most about your host country?
- What hard skills did you learn or build upon while abroad? How can these be reflected on your resume? How will these skills support your desired employment? How will you articulate these skills during an interview with a prospective employer?
- What soft skills did you learn or build upon while abroad? How can these be reflected on your resume? How will these skills support your desired employment? How will you articulate these strengths during an interview with a prospective employer?
These questions are part of a new journal assignment, the most significant change in the capstone project. They are designed to help students reflect back. Pre-departure questions plant seeds, which students have to cultivate by reflecting back while abroad and then again after returning.
Results of Current Research Project
Students who had completed both pre and post, online study abroad student assessment questionnaires (N= 136) were divided into two groups (median split) based on their report of how well they attained their personal goals during their study abroad sojourn. Overall goal attainment was based on the sum of the 13 item Study Abroad Goals Scale (Kitsantas, 2004) which included sub-scales on Cross-cultural competence, Subject interest and competence, and Social gathering goals.
Conclusions
Linkages between program design and assessment, evaluation, and research projects have enhanced and will continue to enhance our students’ study abroad experience. Changes in program design have been evidence-based.
The effectiveness of the redesigned capstone project in helping students to cultivate awareness, hone observational
skills, and articulate expression will be assessed in the coming year with particular attention:
- to whether students actually take the time to reflect back in their responses;
- to what degree they succeed or miss the mark in making connections about their experience as a whole. The process continues.
2011/12 These graphs indicate changes overtime on several key variables that have been tracked by the WOU Study Abroad Office since 2006. Only changes that were statistically significant are reported. Graphs accompany each of the reported assessment results for ease of visualizing the outcomes.