By Eric Hartman 

The 2013 International Service-Learning Summit at Northwestern University featured a vibrant community of researchers and practitioners dedicated to global service-learning (GSL; See global / international terminology discussion in Hartman & Kiely, 2014). Among other documented discussions (Lough & Toms, 2015), participants called for clear, credible, and accessible evaluations that can help support an emerging community of practice. This desire developed from institution-specific interest in continuous improvement as well as field-level interests in considering the power of systematic learning interventions across populations and institutional contexts.

In response to this challenge, an interdisciplinary team of researchers considered the framing of global learning provided through the American Association of Colleges and Universities (AAC&U) and NAFSA: Association of International Educators (Hovland, 2014; McTighe Musil, 2006), established assessment methods in international education (Bennett, 1993; Braskamp, Braskamp, & Engberg, 2014; Morais & Ogden), and existing efforts to assess the impact of global volunteering or service on the participating volunteers or students (Bowman, Brandenberger, Snyder Mick, & Toms Smedley, 2010; Lough, 2010; Niehaus & Crain, 2013). While considerable insights have been amassed in these areas, institutional needs in the context of broad articulations of global learning along with GSL field-building desires led to interests in a survey that more explicitly (a) engages the civic component of global learning; (b) integrates qualitative feedback opportunities throughout the survey; (c) does more to isolate program factors; and (d) is brief and accessible across a variety of institutions.

The Global Engagement Survey 

The result of these desires was the creation of the Global Engagement Survey, a multi-institutional effort coordinated through globalsl.org. During the pilot year (Summer 2014) that just passed, institutional participants were secular and faith-based, public and private, liberal arts and Research 1. While more than 500 students participated in the thirty programs included in the GES, slightly more than 100 were matched through their completion of the pre- and post- experience survey. The survey includes 46 closed questions (strongly agree to strongly disagree), along with 9 opportunities for open-ended sharing. The image below captures the structure of the survey.

Considerable effort was made to isolate program factors, as the diversity among institutional interventions that are intended to advance global learning is incredible. Participating programs included: those with and without service-learning / community engagement, immersive programming inside the United States and outside of it, co-curricular and academic programming, programs less than and more than 4 weeks, and programs that required second language capacity as well as those that did not, among several other factors.

Following the initial pilot survey, we briefly share our sense of results before offering some individual reports below.

While principal component analysis suggest several valid constructs as featured in the image above, results suggest the need to continue developing quantitative constructs. Additionally, qualitative results suggest the importance of institutional and programmatic characteristics in framing and determining student learning. This research breaks new ground by (1) developing and demonstrating a multi-institutional approach to isolating the effects of program factors on global learning outcomes, (2) combining the insights of the international education and civic engagement fields, (3) integrating opportunities for paragraph-length reflections for evidence of behavioral choices and demonstrable student learning that support self-report assertions, and (4) beginning to reveal the impact of institutional and program factors on student learning outcomes.

What emerged through analysis of the qualitative reflections was the extent to which program or institutional culture, as well as explicit learning goals, influenced students’ reflections and insights. This lesson builds on Kiely’s (2004, 2005) work with transformational learning as border crossing, which demonstrated the highly personal and contextual nature of various types of borders. For a partial list, border crossing includes personal (biography, learning style, prior travel experience), structural (race, class, gender, culture), and historical (socioeconomic and political history of communities involved) borders. The data gathered here suggest how institutional culture and classroom or program learning goals can contribute to the creation and navigation of these various borders.

When reflecting upon the post-immersion prompts, participants relied on terminology, theories, and vernacular introduced in their GSL courses. In response to the question about dominant cultural assumptions, for example, one cohort of students focused explicitly on concepts and practices of international development in their coursework.

In global development, the difference between a deficit-based approach or an asset-based approach lies in the cultural assumptions that are made. Aide organizations, if they are acting upon the community instead of from the inside out, will assume who deserves to be worked with and what that population needs.

Another reflection from the same cohort of students provides a further example:

While I was in India, I saw this phenomena at work with regards to the caste system. Even well-intentioned high-caste Indians – even those working with development organizations – sometimes have detrimental views about people in the lower castes being “backwards” or undeserving of the government positions they receive through India’s “reservation system” (kind of like affirmative action for caste).

These two comments explicitly tied to development thinking or practice might seem unremarkable as a response to the prompt, but these were the only two responses to focus explicitly on development thinking out of six pages of such open-ended responses. Other responses from the same cohort also demonstrated relatively sophisticated, structural thinking in respect to both history and whiteness that seemed consistent with informed and self-aware development thinking and practice. Although participants from a number of institutions exhibited nuances in respect to whiteness and history, one institution stood out from the others in respect to its students’ capacities to articulate learning through an international development lens.

More information on the quantitative components of the study can be gathered from the individual institutional reports shared below. Kansas State University, Northwestern University, and Westmont College have agreed to share their reports. Without going too far into institution-specific conversations, faculty and staff members at these institutions have shared how the reports have helped them confirm existing strengths and see specific areas for improvement. The leadership focus at one institution, for example, may have played a role in influencing students to look inward at self and team dynamics, while the specific version of faith embraced at another institution may have led to an institutional culture that embraces a charity approach to service to a greater extent than is the case at other institutions.

 Assessing Global Learning: Global Engagement Survey (Kansas State University) Assessing Global Learning: Global Engagement Survey (Northwestern University)

The GES was initiated with the intent of providing comparative, financially accessible global learning evaluation opportunities to institutions dedicated to advancing students’ intercultural competence, (global) civic engagement, and critical thinking in tandem. Founding sponsor institutions provided funding to make this multi-institutional work possible. For summer 2015, additional institutions may opt into the globalsl.org network by becoming a sponsor, partner, or grant recipient. Additional information on these opportunities will be available in a forthcoming post, now here.

Many thanks to the GES Researchers and Founding Sponsor institutions:

GES Director: Eric Hartman, Staley School, Kansas State University
GES Quantitative Research Director: Benjamin J. Lough, University of Illinois at Urbana – Champaign
GES Qualitative Research Director: Cynthia Toms, Westmont College
GES Director of Evaluation: Nora P. Reynolds, Temple University

The GES Researchers would like to thank the institutional sponsors of globalsl.org; their support made this reporting possible. Sponsoring institutions include:

Cornell University

Duke University

Kansas State University

Northwestern University

Washington University in St. Louis

AAC&U. (2014). VALUE: Valid assessment of learning in undergraduate education. Washington, DC: American Association of Colleges and Universities . Downloaded from https://www.aacu.org/value/rubrics/global-learning

Bennett, M. J. (1993). Towards a developmental model of intercultural sensitivity In Michael Paige, ed. Education for the Intercultural Experience. Yarmouth, ME: Intercultural Press.

Bennett, M. J. (2012). Paradigmatic assumptions of intercultural learning. In M. Vande Berg, R. Paige, & K. Hemming Lou (Eds.), Student learning abroad: What our students are learning, what they’re not, and what we can do about it. Sterling, VA: Stylus Publishing.

Bowman, N., Branderberger, J., Snyder Mick, C, & Toms Smedley, C. (2010). Sustained immersion courses and student orientations to equality, justice, and social responsibility: The role of short-term service-learning. Michigan Journal of Community Service Learning, 17(1), 20-31.

Braskamp, L., Braskamp, D., & Engberg, M. (2014). Global perspective inventory (GPI): Its purpose, construction, potential uses, & psychometric characteristics. Global Perspective Institute, Inc. www.gpi.central.edu

Hartman, E. & Kiely, R. (2014). Pushing boundaries: Introduction to the global service-learning special section. Michigan Journal of Community Service Learning, 21(1).

Hovland, K. (2014). Global learning: Defining, designing, demonstrating. American Association of Colleges and Universities.

Kiely, R. (2004). A chameleon with a complex: Searching for transformation in international service-learning. Michigan Journal of Community Service Learning, 10(2), 5-20.

Kiely, R. (2005). A transformative learning model for service-learning: A longitudinal case study. Michigan Journal of Community Service Learning, 12(1), 5-22.

Lough, B. (2010). Predictors of intercultural competence among international volunteers. Electronic Theses and Dissertations, Paper 214, Washington University of St. Louis.

Lough, B.J. & Toms, C. (forthcoming) Global Service-Learning: Concerns from a Community of practice.

McTighe Musil, C. (2009). Educating students for personal and social responsibility: The civic learning spiral. In Barbara Jacoby & Associates, Civic Engagement in Higher Education, pp. 49-65. San Francisco, CA: John Wiley & Sons.

Morais, D.B., & Ogden A.C. (2011). Initial development and validation of the global citizenship scale. Journal of Studies in International Education, 15(5), 445-466.

Niehaus, E. & Crain, L. K. (2013). Act local or global? Comparing student experience in domestic and international service-learning programs. Michigan Journal of Community Service Learning, 20(1), 31-40.