University student’s engagement: development of the University Student Engagement Inventory (USEI)

In recent years, student engagement, broadly defined as the “students’ willingness to participate in routine school activities, such as attending classes, submitting required work, and following teachers’ directions in class” (Nystrand & Gamoran 1992, p. 14) has received increasing attention by researchers, practitioners, and policy makers (Christenson et al. 2012; Fredricks et al. 2004; Reschly & Christenson 2012; Salanova et al. 2010). The growing body of research, primarily conducted in the United States and Australia, shows that student engagement can act as an antidote to low academic achievement, student burnout, student lack of resilience, dissatisfaction, and school dropout (Christenson & Reschly 2010; Elmore & Huebner 2010; Finn & Zimmer 2012; Wang & Eccles 2012; Krause & Coates 2008). It can also act as a strong mediator between out-of-school variables (e.g., home and family, friends and class-inmates,) (Chen & Astor 2011), teacher-student interactions, academic achievement, school success and life-long learning (Christenson et al. 2012; Gilardi & Guglielmetti 2011; Reschly & Christenson 2012). Recent studies have found a correlation between different engagement profiles and students’ learning (Wang & Eccles 2012) and physical and psychological health and well-being (Li & Lerner 2011; Wang & Eccles 2012).

In Europe, the engagement construct has emerged mainly in professional and occupational activities, and has been researched primarily in relation to business organizations (Bakker et al. 2008; Hirschi 2012; Schaufeli & Bakker 2010). From the business organizations' perspective, work engagement has been defined as a positive, fulfilling, affective-motivational psychological state that is characterized by vigour, dedication and absorption associated with work-related well-being (Bakker et al. 2008). This conceptualization is supported by Maslach and Leiter’s (1997) initial definition of work engagement, which is the opposite of work burnout. Several different conceptualizations of burnout and its relations to work engagement coexist (Hirschi 2012). However, most professionals would agree that engagement is an independent and distinct construct that is negatively correlated with burnout, and that engaged employees have high levels of energy, low burnout and are strongly identified with their organization (González-Romá, Schaufeli, Bakker & Lloret 2006; Bakker et al. 2008; Demerouti et al. 2010). In this organizational perspective, work engagement is operationalized by three correlated constructs: vigour, dedication and absorption. According to Bakker et al. (2008), vigour is characterized by high levels of physical energy, ability to deal with the job’s challenges, and a willingness to invest the required effort to overcome difficulties. Dedication is characterized by a strong involvement in one’s work with a sense of personal realization and pride. Finally, absorption is defined as concentration and happiness in performing the job related tasks (Bakker et al. 2008; Hirschi 2012). Grounded in these definitions, Schaufeli and co-workers have developed the Utrecht Work Engagement Scale (UWES) to measure these three dimensions of work engagement (Schaufeli & Bakker 2003). Nowadays, the UWES is the most used instrument to measure engagement in the work place (Bakker et al. 2008). There have been attempts to expand this construct to non-workplace settings. For example, the UWES was adapted to measure school engagement in university settings, around its three dimensions: vigour, dedication and absorption (Salanova et al. 2010). The UWES-Student Survey has been used in a few studies (Bresó et al. 2011; Gilardi & Guglielmetti 2011). However, concerns have been raised about simply rephrasing items for the work-place to the university setting (Mills et al. 2012; Schaufeli & Bakker 2010), as well as its dimensionality in different age-groups of students (Upadaya & Salmela-Aro 2012).

Indeed, there is a growing consensus among researchers that engagement is a multidimensional construct with both behavioural, emotional and psychological components (see Finn & Zimmer 2012 for a review). However, there is still no clear consensus on the construct precise definition and its dimensionality (Christenson et al. 2012; Fredricks & McColskey, 2012; Reschly & Christenson 2012; Wolf-Wendel et al. 2009; Kahu 2013). For example, proposals for the construct dimensionality have ranged from two- (emotional and behavioural) (Skinner et al. 2009) to eight- (Learning strategies, Academic integration, Institutional emphasis, Co-curricular activity, Diverse interactions, Effort, Overall relationships and Workload) first order factors (LaNasa et al. 2009). Higher dimensional construct conceptualizations have been also proposed. For example, Martin (2007) documented 11 first-order factors with a second order 4-factor model of student engagement and motivation with Australian high school students. There is also large variation in how engagement has been measured (e.g. questionnaires, self-report measures, teacher ratings, interviews and observations) (Fredricks et al. 2011). In another study with first year Australian Universities’ undergraduates, seven ‘calibrated’ measures of engagement were described (Krause & Coates 2008). This variation in both the operationalization and measurement of the engagement construct has made it difficult to make comparisons between studies’ findings (Fredricks & McColskey 2012; Kahu 2013). Therefore, consensus on the different measuring instruments, their dimensionality and concurrent validity is an important area of future research (Reschly & Christenson 2012; Wolf-Wendel et al. 2009; Krause & Coates 2008; Kahu, 2013).

In this paper, we expand Nystrand and Gamoran (1992) broad definition of students’ engagement, by adding the Fredricks et al. (2004) conceptualization of engagement as a three factor construct including behavioural, emotional and cognitive dimensions. This is one approach, to measure students’ engagement, that tries to integrate the four dominant research perspectives on this important construct, namely, the behavioural perspective, the psychological perspective, the socio-cultural perspective and the holistic perspective (for a review and criticism of the different perspectives on student engagement see Kahu 2013). Cognitive engagement is defined as the students’ investment and willingness to exert the necessary efforts for the comprehension and mastering of complex ideas and difficult skills. The emotional engagement dimension reflects both the positive and negative reactions to teachers’ instructions, classmates and school, perceptions of school belonging, and beliefs about the value of schooling. Finally, behavioural engagement is defined in terms of student’s participation in classroom tasks, conduct, and participation in school-related extracurricular activities (see also Carter et al. 2012; Sheppard, 2011).

The majority of studies of student engagement have been conducted in the USA and Australia, with elementary, middle and high school students. One exception is the USA’s National Survey of Student Engagement (NSSE) (NSSE 2012) and the Beginning College Survey of Student Engagement (BCSSE) (BCSSE 2013), which measures engagement in first-year US college students (Chambers & Chiang 2011; Kuh 2009; McCormick & McClenney 2012). However, the NSSE has been strongly criticized. Some scholars emphasize that the NSSE lacks good psychometric properties (namely construct and predictive validities and overall reliability) (Campbell & Cabrera 2011; LaNasa et al. 2009), and that it does not directly measure the “student engagement” psychological construct, but rather students’ studying habits, gains from their college experiences, and other aspects of student life (Wefald & Downey 2009). Although most of these criticisms on the NSSE have been addressed (see McCormick & McClenney 2012), there are still unanswered questions regarding the psychological “student engagement” construct definition and its dimensionality in different schooling contexts, student minority groups and family and social contexts where schools are immersed. Another example, is the seven ‘calibrated’ scales of first-year undergraduates’ engagement in Australian Universities (Krause & Coates 2008). However, this Australian study is not conclusive regarding the construct dimensionality. On the contrary, it points out to the imperative necessity for developing a broader understanding of engagement as a construct with several dimensions (Krause & Coates 2008). Accordingly to these scholars, the multidimensional feature of students’ engagement must be acknowledged in any measurement and monitoring of this construct in higher education.

Interventions aimed at improving university students’ academic achievement, reduce burnout improve student well-being, engagement and graduation success have been pursued in several high schools and universities all over the world (see e.g., Bresó et al. 2011; Chen & Astor 2011; Christenson & Reschly 2010; Elmore & Huebner 2010; Fredricks et al. 2004; Harlow et al. 2011; Kuh 2009; Li & Lerner 2011). Increasing engagement among university students has become crucial to improving their learning experiences, well-being and return in the investment in higher education (Bresó et al. 2011; Christenson & Reschly 2010; Kuh 2009; Salmela-Aro et al. 2009). Therefore, data with good psychometric properties on university students’ engagement and its correlates with academic achievement, course-work fulfilment, graduation and school integration are fundamental for educational psychologists, school counsellors as well as education policy-makers.

To the best of our knowledge, with the exception of the UWES-SS developed in Europe, the Beginning College Survey of Student Engagement (BCSSE) (BCSSE 2013) in the USA and seven ‘calibrated’ scales of first year undergraduate students in Australian Universities (Krause & Coates 2008) both of which have suffered from several criticisms, there are no psychometrically valid and publically available measures of student engagement in university settings. The UWES-SS mimics the professional work-place related engagement and only some of its items comply with studying and university activities (e.g., vigour). Thus, its use in the university context has suffered some criticisms, either due to methodological limitations in items’ construction or non-adaptation to the university context (Mills et al. 2012). On the other hand, surveys developed for high school students (see e.g. Fredricks & McColskey 2012) leave out some key facets of students’ engagement that are important at the university level (e.g., attendance at conferences and seminars). Also, the engagement construct’s dimensionality may vary with different students’ ages (Upadaya & Salmela-Aro 2012). Therefore, surveys developed for younger students may lack the appropriate construct definition and dimensionality, as well as adequate psychometric properties for evaluating student engagement in university students.

The present study draws from the earlier work of Schaufeli et al. (2002) on the UWES-SS, the work by Fredricks et al. (2004) with upper elementary school students in an effort to integrate the 4 perspectives on student engagement described by Kahu (2013) into a single measure instrument. In this study, we developed a new measure, – the University Student Engagement Inventory (USEI) and describe its psychometric properties in a sample of Portuguese college students. We examine content, construct and criterion related validities, and measurement invariance in two independent samples of students, from public and private universities and several study areas. Concurrent validity with the UWES-SS was assessed, as well as predictive validity for self-reported academic achievement and intention to dropout. We hypothesize that ‘university student engagement’, conceptualized as the students’ involvement and motivation towards his or her course work and academic activities within the university, is a second-order construct that is reflected in 3 first-order emotional, behavioural and cognitive dimensions expanding the Nystrand and Gamoran (1992); Fredricks et al. (2004) and Kahu (2013) conceptualizations. We anticipate that this construct is predictive of students’ academic achievement and intention to drop out of school. This new measure, focused on behaviors, cognitions, emotions and actions, that is, on the behavioral, psychological and holistic aspects of engagement can be used both for student counselling as well as research aimed at educational policy and good practices development to improve academic success and school retention.

A growing body of research has shown that student engagement is predictive of students’ well-being, satisfaction with school and peers, academic achievement, and lower burnout and school dropout rates (see e.g. Chen & Astor 2011; Finn & Zimmer 2012; Kuh 2009; Salmela-Aro et al. 2008). In studies with University students, engagement has been measured primarily with the UWES-SS in Europe and the BCSSE in the USA. However, these two instruments have suffered several criticisms ranging from the construct definitions and dimensionality to its application to university students (Campbell & Cabrera 2011; LaNasa et al. 2009; Mills et al. 2012; Wefald & Downey 2009). However, there is still a large debate on the definition (Finn & Zimmer 2012; Fredricks et al. 2011; Schaufeli & Bakker 2010; Schaufeli et al. 2002) and dimensionality (Fredricks & McColskey 2012; LaNasa et al. 2009) of student engagement construct and the need for validated measures is necessary for research in psychology and education in the area of student motivation, burnout and academic success.

In this study, we aimed to evaluate student engagement at the university level, using the expanded three dimensions (behavioural, emotional, and cognitive engagement) previously addressed with students from elementary and high schools in the US. From the different construct dimensionalities observed in the literature (see Fredricks et al. 2011; Kahu 2013; Krause & Coates 2008), our proposed USEI was built upon a scale initially developed by Fredricks and her colleagues for use with elementary students, and which has been also used with older students. Since contextual factors specifically related to the university context have been left out from these previous adaptations, a focus group analysis was conducted to identify specific university activities that could impact on student engagement and a set of new items were developed. Furthermore, there is evidence that the construct dimensionality can be also age-related (Upadaya & Salmela-Aro 2012). Thus, reported differences in the constructs dimensionality may be not only due to school and family related contexts, but also to students’ developmental processes.

Our study adds to the definition of university students’ engagement from an educational perspective in opposition it with the organizational perspective proposed by Schaufeli and colleagues, currently, the most used in Europe (Bresó et al. 2011; Salanova et al. 2010; Schaufeli et al. 2002). The Schaufeli et al. inventory has been criticized for its’ methodological approach (Mills et al. 2012), and difficulties in applying the organizational dimensions to the educational context (e.g., absorption and vigour), (Upadaya & Salmela-Aro 2012; Wefald & Downey 2009). Our proposed inventory addresses these criticisms.

Psychometric data analyses support the behavioural, cognitive and emotional tri-factorial conceptualization of student engagement. Evidence for strong measurement and structural invariance for this 3-factor model was demonstrated in two independent samples of Portuguese university students. Moderate to strong correlations were observed between the three factors, allowing for an empirically supported second-order engagement construct that is reflected on first order behavioural, emotional and cognitive dimensions. This second-order model showed a good fit to university students from both public and private universities majoring in social and human sciences, biological sciences and exact sciences.

Furthermore, the USEI produced reliable estimates of engagement and showed evidence for concurrent validity with the UWES-SS. Although both scales assess overall student engagement, the moderate correlations between their dimensions suggest that different facets of student engagement are being assessed by USEI and UWES-SS as an item’s content analysis would support. The USEI also showed evidence for predictive validity over a set of indicators of student academic achievement, courses failed and intention to dropout. For each extra point in the engagement scale the odds of improving academic achievement, as self-perceived by the student, increases significantly almost 3 times while the chances of failing one or more courses or dropping out of school decreases 2 times.

Our results show that data collected from a new instrument of student engagement has predictive value for important academic variables and can be used to further develop research and psycho-educational interventions on student engagement. Further research should be aimed at developing student engagement as a mediator to improve student achievement, reduce academic failure, promote psychological well-being and reduce university dropout rates.

However, this study has some limitations that need to be pointed out. First, the data of this study is only for a Portuguese population. Therefore, other cross-cultural studies are necessary before the USEI can be used more broadly in research and interventions aimed at improving student engagement. Second, a limitation of this instrument for a U.S. college student population is that it has no items applying to residential life which is a very important aspect of the college experience in the U.S for first year students. Thus the USEI can be used for students living out-of-campus, but may miss an important dimension for the other students’ academic experience. It also did not escape our attention that some items targeted initially to evaluate facets of behavioral engagement (e.g., items encompassing both classroom tasks and school-related extracurricular activities) did not make it to the final 15-item format, either because the experts did not judge these items as fundamental and/or either because they showed weak factor loadings in their proposed factors.

The results observed with the USEI also point to some new research direction to which the USEI maybe useful. For example, when is a student’s engagement at risk during his/her course works? In the earlier semesters or near the end of the academic work? Can the USEI be used as a predictor of student drop-out and or failing courses? Can a relationship between student academic performance, physical and psychological well-being and the students’ engagement in his/her course work be demonstrated? Can psychoeducational interventions aimed at improving students’ engagement have beneficial outcomes for the students’ proficiency and well-being? Can the USEI be used to produce valid and reliable measures of engagement during interventions? These are just a few research questions for an area where much research, supported on valid and reliable data, is needed.

The validated USEI was composed of 15 items, supports the tri-factorial structure of student engagement. We documented evidence of adequate reliability, factorial, convergent and discriminant validities in a sample of Portuguese college students. USEI’s concurrent validity, with the Utrecht Work Engagement Scale-Student Survey, and the predictive validity for self-reported academic achievement and intention to dropout from school were also observed.