Five approaches to measuring engagement: comparisons by video game characteristics

Abstract

Engagement has been identified as a crucial component of learning in games research. However, the conceptualization and operationalization of engagement varies widely in the literature. Many valuable approaches illuminate ways in which presence, flow, arousal, participation and other concepts constitute or contribute to engagement. However, few studies examine multiple conceptualizations of engagement in the same project. This paper discusses the results of two experiments that measure engagement in five different ways: survey self-report, content analyses of player videos, electro-dermal activity, mouse movements, and game click logs. We examine the relationships among these measures and assess how they are affected by the technical characteristics of a 30 minute custom-built educational game: use of a customized character, level of narrative complexity, and level of art complexity. We found that the five measures of engagement correlated in limited ways, and that they revealed substantially different relationships with game characteristics. We conclude that engagement as a construct is more complex than is captured in any of these measures individually and that using multiple methods to assess engagement can illuminate aspects of engagement not detectable by a single method of measurement.