Guessing or solving? Exploring the use of motion features from educational game logs

A learner's guessing behavior while playing educational games can be a key indicator of her disengagement that impacts learning negatively. To distinguish a learner's guessing behavior from solution behavior, we present an explorative study of using motion features, which represent a learner's finger movements on a tablet screen. Our data was collected from the Missing Number game of KitKit School, a tablet-based math game designed for children from pre-K to grade 2 in elementary school. A total of 5,040 problem solving logs, which were collected from 168 students, were analyzed. A two-sample t-test showed a significant difference between guessing and solution behavior for four groups of motion features that indicate distance, curvedness, complexity, and pause (p<0.001). Additionally, our empirical results showed the possibility of using motion features in automatic detection of guessing behavior. Our best model yielded an accuracy of 0.778 and AUC value of 0.851 by using the random forest classifier.