The impact of different personalisation algorithms on literacy and numeracy in Kenyan pre-primary education: A comparative study of summative and formative assessments results

Digital personalisation has demonstrated potential to enhance learning. However, there is limited evidence on the comparative impact of different content personalisation algorithms on early-year numeracy and literacy outcomes, especially in low- and middleincome countries (LMIC). This paper reports an A/B/C test conducted over three weeks via a digital personalised learning tool used by 6479 Kenyan pre-primary learners. Two personalisation algorithms were implemented (maximising learner engagement or score), while expert-curated sequence was used as a control. Learners from 1509 classes were randomly divided across the three partitions. Learning in numeracy and literacy was compared across three metrics: summative assessment, curriculum progress, and formative assessment. Results showed no difference between partitions in the summative assessment. Different partitions tended to progress through the digital curriculum at a different pace. Significant differences between partitions were found in formative assessment scores, with the impact of each algorithmic approach varying according to different learning strands. Findings contribute to a deeper understanding of how different algorithms impact pre-primary education in LMIC contexts, with implications for designing personalised learning approaches tailored to specific learning content and learner profiles. Keywords: Digital personalised learning, pre-primary, low- and middle-income country, literacy, numeracy