Hao, Q., D. H. Smith IV, N. Iriumi, M. Tsikerdekis, and A. J. Ko (2019).
A Systematic Investigation of Replications in Computing Education Research
ACM Transactions on Computing Education. vol. 19, no. 4, pp. 1–18, 2019, issn: 19466226. doi: 10.1145/3345328
Impact Factor: 1.670
As the societal demands for application and knowledge in computer science (CS) increase, CS student enrollment keeps growing rapidly around the world. By continuously improving the efficacy of computing
education and providing guidelines for learning and teaching practice, computing education research plays a
vital role in addressing both educational and societal challenges that emerge from the growth of CS students.
Given the significant role of computing education research, it is important to ensure the reliability of studies
in this field. The extent to which studies can be replicated in a field is one of the most important standards
for reliability. Different fields have paid increasing attention to the replication rates of their studies, but the
replication rate of computing education was never systematically studied. To fill this gap, this study investigated the replication rate of computing education between 2009 and 2018. We examined 2,269 published
studies from three major conferences and two major journals in computing education, and found that the
overall replication rate of computing education was 2.38%. This study demonstrated the need for more replication studies in computing education and discussed how to encourage replication studies through research
initiatives and policy making.