Data-Use Profiles in Primary Mathematics: A Think-Aloud Analysis of Teachers’ Interpretations of Learning Progress

Petra Ackerlauer; Zsolt Lavicza; Tony Houghton

doi:10.71514/jssal/2026.280

Authors

Petra Ackerlauer Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany https://orcid.org/0000-0002-6586-9634
Zsolt Lavicza Johannes Kepler University, Austria https://orcid.org/0000-0002-3701-5068
Tony Houghton Johannes Kepler University, Austria

DOI:

https://doi.org/10.71514/jssal/2026.280

Keywords:

Data Literacy, Teacher Data Use, Primary Education, Think-Aloud Method, Teacher Professional Development

Abstract

This study investigates Austrian primary in-service teachers’ understanding, interpretation, and sense-making of their students’ learning progress graphs generated by the program QUOP. To this end, teachers were asked to verbalize their thoughts using a spontaneous think-aloud technique. Their verbalizations were analyzed using predefined semantic markers derived from three theory-informed categories (data literacy, school organizational factors, and emotional factors), complemented by a context-interpretive synthesis. This approach enabled us to reconstruct how contextual variables were reflected qualitatively in teachers’ articulations. The results showed that Teachers 1 and 2, who worked in performance-oriented school environments, tended to interpret the data primarily as summative evaluations of students. In contrast, Teachers 3 and 4 from other school environments articulated their interpretations in more differentiated ways, reflecting a more pronounced formative interpretation of the data. Despite contextual differences, all teachers tended to reach definitive conclusions quickly, limiting the cyclical potential of this data source for adaptive teaching. The methodological approach combining semantic marker analysis of think-aloud protocols with context synthesis interpretation appeared adequate for assessing integrated factors of data use yet requires further development for systematic assessment.

Downloads

Download data is not yet available.

References

Ackerlauer, P., Diego-Mantecón, J. M., & Lavicza, Z. (2026). Digital assessment data to provide formative feedback in mathematics: a longitudinal study of teachers’ engagement with Quop in Austria. Teacher Development, 1–17. https://doi.org/10.1080/13664530.2026.2616491

Altrichter, H., Moosbrugger, R., & Zuber, J. (2016). Schul- und Unterrichtsentwicklung durch Datenrückmeldung. In K. Maag Merki & H. Altrichter (Eds.), Handbuch neue Steuerung im Schulsystem (pp. 235–277). Springer. https://doi.org/10.1007/978-3-531-18942-0

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. https://doi.org/10.1007/s11092-008-9068-5

Blumenthal, S., Gebhardt, M., Förster, N., & Souvignier, E. (2022). Internetplattformen zur Diagnostik der Lernverläufe von Schülerinnen und Schülern in Deutschland: Ein Vergleich der Plattformen Lernlinie, Levumi und Quop. Zeitschrift für Heilpädagogik, 73(4), 153–167. https://doi.org/10.5283/epub.52069

Bowen, G. M., & Roth, W.-M. (2005). Data and graph interpretation practices among preservice science teachers. Journal of Research in Science Teaching, 42(10), 1063–1088. https://doi.org/10.1002/tea.20086

Charters, E. (2003). The use of think-aloud methods in qualitative research: An introduction. Brock Education Journal, 12(2). https://doi.org/10.26522/brocked.v12i2.38

Coburn, C. E., & Turner, E. O. (2011). Research on data use: A framework and analysis. Measurement: Interdisciplinary Research and Perspectives, 9(4), 173–206. https://doi.org/10.1080/15366367.2011.626729

Corno, L. (2008). On teaching adaptively. Educational Psychologist, 43(3), 161–173. https://doi.org/10.1080/00461520802178466

Demski, D., van Ackeren, I., & Clausen, M. (2016). Zum Zusammenhang von Schulkultur und evidenzbasiertem Handeln: Befunde einer Erhebung mit dem „Organizational Culture Assessment Instrument“. Journal for Educational Research Online, 8(3), 39–58.

Demski, D., & Racherbäumer, K. (2015). Principals’ evidence-based practice: Findings from German schools. International Journal of Educational Management, 29(6), 735–748. https://doi.org/10.1108/IJEM-06-2014-0086

Dweck, C. S., Chiu, C. Y., & Hong, Y. Y. (1995). Implicit theories and their role in judgments and reactions: A word from two perspectives. Psychological Inquiry, 6(4), 267-285. https://doi.org/10.1207/s15327965pli0604_1

Ericsson, K. A., & Simon, H. A. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5(3), 178–186. https://doi.org/10.1207/s15327884mca0503_3

Förster, N., & Souvignier, E. (2015). Effects of providing teachers with information about their students’ reading progress. School Psychology Review, 44(1), 60–75. link.gale.com/apps/doc/A435358918/AONE?u=anon~855698fe&sid=googleScholar&xid=cb66026a.

Förster, N., Kuhn, J., & Souvignier, E. (2017). Normierung von Verfahren zur Lernverlaufsdiagnostik. Empirische Sonderpädagogik, 9(2), 116–122.

Gee, J. P. (2017). Introducing discourse analysis: From grammar to society (5th ed.). Routledge.

Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013). Seeking qualitative rigor in inductive research: Notes on the Gioia methodology. Organizational Research Methods, 16(1), 15–31. https://doi.org/10.1177/1094428112452151

Grillenberger, A., & Romeike, R. (2018). Developing a theoretically founded data literacy competency model. In Proceedings of the 13th Workshop in Primary and Secondary Computing Education (pp. 1–10). ACM. https://doi.org/10.1145/3265757.3265766

Groß Ophoff, J. (2007). Nutzung und Nutzen von Evaluationsstudien in Schule und Unterricht. Empirische Pädagogik, 21(4).

Gummer, E. S., & Mandinach, E. B. (2015). Building a conceptual framework for data literacy. Teachers College Record, 117(4), 1–22. https://doi.org/10.1177/016146811511700401

Hargreaves, A. (1998). The emotional practice of teaching. Teaching and teacher education, 14(8), 835-854. https://doi.org/10.1016/S0742-051X(98)00025-0

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112. https://doi.org/10.3102/003465430298487

Kemethofer, D., Zuber, J., Helm, C., Demski, D., & Rie, C. (2015). Effekte von Schulentwicklungsmaßnahmen auf Schüler/innenleistungen im Fach Mathematik. SWS-Rundschau, 55(1), 26–47.

Kuckartz, U. (2019). Qualitative text analysis: A systematic approach. In G. Kaiser & N. Presmeg (Eds.), Compendium for early career researchers in mathematics education (pp. 181–197). Springer.

Maag Merki, K., & Altrichter, H. (2015). Educational governance. DDS – Die Deutsche Schule, 107(4), 396–410.

Maier, U. (2010). Vergleichsarbeiten im Spannungsfeld zwischen formativer und summativer Leistungsmessung. Die Deutsche Schule, 102(1), 60–69.

Mandinach, E. B., & Gummer, E. S. (2016). What does it mean for teachers to be data literate? Laying out the skills, knowledge, and dispositions. Teaching and Teacher Education, 60, 366–376. https://doi.org/10.1016/j.tate.2016.07.011

Merk, S., Ophoff, J. G., & Kelava, A. (2023). Rich data, poor information? Teachers’ perceptions of mean differences in graphical feedback from statewide tests. Learning and Instruction, 84, Article 101717. https://doi.org/10.1016/j.learninstruc.2022.101717

O’Day, J. A. (2004). Complexity, accountability, and school improvement. In S. H. Fuhrman & R. F. Elmore (Eds.), Redesigning accountability systems for education (pp. 15–43). Teachers College Press. https://doi.org/10.17763/haer.72.3.021q742t8182h238

Reeves, T. D. (2017). Equipping preservice elementary teachers for data use in the classroom. Action in Teacher Education, 39(4), 361–380. https://doi.org/10.1080/01626620.2017.1336131

Schildkamp, K., Lai, M. K., & Earl, L. (Eds.). (2012). Data-based decision making in education: Challenges and opportunities. Springer.

Schildkamp, K., & Poortman, C. (2015). Factors influencing the functioning of data teams. Teachers College Record, 117(4), 1–42. https://doi.org/10.1177/016146811511700403

Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26(3), 482–496. https://doi.org/10.1016/j.tate.2009.06.007

Souvignier, E., Förster, N., & Salaschek, M. (2014). Quop: Ein Ansatz internetbasierter Lernverlaufsdiagnostik mit Testkonzepten für Lesen und Mathematik. In M. Hasselhorn & U.Trautwein (Eds.), Lernverlaufsdiagnostik (pp. 239–258).

Souvignier, E., Förster, N., Hebbecker, K., & Schütze, B. (2021). Using digital data to support teaching practice—Quop: An effective web-based approach to monitor student learning progress in reading and mathematics in entire classrooms. In A. Wilmers & S. Jornitz (Eds.), International perspectives on school settings, education policy and digital strategies (pp. 283–298). Budrich.

Spillane, J. P. (2012). Data in practice: Conceptualizing the data-based decision-making phenomena. American Journal of Education, 118(2), 113–141. https://doi.org/10.1086/663283

Stamm, M. (2008). Evaluation: Wirksame Wege zur Nutzung – Wege zur wirksamen Nutzung. In M. Ant, A. Hammer, & O. Löwenbein (Eds.), Nachhaltiger Mehrwert von Evaluation (pp. 145-158). W. Bertelsmann Verlag

van den Bosch, R. M., Espin, C. A., Chung, S., & Saab, N. (2017). Data-based decision-making: Teachers’ comprehension of curriculum-based measurement progress-monitoring graphs. Learning Disabilities Research & Practice, 32(1), 46–60. https://doi.org/10.1111/ldrp.12122

van Geel, M., Keuning, T., Visscher, A., & Fox, J. P. (2017). Changes in educators’ data literacy during a data-based decision-making intervention. Teaching and Teacher Education, 64, 187–198. https://doi.org/10.1016/j.tate.2017.02.015

Visscher, A. J., & Coe, R. (2003). School Performance Feedback Systems: Conceptualisation, Analysis, and Reflection. School Effectiveness and School Improvement, 14(3), 321–349. https://doi.org/10.1076/sesi.14.3.321.15842

Wagner, D. L., Hammerschmidt-Snidarich, S. M., Espin, C. A., Seifert, K., & McMaster, K. L. (2017). Pre-service teachers’ interpretation of CBM progress monitoring data. Learning Disabilities Research & Practice, 32(1), 22–31. https://doi.org/10.1111/ldrp.12125

Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92(4), 548–573. https://doi.org/10.1037/0033-295X.92.4.548

Zeuch, N., Förster, N., & Souvignier, E. (2017). Assessing teachers’ competencies to read and interpret graphs from learning progress assessment: Results from tests and interviews. Learning Disabilities Research & Practice, 32(1), 61–70. https://doi.org/10.1111/ldrp.12126