Shamya Karumbaiah
Lab Director
Visit websiteDr. Shamya Karumbaiah is an assistant professor of learning sciences and learning analytics in the Department of Educational Psychology at the University of Wisconsin-Madison and the director of The Responsible AI for Learning (TRAIL) Lab. Her research, at the intersection of learning sciences, learning analytics, AI, and human-centered design: (1) supports students to bring diverse linguistic and cultural assets into their learning, (2) enables K–12 teachers to develop equitable practices, and (3) improves methods to identify and mitigate AI biases. In 2022, Dr. Karumbaiah earned her PhD in Learning Sciences and Technology from the University of Pennsylvania. Before joining UW-Madison in 2023, she spent a year as a postdoctoral fellow in the Department of Computer Science at Carnegie Mellon University. For her dissertation work on biases in AI-based learning systems, she was selected in the 2021 cohort of EECS (Electrical Engineering and Computer Science) Rising Stars by MIT (Massachusetts Institute of Technology). Dr. Karumbaiah's research has been nominated for Best Research Paper awards in premier conferences on educational AI such as Artificial Intelligence in Education (2025), Computer Supported Collaborative Learning (2025), Learning Analytics and Knowledge (2021), International Conference on Quantitative Ethnography (2020), International Conference for Computers in Education (2018), and Educational Data Mining (2018). She has 63 peer-reviewed publications—24 as first author and 21 with her graduate students as first author—in stringently refereed proceedings with an average acceptance rate of ~20%. More recently, she was awarded an Early-Career Investigator award (2026) by the Vilas Trust, Racial Equity grant (2025) by the Spencer Foundation, an American Family Funding Initiative award (2024), and an Early Career Researcher award by the Society of Learning Analytics Research (2024) for research on topics related to identifying and mitigating AI inequities in multilingual and multicultural learning contexts. She is currently a finalist on Multilingual by Design: Co-Building AI Tools for Multilingual K-8 Classrooms award by Gates Foundation (2026). In 2026, Dr. Karumbaiah received the prestigious Award for Mentoring Undergraduates in Research, Scholarly and Creative Activities by the Office of the Provost and Division for Teaching and Learning at UW-Madison. In 2025, through a School of Education Outreach Development Fund, Dr. Karumbaiah translated her lab’s research into a professional learning opportunity for educators called the Disillusioning AI for Teachers Fellowship program. She received new funding in 2026 from the university’s Office of Civic Engagement to offer the third iteration of the fellowship to higher education instructors at all the Universities of Wisconsin. For her educational AI expertise, Dr. Karumbaiah has been invited to talk at several notable events, including the presidential session (2026) by the International Society of Learning Sciences (ISLS) to talk about how AI Exacerbates Educational Inequities for Multilingual Learners, UW Now at New York City (2026) by the Wisconsin Alumni Association to talk about Responsible Use of AI in Teaching and Learning, Bloomberg’s Wall Street Week (2025) on Can AI Replace Teachers?, and the National Academy of Education (NAEd) annual meeting (2023) to talk about Challenges and Opportunities for AI in Teaching and Learning.
2025
Teacher Agency in Implementing Automated Writing Evaluation Systems in a Science Classroom: A Focus Group Study
Yuxuan Fu, Shamya Karumbaiah, Xiaoying Gao, Ipshita Dey, Debbie Gnesdilow, Sadhana Puntambekar
Proceedings of the 18th European Conference on Technology Enhanced Learning (ECTEL)
Teacher Feedback on Linguistic Variations as a Source for Evaluating Language Models
Kaycie Barron, N. Tseng, Shamya Karumbaiah, Cynthia Baeza
International Conference of the Learning Sciences (ICLS)
- Exploring Out-of-Tutor Interventions by Teachers in Supporting Student Learning with AI Tutors
Shamya Karumbaiah, Anand Syamkumar, Vincent Aleven
International Conference of the Learning Sciences (ICLS)
A Comparative Analysis of LLM and Specialized NLP System for Automated Assessment of Science Content
X. Gao, Shamya Karumbaiah, A. Dalal, I. Dey, D. Gnesdilow, S. Puntambekar
Proceedings of the 26th International Conference on Artificial Intelligence in Education (AIED)
Authenticity or alienation: Latine students' perceptions of stereotypes in AI-generated educational materials
Proceedings of the 26th International Conference on Artificial Intelligence in Education (AIED)
Teacher Perceptions on AI Support for Bi/multilingual Learners: Superpowers and Constraints
Alina Guha, Shamya Karumbaiah, Cynthia Baeza, K. White, Mariana Castro
Proceedings of the 26th International Conference on Artificial Intelligence in Education (AIED)
- Bias or Insufficient Sample Size? Improving Reliable Estimation of Algorithmic Bias for Minority Groups
Jaeyoon Choi, Shamya Karumbaiah, Jeff Matayoshi
ACM Learning Analytics and Knowledge (ACM LAK)
Toward Teacher-Centered AI Design: Exploring the Role of Pedagogical Values and Contextual Factors in K-12 Teachers’ Perceptions of Responsible AI
Yaxuan Yin, Shamya Karumbaiah, Ju Lim, Schwartz, S., Acquaye, S.
Computer Supported Collaborative Learning (CSCL)
- AIBAT: AI Behavior Analysis Tool for Teacher-Driven Contextual Evaluation of Language Models in Education
Shamya Karumbaiah, Yaxuan Yin, Aayush Bharadwaj
26th International Conference on Artificial Intelligence in Education (AIED)
Responsible AI in Education: Understanding Teachers’ Priorities and Contextual Challenges
Yaxuan Yin, Shamya Karumbaiah, Acquaye, S.
ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT)
Prompt vs. Supervise: A Case of Using Language Models to Assess Students’ Science Explanations
Shamya Karumbaiah, Ananya Ganesh, Yang, S., Tseng, N.
In European Conference on Technology Enhanced Learning (ECTEL)
A Multiple Case Study of Translanguaging and Systemic Functional Linguistics in K-12 Science Education
Kaycie Barron, Shamya Karumbaiah, Cynthia Baeza, Mariana Castro, Diego Roman
European Science and Education Research Association (ESERA)
- Identifying Biases in Large Language Model Assessment of Linguistically Diverse Texts
Lionel Hsien Meng, Shamya Karumbaiah, Vivek Saravanan, Daniel Bolt
Artificial Intelligence in Measurement and Education Conference (AIME-Con)
- Evaluating Language Models Using Linguistic Variations
Kaycie Barron, Nora Tseng, Shamya Karumbaiah, Cynthia Baeza
International Conference of the Learning Sciences (ICLS)
2024
Leveraging Multimodal Classroom Data for Teacher Reflection: Teachers’ Preferences, Practices, and Privacy Considerations
Kexin Bella Yang, Conrad Borchers, Ann-Christin Falhs, Vanessa Echeverria, Shamya Karumbaiah, Nikol Rummel, Vincent Aleven
In European Conference on Technology Enhanced Learning (ECTEL)
- Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis
Conrad Borchers, Yeyu Wang, Shamya Karumbaiah, Muhammad Ashiq, David Williamson Shaffer, Vincent Aleven
14th ACM Conference on Learning Analytics and Knowledge (ACM LAK)
Evaluating Behaviors of General Purpose Language Models in a Pedagogical Context
Shamya Karumbaiah, Ananya Ganesh, Aayush Bharadwaj, Lucas Anderson
25th International Conference on Artificial Intelligence in Education (AIED)
- Analyzing Transitions in Sequential Data with Marginal Models
Jeff Matayoshi, Shamya Karumbaiah
Journal of Educational Data Mining (JEDM)
- Gaming the System Mediates the Relationship Between Gender and Learning Outcomes in a Digital Learning Game
Ryan S Baker, J Elizabeth Richey, Jiayi Zhang, Shamya Karumbaiah, Juan Miguel Andres-Bray, Huy Anh Nguyen, Juliana Maria Alexandra L Andres, Bruce M McLaren
Instructional Science
- Evaluating Behaviors of General Purpose Language Models in a Pedagogical Context
Shamya Karumbaiah, Ananya Ganesh, Aayush Bharadwaj, Lucas Anderson
25th International Conference on Artificial Intelligence in Education (AIED)
2023
- Multimodal Analytics for Collaborative Teacher Reflection of Human-AI Hybrid Teaching: Design Opportunities and Constraints
Shamya Karumbaiah, Pinyang Liu, Alisa Maksimova, Lea De Vylder, Nikol Rummel, Vincent Aleven
Proceedings of the 18th European Conference on Technology Enhanced Learning (ECTEL)
- Optimizing Parameters for Accurate Position Data Mining in Diverse Classrooms Layouts
Tianze Shou, Conrad Borchers, Shamya Karumbaiah, Vincent Aleven
Proceedings of the 15th International Conference on Educational Data Mining (EDM)
- Teacher Noticing and Student Learning in Human-AI Partnered Classrooms: A Multimodal Analysis
Shamya Karumbaiah, Conrad Borchers, Ann-Christin Falhs, Kenneth Holstein, Nikol Rummel, Vincent Aleven
Proceedings of the 18th International Conference of the Learning Sciences (ICLS)
- A Spatiotemporal Analysis of Teacher Practices in Supporting Student Learning and Engagement in an AI-enabled Classroom
Shamya Karumbaiah, Conrad Borchers, Tianze Shou, Ann-Christin Falhs, Pinyang Liu, Tomohiro Nagashima, Nikol Rummel, Vincent Aleven
Proceedings of the 24th International Conference on Artificial Intelligence in Education (AIED)
- Using Demographic Data as Predictor Variables: a Questionable Choice
Ryan S Baker, Lief Esbenshade, Jonathan Vitale, Shamya Karumbaiah
Journal of Educational Data Mining (JEDM)
2022
Upstream Biases in Adaptive Learning Systems
Graduate Summer School on Algorithmic Fairness
- Quick red fox: An App Supporting a New Paradigm in Qualitative Research on AIED for STEM
Stephen Hutt, Ryan S Baker, Jaclyn Ocumpaugh, Anabil Munshi, JMAL Andres, Shamya Karumbaiah, Stefan Slater, Gautam Biswas, Luc Paquette, Nigel Bosch, Martin van Velsen
In Artificial Intelligence in STEM Education: The Paradigmatic Shifts in Research, Education and Technology
- Using Neural Network-Based Knowledge Tracing for a Learning System with Unreliable Skill Tags
Shamya Karumbaiah, Jiayi Zhang, Ryan S Baker, Richard Scruggs, Whitney Cade, Margaret Clements, Shuqiong Lin
Proceedings of the 14th International Conference on Educational Data Mining (EDM)
- How does Students’ Affect in Virtual Learning Relate to Their Outcomes? A Systematic Review Challenging the Positive-Negative Dichotomy
Shamya Karumbaiah, Ryan Baker, Yan Tao, Ziyang Liu
Proceedings of the 12th ACM International Learning Analytics and Knowledge Conference (ACM LAK)
- Analyzing Transitions in Sequential Data with Marginal Models
Jeff Matayoshi, Shamya Karumbaiah
Journal of Educational Data Mining (JEDM)
- Improving Phishing Reporting Using Security Gamification
Matthew L Jensen, Ryan T Wright, Alexandra Durcikova, Shamya Karumbaiah
Journal of Management Information Systems (JMIS)
2021
Optimizing philosophies for predictive models in learning analytics
Shane Hutt, Shamya Karumbaiah, Joshua Ocumpaugh
Proceedings of the Workshop on Towards a Philosophical Framework for Learning Analytics at the 11th International Learning Analytics and Knowledge Conference (LAK)
Decoloniality as a Lens of Ethical Foresight for Learning Analytics
Shamya Karumbaiah, Jillian Brooks
Proceedings of the Workshop on Towards a Philosophical Framework for Learning Analytics at the 11th International Learning Analytics and Knowledge Conference (LAK)
- Using Marginal Models to Adjust for Statistical Bias in the Analysis of State Transitions
Jeff Matayoshi, Shamya Karumbaiah
Proceedings of the 11th International Learning Analytics and Knowledge Conference (LAK)
- Using Past Data to Warm Start Active Machine Learning: Does Context Matter?
Shamya Karumbaiah, Andrew Lan, Sachit Nagpal, Ryan S Baker, Anthony Botelho, Neil Heffernan
Proceedings of the 11th ACM International Learning Analytics and Knowledge Conference (ACM LAK)
- How Colonial Continuities Underlie Algorithmic Injustices in Education
Shamya Karumbaiah, Jamiella Brooks
Proceedings of the IEEE Research in Equity and Sustained Participation in Engineering, Computing, and Technology (IEEE RESPECT)
- Investigating the Validity of Methods Used to Adjust for Multiple Comparisons in Educational Data Mining
Jeff Matayoshi, Shamya Karumbaiah
Proceedings of the 13th International Conference on Educational Data Mining (EDM)
- Four paradigms in learning analytics: Why paradigm convergence matters
Ryan S Baker, Dragan Gašević, Shamya Karumbaiah
Computers and Education: Artificial Intelligence
- Context Matters: Differing Implications of Motivation and Help-Seeking in Educational Technology
Shamya Karumbaiah, Jaclyn Ocumpaugh, Ryan S Baker
International Journal of Artificial Intelligence in Education (IJAIED)
- A Re-Analysis and Synthesis of Data on Affect Dynamics in Learning
Shamya Karumbaiah, Ryan S Baker, Jaclyn Ocumpaugh, Alexandra Andres
IEEE Transactions on Affective Computing (IEEE TAC)
2020
- Accurate and Interpretable Sensor-free Affect Detectors via Monotonic Neural Networks
Andrew S Lan, Anthony Botelho, Shamya Karumbaiah, Ryan S Baker, Neil Heffernan
Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK)
- Relationships Between Math Performance and Human Judgments of Motivational Constructs in an Online Math Tutoring System
Rurik Tywoniw, Scott A Crossley, Jaclyn Ocumpaugh, Shamya Karumbaiah, Ryan Baker
Proceedings of the 21st International Conference on Artificial Intelligence in Education (AIED)
- Studying Affect Dynamics using Epistemic Networks
Shamya Karumbaiah, Ryan S Baker
Proceedings of the 2nd International Conference on Quantitative Ethnography (ICQE)
- Affective Sequences and Student Actions Within Reasoning Mind
Jaclyn Ocumpaugh, Ryan S Baker, Shamya Karumbaiah, Scott A Crossley, Matthew Labrum
Proceedings of the 21st International Conference on Artificial Intelligence in Education (AIED)
- Predicting Math Identity through Language and Click-stream Patterns in a Blended Learning Mathematics Program for Elementary Students
Scott A Crossley, Shamya Karumbaiah, Jaclyn Ocumpaugh, Matthew J Labrum, Ryan S Baker
Journal of Learning Analytics (JLA)
- Adjusting the L Statistic when Self-Transitions are Excluded in Affective Dynamics
Jeff Matayoshi, Shamya Karumbaiah
Journal of Educational Data Mining (JEDM)
2019
- Using Participatory Design to Facilitate In-service Teacher Learning in Computational Thinking
Shamya Karumbaiah, Sugat Dabholkar, Jooeun Shim, Susan Yoon, Betty Chandy, Andy Ye
Proceedings of the 13th International Conference on Computer Supported Collaborative Learning (CSCL)
- Temporally Rich Features Capture Variable Performance Associated with Elementary Students’ Lower Math Self-concept
Shamya Karumbaiah, Joshua Ocumpaugh, Michelle Labrum, Ryan S Baker
Proceedings of the Workshop on Online Learning and Social-Emotional Learning at the 9th International Conference on Learning Analytics and Knowledge (LAK)
- Affect Sequences and Learning in Betty’s Brain
Juliana Ma Alexandra, L Andres, Jaclyn Ocumpaugh, Ryan S Baker, Stefan Slater, Luc Paquette, Yang Jiang, Shamya Karumbaiah, Nigel Bosch, Anabil Munshi, Allison Moore, Gautam Biswas
Proceedings of the 9th ACM International Conference on Learning Analytics and Knowledge (ACM LAK)
- Using Reinforcement Learning to Optimize the Policies of an Intelligent Tutoring System for Interpersonal Skills Training
Kallirroi Georgila, Mark G Core, Benjamin D Nye, Shamya Karumbaiah, Daniel Auerbach, Maya Ram
Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
- Predicting Math Success in an Online Tutoring System Using Language Data and Click-stream Variables: A longitudinal analysis
Scott A Crossley, Shamya Karumbaiah, Joshua Ocumpaugh, Michelle Labrum, Ryan S Baker
Proceedings of the 2nd Conference on Language, Data and Knowledge (LDK)
- The Influence of School Demographics on the Relationship Between Students’ Help-Seeking Behavior and Performance and Motivational Measures
Shamya Karumbaiah, Joshua Ocumpaugh, Ryan S Baker
Proceedings of the 11th International Conference on Educational Data Mining (EDM)
- The Case of Self-Transitions in Affective Dynamics
Shamya Karumbaiah, Ryan S Baker, Jaclyn Ocumpaugh
Proceedings of the 20th International Conference on Artificial Intelligence in Education (AIED)
- Using Epistemic Networks with Automated Codes to Understand Why Players Quit Levels in a Learning Game
Shamya Karumbaiah, Ryan S Baker, Amanda Barany, Valerie Shute
Proceedings of the 1st International Conference on Quantitative Ethnography (ICQE)
2018
- Is Student Frustration in Learning Games More Associated with Game Mechanics or Conceptual Understanding?
Shamya Karumbaiah, Seyedahmad Rahimi, Ryan Baker, Valerie Shute, Sidney Dmello
Proceedings of the 13th International Conference of the Learning Sciences (ICLS)
- The Implications of a Subtle Difference in the Calculation of Affect Dynamics
Shamya Karumbaiah, Juliana Ma Alexandra L Andres, Anthony F Botelho, Ryan S Baker, Jaclyn Ocumpaugh
Proceedings of the 26th International Conference on Computers in Education (ICCE)
- Engaging with the Scenario: Affect and Facial Patterns from a Scenario-Based Intelligent Tutoring System
Benjamin D. Nye, Shamya Karumbaiah, S. Tugba Tokel, Mark G. Core, Giota Stratou, Daniel Auerbach, Kallirroi Georgila
Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED)
- Predicting Quitting in Students Playing a Learning Game
Shamya Karumbaiah, Ryan S Baker, Valerie Shute
Proceedings of the 11th International Conference on Educational Data Mining (EDM)
2017
- Analyzing Learner Affect in a Scenario-Based Intelligent Tutoring System
Benjamin Nye, Shamya Karumbaiah, S Tugba Tokel, Mark G Core, Giota Stratou, Daniel Auerbach, Kallirroi Georgila
Proceedings of the 18th International Conference on Artificial Intelligence in Education (AIED)
- Addressing Student Behavior and Affect with Empathy and Growth Mindset
Shamya Karumbaiah, Rafael Lizarralde, Danielle Allessio, Beverly Woolf, Ivon Arroyo, Naomi Wixon
Proceedings of the 10th International Conference on Educational Data Mining (EDM)
2016
- Phishing training: A Preliminary Look at the Effects of Different Types of Training
Shamya Karumbaiah, Ryan T Wright, Alexandra Durcikova, Matthew L Jensen
Proceedings of the 11th Pre-ICIS Workshop on Information Security and Privacy (WISP)