Student Reasoning and Revision in an AI-enabled Science Conversation-based Assessment

Advances in artificial intelligence (AI) have created new opportunities to design educational assessments that move beyond fixed responses to capture how students reason, make decisions, and revise their thinking. Conversation-based assessments (CBAs), which incorporate conversational AI agents, can elicit richer evidence of students’ scientific reasoning than traditional formats. However, there is limited empirical evidence about when and how conversational agents meaningfully add value in assessment contexts. This project addresses this gap by examining how AI-based conversational agents influence students’ reasoning, decision-making, and revision behaviours within a high-school science CBA focused on storm formation and prediction. By directly comparing agent-supported and non-agent versions of the same task, the research will provide evidence on the impact of conversational AI on students’ interaction trajectories and learning processes.
The project is conducted in partnership with ETS, which supports applied research internships that provide hands-on training in real-world assessment challenges. Findings will inform the responsible design of next-generation AI-enabled assessments that are rigorous, valid, scalable, and better aligned with how students think and solve complex problems.

Faculty Supervisor:

Ying Cui

Student:

Partner:

ETS Canada

Discipline:

Sociology

Sector:

Education; Professional, scientific and technical services

University:

University of Alberta

Program: