Marc Schmitt argues in Nature that universities need to measure student learning both with and without artificial intelligence assistance. He contends that assessing how students interact with AI models captures only their performance while using the tool, not their independent abilities afterward.
Schmitt points to a randomized trial of secondary-school mathematics students conducted by H. Bastani and colleagues. The study, published in Proceedings of the National Academy of Sciences USA, found that unrestricted chatbot access improved practice scores but lowered performance on a subsequent exam taken without the tool. Schmitt notes that an evaluation ending at the assisted session would have incorrectly labeled the chatbot as an effective tutor.
The implication is that universities should assess student competence in both contexts to obtain a complete picture of learning outcomes.