Marc Schmitt makes a compelling case in Nature that universities should teach students to argue with AI models rather than passively accept their outputs. His argument has merit—engaging critically with artificial intelligence is undoubtedly a valuable skill for the modern world. But Schmitt's framework for measuring success is dangerously incomplete. Assessing student performance only while AI assistance is available tells us nothing about what students can actually do when that digital partner is no longer in the room.
The evidence for this gap is stark and recent. A randomized trial of secondary-school mathematics students, reported by H. Bastani and colleagues in the Proceedings of the National Academy of Sciences USA (volume 122, article e2422633122; 2025), found that unrestricted chatbot access raised practice scores—suggesting immediate success. Yet when these same students took an exam without AI assistance, their performance declined. By Schmitt's measurement framework, the chatbot would be celebrated as an effective tutor. The reality is that it may have created an illusion of learning.
This distinction matters profoundly for higher education. Universities exist to develop students' independent capabilities, not to create dependence on technological scaffolding. If we measure only assisted performance, we risk certifying graduates who appear competent in controlled environments but flounder when forced to rely on their own knowledge and reasoning. That's not education; it's performance theater.
The solution is straightforward but administratively demanding: universities must implement dual assessment regimes. Students should absolutely learn to collaborate productively with AI—Schmitt is right about that. But their mastery must be measured in two conditions: with AI assistance and without it. Only when students demonstrate competence in both scenarios can we confidently say they have genuinely learned. The gap between the two measurements tells us something crucial about the quality and depth of their understanding.
This isn't an argument against using AI in education. It's an argument for honest measurement. If AI-assisted learning produces a 20-point gap between performance with and without the tool, universities should know that, acknowledge it, and adjust their pedagogy accordingly. Perhaps that gap is acceptable for certain skills; perhaps it signals that students need different instruction. But we cannot make that judgment if we simply stop assessment at the assisted session, as the current trend encourages.
Universities must radically rethink not just how they teach with AI, but how they measure what students have learned. Anything less is a disservice to students, employers, and the institutions themselves.