The AI Who Taught Me

There has been a lot of discussion about a recent article in NY Magazine, titled Everyone Is Cheating Their Way Through College. It contains many anecdotes about the nearly omnipresent use of ChatGPT and other AI tools on college campuses. Even two years ago, a survey found that nearly all students had used ChatGPT in some form, and the use has just expanded. "Generative-AI chatbots — ChatGPT but also Google’s Gemini, Anthropic’s Claude, Microsoft’s Copilot, and others — take their notes during class, devise their study guides and practice tests, summarize novels and textbooks, and brainstorm, outline, and draft their essays."
One of the students hit the nail on the head: “Most assignments in college are not relevant,” he told me. “They’re hackable by AI, and I just had no interest in doing them."
One teacher interviewed claims that "Massive numbers of students are going to emerge from university with degrees, and into the workforce, who are essentially illiterate. ... It’s short-circuiting the learning process, and it’s happening fast.”
In this episode, we'll examine whether this is true: is the use of AI in schools really short-circuiting the learning process? We look at two repositories of incredible research that help us answer that question.
First, we look at the meta-analysis published on May 6 in Nature: The effect of ChatGPT on students' learning performance, learning perception, and higher order thinking. The authors looked at 51 experimental studies from around the world, and they found "that ChatGPT has a large positive impact on improving learning performance and a moderately positive impact on enhancing learning perception and fostering higher-order thinking."
 
* In Taiwan, researchers saw strong learning when they used ChatGPT embedded in a game to help seventh graders learn science concepts.
* In Dubai, a set of eleventh graders were studying electromagnetism in their physics class. Those who used ChatGPT saw larger learning gains.
* In Australia researchers found that AI helped boost scores on student writing - but only when students actually made edits and engaged with the suggestions. When they just copy/pasted the text, their learning was much worse.
 
We also discuss the Stanford GenAI repository which serves as a great resource for finding up to date studies that analyze what's happening with AI in education.
 
See more on my blog:
https://lukeshepard.com/blog/chatgpt-in-schools

In this kickoff episode, host Luke Shepard and his AI co-host Emerald dive into how AI is entering classrooms. We will be looking at how AI is used in classrooms, starting with some recent research out of Harvard, Standford and Penn.
 
Studies
We look at four studies published in the last year:
Pew Research study on use of chatbots in school. About a quarter of US teens have used ChatGPT for schoolwork
A pre-print of the pilot in the Harvard Physics class in which students used a chatbot to help with studying. AI Tutoring Outperforms Active Learning
From University of Pennsylvania, How kids who used ChatGPT did better on practice but worse on the eventual tests. Kids who use ChatGPT as a study assistant do worse on tests
Stanford saw gains from a study of 1,800 students across multiple school districts in the use of a new AI tutor assistant. How AI can improve tutor effectiveness and the paper PDF
 
Further Listening:
 Listen to this recent interview between Jane Rosenzweig and Michael Horn about how AI is affecting writing.
 
Sal Khan's TED talk: How AI Could Save and Not Destroy Education