A few charts for you

tell me again why only teachers should have access to AI?

I’d like to share two charts/graphs with you. Forgive me when I unintentionally refer to graphic representations of data as graphs - I finished my education before Microsoft changed the language. But it continues to be true that you can put a LOT of data into a picture. And one thing that we are pretty good at is making sense of graphical data. Here they are:

Figure 1 is a comparison of success on well-known tests for GPT3.5 and GPT4.0. The format of the stacked bar graph is a bit odd, but the intent is to show the increase in achievement for the GPT4.0 LLM.

Figure 2 is the well-known bell curve of student achievement, overlaid by achievement increases when a mastery learning approach or a one-to-one tutoring approach is used.

I’ve heard recently that teachers are VERY interested in using AI to develop interactive assignments, create tests/quizzes, assess student writing, save time in planning, and more. But, those same teachers are leery about having students use the same tool because of worries about cheating, taking shortcuts, submitting work that’s not their own, not learning the right (or hard) way, etc. While I understand the uncertainty about how this new technology will affect learning, it seems a bit rich to say “for me but not for thee”. Let’s call it for what it is - a fear that all the lesson and unit plans that we spent so much time creating (perhaps years ago) will not be very useful anymore. Or that students will default to the easy way of doing things, rather than the hard way (the way WE had to learn).

I get it; it’s hard to know what to do with tools like LLMs that have comprehensive and instant access to much of the our collective knowledge. I am old enough to remember the World Book Encyclopedia shelf in the family room that I would use for research projects - one neatly-organized set of books containing the sum of human knowledge. My parent sure knew a good value when it literally came knocking at the front door!

First, what are the charts showing? And second, what does that say about the potential for AI use in education? Figure 1 shows how much improved ChatGPT-4 is over 3.5. The red extension shows how much 4.0 has improved in answering questions from those entrance exams. To the point that AI has conquered yet another closed-system challenge - one in which there is a finite set of information to master. Sam Altman (OpenAI) recently said “GPT-4 is the dumbest model any of you will ever have to use”. What he means is NOT that GPT-4 is a bad product, but rather that given the jump in performance from 3.5 to 4 should give us reason to believe that version 5 will address many of the weaknesses in the GPT-4 model - things like hallucinations (made-up information, outright lying, stuff like “dogs in the NBA” and “glue on pizza”), as well as not understanding context, metaphor, abstraction, and intuition.

While the improvement of GPT-4 on 3.5 is pretty impressive, I think figure 2 should impress educators and parents even more. There is a lot of research and writing on the “2-Sigma Problem” (originally popularized by Benjamin Bloom), but here’s the important takeaway: students do MUCH BETTER when they are given a tutor compared to when they ONLY have whole-group instruction or mastery-learning instruction. How much better? You can see that the lowest end of the right curve is at the middle of the standard bell curve (somewhere between a C and a B). So there is lots of data that says students perform much better when given one-on-one assistance in the teaching-learning process (yes, two standard deviations of improvement is impressive).

Well that’s pretty neat, because one-on-one assistance is something that AI LLMs are really good at. But don’t take it from me; hear it from Sal Khan (of Khan Academy), who writes and talks about the opportunities for using AI tutors in education - including khanmigo, their AI assistant who will help you understand your homework without doing it for you. If you look online, you will see lots of custom GPTs that are designed to act as tutors. One of the required skills in AI right now is knowing how to write an effective prompt for the task you have in mind. I do expect that within the next year or two, the skills of prompt engineering will become less important (as the LLMs get better at interacting with end users (and the current difficulties in Natural Language Processing).

One of my go-to people on AI in education is Ethan Mollick, a business professor at Wharton Business School (UPenn) who writes the “one useful thing” substack. Mollick sees a LOT of potential for using AI in education - from tutoring to developing case studies to playing the “devil’s advocate on your chosen topic”, to helping you brainstorm topics. Of value to teachers is his “prompt library”. Currently there is some hard work required to get the output you need from a LLM; you can’t just ask for a lesson plan, review activity, or test on “topic”, and expect to get something you can use in class. You have to do the hard work of setting the stage for what you want to do, how you want the AI to be involved, and what constraints you want on the input or output.

Take the example below, edited briefly by me, based on Mollick’s work. I wanted to get a bit of tutoring in my re-learning of French: specifically, when to use Avoir and when to use Etre. I know that there is a difference in French, where you say “I am nervous” and “I have hunger”. We don’t see that distinction in English, so I wanted a tutor to help me through that process. And here’s the suggested prompt, which could be adapted for ANY topic in ANY course that a student is struggling in:

You are an AI tutor and your job is to help the user learn about the difference between "être" and "avoir" verbs in French. First, introduce yourself to the user. Your goal is to help the user learn when to use "être" and when to use "avoir" in French sentences. Ask: What do you already know about the difference between "être" and "avoir" verbs? Wait for the student to respond. Do not move on until the student responds. Given this information, help students understand the difference between "être" and "avoir" verbs by providing explanations, examples, and analogies. These should be tailored to the student's prior knowledge. Note: key elements of the topic are understanding the contexts in which "être" (to be) and "avoir" (to have) are used, such as "J'ai froid" (I am cold). Common misconceptions about the topic are when to use each verb since there's no direct equivalent in English. You should guide students in an open-ended way. Do not provide immediate answers or solutions to problems but help students generate their own answers by asking leading questions. Ask students to explain their thinking. For example, ask them to give examples of sentences where "être" is used and where "avoir" is used. If the student is struggling or gets the answer wrong, try giving them additional support or give them a hint. For instance, explain that "être" is often used for states of being or identity (e.g., Je suis fatigué - I am tired), whereas "avoir" is used for possession or certain states (e.g., J'ai faim - I am hungry). If the student improves, then praise them and show excitement. If the student struggles, then be encouraging and give them some ideas to think about. When pushing the student for information, try to end your responses with a question so that the student has to keep generating ideas. Once the student shows an appropriate level of understanding, ask them to explain the concept in their own words (this is the best way to show you know something), or ask them for examples or give them a new problem or situation and ask them to apply the concept. For example, "Can you create a sentence using 'être' to describe a state of being and one using 'avoir' to describe a physical state?" When the student demonstrates that they know the concept, you can move the conversation to a close and tell them you’re here to help if they have further questions. Mention that mastering this concept should enable them to score at least 7/10 on a practice quiz designed to test their understanding of "être" and "avoir." Remember: do not get sidetracked and discuss something else; stick to the learning goal. In some cases, it may be appropriate to model how to solve a problem or create a scenario for students to practice this new skill.

I wrote a similar prompt for reviewing high school algebra - specifically how to convert quadratics from one form to another (depending on what information you want to find - y-intercept, zeros, or vertex. And with a bit of tweaking, I had a ready-made tutor that was helpful, patient, and willing to stick by me for as long as it takes to improve my performance.

I can only imagine the possibilities when science students get to have virtual conversations with famous scientists, when language students co-write a “choose your own adventure” story with an AI, when social studies students brainstorm ideas for addressing real-life problems, and when all students use AI to improve their research, analysis, and writing skills. I’m not suggesting that we leap before we look; however, we shouldn’t be paralyzed by the disruption that each new technology brings. We found out in a bad way what happens when students and corporations with social media; I suggest that we not hand this AI world over to them in the same way. As always, I invite your feedback, questions, and perspective on AI in education.

that’s all for now,

cheers,