Using AI for teaching is one of the hottest topics in education right now. I’ve been curious about how using AI in the classroom stacks up to using the Virginia Teacher Club®. Recently, we ran three tests to determine how ChatGPT compares to the Virginia Teacher Club®. For each test, we assessed the following:
- the quality of the resources produced
- alignment to the standards
- the amount of time taken to produce each resource
Keep reading to find our surprising results!
Major Differences Between the Virginia Teacher Club® and ChatGPT
In case you are unfamiliar, the Virginia Teacher Club® is a one-of-a-kind program that offers upper elementary teachers in Virginia access to a curriculum library full of SOL-aligned resources organized by SOL (Standard of Learning). All you have to do is click on the SOL you are teaching to instantly open all of the lesson plans, worksheets, games, assessments, etc. that you need to teach that SOL.
So, before diving into our test results, let’s review the key differences between the Virginia Teacher Club® and ChatGPT for teachers.
Virginia Teacher Club®:
- All resources are specifically aligned to the Virginia SOLs and any supporting documents provided by the Virginia Department of Education (VDOE), such as the curriculum frameworks and instructional guides.
- All resources are created by experienced teachers who understand what it is like to teach a room full of students with diverse backgrounds and needs.
- Thousands of teachers throughout Virginia have used the resources, and they have received thousands of 5-star reviews.
ChatGPT:
- AI only aligns to what you tell it to align to. So, if you forget to copy and paste a SOL or information from a supporting document, it is not included.
- All resources are created by machines that have no experience working with actual students.
- Using AI for teaching is still a new concept, and we do not have much information about its effectiveness.
Test #1: Lesson Planning
The first test we ran compared ChatGPT lesson plans with those in the Virginia Teacher Club®. For this test, we gave ChatGPT a 5th-grade science SOL and asked it to create a lesson with an introduction activity, a hands-on activity, a written activity, and a quick formative assessment to end the lesson.
Quality of Resources: Initially, ChatGPT produced a long, detailed lesson plan with everything I asked for. The activities were engaging. However, when I asked it to turn the lesson plan into a printable PDF, it removed all the details and provided only a short, half-page summary of the lesson. It was like I had to keep my computer on to view the full lesson, and I couldn’t figure out how to print the detailed lesson plan. Also, ChatGPT had many good ideas but did not provide any of the materials needed to implement them (worksheets, exit tickets, etc.).
The Virginia Teacher Club® lesson plan also had engaging ideas for teaching the SOL. But it clearly listed the instructions for each activity for the teacher and was easy to print. The Virginia Teacher Club® also provided all of the supplemental resources to teach the lesson, including worksheets, experiment observation worksheets, and exit tickets.
Alignment to Standards: Both ChatGPT and the Virginia Teacher Club® did a good job of aligning the lessons to the SOL.
Time: It took me over three minutes to tell ChatGPT everything to incorporate into the lesson and for it to produce the lesson. However, it will still take the teacher more time, since they will have to create all the worksheets and supplemental materials to accompany the lesson.
It took me exactly 23 seconds to log in to the Virginia Teacher Club®, locate the SOL, and download the lesson and supporting materials.
You can see more of our comparison of ChatGPT lessons to Virginia Teacher Club® lessons in the following video.
Test #2: Worksheets
Next, we tested how AI-generated worksheets compare to the worksheets inside the Virginia Teacher Club®. I gave ChatGPT a 4th-grade math SOL related to comparing fractions with like and unlike denominators. I told the AI that I wanted a worksheet with a specific number of questions, both with and without models, and with number lines to compare benchmark fractions.
Quality of Resources: I was pretty disappointed by the ChatGPT worksheet. It created a five-page worksheet, which is not reasonable to print or for students to complete. I also found that the fraction models and number lines were very small and did not look good.
The Virginia Teacher Club® worksheet was perfect. Everything fit onto one page. The fraction models and number lines looked good and were big enough for students to analyze.
Alignment to Standards: While the Virginia Teacher Club® worksheet aligned perfectly to the SOL, I found that the ChatGPT worksheet included questions about ordering fractions. I thought this was strange because ordering fractions was not part of the SOL or the instructions that I provided the AI.
Time: It took me almost five minutes to tell ChatGPT everything I wanted on the worksheet and for the AI to produce it.
It took me exactly 27 seconds to log in to the Virginia Teacher Club®, locate the SOL, and download the worksheet.
You can see a full comparison of the ChatGPT and Virginia Teacher Club® worksheets in the following video.
Test #3: Assessments
Lastly, we tested how AI-generated assessments compare to teacher-made assessments within the Virginia Teacher Club®. I gave ChatGPT a Virginia Studies SOL related to Virginia’s geography and asked it to create a summative assessment with multiple-choice, short-response, true/fale, and map questions. I specifically told it to create maps of Virginia within the assessment that students would need to use to answer some questions and to turn the assessment into a PDF that I could easily print and give to students.
What ChatGPT created was honestly shocking…
Quality of Resources: The ChatGPT assessment was a mess and unusable. For the map questions, it just left blank spaces and said the teacher needed to insert maps. Many of the multiple-choice questions were not even complete sentences. For example, “Mountain regions?” with a list of Virginia regions to choose from. All the questions were also packed tightly together on one page, leaving no room for students to answer.
The Virginia Teacher Club® assessment is well-made, easy for students to read questions and analyze maps, and provides plenty of space for students to answer. The maps of Virginia are already made and embedded into the correct places on the assessment.
Alignment to Standards: Both the ChatGPT and Virginia Teacher Club® assessments were aligned to the Virginia Studies SOL.
Time: It took me four and a half minutes to tell ChatGPT everything I wanted, and for it to produce this unusable assessment.
It took exactly 31 seconds to log in to the Virginia Teacher Club®, locate the SOL, and download the assessment.
If you want to see the horrific ChatGPT assessment, check out the video below.
Conclusion
So, how does ChatGPT stack up to the Virginia Teacher Club®? While AI for teaching can be helpful in some areas, creating high-quality resources is not one of them. I consistently found that the quality and usability of the ChatGPT resources could not begin to touch the quality of resources inside the Virginia Teacher Club®.
Also, AI may be fast, but it’s not as fast as the Virginia Teacher Club®. It consistently took me several minutes longer to produce something of lesser quality using ChatGPT than to find the SOL inside the Virginia Teacher Club® and download the resource.
Interested in learning more about the Virginia Teacher Club®? Go to the Virginia Teacher Club® section of our website to join now or get on the waitlist for when enrollment opens again!
