AI in Schools and Student Data Privacy

Vicki Davis (00:00): An AI site promises, we don’t share your data. But why does an attorney who teaches say don’t trust that? What counts as personally identifiable information that you’d never guess? And when a teacher accidentally types student information into AI, who’s liable? This is Thought Leader Thursday, Episode 1014. And remember, intent doesn’t matter. Once it’s typed, it’s typed.

Voiceover (00:34): This is the 10 Minute Teacher Podcast with your host, Vicki Davis.

Vicki Davis (00:39): There is exactly one person I know who can tell you what AI is doing to your classroom and what it’s doing to your legal liability in the same breath. Dr. Rachelle Dené Poth is an attorney. She holds a Juris Doctor from Duquesne University School of Law. She’s a teacher with a doctorate in instructional technology, author of 10 books, and she’s been working with AI in education back when most people weren’t paying attention yet.

Vicki Davis (01:10): She’s been on the show many times, and, Rachelle, I’m so grateful that you agreed to come back.

Rachelle Dené Poth (01:15): Well, I am so grateful that you invited me to come back on this show. I was thinking how many years ago it was since I was first on. I would bet it was probably eight or nine years, I think.

Vicki Davis (01:27): Since the beginning. We started in 2017, and I think it was either 2017 or 2018.

Rachelle Dené Poth (01:32): I know. I’m a frequent guest and frequent listener, I will say.

Vicki Davis (01:36): Rachelle, you’re an attorney and a teacher. When you look at how schools are using AI right now, what do you see that scares the lawyer in you?

What worries an attorney most about how schools are using AI?

Rachelle Dené Poth (01:46): Yeah, so when I made the shift to, you know, presenting on AI itself, how I was teaching my students, because when I started, there weren’t all the things we have now. About, I guess, three and a half years ago or so, I started to think there are bigger issues than academic integrity, because after ChatGPT, I was getting text messages, LinkedIn, you name all the different platforms, people at my door, should we ban, should we stop, what’s going to happen, kids are going to cheat.

Rachelle Dené Poth (02:11): And I kept saying, like, you’ve been able to cheat since I was in high school. But that was the big issue for a while. But then I started to think about all of the other things that were happening and just personal stories. And presenting on AI and the Law is one of my favorite sessions because I’m a storyteller. And every session that I do, I get another story or my friends will tell me something that they have done or that was done in their school.

Is it safe to put IEP information into ChatGPT?

Rachelle Dené Poth (02:34): And so I have these other stories. And the biggest thing that scares me, and that scares others is when educators or anybody, for that matter, uses a platform, whatever it is, whether it’s a, you know, ChatGPT, Gemini, one of the other large language models or educational-based platform, and they see that there’s something they need to create, like an IEP, for example.

Rachelle Dené Poth (02:55): Well, this is going to save me tons of time. And they just put all the information in, but they don’t think about what information could be personally identifiable. And so that comes back to like when I did my doctoral work, it was on the need for PD specific to AI and any other emerging technology at this point because it’s moving so quickly. But that’s when I started to look at the bigger picture.

Rachelle Dené Poth (03:14): And so what scares me is that educators are not getting enough training to understand what might be considered that PII because that has changed from like back in the day when it was just things like birth date, social security number, address, and so forth. There are a lot of other things out there that you may not consider.

Rachelle Dené Poth (03:33): I think what scares me is that data privacy is usually like the number one answer. Data safety, anything like that is when I say, you know, why do we need to know about this? What comes to mind first? And so many times it’s data privacy.

Vicki Davis (03:45): So, Rachelle, when we talk about data privacy, what are some of the personally identifiable information that people don’t realize is, and we’re just going to call it PII from now on, because that is a term that we all need to know what it is.

What counts as personally identifiable information (PII) for students?

Rachelle Dené Poth (03:58): When I do my sessions, I have like scenarios of things that, you know, could potentially happen. And there was a woman in a session that I had done back in November in Pittsburgh, and she said after the scenario… You could still have enough stars that form a constellation. And I wish I could take credit for that, but I’ve told her story so many times because I gave the setup and I said, well, could they identify the person?

Rachelle Dené Poth (04:22): So things that we think of normally are your name, your address, email address, phone number, social security number, family member names, date of birth. But now we’re moving into looking at things like biometric information and your photo, your voice, anything that can identify you. And then also grades, specific activities, health information for a student. And it can be things that are not even considered to be like direct.

Rachelle Dené Poth (04:48): I have a class and in the class I have one student that lists information about this. And that might be the only student in that school system that has that specific disability potentially. And that could still identify that student. And so it’s not just those initial things that we would think of. It’s a bunch of information. Even when I talk to my students and families, I’ve done some work with families about like what you’re sharing on social media, even sharing somebody’s athletic, academic activities, anything they’re involved in, that is personally identifiable.

Rachelle Dené Poth (05:21): So it’s like you really have to think before you post and it’s becoming more obvious, I guess, just because the different things that are happening, how quickly you can access information. But it can be a little bit scary to think about like all of the ways that you can be identified.

Vicki Davis (05:36): Well, and here’s the other thing. If you’ve got kids using it and then we’re using it and it’s all going into the same mishmash of training data. What’s to say that the unique combination of multiple people putting their own little pieces in there could have come back? And, you know, some people would say, oh, but Rachelle, everything we type in AI in an LLM is private.

Vicki Davis (06:02): Right. It’s just in my account. What’s your answer to that?

Is what I type into ChatGPT private?

Rachelle Dené Poth (06:07): It’s not just in your account. So that’s the thing, because a couple of years ago, I was in L.A., and I think it might have been the first time I did a session on AI and the law, and I thought I had plenty of time to prepare for it. It was mostly administrators, and there were some classroom educators, but predominantly administrators.

Rachelle Dené Poth (06:26): And I had one woman that came in, and she worked with the special education teachers, and she came up to me after, and she said, So I’m looking for something that the teachers I work with can use because they’re creating. And she listed all the things that we’re aware that special education teachers work with. And she said, I found this site. And it says, we do not share, store, this whole list. Share, store, buy, sell, borrow, anything, your information.

Should I trust an AI tool that says “we don’t share your data”?

Rachelle Dené Poth (06:49): And she said, should I trust it? I went, no. And she just gave me this look. She’s like, really? Why? And I said, because they’re saying that they don’t.

What are subprocessors, and why do they matter for student data?

Rachelle Dené Poth (06:59): However, when you look at different platforms, there are usually third parties or subprocessors or something behind the scenes that are creating or making that product function in a certain way. Hypothetically, if it’s a free platform, but also a paid platform, they have to have something to process your payment. So when that person has access, that group has access to your information. Are they doing anything with it? And so I always say, like, look at the policy, because all of these companies, they need to disclose what they’re taking, where it’s going, and they can’t make guarantees. Nothing is, we know, 100% perfect as far as privacy.

Rachelle Dené Poth (07:38): Things can happen, but looking and clicking to see if they have access and what they’re then doing with it. I love reading legalese, too. When I look at different platforms that were used in education, I look to see, do they say, we are using these subprocessors? If you have questions, concerns, it’s like informed consent, basically. We’re not, but they might. And if you have concerns, you should follow up with them.

Rachelle Dené Poth (08:02): Those are like the best case scenarios where you feel a little bit more confident when you’re using something.

Vicki Davis (08:08): Well, and look at all the student information systems that have been hacked and the learning management systems. And right now, ransomware just seems to be tearing through our society. And so the best way not to have your information get out is not to have it online.

What happens when a school has a data breach?

Rachelle Dené Poth (08:24): There is such an increase in cybersecurity attacks. That’s why I started to do cybersecurity as well, because my school had experienced it and they are on the rise. There was one that happened when they first investigated it. They thought that the information that could have been, access went back maybe two to three years. And when they went further into investigating it, they thought that it could have gone back eight school years.

Rachelle Dené Poth (08:47): So things have been put into place like incident response plans. We have to inform all of the potential people who are involved in it.

Vicki Davis (08:53): Who is liable when things go wrong with personally identifiable information or when people are using artificial intelligence and it, quote, goes off the rails? I’ll tell you, I was using a very popular music generation site that I’ve used for two to three years and never had any trouble with it. And then all of a sudden, one of my students generated some profanity with it.

Vicki Davis (09:22): That went against everything I knew about how to set it up and what to do. And I really don’t know what the student did, but I deleted the assignment. I sent everybody off the site and I had my school blocking immediately. So even if you’ve used it, things can be jailbroken, right?

Who is liable if a teacher puts student data into AI?

Rachelle Dené Poth (09:39): They can. And the liability, like one of the scenarios that I use, hypothetically, what happens if a teacher unknowingly puts in PII accidentally into some system to generate whatever? We always hear about intent in law, like there was the intent, there wasn’t the intent. And this is a case where intent doesn’t really matter because the fact is that it was done.

Rachelle Dené Poth (10:01): However, depending on what information, you know, the liability. It could be a variety of people. I mean, one, the educator, because they didn’t know and realize, but then that backs up to the school that should have provided the guidance, the training, the tools that were vetted for it. The school itself is responsible, or the district, of course, depending on, you know, some are individual schools or districts, but they are responsible for student privacy, the education records of students, and the PII that they’re sharing with other outside third-party providers.

Rachelle Dené Poth (10:36): And then the other part of that is the company itself, who is like, hey, we have this product. If they don’t have great security, it’s not up to date, they’re not in compliance with all of the laws that things have to be in compliance with. That’s an issue that can leave them liable. And then, like I said, the employee, it happens sometimes accidentally. I’ve told a story about a friend years ago that did something with creating an IEP using ChatGPT, but they didn’t have the training.

Rachelle Dené Poth (11:06): And so it could be any number of people or it could be all of the above.

Does FERPA apply when teachers use AI tools?

Rachelle Dené Poth (11:11): But when we look at things like FERPA, for example, that’s something that educators need to be mindful of. And schools looking at products and platforms have to be mindful of, okay, is it up to date? Is it in compliance with this? But at the end of the day, and this is what I love doing professional development on, is teaching teachers and students also because they have to learn how to use this as well because if they don’t, they’re going to be putting the PII into these systems.

Should teachers teach students what not to type into AI?

Rachelle Dené Poth (11:38): So I always said back even in the day, should we be on ChatGPT? I’m like, we’re educators. We’re in the best position to help students understand what to put into these systems and what not to because we know they’re going to use it. So we have to understand it ourselves so we can guide them. And sometimes, I mean, I’m not the expert, but being able to guide them and using it in a way that’s safe and ethical, responsible, and modeling that for them is going to make a big difference.

Vicki Davis (12:05): Rachelle Dené Poth, an incredible resource for artificial intelligence, the law. You’re a rock solid classroom teacher, too. Thanks for coming on the show, Rachelle.

Rachelle Dené Poth (12:15): Yeah, thank you. I always enjoy it.

Voiceover (12:17): Thank you for tuning in to the 10 Minute Teacher Podcast. Join us here every weekday and subscribe to the Classroom Matters newsletter. See you later, educator.