The following transcript has been edited for readability. Timestamps have been removed and minor transcription errors corrected. Speaker comments and context have been preserved.

Opening Remarks

Rich Portelance

So, welcome, everybody, to the AI Enhanced Standards Alignment webinar. We're dialing in precision at scale. We're going to give attendees a few minutes to come in. So let's say two minutes or so, and then we're going to kick into the webinar, so hang tight, we're going to be on mute, and we're just letting people flood in. It's going to be an exciting conversation, so thank you very much for joining us, and we'll get started in just a minute.

Okay, we're going to get going here. And again, my name is Rich Portelance. I am the host of the EdGate Powers webinar series. Today, we are talking about AI-enhanced standards alignments and how do we dial in precision at scale?

With me, as always, we have some great panelists, and I want to let everybody know, if you're here, and if you know someone who you think would be a great panelist, please reach out, let us know, because we're always looking for people. And today is an exceptional group. 

We have Amber Berry, Vice President of AI and Strategy at the Middle States Association. Brett Hodus, the CEO and co-founder of Book Buddy Media, and Jerry Ashcraft, the Director of Technology here at EdGate. And I will let all three of them provide a little introduction in just a second.

Today's conversation focuses on a question many educational leaders, publishers, and edtech organizations are wrestling with: how do we leverage AI to improve standards alignment without sacrificing quality, trust, or educational outcomes? We're going to explore the intersection of leadership, technology, and content development through three unique perspectives.

Before we get started, a couple of housekeeping items. All the attendees will be muted. Please use the Q&A button at the bottom of your screen if you have a question. We will either answer those in line if we can or at the end of the session. If we don't get your question while on the webinar, we will get to it via email afterwards. And thank you in advance for asking questions. We'll also send out a deck with content and contacts of our panelists after the webinar. So please, you know, watch your email for that in the coming days. Of course, we'll also send out a link to the webinar so you can share it with your friends.

So without further ado, why don't we start off with Amber Berry? Could you give us an introduction?

Amber Berry

Sure, again, I'm Vice President of AI and Strategy at Middle States, and I'm really looking forward to this conversation today, so I appreciate the invitation. I worked for 15 years full-time in schools. I was a middle school French teacher, world language department chair, and principal. A few years ago, I transitioned to working for the Middle States Association in AI, and I am the co-founder of RAIL (Responsible AI and Learning). And this is an endorsement series that we use with schools globally. We've worked with 100 schools so far, and the two endorsements are AI literacy, safety, and ethics, followed by the essential learning experience with AI. 

And I come to this conversation not forgetting my roots in the classroom and what it means to have good, safe practices, to have standards for our students to be held accountable for that, and also understanding fully that the tension right now is that everything is so new. It's hard to set a standard in a new arena. This is not like technology from before. It's not like things we've studied before. I think a lot of us are building the ship as we fly it, and we need to do right by children, so that tension brings me to this conversation with excitement.

Rich Portelance

Thank you so much, Amber. 

And Brett, little intro?

Brett Hodus

Hi there, thanks for having me. 

I am the CEO and co-founder of Book Buddy Media. We are a content developer for K-12 publishers. I've been doing this for almost 25, 30 years, so a long time. I've seen a lot of changes in the market. We have about 100 publishing clients, and we're under a lot of pressure to find ways to integrate AI into workflows to reduce costs for publishers without negatively impacting the student experience. So, that's kind of our attention is trying to figure out how to keep publishers responsible and be competitive at the same time. But we don’t want to have any of this stuff impact education or impact the quality of instruction or alignment.

Rich Portelance

Excellent. Thank you, Brett. 

And Jeremy?

Jeremy Ashcraft

Hi, I'm Jeremy Ashcraft. I'm the Director of Technology for EdGate. I've been with EdGate for over 25 years now, so I've been with the company all through kind of the evolution of our alignment business, even when we weren't an alignment based company, and I think that makes us uniquely positioned in kind of the transition kind of into an AI-based, you know, economy here. In that, you know, we're kind of uniquely positioned to be able to look at how we use our existing processes and smartly use AI to help enhance that.

Rich Portelance

And as we know, EdGate has been around for about 29 years. So there's a lot of history there and a lot of experience with standards, alignment and kind of that evolution. So we appreciate your passion for this, Jeremy, and your perspective. 

Discussion

Rich Portelance

What I want to do, I'm going to ask each panelist a question, an opening perspective question, and then we break this conversation out into a couple segments. 

So I'm going to start off with Amber. Schools are feeling pressure to adopt AI quickly. What's the biggest mistake leaders make when they begin that journey?

Amber Berry

I think school leaders are making a common mistake, and that's the same one we've made with edtech for, at this point, many decades, and that's they're leading with “How do we integrate AI into what we already do?” And there is a really important opportunity here that at Middle States, our perspective is that it's not about the integration, it's about redefining and reminding ourselves what is powerful learning for students anyway. And starting from there, in what ways might AI help that process, or maybe not? 

And when you're jumping right to tools, you're really at the surface layer of what we're doing, and we feel strongly that it's important to go back to the foundations of a school, the foundations of a community, and that's your mission and your values. So we have modeled RAIL, Responsible AI and Learning, after Stuart Brand's economic model, the Pace Layer Model. So that bottom layer of any society, that culture doesn't change very quickly, and everything should align to that. So in a school, that's your culture, your mission statement, your “Portrait of a Graduate”, and your profile of a teacher. That's where you start, and then you move up to the governance layer, and you think about what are the policies that are aligned with the community that we're trying to create. Then you have infrastructure, and infrastructure is not about just the director of technology. Jeremy's not on an island at EdGate doing this work. It's about everybody's capacity. Maybe he's leading that charge, but it's about building everyone's capacity forward. And then you get to program and practices, which is what does this look like in the classroom? But a lot of folks are rushing to that top piece and, you know, AI could make a billion worksheets an hour, but that's not powerful learning, and that's not what we should take away for our schools. We should improve the system.

Rich Portelance

And just a note on that, Amber, because I had an interview or a conversation with a consultant who is working with Fortune 500 companies. She has 30 odd years of experience, and she was validating a proof of concept of what she's doing going into companies today, and she was asking my perspective. And she said exactly what you just said, what she was trying to validate is, do I go in and talk about the culture? You have to prepare people for this adoption. It's not just about the technology, it's about the people using it. So I appreciate that perspective.

Jeremy, everyone's talking about AI-powered alignment. What separates meaningful alignment from simply generating faster outputs?

Jeremy Ashcraft

Yeah, so, what's important with alignment is, again, like you said, not just getting it done faster, but maintaining the quality and the precision… Making sure that's the word I'm looking for here, making sure that there's kind of repeatability, that's the word I'm looking for, and reusability. 

So, you know, having an AI-driven process is not just about getting more alignments done. You need to be able to have the base process in place so you know that results are repeatable and reusable. You know, in the future, you're not just doing it as a one-off, and then two weeks later, you have to go back and do it again for some other RFP (Request for Proposal) or whatever. You're doing or there's new adoptions, and then we have to go in and do it over again. It's just being smart with your process.

Rich Portelance

No, I appreciate that perspective. Building systems and repeatability is critical. 

Brett, from the publisher's perspective, why has standards alignment become even more important in the age of AI?

Brett Hodus

I think it's a differentiator when you're creating products that are aligned to standards, but also having that nuanced understanding of having a human being on the ground in that district, in that state, to understand the way those standards are being used on the ground, and what's important to teachers. An AI will look at a standard and be very generic, very, very high level, and produce something that is technically aligned, but may not speak to any of that nuance that a human being would say, “Wait, we're really having issues with our writing standards, we need to be more robust here, or more robust there”, or some of these specifics that'll come out of experience and having human beings kind of weigh in. 

The other thing is that as the temptation to create alignment documents with AI comes up and people will do that as an afterthought rather than building from the standard backwards, they'll try to take something and retroactively align it using AI, and create some type of AI gap analysis that's usually really flawed, and it'll put them in another direction so they can check a box off. But it's not aligned in the true sense. It's not aligned to the way students are learning on the ground, the way that things are being taught on the ground, and the way that those values and that culture that Amber was talking about is sort of integrated throughout curriculum and instruction on the ground level. So, AI can do a lot of amazing things, but it's not in the classroom. And you have to have people who are in that classroom to really align on those nuances.

Rich Portelance

I appreciate that, Brett. And it's interesting. I think when a lot of people think about AI, they think about the LLMs, the ChatGPTs and the Claudes. But really, when we talk about customized solutions, we're talking about something that was built with other technologies leveraging AI potentially to do something like alignment, that's inherently different than just going to ChatGPT to say, “I want to align this”. And like you said, those you may get it right once, but there's a lot of flaws that can occur, and it can send you off in the wrong direction.

Brett Hodus

Yeah, and also, there's a genius factor with AI, that it's so smart that it can take content that's not aligned and it can tell you how it is aligned. And we know instructionally it's actually not aligned, but AI can tell that story because it's smarter than a human being a lot of times, and it's also stupider than a human being in a lot of ways. But it can tell that story, and it's actually not a true story in some cases.

Rich Portelance

That's a really good point. I want to get into building the right foundation. 

And Amber, you've said successful AI adoption starts with mission and culture rather than technology. What does that look like in practice?

Amber Berry

So, when a school engages in the RAIL process with us, and even when I'm kind of out on the road and doing keynotes and workshops with folks, when you get back to your roots and the “why” of things, that's when you stay human-centered. When you get away from, you know, “How do I catch a student if they're cheating?” When you get out of that language, you get back into why we are educating these kids to begin with. What is our purpose? You're really able to start to conceptualize what this looks like when you can do it right. And that's the mind, that's just the mindset that we need all educators, all leaders to be in and all communities, frankly, because this also applies to organizations. We just happen to be talking about school.

So what this looks like in practice is something really important when we are uncertain about the future of higher education, the future of jobs, the future of, you know, what do we need us as panelists and adults as skills in the next five, 10, 20 years, but certainly for our kids for the next 40, 50, 60 years. It's so important for us to go back to that “Portrait of a Graduate”. And so what really matters in any school worth its salt is that going back to character matters, critical thinking matters. You know, there are certain things that are evergreen that AI is not going to disrupt. Maybe kids are going to be learning how to manage multiple AI agents someday as opposed to doing certain things that we would have expected ourselves.

But ultimately, when they do that reflection as a community of who are we trying to graduate and what are we preparing them for, even though the future is in some ways ambiguous, that is where the best work happens with the schools that we are interacting with; where they're able to come together, commit to that, and deliver on what they've promised to families and promise to their community.

Rich Portelance

Thank you for that answer. And I agree, you know, those durable skills are so critical, and understanding how to embed those is going to be a real difference maker. 

Jeremy, organizations often view AI as a shortcut, which it can be. Why does precision become even more important as we scale alignment efforts?

Jeremy Ashcraft

Well, as you kind of grow your efforts, alignment becomes more than just a way to sell more content into a school district. You know, it becomes… You know, you're now affecting outcomes within that school with the students. So having proper context and understanding of the standards and the content and how those alignments are made is critical.

Rich Portelance

Okay.

And Brett, when publishers begin developing new products, how early should alignment considerations enter the process? You kind of touched on this earlier

Brett Hodus

Yeah, and I think, you know, obviously in the age of AI, even more so, you know, at the beginning, kind of coming up with what is this structure? Why are we building it? What's the product? What problem is it solving in the market? And I think that really comes back to alignment at the end of the day because a teacher has a problem to solve, whether it's reading or math, or science, or whatever the subject is, they have a set of rubrics and standards that they have to meet. I mean, that's their job, is to educate kids in a specific way. They don't just get to do it any way they want; there's a specific way of doing it. 

And a lot of times, when we build products, we start with engagement, which is a great thing. And, you know, “How are we going to get kids to give effort and try to lean in?” which I think is important, but it has to be within the framework of what is the teacher going to be teaching in the classroom? Because at the end of the day, from a publisher's perspective, we have to educate, we have to sell the product, and a teacher has to buy it, and they're not going to buy it if it doesn't check that box and say, “Yes, we are going to hit all of my standards. This is all aligned, and now we can get into engagement, pedagogy specifically. How am I going to teach it? How much time do I have in a day?” Some of those more nuanced questions, but it has to start with, you know, “I have a mandate, and my mandate is these standards, and I have to meet them”.

Rich Portelance

Yep, and that's not going to change or go away necessarily. So, you know, you got to start there. 

I have a question for all the panelists, so use your little raised hand button at the bottom of the screen for the first person, and we'll kind of start there. Because this is open, anybody can answer it. Is alignment becoming more of a strategic necessity than a compliance exercise?

Jeremy?

Jeremy Ashcraft

Yeah, I mean, you know, it's sort of like my last answer. It's more than just a way to sell content. I mean, if you don't have alignment as a publisher, you know, you're not going to win RFPs, period. I mean, it's essential for, you know, a K-12 based publishing business.

And Brett, when we look at that being a necessity, you deal with publishers, you're dealing with state level compliance, I mean, with 50 states coming in and a lot of rules and regulations, I imagine you have a perspective here.

Brett Hodus

Well, yeah, and I'll just use the California adoption as an example right now. California is having a massive adoption and publishers are all coming to us. And it's interesting the way these conversations start. A lot of times you'd be surprised that the conversation starts with “Who else is competing in this space? Who else are we going to have to go up against in this space? How are they doing it?” But then it always does come back to, “Do we have something that already aligns? Can we retrofit something we've already built and make it align?”

And that's, again, a danger of AI. Ask AI to do that, the answer is always going to be yes. Oh, yeah, this aligns because AI is trying to please you. They'll say, “Oh, this is work. Yeah, this works right here. This works right there”. But it can be a real dangerous, a real dangerous trap. 

And so I think what's happening now is the publishers who are building from scratch, seeing an opportunity like in California to build from scratch, and they'll say, “Okay, these are the standards, this is the alignment, let's get the engagement, and let's build the product going backwards”. Rather than some publishers that we've bumped into have said, “There's all this money in California, we have this product, it sort of works. How do we make it a fit?” 

And I think that's a dangerous thing. But it's also capitalism and that's what happens in educational publishing, and I think with AI, there's more of a danger there, because there's more of this sort of easy path to retrofit things. But I also think publishers that are bringing some of those things to market are starting to see that it doesn't sell, so it becomes a strategic advantage to have an actual aligned product and not a compliant product. When a teacher digs in, says, this is not really aligned. I see that. The strategy of hitting a big RFP or winning at a real scalable level, it really does start with true alignment, where a teacher in that state will look at the product and say, “Yes, a human being who has taught in my state has seen this, and I'm agreeing with their assessment. This does align”.

Rich Portelance

Amber, anything to throw in there, or should I keep going?

Amber Berry

You can keep going.

Rich Portelance

Okay. So in the next segment, we're going to talk a little bit about human expertise versus artificial intelligence. I'm going to start with Jeremy. 

You've said AI should augment processes rather than replace them, which I think, you know, we're starting to understand better. Where do you see AI delivering the greatest value today? We've kind of bashed it a little bit, but it does have value.

Jeremy Ashcraft

Yeah, absolutely. I mean, it does have value. Otherwise, we wouldn't be trying to use it. I think the biggest thing for us is we see AI kind of like, you know, like I've said, we're augmenting our processes, so we're already taking kind of best practices and putting a force multiplier on top of it. It's like we're able to achieve a kind of higher bandwidth with individuals, you know, using AI, identifying what the tedious tasks are? What are the things that you spend a lot of time doing? And how can we automate that process, you know, in a smart way, not just… You know. Go analyze all my emails or anything. You need to have a plan in place for your AI adoptions. 

And I mean, there's a lot of other… A lot of other things like just being able to do deep research, you know it's very valuable just to be able to go out, ask a question and not have to sift through a thousand search results. You know, you can get a nice concise answer. And some of what we're building into our own processes, which works really well, is actually having AI check itself. So, we're using it to augment our QA processes where, okay, we asked you to do a task, now go back and review what you just did, and then also to be able to take direction and have a little bit of an audit trail in its actions. So then, when you do find something that is like, hey, the quality of this isn't very good, you can go back and say, “Okay, where was the disconnect? What do we need to do differently this time? How do I need to change the prompt?”

Rich Portelance

Yeah.

Jeremy Ashcraft

You know, so those are the things it does well.

Rich Portelance

One of the things I know we talk about internally at my company is amplified human intelligence. And I think you just described that. It's great getting you there, but it's a mechanism to enhance what you do or be able to amplify or make it done quicker, but you can't take the human out of the loop in most cases because, you know, really there's inherent flaws, right? But it can make you move faster, or like you said, get more deep research or concise language, which are real advantages, Brett.

Jeremy Ashcraft

Yeah, something else I want to add real quick, sorry, is one of the other nice things it can do for you is just being part of that human review process, especially dealing with the volumes of data that we have to deal with. It can help with the review process, but it can't replace it. It can, however, help you identify the things that need to be reviewed. You know, so if I've got a million records in a data set, having a human review that is not impossible, but it's very tedious and very long. But if you give us some parameters, it can go identify the things. It's like, hey, this doesn't look right. This doesn't look right, this doesn't look right, please review and tell me what to do. It's really good for doing that.

Rich Portelance

So, Brett, during our podcast interview a few weeks back, and I encourage everybody, if you're on this, if you're listening here, we do create podcasts. Each of the panelists graced me with about an hour of time, and those are posted on our YouTube channel, please check them out, because there's even a deeper level of information that was provided. And Brett, you said something along the lines of describing AI as a brilliant assistant that lacks common sense. Where should humans remain firmly in control, in your opinion?

Brett Hodus

At the beginning and at the end. That's kind of the simplest way to say it. 

So when you're deciding in the context of product development, when you're deciding about what you're going to produce, that's what a human being should be thinking about. They should be looking at the market, the need, the why, all of those sort of strategy elements, budget, etc. And then when you get into it, then you can have AI helping, and I'll just give you an anecdotal example. I had AI help us with a product that we were doing for K-2 literacy, and we had put together a proposal for a publisher, and one of the things was going to be flashcards that were going to be part of this product. And when we brought it to the publisher right away, everyone in the room said, “I can't give these flashcards to kindergarteners or first graders. They're going to get ripped and bent. I'll have them for two weeks. You know, this is a bad idea”. And right away we were like, “Yeah. That is a bad idea”. 

And I think that was an example of us maybe leaning a little too much on AI. I thought that was a great idea. And in practice, it is a great idea if flashcards grow on trees or flashcards are made super durable. But that was an example of the AI saying this would be great for students, but not understanding the nuance of a kindergarten classroom or a 1st grade classroom where a paper flashcard is not going to last the whole year, it's not going to work. So that's just an example of AI like, you know, needing a human at the beginning to plan the product, a human at the end to look through everything and make sure that this is sensible and logical and makes sense. In reality, not just in this digital super brain.

Rich Portelance

Makes a lot of sense actually. 

Amber?

Amber Berry

Yeah. Yeah, that's right. They know better, exactly.

Rich Portelance

Brett, can I just say, I love your example because the human in the loop, that kindergarten teacher would say, “Well, that's why we spend so much money on laminators, you know”, because they know better, right? They know that extra step that AI missed.

So, Amber, many school leaders worry about AI literacy among the faculty. How do schools create safe environments for experimentation while maintaining appropriate guardrails?

Amber Berry

Yeah. So the fact is, the kids and the teachers are already experimenting, and school leaders who are paying attention know this. The way you do that is by, of course, getting to the policies. Too many schools, we've encountered still now years in, have yet to really set too many parameters. It might be a couple of bullets added to their acceptable use policy, but they haven't really robustly thought about what are AI guidelines, particularly for the adults. And so, teaching them about protecting data privacy and things like that, teaching them really good, I guess, like outline parameters is important.

A term we use a lot on my team as we started to develop RAIL a few years ago is creating sandboxes. But sandbox is not a global term, and we're a global company. So, for those who don't know what a sandbox is, it's the idea of, like, allowing a child to play in the sand, but there's like this little outline. So you can get messy in the middle, but you're trying not to spill the sand outside and there's a little bit of an outline space and, like, a good play space where you can do things, where you can try and fail, and that's okay to do, and it's encouraged. Schools that do this really well schedule it so they create time for teachers to come together optionally to start, right? It's Tuesday morning, come in and play in that sandbox together, ask questions, bring in a few people who are early adopters to encourage folks, here's something I do, and share and tell stories. 

But truly, we believe in AI literacy training, and that the training doesn't stop as fast as AI is moving. It's not a one and done. It's what does continuous education in this field look like, and how is that different from a full-time classroom teacher to the staff member who welcomes families at the front desk? You know, everyone needs something and like, what does it look like to scale that in an education context?

Rich Portelance

So, you know, what I'm hearing is that all three of you are advocating for human in the loop processes and systems rather than fully automated systems. Are there any decisions that should never be delegated entirely to AI, in your opinion?

Amber?

Amber Berry

I might have an unpopular view on this. Our company tagline is human-led, AI informed, in that order. So, we fully agree that the human has to be in the loop and has to be the leader of the process. At the same time, I believe I'm crediting Eric Hudson with the idea of good questions and better questions when it comes to AI.

The good question everyone's asking my company is, you know, “Should we let the students write using AI?” And the English teachers flinch and say, absolutely not. But there's a big difference between why we ask a seven-year-old to write versus why you ask a junior to write, why you're writing in science class versus why you're writing in English class. And so with all of that nuance, I think my answer really is there could be a really good reason to bring AI in with children as young as seven in the writing process in some way. It's about framing for the educator what is the powerful learning objective in this moment, and does AI serve that or is it going to take away from this moment? And that to me is about the human discretion to make the decision, not yes to AI and writing and no to AI in writing. It can't be a binary.

Rich Portelance

Okay, and Jeremy?

Jeremy Ashcraft

Yeah. So, I think, at least from an alignment perspective, you shouldn't ever just blindly trust what the AI has delivered. There always needs to be at least one layer of human review, mainly because alignment is, as much as we try to layer an objective process on top of it, ultimately alignment's still a subjective area. You know, even human-generated alignments that we do, a lot of times our customers go, “Hmm, I don't know if I agree with that”. You know, and so that's not going to change whether a person did it or AI did it. That's always going to exist.

Rich Portelance

I do want to remind everybody that we do these webinars on a quarterly basis. The Q3 webinar will be coming up in September. We're talking about K-12 adoption advantage and procurement confidence through a scalable system.

You know, if you're looking to win contracts and try to understand in today's day and age, how do you do that in a scalable way, please join us. Again, we'll always have some great panelists, and we hope that you join us as we send out materials. We'll try to include a link to register for that webinar

The third segment that we're going to get into is achieving precision at scale. And Jeremy, EdGate has spent decades, and we talked about 29 years in business, building alignment infrastructure. When you talk about how EdGate does things, you know, and I think if you could give us a little bit of information on that would be great. Why is infrastructure so critical when organizations begin scaling across states, frameworks, and markets?

Jeremy Ashcraft

Yeah, so, kind of what we've spent over the years building out was kind of a system, a taxonomy for identifying the concepts that are addressed within the standards, and creating a system to identify those concepts within content and other types of data as well. Then passing that alignment data onto our clients and helping them build out their own systems for identifying and displaying alignments to their customers and their buyers.

The thing that's so important about that is having the scalability built into it. And when we talk about scalability, in this context, it's having your alignments be applicable across multiple sectors, multiple data sets, and our systems allow you to do that, you know, kind of in one try. You know, and it's very important because there's so much variability across all the different states, all the different NGOs (Non-Governmental Organizations), you know, other countries, other data sets, frameworks, rubrics. There's so much difference in what people are looking for that you need to have the processes and the systems in place to be adaptable across all of those different sectors. 

And even in today's climate, being able to have the flexibility across even… You know, standards might be addressing the same things, but the political agendas in two different states might be different. So, you may have to adapt, you know, your content and your alignments accordingly. And so, you need to have the systems and processes in place in order to do that quickly and efficiently.

Rich Portelance

Well, that makes a lot of sense. I appreciate that. 

Brett, how do publishers balance the demand for speed with the need for engagement? Pedagogical quality and accurate alignment?

Brett Hodus

I mean, the short answer is to plan better. Start things a lot earlier, but that doesn't always happen. I think the biggest thing that publishers are facing, the biggest obstacle right now, and AI is part of this, is there's a little bit of a FOMO (Fear of Missing Out) going on in which the publishers feel like, “Oh, every other publisher is using AI better than me. Every other publisher is integrating it faster than I am. We're behind. We need to get ahead”. And I think that, that fear, which is what it is, it's a fear of missing the market, is actually going to be driving publishers toward all putting out the same product.

And so the way to kind of push against that is to understand that in order to get engagement and in order to have a sound pedagogical process, you have to test, you have to try it out, you have to show a lesson to an educator, and you have to put a lesson in front of students. You have to start small and start small, get people nodding their heads along, and then scale with or without AI. That's a huge thing. 

Now, when it comes to speed and moving quickly and having that same pedagogical consistency, it's not just AI that becomes a temptation. There's also a temptation that if you have 800 lessons to write, ideally, you'd like all those lessons written by one editor. Multiple writers writing them and one editor reviewing all 800 lessons. But when you look at your timeline, you say, “We can't do that”. So, let's have 14 editors each be responsible for a set of lessons. So then, how can you be sure that all your lessons are pedagogically aligned, that they all work the same, they're all written at the same reading level? It's sort of impossible. So, I think speed in publishing is one of the biggest limiters of quality. I see publishers asking us to move too quickly, or publishers themselves moving too quickly, and obviously with AI, there are a lot of times where publishers say, “Well, now that AI's here, we can do it. We're just going to plug in AI here to do this thing that we have never tested, and we don't know if it's capable of doing so, but we think other publishers might be doing it”. There's a danger there if we haven't tested that theory, and at the same time, with these new AI workflows and integrating AI into these kinds of rush workflows is that on the other side of it is a student, who every day and every week is getting better and better at identifying AI. And so our collective consciousness in terms of reading something and being like, oh, that's AI. And what happens to all of us when we think that you watch a cool video of, like, a monkey becoming best friends with a dog, and you're like, “I love this”. Then you're like, “Oh, that's AI. I don't, right?” But if you think it's real, someone's like, “No, that's a real video”. It's like, “Oh, that's amazing. That's so cool”.

The same thing is happening with our lessons, right? Like we, there's something subconsciously, we know it's a computer talking to us. There's a part of our brain that's going to shut off the engagement. And a kid who's six today, and he's 15 in nine years, is going to have such an advanced AI radar that we're gonna have to do really, really well to keep humans in the loop and make sure that the content is engaging, is consistent, and is pedagogically sound. 

So I would say the thing is, publishers need to plan better, slow down, and start earlier. 

Rich Portelance

So, you like the crawl, walk, run scenario rather than in the run, fall, fail scenario.

Brett Hodus

Yeah. Yes, exactly. Or just run. That's what I think we see a lot. Just run.

Rich Portelance

Yeah. 

Amber, many institutions struggle with implementation fatigue. What separates schools that successfully sustain AI initiatives from those that stall?

Amber Berry

Sure. 

So, for schools right now, there are so many forces at play, is how we term it. You know, they're dealing with more than AI, and there's enrollment, and there's mental health crises, and there's just so many things that are happening that can be unique to an institution, or just like everyone's wrestling with the same things, and it is overwhelming. If you say to a group of teachers, “Alright, it's time to get your AI literacy training done”, on top of all the other asks, many schools do not sunset. They don't take anything away, and year over year, they add to the educator's plate. They add to school responsibilities. So, this is a real question that is very, very important for schools to think about. 

One thing that you know, one perspective we have is that AI is a leading, necessary thing that schools need to face, so it needs to be a top priority. At the same time, I think the schools that really get this understand that AI initiatives are not just about teachers in the classroom. It's about the whole institution, it's about your marketing, it's about your admissions department, it's about athletics and the growth of your athletics program, it's about everything. So, if you get AI correct in terms of foundations and mission alignment, then you're really going to be on track to help your entire institution. 

And it's not just about the classroom practices and student outcomes. So, I do believe that, and again, we're talking about schools, but if I just put in the word organization, an organization that really gets it gets that everyone has to be involved. And if we all up our literacy and our, I guess, fluency, our ability to use AI effectively, then that will really help the entire company and not just me and my work on my plate. So, I think that's more holistic. You have to zoom out to see the importance, but I don't want to discount that schools are wrestling with so many things

And so it's not that the schools we've encountered are not, you know, maybe ignoring or stalling their practices on AI because they don't think it's important. It's that there are so many important things they just have yet to prioritize that one.

Rich Portelance

Okay. 

I do have a kind of a cross panel discussion question, but I'm going to bring in a question. And thank you everyone that's typing in questions. They're great and we really appreciate it. There's one in here that kind of, I think, replaces what I was going to ask, and it's so what other processes will allow publishers to accelerate volume, efficacy, and quality.

Anybody?

Jeremy Ashcraft

Sorry, I'm rereading the question here.

Rich Portelance

No worries. I thought Brett was gonna jump all over this.

Brett Hodus

Where do I find that question? I'm trying to see it so I can answer it probably

Rich Portelance

It's in the Q&A box. If you click on that, it should pop up, and it says, so so what other processes will allow publishers to accelerate volume and efficacy slash

Brett Hodus

Okay, yeah. So I'll tell you what we're doing. 

So, what we've done is we've taken every task that we work on at Book Buddy, every individual task from an editorial review, to an illustration review, to a video review, to a script, I mean, every single task, and we've identified over 400 individual tasks that we do. And what we're trying to do is create this giant database of how long does it take a human being to do this, how long does it take AI to do this, what tool should we use with AI, and is this a good idea? And that chart is being filled out every day, and we're learning a lot as we go through testing. And so that's the number one thing I would tell publishers and content creators is test trial and error. Test something out, test it at the subtask level. Get very granular in terms of… You don't write a book with AI. You ask what are the 50 steps of writing the book? Break those down, and then do one step with AI, and then compare that to a human being. 

And I'll give you another, just a quick example. You know, we tested out having AI write a library book about the college, UConn, their sports program, their basketball program. And AI wrote us a draft of that book. But AI did not include more than 20% of the book about the women's team, and the women's team is the best thing about UConn basketball. They've won more national titles than the men's team, and so an editor would have looked at that and said, “Hey, at least half of this book should be about the women, right? Not 20%”. And so that's something that, right away, a human being would have identified, but AI didn't identify it.

And so, checking out things like that on a task-by-task basis, and then other times you'll see, for example, like, there's these, well… I won't get too specific, but there are other tasks that are repetitive, like Jeremy said, that when a human being does once and gives a prompt to an AI, and it's a subtask that's within another test, and a human being is going to check it later, and AI can do it 400 times. That's a great place for AI to be right. So, repetitive tasks, etc. But more than anything, it's a tested workflow. Don't assume the AI is going to be able to do it, and don't assume the AI is not going to be able to do it, because sometimes it's an incredible tool. Sometimes it does things better than you think it's going to do it. 

And so I would say test, trial and error, document the test results, and then you can find small ways to impact. But publishers who think that AI is going to increase their speed by double and reduce their cost by half are not doing it right, right? It should be on the margins. It should help you by 15, 20% in a tested scalable way that's tested and then scaled.

Rich Portelance

Exactly. Exactly. And some say it would probably argue that 85% of that book should be about the women’s team and not even 50%. 

Jeremy?

Jeremy Ashcraft

Yeah, so, just again, something else that's anecdotal, just kind of building on what he said, is like, yeah, I mean, you just have to spend the time testing things out. Like, we're working on something right now internally that, you know, it's a very manual process for a person to do. You know, we have an AI process that can go out and do that for that person instead. And as of right now, most of the results we're getting are fine. Some of them, there's a lot of outliers where we’re, like, “Okay, that's actually not that great”. But the comparison of the agent took 30 minutes to do it versus with a person, it would take 12 hours to do it.

You know, for us right now, it's still worth the time to do that, because then I'm gonna spend an hour cleaning up the mistakes, or I still saved, you know, 10 hours total. So, you know, like, it doesn't have to be perfect, kind of off the rip for it to be useful, I guess I'll put it that way.

Rich Portelance

Fair.

Brett Hodus

And also, the AI can find a way when a person does that first step. Let's say they're writing an outline for something, and the person again, this is what Rich said earlier about making people smarter, right? You're writing an outline, and you get to a moment in the outline where you want to give an example of something, and you go to AI and say, “Give me 10 examples of that”. And then you have a list to choose from and say, “Oh, do I like any of those?” And so you're not just getting an example, you're not saving any time, but you're getting the best example because now you've got 10 to choose from and a human being can pick the best one. The product got better. We didn't necessarily save money or time, but the product got better. So that's another way to use AI. It's not just about cutting costs. It's also about making us smarter, making our products better. Ultimately, we shouldn't have to sacrifice.

Rich Portelance

Cool. I thank you for those additional perspectives. We're trending in a good way. I do want to look ahead a little bit and I want to get back to Amber because I'm curious about what a truly future-ready school looks like. Three to five years from now, what are you seeing in the work that you do?

Amber Berry

A lot of our focus at Middle States is intentionally about helping school leaders enact change. And change is hard. It's hard in an institution that hasn't changed all that much itself in a very long time. So, how do you do that successfully in a community? And AI happens to be one change. But I wouldn't think of a future ready school as the one that has just mastered, harnessed… Whatever we want to call it, AI. It's a school that's ready for the next change because more change is coming in the future of this world, so that's the kind of work that we do. 

We have every school create a change strategy and plan for their institution when it comes to AI, because some schools, they really do have a pretty forward thinking, lots of teachers experimenting. The wave is there, the readiness is there, maybe the infrastructure is there for them to move ahead at an accelerated pace, but some places aren't there. And so, in a world where teacher shortages are real and many schools are really struggling, they'd have to be careful, maybe “go slow to go fast” is the ethos for change for AI for them, learning how to confront change in a strategic way and in a very planful way with that eye toward the future is what we want all schools to be ready for. 

But I guess going back to that “Portrait of a Graduate”, I want us to think about who are the kids that we need to be running this world in 10, 20, 30, 40 years and thinking about what are those durable skills, to use your term, Rich, that we want every child to become an adult and have in order to be ready for whatever is to come. And a lot of that is uncertain right now.

Rich Portelance

I thank you. I think that's a great answer and it makes a lot of sense to prepare the people well for the changes that are coming and we're going to do better.

Jeremy, how do you see standards alignment evolving as AI capabilities continue to mature?

Jeremy Ashcraft

Yeah, so I think alignment's going to evolve past just, “I've got this piece of content or this resource that addresses or teaches something within a standard”. I think we're going to move more towards having some kind of additional metadata about the alignment. It's like, “here's the specific concept, and you can sort of do that now. Here's the specific concepts that are being addressed. Here's why those specific contracts and how those specific contract concepts are being addressed within the content”.

And even moving towards, you know, here's the impact that this can make. You know within the classroom and also identifying not just these kinds of K-12 curricular concepts, but more of the kind of the real world skills. So, that type of skill acquisition if you're addressing this standard, you're not only teaching these concepts, but you're also, you know, helping students acquire these real-world skills at the same time.

Rich Portelance

Now, I think those concepts, underlying concepts and skills that are added to the standards are really critical material that's getting carried along. Do I understand that correctly?

Jeremy Ashcraft

Yes. Yeah.

Rich Portelance

Hmm.

Brett, what trends do you believe publishers and edtech companies should be paying attention to right now?

Brett Hodus

Well, I think there's kind of two parts to it. The first thing I think, you know, to speak to what Amber's saying is we have to have schools that have a way of thinking about AI that's not rejecting it, that's figuring out how to use it as a superpower and how to train kids on how to use it, because we're going to lose trust in kids if we tell them AI is bad and it's not something they should use, and then they see us all using it every day all the time, and they see people in business using it. And so, I think there's a credibility issue with students where we have to give them some AI literacy and have a policy at the school level that is consistent. And, you know, one of the things Amber was talking about was just sort of individual teachers experimenting, which is great, which is a cool thing. But when we don't know the results of those experiments, and some teachers are better than other teachers, kids are going to get sort of a path forward on how to use AI based on who their teacher is, and that shouldn't be the case. It really should be at the school or district, or even state level, where we're getting some policies in place that say, hey, this is how you're allowed to use it. This is how you should use it, and we need some research to be done to show, “How do I use this to make myself a better writer and a better learner, but also to actually learn, right?” And so, and I think that's a skill that we have to teach kids how to do. 

I have two teenagers, so I can tell you personally about that, you know, it's very easy to just write your paper with ChatGPT and not know anything about it, and then edit it afterwards so it doesn't look like ChatGPT. That's what they're getting good at, and that's not really the way we want them to use ChatGPT. That's kind of a scam. And so, we have to, like, teach them the correct way to use it, some of the stuff that we've talked about. So, that's kind of part one of that. 

And then part two is on more of the publisher side of having their own plan of how they're going to use it and what risk they're going to take, what disclosures they're going to give to educators about how they've used AI, and I think that's another thing that we need to talk about at some point as a country in terms of, like, when we create educational content, just like when we sell food in the store, we should tell you what's in it, right? Right? We should let you know, “Hey, this is how AI was used to build this content, and this is how we used real human beings”. And so we do that with everything we do with our publishers, but it's up to them to disclose it to educators. But we make sure to let publishers know, “Hey, we're going to give you an option with AI and an option without for everything, and you get to choose how you want to build your product, we're going to fully disclose it to you. We're advising you to fully disclose it to the students and to the teachers. I don't know if they always do, but I think that would be a net positive. So I think one thing for everyone to pay attention to is, let's get this stuff in the light of day, let's not make it the big bad wolf, and let's not make it the silver bullet that solves everything. Let's have a real conversation about, “Hey, kids, you should be using it. Here's how you should use it. Don't use it like this, use it like that”.

Rich Portelance

Yeah.

Amber Berry

Rich, could I jump in there for both parts, Brett, of what you just shared? That first part is about how our second endorsement for RAIL is all about action research. So, it's teachers documenting the journey of how AI was used and then what are the results of that outcome. 

And then for the accreditation side of my company, we do have a protocol, sustaining excellence is like all about action research to attain sustained over years, and that is the critical piece that I hope all edtech companies get held accountable for what are the gains, the measurable gains with and without AI that you are noticing, but also that schools are doing. 

And I wish we weren't working at Middle States on a school-by-school basis, and we were working at the state, at the national level with these schools around the world that we are interacting with right now. That lack of alignment, that it can be an individual choice, is disheartening to me because I don't like that the experience from one school to another is different, let alone one classroom to another. So I have hope for the future that we get there, Brett.

Rich Portelance

I want to address Doug, who's out there in the ether. I see your comments. I’d love to talk to you about these offline. We're not going to address it because it's a little askew from where we are with this podcast, but it's relevant, I think, going forward, and I certainly agree with a lot of your positions here. I have some anecdotes I would love to share, so that's something we can have some conversations on. 

The final lightning round is here. Because we are getting to the end of our time, I’ll ask each of you to complete the sentence. The organizations that will thrive in the AI era are the ones that blank.

Amber?

Amber Berry

I would say our human-led and AI are informed in that order. I think if we lose sight of that, we're doomed. Truly. And so, I really hope that that can be true. And I know that it's about modeling. 

Brett, when you talked about showing the students ethical consent, you know, sharing with families, these are the tools that we use with your children and why that families in the home are modeling. I'm side-by-side with technology with my children, and so we use AI together on a regular basis and talk about the garbage, the good points, the “how is this useful”, and how is it not, and the ethical boundaries. 

So, I really hope that as a society, we really move forward as a world, we move forward in a way that is human-led, AI-informed.

Rich Portelance

I love it. 

Jeremy?

Jeremy Ashcraft 

Yeah, so I think, you know, to piggyback on something that Amber said, it's like, organizations where kind of everyone's involved in the policy. And you're getting everyone to identify, okay, these are the issues we see, these are the pain points, and these are the things that are hard. And in figuring it out, how do we best automate those processes? How can we automate it? Do we use AI for this? Does it make sense here? Does it not make sense here? And just getting everyone to buy in because it's not going to be effective if you're just foisting and saying, “Hey, everybody, we're going to use AI. Figure it out”.

You know, you have to collectively come up with a plan of attack as to, you know, what's going to be best for the entire organization.

Rich Portelance

Great advice. Have a plan. 

And Brett?

Brett Hodus

Yeah, I would say the organizations that are willing to crawl, walk, run, test, be curious, be optimistic, and be ethical.

Rich Portelance

Excellent. 

Closing Thoughts

Rich Portelance

So, we're going to have a final takeaway question in a second, but thank you, our attendees, for coming today, for asking great questions and keeping engaged. Thank you to our wonderful panel. You provided a lot of, just, really well-informed insights, so we appreciate your time today. Today's discussion reinforced a common theme. AI may accelerate the work, but precision, expertise, and trust remain essential in what we do. 

Amber, what is an action that school leaders should take this summer?

Amber Berry

If you are a school leader who is not quite as versed in this experience yourself in terms of literacy and fluency, start using it and start leveling up your own skills. And for those who are in the work, I mean, it's summer, summer break, right, in the U.S. at least, think about ways to invite people in. What are some optional ways that your staff and you can make some progress forward this summer? Read a book together, try something together, but take advantage of the weeks you have ahead.

Rich Portelance

Excellent. Thank you very much. Jeremy? 

Jeremy Ashcraft

One action that publishers or an edtech provider should take, I think they should take a nice hard look at how they're using their alignments. You know, are we using it as just a sales tool? Or are we using it as something that does effectively, you know, connect the dots between content and standards in the classroom and students

You know, much to like what Amber said, you know. To be able to go in and say, “Hey, a student in your state, if they use our materials, if they move halfway through the school year, you know, to a completely different district, they're still going to learn the same things. They're going to be relatively in the same place”. You know, and just have that assuredness that students aren't going to fall behind just because they move.

Rich Portelance

And Brett. One action content teams should take?

Brett Hodus

This is really specific, breakdown all the things you do into tasks and subtasks, test AI workflows, and document it. And it's not ready for prime time until you've tested it and proven it. So don't sell that to a teacher until you know it works. Try it out first and test it.

Rich Portelance

Thank you, everyone, for joining the EdGate Powers Podcast. Have a great summer.

Take care, everyone.