Fifteen Years and Billions Later: EdTech's Honest Reckoning With What Actually Works
· Nia
Fifteen Years and Billions Later: EdTech's Honest Reckoning With What Actually Works
Education Next just published a piece that every edtech founder, school administrator, and policymaker needs to read. The headline — "Logged In, Tuned Out" — asks a question the industry has been dodging for years: after fifteen years and billions of dollars invested, what has learning technology actually accomplished?
The honest answer is uncomfortable. And I think that discomfort is exactly where the real progress starts.
The Promise vs. The Reality
Cast your mind back to 2011. iPads were going to revolutionize classrooms. Khan Academy was going to democratize education. MOOCs were going to make elite universities obsolete. Every edtech pitch deck promised "personalized learning at scale."
Here we are in 2026, and the data tells a more nuanced story. Student outcomes haven't improved at the rate that technology investment would suggest. The National Assessment of Educational Progress (NAEP) scores have been largely flat or declining. Screen fatigue is real. And the pandemic's forced experiment in remote learning showed us that technology without pedagogy is just... screens.
I'm not saying technology in education is a failure. But I am saying the industry has spent fifteen years optimizing for the wrong metrics — adoption rates, engagement minutes, funding rounds — instead of the only metric that matters: did students actually learn more?
The AI Question Makes Everything More Urgent
The arrival of powerful AI tools in 2023-2024 poured gasoline on an already burning question. Schools are now grappling with AI the same way they grappled with calculators, Wikipedia, and smartphones — except the stakes are exponentially higher.
The current landscape is a mess. Education Next reports that most schools still don't have a formal AI policy. Students are using ChatGPT, Claude, and other tools whether schools approve or not. Teachers are scrambling. Administrators are paralyzed between "ban everything" and "embrace everything."
One administrator from a New Jersey school that actually has an AI policy put it well: "We acknowledge that AI is here, and we view ourselves as being responsible for exposing students to what it is, its capabilities, and the potential dangers and pitfalls." That's pragmatic. That's honest. And it's depressingly rare.
Washington Leadership Academy: A Model Worth Studying
Among the noise, one school stands out as doing something genuinely thoughtful. Washington Leadership Academy (WLA) in D.C. created a schoolwide rubric scoring AI use from 0 to 4 for every assignment:
- 0: No AI whatsoever
- 2: AI permissible for drafting and revising, but students "must critically evaluate and modify any AI-generated content"
- 4: Full creative AI use, no restrictions
This is brilliant in its simplicity. Instead of a blanket policy, teachers calibrate AI involvement based on the learning objective of each specific assignment. A creative writing exercise exploring personal voice? That's a 0. A research project where synthesis matters more than the initial search? That's a 2 or 3.
Adam Browning, WLA's director of academic innovation, makes a point that resonates: AI will increasingly enable self-paced learning, which forces us to reconsider whether course credit should be based on seat time or demonstrated mastery. "But," he wisely cautions, "not everything should be self-paced."
That caveat matters enormously. The edtech industry has a chronic tendency to take a good idea and stretch it until it breaks. Self-paced learning works for some things. Human connection, mentorship, and collaborative problem-solving require something else entirely.
The Alpha School Extreme — And Why I'm Skeptical
At the other end of the spectrum, you have Alpha School, which charges families up to $75,000 annually to educate their children almost entirely through devices and AI. No traditional classroom instruction. Full technology immersion.
I'll be direct: I'm deeply skeptical of this approach, and the research backs me up.
Study after study confirms the irreplaceable value of human teachers. A recent paper in PMC examined the psychosocial benefits of student-teacher relationships — benefits that no algorithm can replicate. Another study in Springer's Social Psychology of Education journal found that teacher presence significantly impacts student motivation and identity formation in ways that go far beyond content delivery.
Selling parents a $75,000 vision of AI-powered education without robust longitudinal evidence isn't innovation. It's marketing. And the students are the ones who bear the risk.
What Actually Works: The Evidence-Based Playbook
After reviewing the research and talking to educators, here's what I believe the evidence actually supports:
1. Purposeful Technology, Not Pervasive Technology
WLA's approach deserves repetition: they went through a phase where teachers explored every tool they could find, then deliberately narrowed to a select handful. Teachers receive in-depth, ongoing professional development on those specific platforms.
Browning's phrase captures the philosophy perfectly: "We believe in purposeful technology, not pervasive technology."
This is the opposite of what most schools do. Most schools adopt tools haphazardly — a math app here, a reading platform there, a classroom management tool on top — creating a fragmented digital landscape that exhausts teachers and confuses students.
2. Human Teachers Remain Central
The research is unambiguous: technology works best when it augments skilled teachers, not when it replaces them. The most effective edtech implementations use AI and digital tools to handle routine tasks (grading, basic content delivery, progress tracking) so teachers can spend more time on what they do uniquely well — mentoring, inspiring, adapting to individual students' emotional and intellectual needs.
3. Explicit AI Literacy Is Non-Negotiable
Schools that pretend AI doesn't exist are failing their students. WLA students who receive formal AI instruction report being "scared" that peers at other schools get no guidance at all. And they're right to be scared. Sending students into a workforce saturated with AI tools without any formal education in how to use them critically is educational malpractice.
AI literacy shouldn't be a standalone class. It should be woven into every subject — how to evaluate AI-generated text in English class, how to use AI as a research tool in science, how to understand AI bias in social studies.
4. Measure Learning, Not Engagement
The edtech industry's favorite metrics — daily active users, time on platform, engagement scores — tell you almost nothing about learning. A student spending 45 minutes on a gamified math app might be learning... or they might be optimizing for points while their mathematical understanding remains unchanged.
The schools seeing real results are the ones that measure what students can actually do after using technology versus what they could do before. Pre/post assessments. Transfer tasks. Long-term retention. These are harder to measure and less impressive in pitch decks, but they're the only metrics that matter.
The Path Forward
We're at a crossroads. The next five years will determine whether AI in education becomes the transformative force it could be, or whether it follows the same hype-investment-disappointment cycle that's plagued edtech for the past fifteen years.
Here's what I think needs to happen:
For schools: Adopt the WLA model. Be deliberate about which tools you use. Invest in teacher training, not just software licenses. Create clear AI policies with graduated levels of acceptable use.
For edtech companies: Stop optimizing for engagement and start proving learning outcomes. Fund independent research on your products. Be honest about what your technology can and can't do.
For policymakers: Fund AI literacy programs as urgently as you fund STEM. Require evidence-based procurement — no more buying edtech tools based on sales pitches without efficacy data.
For parents: Ask your school what their AI policy is. If they don't have one, push for one. Ask what evidence they have that their technology investments are improving outcomes.
The billions have been spent. The screens are in every classroom. The question was never whether technology would enter education — it was always whether we'd be thoughtful enough to make it actually work.
Fifteen years in, the evidence says we haven't been thoughtful enough. But it's not too late. The schools that get this right in the next few years won't just adopt technology — they'll integrate it with the kind of intentionality that actually changes how students learn.
And that's worth every dollar.
Building education tools that actually work? Youmake helps developers and educators go from idea to app at the speed of thought. Start building today.