In December 2019, NYT published a piece about an EdTech product called Bakpax. It is a 1,500-word, feature-length article that provides neither accuracy, nor balance, nor context.

It is sourced almost entirely from company spokespeople, and the author borrows liberally from Bakpax's PR materials to exaggerate the role of AI. To keep the spotlight on AI, the article downplays the human labor from teachers that keeps the system running—such as developing and digitizing assignments.

Bakpax shut down in May 2022.

This is hardly a surprise: EdTech is an overhyped space. In the last decade, there have been hundreds of EdTech products that claim to "revolutionize" education. Despite billions of dollars in funding, many of them fail. Surprisingly, the article does not provide any context about this history.

See for yourself where the article goes wrong. The contents of the article are reproduced in their entirety on the left, and our annotations are on the right. Link to the original article.

For more details about the pitfalls, read the blog post which introduces this article here.

The Machines Are Learning, and So Are the Students
Artificial intelligence is starting to take over repetitive tasks in classrooms, like grading, and is optimizing coursework and revolutionizing the preparation for college entrance exams.
Jennifer Turner, an algebra teacher at Gloucester County Christian School in Sewell, N.J., uses an artificial intelligence-powered program that is helping her teach.
Jennifer Turner, an algebra teacher at Gloucester County Christian School in Sewell, N.J., uses an artificial intelligence-powered program that is helping her teach.
Jennifer Turner’s algebra classes were once sleepy affairs, and a lot of her students struggled to stay awake. Today, they are active and engaged, thanks to new technologies, including an artificial intelligence-powered program that is helping her teach.
She uses the platform Bakpax that can read students’ handwriting and auto-grade schoolwork , and she assigns lectures for students to watch online while they are at home. Using the platform has provided Mrs. Turner, 41, who teaches at the Gloucester County Christian School in Sewell, N.J., more flexibility in how she teaches, reserving class time for interactive exercises.
“The grades for homework have been much better this year because of Bakpax,” Mrs. Turner said. “Students are excited to be in my room, they’re telling me they love math, and those are things that I don’t normally hear.”
For years, people have tried to re-engineer learning with artificial intelligence, but it was not until the machine-learning revolution of the past seven years that real progress has been made. Slowly, algorithms are making their way into classrooms, taking over repetitive tasks like grading, optimizing coursework to fit individual student needs and revolutionizing the preparation for College Board exams like the SAT. A plethora of online courses and tutorials also have freed teachers from lecturing and allowed them to spend class time working on problem solving with students instead.
While that trend is helping people like Mrs. Turner teach, it has just begun. Researchers are using A.I. to understand how the brain learns and are applying it to systems that they hope will make it easier and more enjoyable for students to study. Machine-learning-powered systems not only track students’ progress, spot weaknesses and deliver content according to their needs, but will soon incorporate humanlike interfaces that students will be able to converse with as they would a teacher.
“Education, I think, is going to be the killer app for deep learning,” said Terrence Sejnowski , who runs the Computational Neurobiology Laboratory at the Salk Institute for Biological Studies in La Jolla, Calif., and also is the president of the Neural Information Processing Systems Foundation , which each year puts on the largest machine-learning conference in the world.
Mrs. Turner takes a picture of a lesson to upload to the software program Bakpax in her classroom.
Mrs. Turner takes a picture of a lesson to upload to the software program Bakpax in her classroom.
It is well established that the best education is delivered one-to-one by an experienced educator. But that is expensive and labor-intensive, and cannot be applied at the scale required to educate large populations. A.I. helps solve that.
The first computer tutoring systems appeared in the 1960s, presenting material in short segments, asking students questions as they moved through the material and providing immediate feedback on answers. Because they were expensive and computers far from ubiquitous, they were largely confined to research institutes.
By the 1970s and 1980s systems began using rule-based artificial intelligence and cognitive theory . These systems led students through each step of a problem, giving hints from expert knowledge bases. But rule-based systems failed because they were not scalable — it was expensive and tedious to program extensive domain expertise.
Since then, most computer teaching systems have been based on decision trees , leading students through a preprogrammed learning path determined by their performance — if they get a question right, they are sent in one direction, and if they get the question wrong, they are sent in another. The system may look like it is adapting to the student, but it is actually just leading the student along a preset path.
But the machine-learning revolution is changing that. Today, learning algorithms uncover patterns in large pools of data about how students have performed on material in the past and optimize teaching strategies accordingly. They adapt to the student’s performance as the student interacts with the system. Bakpax asks teachers to notify parents how their children’s data will be used, and parents can opt out. But Bakpax and other companies say they mask identities and encrypt the data they do collect.
Studies show that these systems can raise student performance well beyond the level of conventional classes and even beyond the level achieved by students who receive instruction from human tutors. A.I. tutors perform better, in part, because a computer is more patient and often more insightful.
One of the first commercial applications of machine learning to teaching was by the company Knewton , founded by Jose Ferreira , a former executive at the private education company Kaplan . Knewton uses a mix of learning algorithms to evaluate students and match material to their needs.

“After a few questions we could very quickly figure out what level you are at and the optimal piece of content for teaching,” Mr. Ferreira said. “The more you worked with the system, the better our profile of you got and the more we could give you better and better content.”

Nonetheless, Knewton ran into financial difficulties and was sold in May to the education publisher Wiley . Mr. Ferreira said the company’s troubles were not because its technology did not work, but because the company had relied heavily on one customer, which dropped Knewton in favor of an in-house system. Mr. Ferreira, 51, left to start Bakpax.
At its core, Bakpax is a computer vision system that converts handwriting to text and interprets what the student meant to say . The system’s auto-grader teaches itself how to score.
“Instead of handing your homework in, you just take a picture of it on your phone, and a few seconds later we can tell you what you got right and what you got wrong,” Mr. Ferreira said. “We can even tell you what the right answer is for the ones you got wrong.”
Mrs. Turner said her students loved the immediacy. The system also gathers data over time that allows teachers to see where a class is having trouble or compare one class’s performance with another. “There’s a lot of power in all this information that, right now, literally is just thrown in the trash every day,” Mr. Ferreira said.
A screen showing feedback on a student’s algebra homework through Bakpax in Mrs. Turner’s classroom. Since using the software, she said, “The grades for homework have been much better.”
A screen showing feedback on a student’s algebra homework through Bakpax in Mrs. Turner’s classroom. Since using the software, she said, “The grades for homework have been much better.”
Not surprisingly, machine-learning solutions are making their way into the test preparation market, a multibillion-dollar global industry . Riiid , a Korean start-up, is using reinforcement learning algorithms — which learn on their own to reach a specified goal — to maximize the probability of a student achieving a target score in a given time constraint.
Riiid claims students can increase their scores by 20 percent or more with just 20 hours of study. It has already incorporated machine-learning algorithms into its program to prepare students for English-language proficiency tests and has introduced test prep programs for the SAT. It expects to enter the United States in 2020.

Still more transformational applications are being developed that could revolutionize education altogether. Acuitus , a Silicon Valley start-up, has drawn on lessons learned over the past 50 years in education — cognitive psychology, social psychology, computer science, linguistics and artificial intelligence — to create a digital tutor that it claims can train experts in months rather than years.

Acuitus’s system was originally funded by the Defense Department’s Defense Advanced Research Projects Agency for training Navy information technology specialists. John Newkirk , the company’s co-founder and chief executive, said Acuitus focused on teaching concepts and understanding.

The company has taught nearly 1,000 students with its course on information technology and is in the prototype stage for a system that will teach algebra. Dr. Newkirk said the underlying A.I. technology was content-agnostic and could be used to teach the full range of STEM subjects.

Dr. Newkirk likens A.I.-powered education today to the Wright brothers’ early exhibition flights — proof that it can be done, but far from what it will be a decade or two from now.
The world will still need schools, classrooms and teachers to motivate students and to teach social skills, teamwork and soft subjects like art, music and sports. The challenge for A.I.-aided learning, some people say, is not the technology, but bureaucratic barriers that protect the status quo.
“There are gatekeepers at every step,” said Dr. Sejnowski, who together with Barbara Oakley , a computer-science engineer at Michigan’s Oakland University, created a massive open online course, or MOOC, called “ Learning How to Learn .”

He said that by using machine-learning systems and the internet, new education technology would bypass the gatekeepers and go directly to students in their homes. “Parents are figuring out that they can get much better educational lessons for their kids through the internet than they’re getting at school,” he said.

Copyright to the original article belongs to the New York Times, and the article can be read in its original form in full at nytimes.com. It is republished here for the purposes of critical commentary.

We re-used the code and the copyright notice above from Molly White et al.’s The Edited Latecomers’ Guide to Crypto to generate our annotated article.