By Julia Schwarz
Colleges use AI tools for academic advising, analyzing data to predict which students need additional help. But research from Princeton Engineering suggests that helping students requires judgment and expertise that go beyond the algorithm.
The research, led by Lydia Liu, assistant professor of computer science, shows that advisers used information unavailable to the algorithm in two out of three actions taken during student meetings. These included things like rearranging a student’s schedule because of a health condition, providing encouragement to a student who is struggling in math or suggesting courses that align with a student’s career goals.
While the AI tool may have prompted an adviser to check in, Liu said, advising a student usually requires human expertise.
The paper, presented at the Equity and Access in Algorithms, Mechanisms, and Optimization conference last year, uses data from a six-year study conducted at Georgia State University on the impact of AI-assisted advising.
The randomized controlled trial showed that students who were enrolled in AI-assisted advising had slightly higher grades, 0.16 points on average, and a 7% higher graduation rate than those in the control group.
While the project was described as a successful example of using predictive analytics to boost student success, as reported by the Chronicle of Higher Education in 2022, Liu wanted to understand how much of this improvement was due to the algorithm alone versus human expertise.
Combining student data and extensive notes provided by advisers, the researchers analyzed 8,385 advising actions taken across 5,269 meetings. Through statistical testing of student outcomes and qualitative analysis, the researchers determined that, in 67% of cases, advisors drew on information accessible only to humans to target their interventions to help students.
According to Liu, the AI tools used in the Georgia State study are the same type of tools used to predict sepsis in hospitals and recidivism in the criminal justice system. “All these use cases share an underlying problem formulation, which is this idea of predicting risk from an individual’s data,” she said.
As AI tools are implemented across more fields, Liu said, it’s critical to assess the human expertise required to use these tools effectively. “Human intervention really determines what the downstream impact of that algorithmic prediction will be,” Liu said. The predictive algorithm flags students, but in most cases helping a student was something only a human could do.
In complex fields like health, education and criminal justice, the data gathered by AI tools often creates more opportunities for human intervention, not less, she added.
College advising is a compelling example of this, Liu said, because it depends on human relationships. “Advising is not just transactional. It’s not just about delivering information and getting information and processing information. There’s also this relational piece, when students feel like they’ve been seen by somebody.”
She added: “I think it would be really sad if the way we implement automation takes away that important piece of the experience.”
The paper, Discretion in the Loop: Human Expertise in Algorithm-Assisted College Advising, was presented at the EAAMO: Equity and Access in Algorithms, Mechanisms, and Optimization conference in November 2025. In addition to Liu, authors include Kara Schechtman from Princeton University, Benjamin Brandon and Jenise Stafford at Georgia State University and Hannah Li at Columbia Business School.