10-20

Mike Wong will present his FPO "FINDING THE NEEDLE IN THE HAYSTACK: APPLICATION-AWARE DATA SYSTEMS FOR EFFICIENT AI INFERENCE" on 10/20/26 in Friend 108 at 10am.

Committee members: Ravi Netravali (adviser)

Examiners: Jialin Ding, Wyatt Lloyd, Kai Li, Ravi Netravali (adviser)

Readers: Jialin Ding, Ravi Netravali

 

Abstract:

Modern Al systems are increasingly deployed in settings where inference cost dominates opera- tional expenditure. Consequently, a large body of research has focused on improving the efficiency of Al systems through techniques such as model compression, quantization, caching, kernel opti- mization, and specialized hardware. While these approaches reduce the cost of executing inference, they largely treat the data presented to the model as fixed. In practice, every Al system operates downstream of a data system that determines what information is collected from the environment, how it is processed, and how it is ultimately represented to the model. The Al system never ob- serves the environment directly; it only observes the outputs of the data system. This thesis argues that the data system is an equally important and largely underexplored target for inference opti- mization. A central observation motivates this view: across diverse domains, only a small fraction of observed information is relevant to the task being performed. Most collected data does not influ-  ence the final output, yet it still incurs collection, transmission, storage, and inference costs. This observation creates an opportunity for application-aware specialization that identifies task-relevant information before expensive Al inference occurs. To study this opportunity, this thesis introduces a framework that decomposes data-system  specialization into three stages: Planning, Adaptation, and Representation. Planning determines what information should be acquired and how it should be collected. Adaptation dynamically adjusts data acquisition in response to environmental changes. Representation transforms collected information into forms that are more informative and efficient for downstream inference. This thesis instantiates this framework through three systems spanning web agents, video an-  alytics, and network security. First, Skim addresses planning for web agents by learning reusable website structure and using it to speculatively predict efficient data acquisition strategies before execution begins, reducing per-task cost by 1.9x and latency by 33.4% while preserving ac- curacy. Second, MadEye addresses adaptation for live video analytics by dynamically steering pan-tilt-zoom cameras toward task-relevant regions of a scene, improving workload accuracy by 2.9-25.7% relative to the best fixed-orientation strategy. Third, NetVigil addresses representation for east-west network anomaly detection by constructing domain-aware graph representations of flow-level network telemetry and applying graph neural networks for robust anomaly detection, improving average AUC by 0.22 over prior systems while substantially reducing operational cost. Together, these systems demonstrate that application-aware data processing can significantly reduce the cost of Al-driven systems by identifying and prioritizing task-relevant information be- fore expensive inference occurs. More broadly, this thesis establishes data-system specialization as a general framework for improving the efficiency of Al-enabled applications across diverse domains.