“Not All Samples Are Created Equal: Task-Aware Informative Sampling and Adaptive Inference for Efficient Edge AI”

The rapid proliferation of Internet of Things (IoT) devices has resulted in an overwhelming influx of data generated at the edge by a myriad of sensors. Traditional methods of transmitting all data to the cloud for processing are increasingly untenable due to constraints in bandwidth, latency, and privacy concerns. Consequently, edge computing has emerged as a pivotal paradigm that brings computation closer to the data source, enabling real-time processing and decision-making. The integration of deep learning techniques within edge computing, leveraging local data and computation, has paved the way for edge intelligence, facilitating personalized, autonomous, and seamless interactions. However, this paradigm faces significant challenges, including limitations in computation, storage, and energy resources, as well as data and computation redundancies, which impede its full potential. This thesis proposes novel frameworks to address these challenges through three primary contributions. First, it introduces hardware-aware model compression techniques that incorporate advanced 5-bit quantization and weight-sharing strategies applied to modern deep models. To support these compressed models, the thesis develops a highly parallel and energy-efficient, Processing in Sensor (PIS) architecture, named APRIS, designed specifically for the efficient handling of the compressed neural network layers. Second, in response to the redundancies in data, computation, and storage, a task-aware informative sampling framework is proposed. This framework, named EnCoDe, utilizes an intelligent sampler network trained in conjunction with the downstream task in an adversarial manner to identify and select the most informative and relevant samples, thereby reducing computational and storage overhead, while improving model compactness and generalization performance. Third, to address computational inefficiencies in energy-constrained environments—especially in battery-less IoT devices—an adaptive network inference framework is developed. This framework features multiple task-specific modules with variable computational and energy profiles, all sharing a common feature extractor, which optimizes resource utilization. This thesis contributes to advancing the field of edge intelligence by introducing novel techniques for model compression, intelligent sampling, and adaptive inference, all aimed at enhancing efficiency, resource utilization, and performance in edge computing environments.

Committee Members:
Dr. Arman Roohi, Advisor
Dr. Sasitharan Balasubramaniam
Dr. Stephen Scott