Real-Time Deep Learning Inference on the Edge

Dr. Hamed Tabkhivayghan
Research Highlight

Dr. Hamed Tabkhivayghan is also leading a project on deep learning. This project aims to build an ultra-efficient real-time deep learning inference at the tactical edge. It enables high-performance, power-efficient data analytics and cognitive processing near sensors (e.g., video camera). To this end, this research proposes a reconfigurable deep learning processor to perform real-time deep learning inference with a fraction of a Watt. The key insight to increase efficiency is to minimize data movements at any architecture hierarchy level, exploit parallelism (spatial and temporal) across deep learning functional blocks, and remove the overhead of instruction-level programmability while maintaining enough flexibility. At the same time, this research proposes a Transparent Extensible Deep Learning Framework as a multi-layer software abstraction between the deep learning programming development and real-time execution on edge across heterogeneous devices. The aim is to enable both algorithm-dependent and architecture-dependent optimization to realize full AI and data analytics on the edge devices. Overall, this research opens a path to perform real-time complex stream processing near the sensors offering human-like and beyond human cyber cognitive abilities.