Abstarct

Video analytics systems conduct video preprocessing to filter out unnecessary frames and model inference using appropriately selected neural networks for high analytics speed. Video preprocessing is instruction-intensive computing (IIC) executed by CPU, and model inference is data-intensive computing (DIC) executed by GPU. In this paper, we show the analytics accuracy of existing systems can largely vary in fields, caused by the dynamic IIC and DIC workloads of different contents in applications. Unfortunately, cameras have fixed CPU/GPU resources and cannot effectively adapt to workload dynamics. We develop Gemini, a new edge-side real-time video analytics system enhanced by a dual-image FPGA. We take the advantage of negligible image switching time of dual image FPGAs, pre-configure one CPU image and one GPU image and elastically multiplex the dual CPU-GPU resources in time dimension. Gemini requires both hardware and software revisions. In hardware, we overcome challenges of hardware dependent application development, low communication efficiency between the microprocessor and FPGA, and high programming complexity by hardware abstraction, asynchronous data transfer mechanism and stub-skeleton middleware. In software, we overcome the challenge of adapting to the dynamic workloads by a bandit learning approach. We implement Gemini and show that Gemini can improve the analytics accuracy to 90.35%.

Problem to solve

Edge-side real-time video analytics.

Method

We propose Gemini, a new real-time video analytics system enhanced by a dual-image FPGA. An image in FPGA is a bit file to configure every Logic Unit to the target functions. The newly developed dual-image FPGA, e.g., Intel Max10, can pre-store two or more images in the FPGA image flash and fast switch between them. The dual-image FPGA has a key advantage over current FPGAs on the reconfiguration time, which can take minutes. Moreover, we can alternatively choose the image switching address to enable more than two images stored on FPGA by simply modifying the source codes. Thus, we can preconfigure CPU and GPU images and switch them in runtime. As a result, we can have elastic CPU-GPU computing resources by multiplexing the dual computing resources in the time dimension.

Result

In software, we overcome the challenge of adapting to the dynamic workloads by a bandit learning approach. We implement Gemini and show that Gemini can improve the analytics accuracy to 90.35%.

Bibtex

@article{hu2023edge,
  title={An Edge-side Real-time Video Analytics System with Dual Computing Resource Control},
  author={Hu, Chuang and Lu, Rui and Sang, Qianlong and Liang, Huanghuang and Wang, Dan and Cheng, Dazhao and Zhang, Jin and Li, Qing and Peng, JunKun},
  journal={IEEE Transactions on Computers},
  year={2023},
  publisher={IEEE}
}