Abstract

Aerial images from drones have been used to detect and track suspects in the crowd for the public safety purpose. However, using a single drone for human identification and localization faces many challenges including low accuracy and long latency, due to poor visibility, varying field of views (FoVs), and limited on-board computing resources. In this paper, we propose SkyNet, a multi-drone cooperative system for accurate and realtime human identification and localization. SkyNet computes the 3D position of a person by cross searching from multiple views. To achieve high accuracy in identification, SkyNet fuses aerial images of multiple drones according to their legibility. Moreover, by predicting the estimated finishing time of tasks, SkyNet schedules and balances workloads among edge devices and the cloud server to minimize processing latency. We implement and deploy SkyNet in real life, and evaluate the performance of identification and localization with 20 human participants. The results show that SkyNet can locate people with an average error within 0.18m on a square of 554m2. The identification accuracy is 91.36%. The localization and identification process is completed within 0.84s.

Problem to solve

How to achieve accurate and real-time human identification and localization?

Method

We design SkyNet, a multi-drone cooperative framework to achieve accurate and real-time identification and localization. In a crowded scene, SkyNet locates and identifies the human target using the Multi-Drone Person Localization (MDPL) and the Multi-View Fusion Identification (MVFI) modules. The Dynamic Task Scheduling for Heterogeneous Devices (DTSH) module is designed to schedule tasks among edge devices and the cloud server to reduce latency and balance workload.

Result

We implement and deploy SkyNet in real life, and evaluate the performance of identification and localization with 20 human participants. The results show that SkyNet can locate people with an average error within 0.18m on a square of 554m². The identification accuracy is 91.36%. The localization and identification process is completed within 0.84s.

Bibtex

@inproceedings{skynet,
  title={SkyNet: Multi-Drone Cooperation for Real-Time Identification and Localization},
  author={Junkun Peng and Qing Li and Yuanzheng Tan and Dan Zhao and Zhenhui Yuan and Jinhua Chen and Hanling Wang and Yong Jiang},
  booktitle={Proceedings of IEEE International Conference on Computer Communications (INFOCOM)},
  pages={1--10},
  year={2023}
}