Current Results and Next Steps

In our recent evaluation, we combined Evidential Deep Learning (EDL) for classification [1] with the Single-Shot Detector (SSD) [2] to get uncertainty estimation at inference time. This approach promises low computational overhead, which is crucial for fast detectors operating in real-time on the edge. However, this method struggles with object detection models, as it is difficult to gather evidence values for the countless possible objects with various shapes, sizes, and locations in an image. Current experiments demonstrate these limitations by comparing several object detection implementations. The base network for all detectors was MobileNetV2 [3], a lightweight and fast model suitable for the edge. Regarding evaluation data, train and test splits were generated from the Oxford Town Centre dataset [4]. Below we can see some results for the mean Average Precision metric, where a higher value (up to 100%) indicates a better performance for the object detector.

On the left (a)) we can see the base SSD detector without uncertainty estimation, providing a baseline of 36.7% mAP. In the middle (b)) we can see the SSD model with additional uncertainty estimation, where the model was trained to predict a Dirichlet distribution, according to [1]. On the right (c)) we can see the latest approach of EDL for object detection, designed by [5], where the authors substitute the classical Dirichlet distribution with a Beta distribution. They argue that the Beta function is better at handling class imbalances and generating evidence for predictions. Indeed, we can see a small improvement going from the traditional EDL implementation in b) with an mAP of 26.7%, to the newer EDL version in c) with an mAP of 28.6%. Unfortunately, this gain in mAP is only marginal, and in both cases, EDL is still far behind the baseline. This suggests that despite trying various approaches, EDL in its current state may not be compatible with the SSD detector. Therefore, the next step is to try a different object detector, like YOLO [6], and see how EDL complies with it.

References

[1] Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 3183–3193, 2018.

[2] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference on computer vision, pages 21–37. Springer, 2016.

[3] Sandler, Mark, et al. "Mobilenetv2: Inverted residuals and linear bottlenecks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

[4] Ben Benfold and I. Reid. Stable multi-target tracking in real-time surveillance video. CVPR 2011, pages 3457–3464, 2011.

[5] Tejas Pandey, Nick Pears, William A P Smith, and John Alexander McDermid. E-DETR: Evidential deep learning for end-to-end uncertainty estimation in object detection, 2025, https://openreview.net/forum?id=tdV1GRkCpZ (accessed 31.01.2025).

[6] Glenn Jocher and Jing Qiu. Ultralytics yolo11, 2024, https://github.com/ultralytics/ultralytics (accessed 31.01.2025).