Investigating the Interpretability of Pedestrian Detection Models via Testing with Concept Activation Vectors (TCAV): A Cross-Architectural Empirical Analysis

Abstract

This study addresses the “black-box” issue in pedestrian detection models deployed in high-risk scenarios, such as autonomous driving and intelligent security systems, by employing the Testing with Concept Activation Vectors (TCAV) framework. We conduct a cross-layer empirical analysis of feature representations across three representative architectures: Faster R-CNN, YOLOv11, and RT-DETR. An automatic cropping strategy based on human anatomical proportions is introduced to construct semantic concept sets (head, torso, and legs), enabling quantitative evaluation of semantic evolution at different network depths. Results show that Faster R-CNN follows a progressive semantic modeling pattern from local textures to global structures while maintaining stable semantic disentanglement in deeper layers; YOLOv11 exhibits semantic latency, with structured human-body representations emerging mainly within the feature fusion module rather than the backbone network; and RT-DETR, despite strong semantic separability, demonstrates gradient sparsity induced by output saturation, limiting the effectiveness of linear interpretability methods. These findings delineate the applicability boundaries of TCAV-based interpretation across heterogeneous architectural paradigms and provide a quantitative basis for designing highly transparent pedestrian detection systems.