PoseTrack: Enhancing Multi-Object Tracking with 3D Pose-Visual Fusion and Adaptive Trajectory Management

In this paper, we introduce PoseTrack, a new multi-object tracking (MOT) framework specifically designed for multi-person sports scenes. Existing methods often struggle due to challenges such as similar player appearances, frequent occlusions, and unpredictable entrances and exits. PoseTrack proposes a new MOT approach by introducing human pose estimation into the tracking algorithm and temporally fusing it with 2D visual cues to address these challenges. Specifically, by leveraging the unique 3D poses of individuals, PoseTrack enhances the association between trajectories and detections across consecutive frames, thereby improving the accuracy of tracking. In addition, our PoseTrack ensures robust tracking performance even when athletes enter or exit the frame by imposing constraints on disappeared trajectories and adaptively adjusting their disappearance time based on the trajectory’s survival duration. Extensive evaluations on two large-scale sports benchmarks—SportsMOT and SoccerNet-Tracking—demonstrate that PoseTrack outperforms previous methods across key metrics (HOTA, IDF1, AssA, DetA). PoseTrack can support applications such as sports tactical analysis and athlete performance evaluation by enabling highly accurate and reliable motion analysis.