Distribution-Aware Coordinate Representation for Human Pose Estimation

Feng Zhang¹ Xiatian Zhu² Hanbin Dai¹ Mao Ye¹ Ce Zhu¹

1.University of Electronic Science and Technology of China; 2.University of Surrey

Abstract

     While being the de facto standard coordinate representation in human pose estimation, heatmap is never systematically investigated in the literature, to our best knowledge.
     This work fills this gap by studying the coordinate representation with a particular focus on the heatmap. Interestingly, we found that the process of decoding the predicted heatmaps into the final joint coordinates in the original image space is surprisingly significant for human pose estimation performance, which nevertheless was not recognised before. In light of the discovered importance, we further probe the design limitations of the standard coordinate decoding method widely used by existing methods, and propose a more principled distribution-aware decoding method. Meanwhile, we improve the standard coordinate encoding process (i.e. transforming ground-truth coordinates to heatmaps) by generating accurate heatmap distributions for unbiased model training. Taking the two together, we formulate a novel Distribution-Aware coordinate Representation of Keypoint (DARK) method. Serving as a model-agnostic plug-in, DARK significantly improves the performance of a variety of state-of-the-art human pose estimation models.
     Extensive experiments show that DARK yields the best results on two common benchmarks, MPII and COCO, consistently validating the usefulness and effectiveness of our novel coordinate representation idea.

Paper

arxiv

Code

We will release the training and testing code and the pretrained model at GitHub.
Our CVPR2019 work Fast Human Pose Estimation can work seamlessly with DARK, which is available at GitHub.

COCO 2019 Keypoint Detection Challenge

Hanbin Dai¹* Liangbo Zhou¹* Feng Zhang ¹* Zhengyu Zhang²* Hong Hu ¹* Xiatian Zhu³* Mao Ye¹

* means equal contribution; 1.University of Electronic Science and Technology of China; 2.Shenzhen University; 3.University of Surrey

AP result: 78.9 on the COCO test-dev set (2nd place in the test-dev leaderboard) and 76.4 on the COCO test-challenge (2nd place entry of COCO Keypoints Challenge ICCV 2019)

Technical report is here.

Visualization

We give four types of challenges: images containing invisible joints, images containing low-resolution persons, images having crowd scenes, images containing partial parts.