Skip to main content

Local keypoint-based Faster R-CNN

Ding, Xintao; Li, Qingde; Cheng, Yongqiang; Wang, Jinbao; Bian, Weixin; Jie, Biao


Xintao Ding

Jinbao Wang

Weixin Bian

Biao Jie


Region-based Convolutional Neural Network (R-CNN) detectors have achieved state-of-the-art results on various challenging benchmarks. Although R-CNN has achieved high detection performance, the research of local information in producing candidates is insufficient. In this paper, we design a Keypoint-based Faster R-CNN (K-Faster) method for object detection. K-Faster incorporates local keypoints in Faster R-CNN to improve the detection performance. In detail, a sparse descriptor, which first detects the points of interest in a given image and then samples a local patch and describes its invariant features, is first employed to produce keypoints. All 2-combinations of the produced keypoints are second selected to generate keypoint anchors, which are helpful for object detection. The heterogeneously distributed anchors are then encoded in feature maps based on their areas and center coordinates. Finally, the keypoint anchors are coupled with the anchors produced by Faster R-CNN, and the coupled anchors are used for Region Proposal Network (RPN) training. Comparison experiments are implemented on PASCAL VOC 07/12 and MS COCO. The experimental results show that our K-Faster approach not only increases the mean Average Precision (mAP) performance but also improves the positioning precision of the detected boxes.

Journal Article Type Article
Journal Applied Intelligence
Print ISSN 0924-669X
Electronic ISSN 1573-7497
Publisher Springer (part of Springer Nature)
Peer Reviewed Peer Reviewed
APA6 Citation Ding, X., Li, Q., Cheng, Y., Wang, J., Bian, W., & Jie, B. (in press). Local keypoint-based Faster R-CNN. Applied Intelligence,
Keywords Keypoint; SIFT; Convolutional neural network; Faster R-CNN
Publisher URL
Additional Information First Online: 28 April 2020