python-3.x - 从 Detectron2 中提取对象特征

Question

我对物体检测真的很陌生，如果这个问题看起来太明显了，我很抱歉。

我在检测对象的 detectron2 上有一个训练有素的 FasterRCNN 模型，我正在尝试提取每个检测到的对象的特征以作为模型预测的输出。可用的教程似乎在寻找 ROI 并做出新的预测，以及它们的框和功能。我有推断的盒子，我只需要提取每个盒子的特征。我在下面添加了我一直在使用的代码。谢谢

# Preprocessing
images = predictor.model.preprocess_image(inputs)  # don't forget to preprocess
# Run Backbone Res1-Res4
features = predictor.model.backbone(images.tensor)  # set of cnn features
        
#get proposed boxes + rois + features + predictions
# Run RoI head for each proposal (RoI Pooling + Res5)
proposal_boxes = [x.proposal_boxes for x in proposals]
features = [features[f] for f in predictor.model.roi_heads.in_features]
proposal_rois = predictor.model.roi_heads.box_pooler(features, proposal_boxes)
box_features = predictor.model.roi_heads.box_head(proposal_rois)
predictions = predictor.model.roi_heads.box_predictor(box_features)#found here: https://detectron2.readthedocs.io/_modules/detectron2/modeling/roi_heads/roi_heads.html        
pred_instances, pred_inds = predictor.model.roi_heads.box_predictor.inference(predictions, proposals)
pred_instances = predictor.model.roi_heads.forward_with_given_boxes(features, pred_instances)


# output boxes, masks, scores, etc
pred_instances = predictor.model._postprocess(pred_instances, inputs, images.image_sizes)  # scale box to orig size
# features of the proposed boxes
feats = box_features[pred_inds]
proposal_boxes = proposals[0].proposal_boxes[pred_inds]

Answer 1

这个问题已经在本期讨论得很好:https://github.com/facebookresearch/detectron2/issues/5 .

documentation还解释了如何实现这一目标。