Evaluation

Data Preparation

The first step of evaluation is to prepare your own data. You need to run your code and generate a GraspGroup for each image in each scene. Then call the save_npy function of GraspGroup to dump the results.

The file structure of dump folder should be as follows:

|-- dump_folder
    |-- scene_0100
    |   |-- kinect
    |   |   |
    |   |   --- 0000.npy to 0255.npy
    |   |
    |   --- realsense
    |       |
    |       --- 0000.npy to 0255.npy
    |
    |-- scene_0101
    |
    ...
    |
    --- scene_0189

You can only generate dump for one camera, there will be no error for doing that.

Evaluation API

Get GraspNetEval instances.

# GraspNetAPI example for evaluate grasps for a scene.
# change the graspnet_root path
import numpy as np
from graspnetAPI import GraspNetEval

####################################################################
graspnet_root = '/home/gmh/graspnet' # ROOT PATH FOR GRASPNET
dump_folder = '/home/gmh/git/rgbd_graspnet/dump_affordance_iounan/' # ROOT PATH FOR DUMP
####################################################################

sceneId = 121
camera = 'kinect'    
ge_k = GraspNetEval(root = graspnet_root, camera = 'kinect', split = 'test')
ge_r = GraspNetEval(root = graspnet_root, camera = 'realsense', split = 'test')

Evaluate A Single Scene

# eval a single scene
print('Evaluating scene:{}, camera:{}'.format(sceneId, camera))
acc = ge_k.eval_scene(scene_id = sceneId, dump_folder = dump_folder)
np_acc = np.array(acc)
print('mean accuracy:{}'.format(np.mean(np_acc)))

Evaluate All Scenes

# # eval all data for kinect
# print('Evaluating kinect')
# res, ap = ge_k.eval_all(dump_folder, proc = 24)

Evaluate ‘Seen’ Split

# # eval 'seen' split for realsense
# print('Evaluating realsense')
# res, ap = ge_r.eval_seen(dump_folder, proc = 24)