HICO & HICO-DET
Benchmarks for Recognizing Human-Object Interactions in Images

We introduce two new benchmarks for classifying and detecting human-object interactions (HOI) in images:

1. HICO (Humans Interacting with Common Objects)
2. HICO-DET

Key features:

1. A diverse set of interactions with common object categories
2. A list of well-defined, sense-based HOI categories
3. An exhaustive labeling of co-occurring interactions with an object category in each image
4. The annotation of each HOI instance (i.e. a human and an object bounding box with an interaction class label) in all images

Task 1: HOI Classification

The input is an image and the output is a set of binary labels, each representing the presence or absense of an HOI class.

Sample annotations in the HICO benchmark

Task 2: HOI Detection

The input is an image and the output is a set of bounding box pairs, each localizes a human plus an object and predicts an HOI class label.

Riding a horse

 

Feeding a horse

 

Eating an apple

 

Cutting an apple

Sample annotations in the HICO-DET benchmark