Instructions — IMPORTANT: Read Before You Start

You will be given pictures and their descriptions. Each description will tell you a specific interaction between people and objects in the picture, for example, a person riding a bicycle. Your task is to draw tight boxes around each person and each object involved in the given interaction, and link a person to an object if you see the interaction happening between them.

Sample annotations for the description "A person riding a bicycle."
tutorial1 tutorial2 tutorial3

Once you start the annotation, your task for involves three steps:

Step 1: Draw a tight box around each person involved in the interaction  Click to expand
For example, draw a tight box around each person riding bicycles; ignore all the people not riding bicycles.

    tutorial1
  • Draw a box
    1. Hold down your left mouse button.
    2. Move the cursor around.
    3. Release your left mouse button.
    • tutorial1
  • Adjust a box
    1. Move the cursor to one vertex of the box you want to adjust. You will see the vertex becomes bigger.
    2. Hold down your left mouse button
    3. Move the cursor around.
    4. Releasing your left mouse button.
    • tutorial1
  • Delete a box
    1. Click on the box once. This will highlight the box.
    2. Click on the box again. This will delete the box.

Step 2: Draw a tight box around each object involved in the interaction  Click to expand
For example, draw a tight box around each bicycle being ridden by people; ignore all the bicycles not being ridden.

    tutorial1
  • Draw a box
    1. Hold down your left mouse button.
    2. Move the cursor around.
    3. Release your left mouse button.
    • tutorial1
  • Adjust a box
    1. Move the cursor to one vertex of the box you want to adjust. You will see the vertex becomes bigger.
    2. Hold down your left mouse button
    3. Move the cursor around.
    4. Releasing your left mouse button.
    • tutorial1
  • Delete a box
    1. Click on the box once. This will highlight the box.
    2. Click on the box again. This will delete the box.

Step 3: Link a person to an object if the interaction is happening between them.  Click to expand
For example, link a person to a bicycle if that person is riding that bicycle.

    tutorial1
  1. Select one person and draw links to all the objects which he/she is interacting with.
    1. Click on one person box. This will highlight the person.
    2. Click on one object box. This will draw a link from the person to the object.
  2. Remove a link by clicking on the object box again.
  3. Switch to another person by clicking on another person box.
  4. Repeat until all the person boxes are done.

If the person or object involved in the interaction is not visible in the picture, you should check the square box under Reset and skip the picture.

Notes

  1. You are free to swtich between steps. You can always go back to previous steps if you need to make changes.
  2. If a person or object is occluded or cut by the picture, draw a box only around its visible part.
  3. Do not draw box around people or object that is not involved in the interaction.
  4. Make sure you link every person-object pair where you see the given interaction.

Click to see more examples

If a person or object is occluded or cut by the picture, draw a box only around its visible part.
A person riding an airplane.
tutorial1
A person feeding a bird.
tutorial1
A person petting a bird.
tutorial1
A person exiting an airplane.
tutorial1
Do not draw box around people or object that is not involved in the interaction.
A person washing a bicycle
tutorial1
A person tying a boat.
tutorial1
A person repairing an umbrella.
tutorial1
A person hosing a car.
tutorial1
Link every person-object pair where you see the given interaction.
A person boarding an airplane.
tutorial1
A person chasing a bird.
tutorial1
A person herding a cow.
tutorial1
A person eating at a dining table.
tutorial1

Keyboard interface

  1. G / A  :  Go to the next/previous picture.
  2. S: Switch to Step 1.
  3. D: Switch to Step 2.
  4. F: Switch to Step 3.
  5. R: Reset annotations on the current picture.

Warning: Rejection is possible if your accuracy is too low.
HIT Starts Here
Description:

        [S]
        [D]
        [F]
        [R]

Check this box if it is impossible to draw a box on person or . For example, if all people or all involved in the interaction is fully occluded or outside the picture.

 

  
[A]                         [G]

/