WBIA

Image ID: 2252
Folder(s):
Filename: New%20Zealand%20Department%20of%20Conservation%2Fsheaven%40doc.govt.nz%2Frn_image_picker_lib_temp_fa70a066-74be-461f-8762-a4d33605196c_1753150878568.jpg

Hold Ctrl for hotkeys

Axis-to-Axis Mid-line (M)

Show Part Assignments (A)

Annotation (T)

(S) Always Show Parts
(D / H) Always Hide Parts

Staged Users: 0

Quick Help

Annotation Part

Viewpoint:

Primary (⇧ / Shift) focus an annotation to edit these values

null (0)

top (1)

bottom (2)

front (3)

back (4)

left (5)

right (6)

Secondary (⌥ / Alt / Option)

null (0)

top (1)

bottom (2)

front (3)

back (4)

left (5)

right (6)

Third

null (0)

top (1)

bottom (2)

front (3)

back (4)

left (5)

right (6)

Quality:

null (E)

(Q)

(W)

(I) Annotation of Interest

(O) Multiple Animals

Species: (C to cycle) Add Species

Viewpoint:

Primary (⇧ / Shift) focus a part to edit these values

inherit (0)

top (1)

bottom (2)

front (3)

back (4)

left (5)

right (6)

Quality:

inherit (E)

(Q)

(W)

Type: (C to cycle) Add Type

Add a Species

Add a Part Type

Example Images of Species Found in WBIA

Here are example images of the different species that WBIA recognizes.

Only annotations for the species specified (top left of the page) should exist in the image. If the species is '____', annotate the most frequent or predominate species in the image. If you are uncertain as to what the specified species looks like, ask a graduate student.

African Elephant

Loxodonta

The African Elephant is distinguished by being the largest terrestrial animal, generally has grey or dark-brown baggy skin, a long trunk that acts as a nose and grabbing appendage, and showcases two white Ivory tusks. The African Elephant can be distinguished against the Asian Elephant by having much larger ears to facilitate efficient heat transfer to the air. [1]

African Wild Dog

Lycaon pictus

The African Wild Dog is canid that can be distinguished by its dog or coyote-like appearance with a dapple / spotted coat of black, tan, brown and white. The ears are round and forward facing and there is generally a black stripe running down from the ears and between the eyes. [2]

Cheetah

Acinonyx jubatus

A Cheetah is the fastest terrestrial animal and is distinguished by the very long and lean torso and elongated legs. The Cheetah is also unique in the small size of its head, the large tail for counter-balance, and dark-brown spots covering the brown to tan fur. [3]

Giraffe

Giraffa camelopardalis

A Giraffe is distinguished by, most obviously, having a very elongated neck and legs. The giraffe is also unique in that it has patches of brown islands over a white, light-yellow, or light-brown coat and showcases a very long, purple tongue. [4]

Grévy's Zebra (Grevy's / Imperial Zebra)

Equus grevyi

The Grevy's Zebra (named after Jules Grévy) are also horse-lie in structure and are distinguished from Plains Zebras by having very thin stripes that do not extend down and wrap under the belly (the underbelly is generally white). [5]

Lion (Male & Female)

Panthera leo

The Male Lion is distinguished by its large size, dark-brown or tan mane around the neck, and sleek, thin, tan coat. The Male Lion has a much larger head compared to the Cheetah and the juvenile males can usually be identified by the presence of testicles and the start of a mane. [6]

The Female Lion (Lioness) is distinguished from the Male Lion by the absence of a mane around the neck. The Lioness is also more lean and has a swayed back for hunting and stalking prey more efficiently. [7]

Plains Zebra (Common / Grant's Zebra)

Equus quagga

A Plains Zebra is the stereotypical zebra found in most zoos and children's books. The Plans Zebra has a horse-like structure and can be distinguished by thick stripes that extend down and wrap under the belly. [7]

Detection Reviewing Instructions

To review the detections for an image, simply draw new rectangle bounding boxes (annotations) around animals that are missing annotations and delete annotations that should not exist based on the following guidelines:

When deciding to add, edit, or remove an annotation, it is important to understand to what ends WBIA will use annotations. WBIA uses the annotation's usable information (a picture is worth a thousand words) to assign keypoint descriptors (see below), which are oval regions on the image that have a lot of information in them. The algorithms in WBIA use the bounding boxes and annotations to detect animals (draw bounding boxes automatically) and distinguish animals (differentiate between two individuals) in an image. In order to capture the most information about an animal, the bounding box should satisfy the following three requirements:
1. An annotation should include as much of the foreground animal as possible; this means cover the entire animal with the bounding box.
2. An annotation should exclude as much of the background landscape or other animals as possible; this means fit the bounding box tightly around the animal.
3. Only annotations for the species specified (top left of the page) should exist. If the species is '____', annotate the most frequent or predominate species in the image. If you are uncertain as to what the specified species looks like, ask a graduate student or click here.
The example images below show how hard it can be to satisfy these three simple requirements.
For example:

A bounding box needs to cover the entire animal without being too large or too small - the green bounding box is ideal. The big red bounding box has too much background and can be fit tighter around the animal, whereas the two small red bounding boxes are losing information that is important (information about the top of the animal's head and its legs). The orange bounding box is slightly too large, but is also a border-line acceptable bounding box. [7]

Keypoint descriptors. The oval regions are extracted within the bounding box based on how much useful information the algorithm can find. As we can see, the keypoints really respond the zebra's face, legs and mid body. The entire body of the zebra produces good, usable, informative keypoints. The background, however, produces small, and not very important keypoints. In order to be the most productive reviewer, always keep in mind what information these keypoints are trying to capture. [7]

Assigning bounding boxes around an animal with severe occlusion can be very challenging. The green bounding boxes are obvious, but the orange and red examples can be harder to decide. For this example, all of the orange bounding boxes are border-line acceptable cases. The red bounding boxes, however, do not have very much usable information. This can be a very difficult decision and the image may have to be reviewed with the help of a graduate student. For this example, all of the red bounding boxes should be excluded because they are too small or do not capture enough information to be useful. [7]

Whenever there is large overlap in animals that share roughly the same bounding box, always add a bounding box for each animal. Even if the resulting bounding boxes are almost duplicates, it gives WBIA a useful notion of how many animals of a particular species are in the image. For this example, both bounding boxes are necessary. [7]

In the event of a significant occlusion of a portion of the animal, try your best to cover as much of the visible portions of the animal as if the occlusion did not exist. This occlusion can be due to other animals, a bush, tall grass, or (in this example) a tree bisecting the entire animal. In this example, two bounding boxes would be incorrect because it would signify that there are two animals in the image. [8]

A new annotation can be added by dragging the mouse cursor around the animal like a selection. To perform a selection, click the left mouse button down on the top left corner where the annotation should start, drag the mouse to the bottom right corner where the annotation should end, and release the left mouse button to add the annotation.
You can always add a new annotation by double clicking. This is particularly helpful when adding an annotation on top (inside) of a current annotation.
When an annotation is highlighted (by hovering the mouse cursor over the annotation), the bounding box will turn orange and the following actions will become available:
- Translation - by dragging the mouse an annotation can be moved. An annotation can also be moved (when highlighted) by using the arrow keys on the keyboard. When using the arrow keys, an annotation will be moved 1 pixel at a time. When using the arrow keys to move an annotation and the Shift key is held, the annotation is moved 10 pixels at a time.
- Scaling - by dragging one of the 8 anchors an annotation can be resized. If the Shift key is held, the annotation's aspect ratio is preserved.
- Rotation - by dragging on the
  rotation anchor an annotation can be rotated. If the Shift key is held, the annotation's theta is constrained to increments of 45 degrees.
- Deletion - by clicking on the
  ×
  button an annotation can be deleted. An annotation can also be deleted (when highlighted) by hitting the Esc key or the k key.
- Send to Back - when an annotation is highlighted, pressing the b key will send the annotation to the "back". This is useful (or required) when a large annotation is covering a smaller annotation and makes highlighting the smaller annotation difficult (or impossible).
Always annotate the animals regardless of viewpoint.
If an image has no animals in it, simply hit Accept with no annotations, because the image should have no annotations.
A new, in-progress annotation selection can be cancelled by pressing the Esc key or the k key.
The Space key deletes the image and does not save any viewpoint information for that image. The Enter key submits all current annotations.
Review annotations as fast as you can while maintaining perfect accuracy.
To bring up these instructions again, click on the button
Known issues:
- Scaling an annotation that has been rotated does not behave as expected
- Translating an annotation that has been rotated does not respect the containment region correctly
- When resizing the left edge of an annotation and the cursor moves off of the image, the annotation is resized

Image Credits
[1] http://de.wikipedia.org/
[2] http://www.fauna-flora.org/
[3] http://www.napha-namibia.com/
[4] http://www.girrytrust.org/
[5] http://www.wanderlustasap.com/
[6] Tanzanian Images / Getty Images / iStockphoto
[7] http://en.wikipedia.org/
[8] http://talk2theanimals.net/

New Features for Detection Reviewing

Here are some of the new features in the redesigned detection interface:

Annotation of Interest (AoI)

We want to capture a simple concept that can sometimes manifest itself in very complicated ways. In particular, we ask the following question:

“Why did the photographer take this particular picture?”

This is not a question of artistic composition, however. Pictures should be contributed to Wildbook for the purposes of identifying individual animals. Therefore, to answer the "why" question, the goal is to understand an image's semantic composition by marking the individual annotation(s) that are the interest, subject(s) or focus of the picture. More specifically, the task is to mark all annotations that have (in general) the following properties:

Is of a distinguishable individual (i.e. free-standing or clearly in the foreground of a heard)
Is relatively large and has decent resolution
Is commonly located near the center of the image
Is in focus and not blurry

Conversely, the annotations that are not considered to be an AoI have (in general) the following opposite properties:

Is a part of an overlapping herd or group of animals
Is relatively small and/or contains few pixels
Is out of focus and is blurry (from being too far from or too close to the camera relative to the plane that is in focus)
Is located around the edges of the image
Is significantly (greater than 25%) occluded by other animals or objects
Is significantly (greater than 25%) off the edge of the frame of the image

These above properties (and counter properties) require that the annotation must not be viewed in isolation. The decision that an annotation is an AoI must be made by weighing the entire context of the image and against any other annotations that may also be in the image. Further, because these conditions are fairly strict, there are rarely more than a few AoIs in a particular image, and some images may have none.

Note: As a small (but important) aside, all individual animals (of the target species) in the image should have an annotated bounding box. The intent is to have the same number of bounding boxes as the number of animals that can realistically and reliably be manually counted in the image. If one can reasonably count 30 animals in the image, there should exist 30 corresponding bounding boxes. This completeness assumption is crucial to the validity of the Annotation of Interest decision for a given image; any image that does not satisfy this condition should first be reviewed to obtain a complete set of annotations.

Beware: it is crucially important to not accidentally conflate this metric with other detection and identification concepts. While the concepts below are related to and may be strong indicators of Annotation of Interest, they are not necessarily identical. For example,

Annotation of Interest ≠ is identifiable by any particular algorithm
Annotation of Interest ≠ is of a desired quality
Annotation of Interest ≠ is of a desired species
Annotation of Interest ≠ is of a desired viewpoint
Annotation of Interest ≠ is free of photo-bombs

How well do these rules work on real-world examples? Let's take a look at a few:

Example 1

First, there is the need to define the color scheme. An annotation bounding box is marked with orange or green unless it is an AoI, which is marked with the color blue.

Tip: Use the hotkey I (for interest) to toggle a hovered box as an AoI

Tip: Sometimes bounding boxes are very close together and overlapping. If you experience difficulty hovering the desired bounding box, use the hotkey B (for background) to send the hovered (undesired) annotation to the background.

This first example (above) is straightforward. There is only one annotation in this image, it is clearly lit, and has no occlusions. A free-standing animal in a field, like in this example, can be considered an ideal AoI. The annotation is therefore marked in blue to signify that it is an AoI.

Remember, not all images are guaranteed to have at least one annotation. Furthermore, for an image with a single annotation, it is not guaranteed that the solo annotation should always be an AoI.

Example 2

This second example gives multiple instances of what is considered to be an out-of-focus annotation.

The single AoI in this image (again, in blue) is smaller by area than in the first example. The blue annotation is in focus and clearly distinguishable, which makes it an AoI for this image. The four additional orange annotations near the top of the image are not AoIs: they are small and out of focus.

Example 3

This example has 4 total annotations, two of which (in blue) are AoIs and two which are just normal annotations (in orange). The two smaller annotations in the background do not qualify as AoIs because they are slightly out of focus, too small, and the left-most orange box has significant occlusion by another animal. The two blue AoIs are both in focus and have no significant occlusions (the small amount of occlusion of the left AoI by the right AoI is not significant nor is the clipping end of the snout by the edge of the frame).

Looking at the AoIs from these first 3 examples, a pattern begins to emerge. This pattern revolves around what can be called the "focus plane" and it helps to outline a simple procedure for deciding which annotations should be considered an AoI.

Find the single best AoI in the image, if it exists. It is important that the best AoI is in focus and has the easiest argument for being an AoI. (For example, the right of the two blue bounding boxes in Example 3 should be chosen as the best AoI because the zebra is more central in the image and it slightly occludes the zebra in the left blue bounding box.) If no such AoI exists, continue to the next image without selecting any AoIs.
Determine the "focus plane" that this best AoI lives within. The focus plane can be decided based on the regions of sharpness and blurriness within the image. This theoretical plane is the depth within the image that the best AoI occupies in 3D space. For example, all other objects that are also within this plane should be all have the same focus and depth within the image as the best AoI. Note that not all focus planes will be the same width; an image taken with a larger camera aperture opening will give a narrower depth of field.
Any annotation within the focus plane should be considered as a candidate AoI and must be reviewed individually. Any annotation that satisfies the AoI properties and is also within the focus plane of the best AoIs should also be marked as an AoI.
Any annotation that is not within the focus plane should not be considered an AoI.

A caveat to the procedure above is when the focus plane is very wide (small aperture opening on the camera) and objects at different actual depths in the image can both appear to be similarly in focus and roughly within the same focus plane. In this scenario, these annotations may be treated as if they were in the same focus plane and occupied the same 3D depth within the image.

Let's apply the procedure above to the next example:

Example 4

In this fourth example, the concept of a "focus plane" is demonstrably clear. Let's follow the decision steps just outlined:

Deciding the single, best AoI in this example is subjective. The most in-focus and free-standing annotations are in the center of the image (marked in blue). For the purpose of discussion, however, we can pick the blue annotation on the right as the best AoI because it is (slightly) more central to the image.
With the best AoI selected, the depth the focus plane for this image can be decided. This example image can be conceptually broken up into three distinct depth zones relative to the camera: in-front of the focus plane, the focus plane itself, and behind the focus plane. Note that not every image has these three distinct depth zones and some images may only have one. For this example, the bottom 1/3 of the image is in-front of the focus plane, the focus plane occupies the middle 1/3, and the top 1/3 is behind the focus plane.
With the three focus zones defined, there are three annotations (the two blue and the cutoff orange annotation off the left border) that are within the focus plane. Next, each box is individually checked against the AoI properties. The left blue annotation is a good AoI, but the left-most orange annotation is largely cutoff and should not be considered as an AoI.
The 4 annotations at the bottom of the image fall outside (in-front) of the focus plane and are therefore should not be considered as AoIs. The 4 annotations at the top of the image also fall outside (behind) of the focus plane and should similarly not be considered as AoIs.

Example 5

Example 5 shows a group of animals in a tight herd. The herd itself is the primary feature of this image rather than any one individual. There are no clearly distinguishable individuals apart from the herd and therefore no AoIs. Also even though all of these annotations occupy roughly the same focus plane, the annotations are also too small to be considered AoIs.

Note: Herds dilute interest. This example's scenario of smaller bounding boxes along with being of a herd is one of the most dangerous in terms of ambiguity and difficulty. Usually these examples will produce no AoI's.

Example 6

The image in Example 6 is striking because there are many annotations, but none should be considered AoIs. This is primarily because the focus plane is clearly centered around the bird stuffed animal. The right-most (blurry) orange annotation would normally be the best AoI, but is ignored for being behind the focus plane.

Example 7

Here is another example that shows many (small) annotations in a herd. None of these annotations should be labeled as AoIs.

Note: Remember that it is crucial for an image to have annotation completeness. To a reasonable effort, every discernible animal in the image should have a corresponding bounding box.

Example 8

With this example we want to reiterate that the decision to label an annotation as an AoI should not be decided by or conflated with other metrics, like viewpoint and pose. The focus plane is fairly wide in this image and also contains all three annotations. The pose of the two lower annotations do not discount them as AoIs.

Example 9

Finishing the instructions, there is an important scenario that needs special attention. If an image contains closely overlapping annnotations of what are clearly a mother and a foal, then the two annotations should both be marked as AoIs, or both not marked as AoIs. This represents a circumstance where it is quite difficult to obtain clear photographs of each animal individually and also where the association between mother and foal annotations is important for later analysis.

Example 10

This final example shows how complex a mother and foal pair of annotations scenario can present itself. In this example, there are two foals and two corresponding mothers. The two foals can be marked as AoIs and the two mother annotations should therefore be marked as well.

P.S.

We have special instructions for sea turtles. We relax the occlusion percentages from 25% to 50%.

Is significantly (greater than 50%) occluded by other animals or objects
Is significantly (greater than 50%) off the edge of the frame of the image

Example 11

Example 12

Examples 11 and 12 shows an AoI of sea turtles. The image of example 11 shows a clearly visible turtle and is not occluded where example 12 shows a turtle that is only partially occluded. Since we can see at least 50% of either animal, we consider both to be AoIs.

Example 13

The next image, example 13, shows a sea turtle that is too occluded to be considered an AoI as it only shows the head of the turtle. The occlusion threshold for sea turtles should be relaxed to 50% compared to 25%.

P.S.S.

We also have special instructions for whale flukes. We relax the occlusion percentages from 25% to 50%.

Is significantly (greater than 50%) occluded by other animals or objects
Is significantly (greater than 50%) off the edge of the frame of the image

Example 14

Example 14 shows an AoI of a whale fluke, where the body of the whale is completely occluded by the water but the fluke is not occluded and clearly visible.

Example 15

The next image, example 15, shows a whale fluke that is too occluded to be considered an AoI. The occlusion threshold for whale flukes should be relaxed to 50% compared to 25%. We consider the starting point of the whale fluke to be where the forward part of the fin meets the main tail body.

WBIA Web