Objects

Supported Shapes
Supervisely Annotation Format supports the following figures:
point
rectangle
polygon
line / polyline 
bitmap
keypoint structures 
cuboid
Coordinate System
For two-dimensional mediums (images and videos) we use the following coordinate system (it's similar to a two-dimensional NumPy coordinate system):
coordinate system
All numberical values are provided in pixels.
General Fields
When generating JSON annotation files, we assign each figure a mix of general fields and fileds unique for each geometric shape. Some of the general fields are optional: the system generates them automatically when the data is uploaded/first created. This means that these fields can be omitted during manual annotation.
Optional fields:
1
"id": 503051990,
2
"classId": 1693352,
3
"labelerLogin": "alexxx",
4
"createdAt": "2020-08-22T09:32:48.010Z",
5
"updatedAt": "2020-08-22T09:33:08.926Z".
Copied!
Fields definitions:
id - unique identifier of the current object
classId - unique class identifier of the current object
labelerLogin - string - the name of user who created the current figure
createdAt - string - date and time of figure creation
updatedAt - string - date and time of the last figure update
Point
Example:
point example
Json format for this shape:
1
{
2
    "id": 503051990,
3
    "classId": 1693352,
4
    "labelerLogin": "alexxx",
5
    "createdAt": "2020-08-22T09:32:48.010Z",
6
    "updatedAt": "2020-08-22T09:33:08.926Z",
7
    "description": "",
8
    "geometryType": "point",
9
    "tags": [],
10
    "classTitle": "point",
11
    "points": {
12
        "exterior": [
13
            [
14
                1334,
15
                907
16
            ]
17
        ],
18
        "interior": []
19
    }
20
}
Copied!
Fields definitions:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "point" - class shape
tags - list of tags assigned to the current object
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
points - object with two fields:
exterior - list of  2 values for coordinates (x and y in that order) for every figure
interior - always an empty field for this type of figure
Rectangle
Example:
rectangle example
Json format for this figure:
1
{
2
  "id": 283051572,
3
  "classId": 1692857,
4
  "labelerLogin": "max",
5
  "createdAt": "2020-08-22T09:32:48.010Z",
6
  "updatedAt": "2020-08-22T09:33:08.926Z",
7
  "description": "",
8
  "geometryType": "rectangle",
9
  "tags": [],
10
  "classTitle": "person_bbox",
11
  "points": {
12
    "exterior": [
13
      [
14
        533,
15
        63
16
      ],
17
      [
18
        800,
19
        830
20
      ]
21
    ],
22
    "interior": []
23
  }
24
}
Copied!
Fields definitions:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "rectangle" - class shape
tags - list of tags assigned to the current object
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
points - object with two fields:
exterior - list of two lists, each containing two coordinates (x and y in that order), with the following structure: [[left, top], [right, bottom]]
interior - always an empty list for this type of figure
Polygon (without holes)
Example:
polygon example
1
{
2
    "id": 503004154,
3
    "classId": 1693021,
4
    "labelerLogin": "alexxx",
5
    "createdAt": "2020-08-21T15:15:28.092Z",
6
    "updatedAt": "2020-08-21T15:15:37.687Z",
7
    "description": "",
8
    "geometryType": "polygon",
9
    "tags": [],
10
    "classTitle": "triangle",
11
    "points": {
12
        "exterior": [
13
            [
14
                730,
15
                2104
16
            ],
17
            [
18
                2479,
19
                402
20
            ],
21
            [
22
                3746,
23
                1646
24
            ]
25
        ],
26
        "interior": []
27
    }
28
}
Copied!
Fields definitions:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "polygon" - class shape
tags - list of tags assigned to the curent object
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
points - object with two fields:
exterior - list of points [point1, point2, point3, etc ...] where each point is a list of two numbers (coordinates) [col, row]
interior - list of elements with the same structure as the "exterior" field. In other words, this is the list of polygons that define object holes. For polygons without holes in them, this field is empty
Polygon (without holes)
Example:
polygon example
1
{
2
    "id": 503004154,
3
    "classId": 1693021,
4
    "labelerLogin": "alexxx",
5
    "createdAt": "2020-08-21T15:15:28.092Z",
6
    "updatedAt": "2020-08-21T16:06:11.461Z",
7
    "description": "",
8
    "geometryType": "polygon",
9
    "tags": [],
10
    "classTitle": "triangle_hole",
11
    "points": {
12
        "exterior": [
13
            [
14
                730,
15
                2104
16
            ],
17
            [
18
                2479,
19
                402
20
            ],
21
            [
22
                3746,
23
                1646
24
            ]
25
        ],
26
        "interior": [
27
        [
28
            [
29
                1907,
30
                1255
31
            ],
32
            [
33
                2468,
34
                875
35
            ],
36
            [
37
                2679,
38
                1577
39
            ]
40
        ]
41
        ]
42
    }
43
}
Copied!
Fields definitions:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "polygon" - class shape
tags - list of tags assigned to the curent object
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
points - object with two fields:
exterior - list of points [point1, point2, point3, etc ...] where each point is a list of two numbers (coordinates) [col, row]
interior - list of elements with the same structure as the "exterior" field. In other words, this is the list of polygons that define object holes. 
Polyline
Example:
polyline example
1
{
2
    "id": 503049791,
3
    "classId": 1693340,
4
    "labelerLogin": "alexxx",
5
    "createdAt": "2020-08-22T08:39:29.386Z",
6
    "updatedAt": "2020-08-22T08:39:34.802Z",
7
    "description": "",
8
    "geometryType": "line",
9
    "tags": [],
10
    "classTitle": "line",
11
    "points": {
12
        "exterior": [
13
            [
14
                211,
15
                2266
16
            ],
17
            [
18
                1208,
19
                1310
20
            ],
21
            [
22
                369,
23
                981
24
            ]
25
        ],
26
        "interior": []
27
    }
28
}
Copied!
Fields definitions:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "line" - class shape
tags - list of tags assigned to the current object 
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
points - object with two fields:
exterior - list of points [point1, point2, point3, etc ...] where each point is a list of two numbers (coordinates) [col, row]
interior - always an empty list for this type of figure
Bitmap
Bitmap is a figure that is described by a point of "origin"(upper left corner), which defines the location of the bitmap within the image and a "data" - Boolean matrix encoded into a string, which defines each pixel of the bitmap.
Example:
bitmap example
1
{
2
    "id": 497489556,
3
    "classId": 1661459,
4
    "labelerLogin": "alexxx",
5
    "createdAt": "2020-07-24T07:30:39.202Z",
6
    "updatedAt": "2020-07-24T07:41:12.753Z",
7
    "description": "",
8
    "geometryType": "bitmap",
9
    "tags": [],
10
    "classTitle": "person",
11
    "bitmap": {
12
        "data": "eJwB ... kUnW",
13
        "origin": [
14
                535,
15
                66
16
            ]
17
    }
18
}
Copied!
Fields description:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "bitmap" - class shape
tags - list of tags assigned to the current object 
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
bitmap - object with two fields:
origin  - points (x and y coordinates) of the top left corner of the bitmap, i.e. the position of the bitmap within the image
data  - string - encoded representation of a string
A few words about bitmap -> data. You can use these two python methods to convert a base64 encoded string to numpy and vice versa.
1
def base64_2_mask(s):
2
    z = zlib.decompress(base64.b64decode(s))
3
    n = np.fromstring(z, np.uint8)
4
    mask = cv2.imdecode(n, cv2.IMREAD_UNCHANGED)[:, :, 3].astype(bool)
5
    return mask
6
​
7
def mask_2_base64(mask):
8
    img_pil = Image.fromarray(np.array(mask, dtype=np.uint8))
9
    img_pil.putpalette([0,0,0,255,255,255])
10
    bytes_io = io.BytesIO()
11
    img_pil.save(bytes_io, format='PNG', transparency=0, optimize=0)
12
    bytes = bytes_io.getvalue()
13
    return base64.b64encode(zlib.compress(bytes)).decode('utf-8')
Copied!
Example:
1
import numpy as np
2
import cv2, zlib, base64, io
3
from PIL import Image
4
​
5
def base64_2_mask(s):
6
    z = zlib.decompress(base64.b64decode(s))
7
    n = np.fromstring(z, np.uint8)
8
    mask = cv2.imdecode(n, cv2.IMREAD_UNCHANGED)[:, :, 3].astype(bool)
9
    return mask
10
​
11
def mask_2_base64(mask):
12
    img_pil = Image.fromarray(np.array(mask, dtype=np.uint8))
13
    img_pil.putpalette([0,0,0,255,255,255])
14
    bytes_io = io.BytesIO()
15
    img_pil.save(bytes_io, format='PNG', transparency=0, optimize=0)
16
    bytes = bytes_io.getvalue()
17
    return base64.b64encode(zlib.compress(bytes)).decode('utf-8')
18
​
19
example_np_bool = np.ones((3, 3), dtype=bool)
20
example_np_bool[1][1] = False
21
example_np_bool[1][2] = False
22
print(example_np_bool)
23
encoded_string = mask_2_base64(example_np_bool)
24
print(encoded_string)
25
print(base64_2_mask(encoded_string))
Copied!
Program output after executing the code:
1
[[ True  True  True]
2
 [ True False False]
3
 [ True  True  True]]
4
​
5
'eJzrDPBz5+WS4mJgYOD19HAJAtLMIMwIInOeqf8BUmwBPiGuQPr///9Lb86/C2QxlgT5BTM4PLuRBuTwebo4hlTMSa44cOHAB6DqY0yORgq8YkAZBk9XP5d1TglNANAFGzA='
6
​
7
[[ True  True  True]
8
 [ True False False]
9
 [ True  True  True]]
Copied!
Keypoint structure
Keypoint structures consist of vertices (also called nodes or points) which are connected by edges (also called links or lines).
Example:
key_point_structurebitmap example
1
{
2
    "id": 503055304,
3
    "classId": 1693357,
4
    "description": "",
5
    "geometryType": "graph",
6
    "labelerLogin": "alexxx",
7
    "createdAt": "2020-08-22T10:50:28.336Z",
8
    "updatedAt": "2020-08-22T10:53:57.760Z",
9
    "tags": [],
10
    "classTitle": "graph",
11
    "nodes": {
12
        "8e20c830-ee86-450f-9d21-833eec53e3c5": {
13
            "loc": [
14
                1017,
15
                1556
16
            ]
17
        },
18
        "bf89e248-7b3b-4732-888a-99d3369fbb2f": {
19
            "loc": [
20
                1024,
21
                394
22
            ]
23
        },
24
        "66502c5b-8d98-492c-bb48-8ce7c4487038": {
25
            "loc": [
26
                1026,
27
                738
28
            ]
29
        },
30
        "56517c2a-6053-442a-9af2-bd6f29bae987": {
31
            "loc": [
32
                668,
33
                574
34
            ]
35
        },
36
        "7a40d5f7-bcc8-4e2f-bf3b-3e52d39c4206": {
37
            "loc": [
38
                1388,
39
                549
40
            ]
41
                }
42
    }
43
}
Copied!
Fields definitions:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "graph" - class shape
tags - list of tags assigned to the current object 
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
nodes - is a dictionary, where keys denote the names of the graph vertices and values in a dictionary, and where values denote location of a node on image
loc - list of single points  (x and y coordinates) of a vertice
Cuboids (2D annotation)
Example:
cuboid 2d example
1
{
2
  "description": "",
3
  "tags": [],
4
  "classTitle": "Cuboid",
5
  "faces": [
6
    [
7
      0,
8
      1,
9
      2,
10
      3
11
    ],
12
    [
13
      0,
14
      4,
15
      5,
16
      1
17
    ],
18
    [
19
      1,
20
      5,
21
      6,
22
      2
23
    ]
24
  ],
25
  "points": [
26
    [
27
      277,
28
      273
29
    ],
30
    [
31
      840,
32
      273
33
    ],
34
    [
35
      840,
36
      690
37
    ],
38
    [
39
      277,
40
      690
41
    ],
42
    [
43
      688,
44
      168
45
    ],
46
    [
47
      1200,
48
      168
49
    ],
50
    [
51
      1200,
52
      522
53
    ]
54
  ]
55
}
Copied!
Fields definitions:
Optional fields id, classId, labelerLogin, createdAt, updatedAt are described above​
description - string - text description (optional)
geometryType: "graph" - class shape
tags - list of tags assigned to the current object 
classTitle - string - the title of the current class. It's used to identify the corresponding class shape from the meta.json file 
points - an array of points that form the cuboid. There are always 7 points in a cuboid. Each Point is presented as an array of X and Y coordinates, i.e. [277, 690] means X is 277 and Y is 690, calculating from the top left corner of the image.
faces - an array of faces that indicates how points from the points array are connected. There are always 3 faces in a cuboid. In the example above, you can see that face number 3 that consists of points 1, 2, 5, 6 with coordinates [840, 273], [840, 690], [1200, 168], [1200, 522]. Check the image below: