Skip to content

Annotation Format (JSON)

We recommend to read this chapter if you are going to export annotations.

Supervisely json-based annotation format supports such figures:

  • rectangle

  • line (polyline)

  • polygon

  • point

  • bitmap

Annotations store for each image seperately.

Let's consider the following example:

original image annotated image

Json for project

Each project has predefined objects classes, image tags, object tags. File meta.json contains this information.

{
    "tags_objects": ["vegetable", "fruit", "other"],
    "tags_images": ["breakfast", "lunch", "dinner"],
    "classes": [
        {
            "title": "tomato",
            "shape": "bitmap",
            "color": "#F6FF00"
        },
        {
            "title": "egg",
            "shape": "polygon",
            "color": "#00FF06"
        },
        {
            "title": "bread",
            "shape": "rectangle",
            "color": "#FF0000"
        },
        {
            "title": "basil",
            "shape": "line",
            "color": "#FF00B1"
        },
        {
            "title": "mozzarella",
            "shape": "point",
            "color": "#0030FF"
        }
    ]
}

Fields explanation:

  • "tags_objects" - list of strings - all possible tags that can be assigned to objects

  • "tags_images" - list of strings - all possible tags that can be assigned to images

  • "classes" - list of objects - all possible object classes. Each object has fields:

    • "title" - string - unique identifier of class

    • "shape" - string - one of ["bitmap", "polygon", "rectangle", "line", "point"]

    • "color" - string - hex color code

Json for image

For each image we keep json file with annotations. Let's consider the structure of this file with the following example:

{
    "description": "tasty food",
    "name": "tomatoes-eggs-dish-the-green-plate-51163",
    "size": {
        "width": 2100,
        "height": 1500
    },
    "tags": ["lunch"],
    "objects": [...]
}

Fields explanation:

  • "description" (optional) - used to store some text we want to assign to the image. For example, it can be useful if you are going to build image captioning system
  • "name" - image basename (filename without extension)
  • "size" - is equal to image size. Mostly, it is used to get image size without actual image reading to speed up some data processing steps.
    • "width" - image width in pixels
    • "height" - image height in pixels
  • "tags" - list of strings, that will interpreted as image tags.
  • "objects" - list of objects, which format will be described below

Figures: elements of "objects" list

Rectangle

{
  "description": "",
  "tags": []
  "bitmap": null,
  "classTitle": "bread",
  "points": {
    "exterior": [
      [
        1075,
        298
      ],
      [
        1517,
        681
      ]
    ],
    "interior": []
  }
}

Fields explanation:

  • "description" - string - text description of object

  • "tags" - list of strings - object tags

  • "bitmap" always equals to null

  • "classTitle" - string - title of class. It is used to identify class shape from file meta.json

  • "points" - object with two fields:

    • "exterior" - list of two lists with two numbers (coordinates) [[left, top], [right, bottom]]

    • "interior" - always empty list for this shape

Line

{
    "description": "",
    "tags": [],
    "bitmap": null,
    "classTitle": "basil",
    "points": {
        "exterior": [
            [
                834,
                848
            ],
            [
                941,
                791
            ]
        ],
        "interior": []
    }
}

Fields explanation:

  • "description" (optional) - string - text description of object

  • "tags" - list of strings - object tags

  • "bitmap" always equals to null

  • "classTitle" - string - title of class. It is used to identify class shape from file meta.json

  • "points" - object with two fields:

    • "exterior" - list of poinst (x and y coordinates), e.g. [[x1, y1], [x2, y2], [x3, y3], [x4, y4], ...]

    • "interior" - always empty list for this shape

Polygon

{
    "description": "",
    "tags": ["other"],
    "bitmap": null,
    "classTitle": "egg",
    "points": {
        "exterior": [ 
            [1126, 753], [1123, 775], ..., [1136, 721], [1126, 753]
        ],
        "interior": []
    }
}

Fields explanation:

  • "description" (optional) - string - text description of object

  • "tags" - list of strings - object tags

  • "bitmap" always equals to null

  • "classTitle" - string - title of class. It is used to identify class shape from file meta.json

  • "points" - object with two fields:

    • "exterior" - list of poinst (x and y coordinates), e.g. [[x1, y1], [x2, y2], [x3, y3], [x4, y4], ...]. Object contour is defined with these points

    • "interior" - list of elements, that has the same structure like "exterior" field. In other words, This is the list of polygons, that define object holes.

Point

{
    "description": "",
    "tags": [],
    "bitmap": null,
    "classTitle": "mozzarella",
    "points": {
        "exterior": [
            [
                1004,
                999
            ]
        ],
        "interior": []
    }
}

Fields explanation:

  • "description" (optional) - string - text description of object

  • "tags" - list of strings - object tags

  • "bitmap" always equals to null

  • "classTitle" - string - title of class. It is used to identify class shape from file meta.json

  • "points" - object with two fields:

    • "exterior" - list of one poinst (x and y coordinates)

    • "interior" - always empty list for this shape

Bitmap

{
    "description": "it looks very good",
    "tags": ["vegetable"],
    "bitmap": {
        "origin": [
          1177,
          931
        ],
        "data": "eJwBvwJA/YlQTkcNC ... AEDm2GYAAAJn"
    },
    "classTitle": "tomato",
    "points": {
        "exterior": [],
        "interior": []
    }
}

Fields explanation:

  • "description" - string - text description of object

  • "tags" - list of strings - object tags

  • "bitmap" - object with two fields:

    • "origin" - poinst (x and y coordinates) of the left top corner of bitmap, i.e. the position of the bitmap within image
    • "data" - string - encoded representation of string
  • "classTitle" - string - title of class. It is used to identify class shape from file meta.json

  • "points" - object with two fields:

    • "exterior" - always empty list for this shape

    • "interior" - always empty list for this shape

Few words about "bitmap" -> "data".

You can use this two python methods to convert base64 encoded string to numpy and vice versa.

How to work with bitmaps outside Supervisely

You can use this two python methods to convert base64 encoded string to numpy and vice versa.

def base64_2_mask(s):
    z = zlib.decompress(base64.b64decode(s))
    n = np.fromstring(z, np.uint8)
    mask = cv2.imdecode(n, cv2.IMREAD_UNCHANGED)[:, :, 3].astype(bool)
    return mask

def mask_2_base64(mask):
    img_pil = Image.fromarray(np.array(mask, dtype=np.uint8))
    img_pil.putpalette([0,0,0,255,255,255])
    bytes_io = io.BytesIO()
    img_pil.save(bytes_io, format='PNG', transparency=0, optimize=0)
    bytes = bytes_io.getvalue()
    return base64.b64encode(zlib.compress(bytes)).decode('utf-8')