Inference

Supervisely supports a few inference modes. These modes differ from each other in what image area will be fed to the NN.
Full image
1
{
2
  "model": {
3
    "gpu_device": 0
4
  },
5
  "mode": {
6
    "name": "full_image",
7
    "model_classes": {
8
      "save_classes": "__all__",
9
      "add_suffix": "_unet"
10
    }
11
  }
12
}
Copied!
model - group contains unique settings for each model:
gpu_device - device to use for inference. Right now we support only single GPU.
mode - group contains all mode settings:
name - mode name defines how to apply NN to image (e.g. full_image - apply NN to full image)
model_classes - which classes will be used, e.g. NN produces 80 classes and you are going to use only a few and ignore others. In that case you should set save_classes field with the list of interested class names. add_suffix string will be added to new class to prevent having same class names as already exisiting classes in the project. If you are going to use all model classes just set "save_classes": "__all__".
For example you have a project where each person is segmented (class 'person'). And you are going to apply a new model to this project to also segment humans, but you want to save both annotations: original and new predictions. Here is the appropriate config:
1
{
2
  "model": {
3
    "gpu_device": 0
4
  },
5
  "mode": {
6
    "name": "full_image",
7
    "model_classes": {
8
      "save_classes": ["person"],
9
      "add_suffix": "_unet"
10
    }
11
  }
12
}
Copied!
All new annotations after NN inference will have class person_unet.
Objects
You can apply NN to image areas that are defined by object bounding boxes. Instead of applying NN to the whole image, it will be applied to a few image parts. It may be very usefull when you are going to apply a few NNs in a sequence.
For example, you are going to segment person instances. But you don't want to use Mask-RCNN (due to low segmentation quality near object edges). Instead you apply Faster-RCNN to detect all persons, and then apply your custom UNet model to segment each person.
1
{
2
  "model": {
3
    "gpu_device": 0
4
  },
5
  "mode": {
6
    "name": "bboxes",
7
    "from_classes": ["person_bbox"],
8
    "padding": {
9
      "left": "5%",
10
      "top": "5%",
11
      "right": "5%",
12
      "bottom": "5%"
13
    },
14
    "save": true,
15
    "add_suffix": "_input_bbox",
16
    "model_classes": {
17
      "save_classes": ["person_bbox"],
18
      "add_suffix": "_unet"
19
    }
20
  }
21
}
Copied!
Many of the fields are similar to those that were already described in the "Full image" chapter. Here is the explanation for the new ones.
mode - group contains all mode settings:
name: bboxes - apply NN to bounding boxes of specified objects
from_classes - list of classes. All objects of these classes will be used for inference: 1. get object bounding box, 2. apply padding to bounding box (slightly encrease its size), 3. feed the image area that is defined with the bounding box to the NN.
padding - how to increase input bounding box. Possible values examples: "5%" or "15px"
save - save input bounding box after inference if set as true
add_suffix - suffix for input bounding box
ROI
This mode allow to use only a part of the input image.
1
{
2
  "model": {
3
    "gpu_device": 0
4
  },
5
  "mode": {
6
    "name": "roi",
7
    "bounds": {
8
      "left": "10%",
9
      "top": "50%",
10
      "right": "25%",
11
      "bottom": "0%"
12
    },
13
    "save": false,
14
    "class_name": "inference_roi",
15
    "model_classes": {
16
      "save_classes": "__all__",
17
      "add_suffix": "_unet"
18
    }
19
  }
20
}
Copied!
Many of the fields are similar to those that were already described in the "Full image" chapter. Here is the explanation for the new ones.
mode - group contains all mode settings:
name: roi - apply NN to the image part defined by bounds
bounds - defines the relevant part of the input image. Below is the graphical explanation.
save - save input bounding box after inference if set as true
class_name - save input bounding box with given class name if the above option save is true.
Sliding window
The mode allows to perform inference on (possibly overlapping) parts of the image. It is useful for getting inference results on large or relatively large images without splitting them and stitching.
Sliding window of fixed size moves on the source image and covers it entirely. In each position the model is applied to the part of the image bounded by the window. After processing the whole image results from different positions are aggregated to get the final prediction.
Now we support the modes for image segmentation and detection models only.
Sliding window for segmentation
Segmentation results are aggregated by averaging output probabilities.
1
{
2
  "model": {
3
    "gpu_device": 0
4
  },
5
  "mode": {
6
    "name": "sliding_window",
7
    "window": {
8
      "width": 1000,
9
      "height": 750
10
    },
11
    "min_overlap": {
12
      "x": 600,
13
      "y": 600
14
    },
15
    "save": true,
16
    "class_name": "sl_window",
17
    "model_classes": {
18
      "save_classes": "__all__",
19
      "add_suffix": "_unet"
20
    }
21
  }
22
}
Copied!
mode - group contains all mode settings:
name: sliding_window - selects the sliding window mode for segmentation models
window - defines fixed size of sliding window in pixels; crop with the size will be passed to network (note that in different NN implementations it would be resized to the fixed network input after that )
min_overlap - defines minimal overlap in pixels between different positions of the sliding window; real overlap may be greater to cover the source image entirely
save - save or not bounds of each sliding window position
class_name - class name for the sliding window bounds
Sliding window for detection
Detection results can be merged by special postprocessing algorithm (described below).
1
{
2
  "model": {
3
    "gpu_device": 0,
4
    "confidence_tag_name": "confidence"
5
  },
6
  "mode": {
7
    "name": "sliding_window_det",
8
    "window": {
9
      "width": 416,
10
      "height": 416
11
    },
12
    "min_overlap": {
13
      "x": 100,
14
      "y": 100
15
    },
16
    "save": true,
17
    "class_name": "sl_window",
18
    "nms_after": {
19
      "enable": true,
20
      "iou_threshold": 0.4,
21
      "confidence_tag_name": "confidence"
22
    },
23
    "model_classes": {
24
      "save_classes": "__all__",
25
      "add_suffix": "_yolo"
26
    }
27
  }
28
}
Copied!
Many of the fields are similar to those that were already described in above chapters. Here is the explanation for new ones.
model - group contains unique settings for each model:
confidence_tag_name - name of the confidence tag for predicted bound boxes.
mode - group contains all mode settings:
name: sliding_window_det - selects the sliding window mode for detection models
nms_after - defines how to apply Non-Maximum-Suppression postprocessing algorithm:
enable - set true to enable postprocessing, or false elsewise.
iou_threshold - minimal Intersection over Union value for merging 2 intersected boxes with same classes.
confidence_tag_name - name of confidence tag. (Most always be equal value of model -> confidence_tag_name )
Training
Next - Neural Networks
Examples
Last modified 2yr ago
Copy link
Contents
Sliding window for segmentation
Sliding window for detection