Losses¶

Loss modules are very important in the training of neural networks, because as they guide and shape the learning process by minimizing the loss. They can affect the speed of convergence during training and the overall robustness of the model.

However, loss functions can vary depending on the task, especially in tasks like detection and segmentation, where specialized loss functions are required. Therefore, NetsPresso Trainer provides a predefined variety of loss modules, designed for flexible use across different tasks. Users can seamlessly apply the appropriate loss function to their desired task through simple configuration settings.

Supporting loss modules¶

The currently supported methods in NetsPresso Trainer are as follows. Since techniques are adapted from pre-existing codes, hence most of the parameters remain unchanged. We note that most of these parameter descriptions are derived from original implementations.

We appreciate all the original code owners and we also do our best to make other values.

CrossEntropyLoss¶

Cross entropy loss. This loss follows the CrossEntropyLoss in torch library.

Field	Description
`criterion`	(str) Criterion must be "cross_entropy" to use `CrossEntropyLoss`.
`label_smoothing`	(float) Specifies the amount of smoothing when computing the loss, where 0.0 means no smoothing.
`weight`	(float) Weight for this cross entropy loss.

Cross entropy loss example

model:
  losses:
    - criterion: cross_entropy
      label_smoothing: 0.1
      weight: ~

SigmoidFocalLoss¶

Focal loss based on Focal loss for dense object detections. This loss follows the sigmoid_focal_loss in the torch library.

Field	Description
`criterion`	(str) Criterion must be "focal_loss" to use `SigmoidFocalLoss`.
`alpha`	(float) Balancing parameter alpha for focal loss.
`gamma`	(float) Focusing parameter gamma for focal loss.
`weight`	(float) Weight for this focal loss.

Focal loss example

model:
  losses:
    - criterion: focal_loss
      alpha: 0.25
      gamma: 2.0
      weight: ~

YOLOXLoss¶

Loss module for AnchorFreeDecoupledHead. This loss follows the YOLOX implementation.

Field	Description
`criterion`	(str) Criterion must be "yolox_loss" to use `YOLOXLoss`.
`weight`	(float) Weight for this YOLOX loss.
`l1_activate_epoch`	(int) Activate l1 loss at `l1_activate_epoch` epoch.

YOLOX loss example

model:
  losses:
    - criterion: yolox_loss
      weight: ~
      l1_activate_epoch: 1

RetinaNetLoss¶

Loss module for AnchorDecoupledHead. This loss follows torchvision implementation, it contains classification loss via focal loss and box regression loss via L1 loss.

Field	Description
`criterion`	(str) Criterion must be "retinanet_loss" to use `RetinaNetLoss`.
`weight`	(float) Weight for this RetinaNet loss.

RetinaNetLoss loss example

model:
  losses:
    - criterion: retinanet_loss
      weight: ~

PIDNetLoss¶

Loss module for PIDNet. This loss follows official implementation repository.

Field	Description
`criterion`	(str) Criterion must be "pidnet_loss" to use `PIDNetLoss`.
`weight`	(float) Weight for this PIDNet loss.
`ignore_index`	(int) A target value that is ignored and does not contribute to the input gradient.

PIDNetLoss loss example

model:
  losses:
    - criterion: pidnet_loss
      weight: ~
      ignore_index: 255