deel.lip.losses module

This module contains losses used in wasserstein distance estimation. See https://arxiv.org/abs/2006.06520 for more information.

deel.lip.losses.HKR_loss(alpha, min_margin=1)

Wasserstein loss with a regularization param based on hinge loss.

\[\inf_{f \in Lip_1(\Omega)} \underset{\textbf{x} \sim P_-}{\mathbb{E}} \left[f(\textbf{x} )\right] - \underset{\textbf{x} \sim P_+} {\mathbb{E}} \left[f(\textbf{x} )\right] + \alpha \underset{\textbf{x}}{\mathbb{E}} \left(\text{min_margin} -Yf(\textbf{x})\right)_+\]
Parameters

  • alpha – regularization factor

  • min_margin – minimal margin ( see hinge_margin_loss )

  • term of the loss. In order to be consistent (Kantorovich-rubinstein) –

  • hinge and KR (between) –

  • first label must yield the positve class (the) –

  • the second yields negative class. (while) –

Returns

a function that compute the regularized Wasserstein loss

deel.lip.losses.HKR_multiclass_loss(alpha=0.0, min_margin=1)

The multiclass version of HKR. This is done by computing the HKR term over each class and averaging the results.

Parameters

  • alpha – regularization factor

  • min_margin – minimal margin ( see Hinge_multiclass_loss )

Returns

Callable, the function to compute HKR loss #Note y_true has to be one hot encoded

deel.lip.losses.Hinge_multiclass_loss(min_margin=1)

Loss to estimate the Hinge loss in a multiclass setup. It compute the elementwise hinge term. Note that this formulation differs from the one commonly found in tensorflow/pytorch (with marximise the difference between the two largest logits). This formulation is consistent with the binary classification loss used in a multiclass fashion. Note y_true should be one hot encoded. labels in (1,0)

Returns

Callable, the function to compute multiclass Hinge loss #Note y_true has to be one hot encoded

deel.lip.losses.KR_loss()

Loss to estimate wasserstein-1 distance using Kantorovich-Rubinstein duality. The Kantorovich-Rubinstein duality is formulated as following:

\[W_1(\mu, \nu) = \sup_{f \in Lip_1(\Omega)} \underset{\textbf{x} \sim \mu}{\mathbb{E}} \left[f(\textbf{x} )\right] - \underset{\textbf{x} \sim \nu}{\mathbb{E}} \left[f(\textbf{x} )\right]\]

Where mu and nu stands for the two distributions, the distribution where the label is 1 and the rest.

Returns

Callable, the function to compute Wasserstein loss

deel.lip.losses.KR_multiclass_loss()

Loss to estimate average of W1 distance using Kantorovich-Rubinstein duality over outputs. Note y_true should be one hot encoding (labels being 1s and 0s ). In this multiclass setup thr KR term is computed for each class and then averaged.

Returns

Callable, the function to compute Wasserstein multiclass loss. #Note y_true has to be one hot encoded

deel.lip.losses.MultiMarginLoss(min_margin=1)
Compute the mean hinge margin loss for multi class (equivalent to Pytorch

multi_margin_loss)

Parameters

  • min_margin – the minimal margin to enforce.

  • has to be to_categorical (y_true) –

Returns

Callable, the function to compute multi margin loss

deel.lip.losses.hinge_margin_loss(min_margin=1)

Compute the hinge margin loss.

\[\underset{\textbf{x}}{\mathbb{E}} \left(\text{min_margin} -Yf(\textbf{x})\right)_+\]
Parameters

min_margin – the minimal margin to enforce.

Returns

a function that compute the hinge loss.

deel.lip.losses.neg_KR_loss()

Loss to compute the negative wasserstein-1 distance using Kantorovich-Rubinstein duality. This allows the maximisation of the term using conventional optimizer.

Returns

Callable, the function to compute negative Wasserstein loss