deel.lip.constraints module

This module contains extra constraint objects. These object can be added as params to regular layers.

class deel.lip.constraints.AutoWeightClip(scale=1)

Bases: tensorflow.python.keras.constraints.Constraint

Clips the weights incident to each hidden unit to be inside the range [-c,+c]. With c = 1/sqrt(size(kernel)).

Parameters

scale – scaling factor to increase/decrease clipping value.

get_config()
class deel.lip.constraints.BjorckNormalizer(niter_spectral=3, niter_bjorck=15, u=None)

Bases: deel.lip.constraints.SpectralNormalizer

Ensure that all singular values of the weight matrix equals to 1. Computation based on BjorckNormalizer algorithm. The computation is done in two steps:

  1. reduce the larget singular value to 1, using iterated power method.

  2. increase other singular values to 1, using BjorckNormalizer algorithm.

Parameters

  • niter_spectral – number of iteration to find the maximum singular value.

  • niter_bjorck – number of iteration with BjorckNormalizer algorithm..

  • u – vector used for iterated power method, can be set to None (used for serialization/deserialization purposes).

get_config()
class deel.lip.constraints.FrobeniusNormalizer(**kwargs)

Bases: tensorflow.python.keras.constraints.Constraint

Clips the weights incident to each hidden unit to be inside the range [-c,+c]. With c = 1/norm(kernel).

class deel.lip.constraints.SpectralNormalizer(niter_spectral=3, u=None)

Bases: tensorflow.python.keras.constraints.Constraint

Ensure that the weights matrix have sigma_max == 1 (maximum singular value of the weights matrix).

Parameters

  • niter_spectral – number of iteration to find the maximum singular value.

  • u – vector used for iterated power method, can be set to None (used for serialization/deserialization purposes).

get_config()
class deel.lip.constraints.WeightClip(c=2)

Bases: tensorflow.python.keras.constraints.Constraint

Clips the weights incident to each hidden unit to be inside the range [-c,+c].

Parameters

c – clipping parameter.

get_config()