deel.lip.constraints module

This module contains extra constraint objects. These object can be added as params to regular layers.

class deel.lip.constraints.AutoWeightClipConstraint(scale=1)

Bases: Constraint

Clips the weights incident to each hidden unit to be inside the range [-c,+c]. With c = 1/sqrt(size(kernel)).

Parameters:

scale – scaling factor to increase/decrease clipping value.

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns:

Python dict containing the configuration of the constraint object.

class deel.lip.constraints.FrobeniusConstraint(eps=1e-07)

Bases: Constraint

Constrain the weights by dividing the weight matrix by it’s L2 norm.

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns:

Python dict containing the configuration of the constraint object.

class deel.lip.constraints.SpectralConstraint(k_coef_lip=1.0, eps_spectral=0.001, eps_bjorck=0.001, beta_bjorck=0.5, u=None)

Bases: Constraint

Ensure that all singular values of the weight matrix equals to 1. Computation based on Bjorck algorithm. The computation is done in two steps:

  1. reduce the larget singular value to k_coef_lip, using iterate power method.

  2. increase other singular values to k_coef_lip, using bjorck algorithm.

Parameters:

  • k_coef_lip – lipschitz coefficient of the weight matrix

  • eps_spectral – stopping criterion for the iterative power algorithm.

  • eps_bjorck – stopping criterion Bjorck algorithm.

  • beta_bjorck – beta parameter in bjorck algorithm.

  • u – vector used for iterated power method, can be set to None (used for serialization/deserialization purposes).

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns:

Python dict containing the configuration of the constraint object.

class deel.lip.constraints.WeightClipConstraint(c=2)

Bases: Constraint

Clips the weights incident to each hidden unit to be inside the range [-c,+c].

Parameters:

c – clipping parameter.

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns:

Python dict containing the configuration of the constraint object.