deel.lip.constraints module

This module contains extra constraint objects. These object can be added as params to regular layers.

class deel.lip.constraints.AutoWeightClipConstraint(scale=1)

Bases: keras.constraints.Constraint

Clips the weights incident to each hidden unit to be inside the range [-c,+c]. With c = 1/sqrt(size(kernel)).

Parameters

scale – scaling factor to increase/decrease clipping value.

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns

Python dict containing the configuration of the constraint object.

class deel.lip.constraints.FrobeniusConstraint(eps=1e-07)

Bases: keras.constraints.Constraint

Constrain the weights by dividing the weight matrix by it’s L2 norm.

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns

Python dict containing the configuration of the constraint object.

class deel.lip.constraints.SpectralConstraint(k_coef_lip=1.0, niter_spectral=3, niter_bjorck=15, u=None)

Bases: keras.constraints.Constraint

Ensure that all singular values of the weight matrix equals to 1. Computation based on Bjorck algorithm. The computation is done in two steps:

  1. reduce the larget singular value to k_coef_lip, using iterate power method.

  2. increase other singular values to k_coef_lip, using bjorck algorithm.

Parameters

  • k_coef_lip – lipschitz coefficient of the weight matrix

  • niter_spectral – number of iteration to find the maximum singular value.

  • niter_bjorck – number of iteration with Bjorck algorithm..

  • u – vector used for iterated power method, can be set to None (used for serialization/deserialization purposes).

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns

Python dict containing the configuration of the constraint object.

class deel.lip.constraints.WeightClipConstraint(c=2)

Bases: keras.constraints.Constraint

Clips the weights incident to each hidden unit to be inside the range [-c,+c].

Parameters

c – clipping parameter.

get_config()

Returns a Python dict of the object config.

A constraint config is a Python dictionary (JSON-serializable) that can be used to reinstantiate the same object.

Returns

Python dict containing the configuration of the constraint object.