deel.lip.callbacks module

This module contains callbacks that can be added to keras training process.

class deel.lip.callbacks.CondenseCallback(on_epoch: bool = True, on_batch: bool = False)

Bases: tensorflow.python.keras.callbacks.Callback

Automatically condense layers of a model on batches/epochs. Condensing a layer consists in overwriting the kernel with the constrained weights. This prevents the explosion/vanishing of values inside the original kernel.

Warning

Overwriting the kernel may disturb the optimizer, especially if it has a non-zero momentum.

Parameters

  • on_epoch – if True apply the constraint between epochs

  • on_batch – if True apply constraints between batches

get_config()
on_epoch_end(epoch: int, logs: Optional[Dict[str, float]] = None)

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Parameters

  • epoch – integer, index of epoch.

  • logs – dict, metric results for this training epoch, and for the validation epoch if validation is performed. Validation result keys are prefixed with val_.

on_train_batch_end(batch: int, logs: Optional[Dict[str, float]] = None)

Called at the end of a training batch in fit methods.

Subclasses should override for any actions to run.

Parameters

  • batch – integer, index of batch within the current epoch.

  • logs – dict. Metric results for this batch.

class deel.lip.callbacks.MonitorCallback(monitored_layers: Iterable[str], logdir: str, what: str = 'max', on_epoch: bool = True, on_batch: bool = False)

Bases: tensorflow.python.keras.callbacks.Callback

Allow to monitor the singular values of specified layers during training. This analyze the singular values of the original kernel (before reparametrization). Two modes can be chosen: “max” plots the largest singular value over training, while “all” plots the distribution of the singular values over training (series of distribution).

Parameters

  • monitored_layers – list of layer name to monitor.

  • logdir – path to the logging directory.

  • what – either “max”, which display the largest singular value over the training process, or “all”, which plot the distribution of all singular values.

  • on_epoch – if True apply the constraint between epochs.

  • on_batch – if True apply constraints between batches.

get_config()
on_epoch_end(epoch, logs=None)

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Parameters

  • epoch – integer, index of epoch.

  • logs – dict, metric results for this training epoch, and for the validation epoch if validation is performed. Validation result keys are prefixed with val_.

on_train_batch_end(batch, logs=None)

Called at the end of a training batch in fit methods.

Subclasses should override for any actions to run.

Parameters

  • batch – integer, index of batch within the current epoch.

  • logs – dict. Metric results for this batch.