deel.lip.callbacks module¶
This module contains callbacks that can be added to keras training process.
-
class
deel.lip.callbacks.
CondenseCallback
(on_epoch: bool = True, on_batch: bool = False)¶ Bases:
tensorflow.python.keras.callbacks.Callback
Automatically condense layers of a model on batches/epochs. Condensing a layer consists in overwriting the kernel with the constrained weights. This prevents the explosion/vanishing of values inside the original kernel.
Warning
Overwriting the kernel may disturb the optimizer, especially if it has a non-zero momentum.
- Parameters
on_epoch – if True apply the constraint between epochs
on_batch – if True apply constraints between batches
-
get_config
()¶
-
on_epoch_end
(epoch: int, logs: Optional[Dict[str, float]] = None)¶ Called at the end of an epoch.
Subclasses should override for any actions to run. This function should only be called during TRAIN mode.
- Parameters
epoch – Integer, index of epoch.
logs – Dict, metric results for this training epoch, and for the validation epoch if validation is performed. Validation result keys are prefixed with val_.
-
on_train_batch_end
(batch: int, logs: Optional[Dict[str, float]] = None)¶ Called at the end of a training batch in fit methods.
Subclasses should override for any actions to run.
- Parameters
batch – Integer, index of batch within the current epoch.
logs – Dict. Aggregated metric results up until this batch.
-
class
deel.lip.callbacks.
MonitorCallback
(monitored_layers: Iterable[str], logdir: str, what: str = 'max', on_epoch: bool = True, on_batch: bool = False)¶ Bases:
tensorflow.python.keras.callbacks.Callback
Allow to monitor the singular values of specified layers during training. This analyze the singular values of the original kernel (before reparametrization). Two modes can be chosen: “max” plots the largest singular value over training, while “all” plots the distribution of the singular values over training (series of distribution).
- Parameters
monitored_layers – list of layer name to monitor.
logdir – path to the logging directory.
what – either “max”, which display the largest singular value over the training process, or “all”, which plot the distribution of all singular values.
on_epoch – if True apply the constraint between epochs.
on_batch – if True apply constraints between batches.
-
get_config
()¶
-
on_epoch_end
(epoch, logs=None)¶ Called at the end of an epoch.
Subclasses should override for any actions to run. This function should only be called during TRAIN mode.
- Parameters
epoch – Integer, index of epoch.
logs – Dict, metric results for this training epoch, and for the validation epoch if validation is performed. Validation result keys are prefixed with val_.
-
on_train_batch_end
(batch, logs=None)¶ Called at the end of a training batch in fit methods.
Subclasses should override for any actions to run.
- Parameters
batch – Integer, index of batch within the current epoch.
logs – Dict. Aggregated metric results up until this batch.