deel.lip.normalizers module
This module contains computation function, for Bjorck and spectral normalization. This is done for internal use only.
- deel.lip.normalizers.bjorck_normalization(w, eps=0.001, beta=0.5)
apply Bjorck normalization on w.
- Parameters
w – weight to normalize, in order to work properly, we must have max_eigenval(w) ~= 1
eps – epsilon stopping criterion: norm(wt - wt-1) must be less than eps
beta – beta used in each iteration, must be in the interval ]0, 0.5]
- Returns
the orthonormal weights
- deel.lip.normalizers.reshaped_kernel_orthogonalization(kernel, u, adjustment_coef, eps_spectral=0.001, eps_bjorck=0.001, beta=0.5)
Perform reshaped kernel orthogonalization (RKO) to the kernel given as input. It apply the power method to find the largest singular value and apply the Bjorck algorithm to the rescaled kernel. This greatly improve the stability and and speed convergence of the bjorck algorithm.
- Parameters
kernel – the kernel to orthogonalize
u – the vector used to do the power iteration method
adjustment_coef – the adjustment coefficient as used in convolution
eps_spectral – stopping criterion in spectral algorithm
eps_bjorck – stopping criterion in bjorck algorithm
beta – the beta used in the bjorck algorithm
- Returns: the orthogonalized kernel, the new u, and sigma which is the largest
singular value
- deel.lip.normalizers.spectral_normalization(kernel, u, eps=0.001)
Normalize the kernel to have it’s max eigenvalue == 1.
- Parameters
kernel – the kernel to normalize
u – initialization for the max eigen vector
eps – epsilon stopping criterion: norm(ut - ut-1) must be less than eps
- Returns
the normalized kernel w_bar, it’s shape, the maximum eigen vector, and the maximum singular value