deel.lip.normalizers module
This module contains computation function, for Bjorck and spectral normalization. This is done for internal use only.
- deel.lip.normalizers.bjorck_normalization(w, niter=15, beta=0.25)
apply Bjorck normalization on w.
- Parameters
w – weight to normalize, in order to work properly, we must have max_eigenval(w) ~= 1
niter – number of iterations
beta – beta used in each iteration, must be in the interval ]0, 0.5]
- Returns
the orthonormal weights
- deel.lip.normalizers.reshaped_kernel_orthogonalization(kernel, u, adjustment_coef, niter_spectral=3, niter_bjorck=15, beta=0.25)
Perform reshaped kernel orthogonalization (RKO) to the kernel given as input. It apply the power method to find the largest singular value and apply the Bjorck algorithm to the rescaled kernel. This greatly improve the stability and and speed convergence of the bjorck algorithm.
- Parameters
kernel – the kernel to orthogonalize
u – the vector used to do the power iteration method
adjustment_coef – the adjustment coefficient as used in convolution
niter_spectral – number of iteration to do in spectral algorithm
niter_bjorck – iteration used for bjorck algorithm
beta – the beta used in the bjorck algorithm
- Returns: the orthogonalized kernel, the new u, and sigma which is the largest
singular value
- deel.lip.normalizers.spectral_normalization(kernel, u, niter=3)
Normalize the kernel to have it’s max eigenvalue == 1.
- Parameters
kernel – the kernel to normalize
u – initialization for the max eigen vector
niter – number of iteration
- Returns
the normalized kernel w_bar, it’s shape, the maximum eigen vector, and the maximum eigen value