Given a symmetric $A \in \mathbb{R}^{n\times n}$, the Jacobi algorithm produces a sequence of orthogonally similar matrices $A_{k+1} = J_{k}^{T} A_{k} J_{k}$ for $k = 1,2,\dots$, $A_{1} = A$, and $J_{k}$ are the carefully constructed Jacobi rotations. In fact, the Jacobi rotation is the same as the Givens rotation, but we give the credit to the inventor, Carl G. J. Jacobi.

We now describe the $k$ th step of the Jacobi algorithm. We first choose a pivot $(i,j)$, and the task of this step is to eliminate $a_{ij}^{(k)}$ and $a_{ji}^{(k)}$. Let us now focused on the operations on the $(i,j)$ plane of iterates, with \begin{equation}\notag A_{k}([i,j],[i,j]) = \begin{bmatrix} \alpha & \gamma \ \gamma & \beta \end{bmatrix}. \end{equation} Then, the Jacobi algorithm will produce \begin{equation}\label{eq1} A_{k+1}([i,j],[i,j]) = \begin{bmatrix} \widetilde{\alpha} & 0 \ 0 & \widetilde{\beta} \end{bmatrix} = \begin{bmatrix} c & s \ -s & c \end{bmatrix}^T \begin{bmatrix} \alpha & \gamma \ \gamma & \beta \end{bmatrix} \begin{bmatrix} c & s \ -s & c \end{bmatrix}, \end{equation} with specially chosen $c$ and $s$. Rutishauser in /Handbook for Automatic Computation/ provided a nice way to compute the Jacobi rotation. He first computes a parameter $t$ by \begin{equation}\notag \displaystyle\zeta = \frac{\beta - \alpha}{2\gamma}, \quad t = {\rm sign}(\zeta)/(|\zeta| + \sqrt{1+\zeta^{2}}), \end{equation} with ${\rm sign}(0) = -1$, and then we have \begin{equation}\notag c = 1/\sqrt{1+t^{2}}, \quad s = ct. \end{equation}

Convergence in general

Define ${\rm off}(A) = \sqrt{\sum_{i > j}^{} a_{ij}^{2}} = |A|{F}^{2} - \sum{k=1}^{n}a_{kk}^{2}$. Since the Jacobi rotation is orthogonal, we have $\widetilde{\alpha}^{2} + \widetilde{\beta}^{2} =
\alpha^{2} + \beta^{2} + 2\gamma^{2}$. More importantly, we have ${\rm off}(A_{k+1})^{2} = {\rm off}(A_{k})^{2} - 2\gamma^{2}$. Although later steps of the Jacobi algorithm can make $\gamma$ to become nonzero again, the above relation ensures the off-diagonal entries are decreasing overall.

Pivot strategies

Classical Jacobi algorithm

Jacobi in his original paper choose the largest in magnitude off-diagonal entries as the pivot element. The Jacobi algorithm with such choice is called the /classical Jacobi algorithm/. It has been proved by Jacobi that, the classical Jacobi algorithm converges linearly, \begin{equation}\notag \displaystyle {\rm off}(A_{k+1}) \le \sqrt{1-1/N} {\rm off}(A), \qquad N = \frac{n(n-1)}{2}. \end{equation} We will call $N$ iterations as a sweep. Later, its local quadratic convergence has been proved by Henrici (/J. Soc. Indust. Appl. Math./ 6, 144–162, 1958), which said for large enough $i$, we have \begin{equation}\notag {\rm off}(A_{i+N}) \le c \cdot {\rm off}(A_{i})^{2}, \end{equation} for some constant $c$. This method does not attractive in practice, due to the searching time is more than the performing time ($O(n^{2})$ flops versus $O(n)$ flops)!

Cyclic Jacobi algorithm

Gregory (/Math. Comp./ 7, 215-220, 1953) discussed choosing $a_{ij}$ in a row-by-row fashion, \begin{equation}\notag (i,j) = (1,2),\dots,(1,n),(2,3),\dots,(n-1,n),(1,2),\dots. \end{equation} This type of the Jacobi algorithm is known as the /cyclic Jacobi algorithm/. One requires the magnitude of the rotation angle to be less than $\pi/4$ in order to achieve convergence. In fact, the difficulty of proving the convergence of the cyclic Jacobi algorithm is the possibility that the larger off-diagonal elements are always moving around /ahead/ of the current element. The cyclic Jacobi algorithm is also asymptotically quadratically convergent like the classical Jacobi algorithm.

Implementation

The Jacobi algorithm for eigenvalues is not implemented in LAPACK. The only related routines in LAPACK Version 3.12.0 are DGESVJ and DGEJSV which use the /one-sided Jacobi algorithm/ to compute the singular values, see Drmač and Veselić (/SIAM J. Matrix Anal. Appl./ 29, 1322-1342, 2008).

** Further readings

• Beresford N. Parlett, The Symmetric Eigenvalue Problem, xxiv+398, Society for Industrial and Applied Mathematics, 1998.
• James Demmel, Applied Numerical Linear Algebra, xi+416, Society for Industrial and Applied Mathematics, 1997

• first, generate singular values,
• then construct a random orthogonal matrix using qmult, and
• finally multiply the orthogonal matrix at both sides of the diagonal matrix that contains these singular values.

We modified the source code of randsvd, and using two additional functions: qmult and mysign, which are all from Prof. Nicholas J. Higham.

• Version: Thursday December 6, 2023
• Acknowledgement: Prof. Nicholas J. Higham

function A = my_randsvd(n, kappa, mode)
%MY_RANDSVD Customized gallery('randsvd')
%  A = MY_RANDSVD(N, KAPPA, MODE) generates
%  a random symmetric matrix A of order N with 
%  cond(A) = KAPPA and singular values from distribution MODE.
%
% Input :
%      N : Size of the matrix.
%
%  KAPPA : pre-defined 2-norm condition number for the output,
%          if kappa is negative, it will return a symmetric
%          positive definite matrix.
%		   Default value is 100.
% https://www.mathworks.com/help/matlab/ref/gallery.html 
%  
%  
%   MODE : one of the following values:
%	       1: one large singular value,
%		   2: one small singular value,
%		   3: geometrically distributed singular values,
%		   4: arithmetically distributed singular values,
%		   5: random singular values with uniformly distributed
%		      logarithm.
%          Default value is 3. 
%
% Reference:
% The MATLAB 'gallery' function.
classname = 'double';

switch nargin
	case 1
		kappa = 100;
		mode = 3;
	case 2
		mode = 3;
end

% check if we need export p.d. matrix
if sign(kappa) == 1, pd = false; else, pd = true; end
kappa = abs(kappa);

switch mode % Set up vector of singular values
	case 1
		sigma = ones(n,1)./kappa;
		sigma(1) = 1;

	case 2
		sigma = ones(n,1);
		sigma(n) = 1/kappa;

	case 3
		factor = kappa^(-1/(n-1));
		sigma = factor.^(0:n-1);

	case 4
		sigma = ones(n,1) - (0:n-1)'/(n-1)*(1-1/kappa);

	case 5    % In this case cond(A) <= kappa.
		sigma = exp( -rand(n,1)*log(kappa) );

	otherwise
		error(message('MATLAB:randsvd:invalidMode'));
end

sigma = cast(sigma,classname);

if ~pd % randomly introducing signs
	[l,j] = size(sigma(2:n-1));
	signs = sign(rand(1,n-2)*2-1)';
	signs = reshape(signs,[l,j]);
	sigma(2:n-1) = sigma(2:n-1) .* signs;
end

Q = qmult(n,1,classname);
A = Q*diag(sigma)*Q';
A = (A + A')/2; % ensure symmetry

end

Here is the code for qmult and mysign:

function B = qmult(A,method,classname)
%QMULT Pre-multiply matrix by random orthogonal matrix.
%   QMULT(A) returns Q*A where Q is a random real orthogonal matrix
%   from the Haar distribution of dimension the number of rows in A.
%   Special case: if A is a scalar then QMULT(A) is the same as QMULT(EYE(A)).
%   QMULT(A,METHOD) specifies how the computations are carried out.
%   METHOD = 0 is the default, while METHOD = 1 uses a call to QR,
%   which is much faster for large dimensions, even though it uses more flops.

%   Called by RANDCOLU, RANDCORR, RANDJORTH, RANDSVD.

%   Reference:
%   G. W. Stewart, The efficient generation of random
%   orthogonal matrices with an application to condition estimators,
%   SIAM J. Numer. Anal., 17 (1980), 403-409.
%
%   Nicholas J. Higham
%   Copyright 1984-2020 The MathWorks, Inc.

if nargin < 2 || isempty(method)
	method = 0;
end
%  Handle scalar A.
if isscalar(A)
	n = A;
	A = eye(n,classname);
else
	n = size(A, 1);
end
if isempty(A) % nothing to do.
	B = A;
	return
end
if method == 1
	[Q,R] = qr(randn(n));
	B = Q*diag(sign(diag(R)))*A;
	return
end
d = zeros(n,1,classname);
for k = n-1:-1:1
	% Generate random Householder transformation.
	x = randn(n-k+1,1);
	s = norm(x);
	sgn = mysign(x(1));
	s = sgn*s;
	d(k) = -sgn;
	x(1) = x(1) + s;
	beta = s*x(1);
	% Apply the transformation to A.
	y = x'*A(k:n,:);
	A(k:n,:) = A(k:n,:) - x*(y/beta);
end
% Tidy up signs.
for i=1:n-1
	A(i,:) = d(i)*A(i,:);
end
A(n,:) = A(n,:)*mysign(randn);
B = A;
end

function S = mysign(A)
%MYSIGN True sign function with MYSIGN(0) = 1.

%   Called by various matrices in elmat/private.
%
%   Nicholas J. Higham
%   Copyright 1984-2013 The MathWorks, Inc.
S = sign(A);
S(S==0) = 1;
end