I’m going through chapter 16 of the book on SVD. On page 479, it says that:
The singular values are all non-negative, so flipping signs of singular vectors may be necessary for reconstructing the matrix.
My question is what do you use to determine which singular vectors to negate? I’m going through the excercises on the next page and it seems completely arbitrary. That is you just find some vectors to negate such that the product UEV^T equals the original matrix.
Hey, yes, that is a confusing point.
“Uniqueness” is an important concept in matrix decompositions. It’s important to know whether a matrix has exactly one SVD (or LU, or QR, or any other decomposition) or whether it has an infinite number of possible SVDs.
For some decompositions, there is no uniqueness. LU is an example: There is an infinite number of lower-upper matrices that can multiply to produce some other matrix A. So how do we guarantee uniqueness? We impose additional constraints. In the case of LU, if you constrain the diagonals of L to be 1, then there is exactly one unique constrained-LU decomposition (which is what MATLAB and Python will compute).
For the SVD, the constraint is that the singular values are non-negative. That’s actually not a constraint that we volutarily impose, but instead comes from the relationship between SVD and eigendecomposition.
But none of this answers your question how how are the signs chosen. I spent a minute looking up the algorithm that LAPACK implements (you can find their guidebook here). Each singular vector is identified using Householder reflections, which is the numerically stable way of implementing Gram-Schmidt. Still, there needs to be something in there to determine the sign, because GS also has sign uncertainty. To be honest, I’ve never looked into that level of detail of numerical implementation. The SVD has more constraints than eigendecomposition because of the pair of singular vectors, so I imagine that once you have the sign for the first pair of singular vectors, the rest of them fall into place.
Anyway, great question