Sanity Check

As SCCA is based on solving an SPCA problem I ran a sanity check by running SPCA on synthetic data as done in Gert and Jordan's paper. Sure enough it does work. It the test they created a synthetic random vector v = ( 1 0 1 0 1 0 1 0 1 0 )' and running SPCA on U + Sig * v * v'. Where U is noise, and Sig is a signal to noise ratio.

You can run SCCA with a cardinality constraint set to k = 1 to 10. Very interesting experiment. The inherent cardinality of the example is 5. This becomes apparent in the SPCA direction when k = 6. When k = 5 sometimes the PCA directions flips negative other times positive. As k goes up to 10 the non-zero elements become less sparse in w. Similarly as k goes to 1.

This bring up an imporant issue to keep in mind while I'm running experiments, and that is that the value of k, the intrinsic cardinality of the problem, is an unknown parameter that will need to be honed in on. (Since this value is bounded by the dimentionality of the data, we can perform a binary search on the data to find this value.)