Sparse CCA on its way...

My initial formulation of a sparse CCA SDP did not lead to sparse solutions when I tested it on synthetic 2-d data. Let me try and explain the issue. Sparse CCA (sCCA) imposes cardinality constraints on two variables, the vectors x, y. The standard formulation for an SDP problem, however, imposes constraints on a single set of input variables. This means that before I can plug in the sCCA program into a solver, I have to wrap up the variables x, and y into a single set of input variables. [ details: I do this by stacking x and y into a single vector w, and then rederive sCCA in terms of this new input vector. ]

Now the problem with imposing cardinality constraints on x and y, is that these cardinality constraints get relaxed when applied to the vector w. For instance, lets say we impose sparsity constraints on x and y so that card(x) = 2 and card(y) = 2. This implies that card(w) = 4, but, at least initially, we have no way to guarantee that the sparsity constraints are evenly distributed between x and y now.

This means that we have to modify the sCCA program to account for this and somewhere in adding these additional constraints sparsity is not acheived.

Luckily I found an alternative fomulation to the problem. CCA can also be formulated as the solution of two independent eigenvalue problems. You solve one system for vector x, and the other for y. Solving CCA using this approach allows us to place sparsity contraints individually on x and y. It turns out that a sparse eigenvalue problem is equivalent to a sparse PCA program, something that I implemented as a sanity check and which works just fine.

So we are getting sparse CCA solutions using this alternate formulation. I still want to look into the initial formulation of the program and see where the two methods are diverging. I'm also going to run some tests of the method on synthetic data to ensure that it is infact finding sparse directions of maximum covarience.