The colored paths are labeled training data, just like in my last post on this.
The title gives the "answer" for a test video:
Could you tell from just this what the sequence of gestures was?
Not perfectly, but way better than chance.
See a couple more examples by clicking:
I'm sure that a prediction method based only on these principal components isn't the best way to go (is it ever? except for reducing the size of your problem for computational purposes), but I'd like to try -- then at least I can visualize my algorithm very nicely and see where it's failing.
And a few more (first ten from "devel01" test data) here (pdf, not an animation).