YB Labs Blog

Linear Algebra and Scientific Computation on OCW

2009-12-11 / Jaba

I was happy to find some neat courses I hadn't seen before on MIT's Open CourseWare (OCW). I've been a huge fan of Gilbert Strang ever since I first watched one of his Linear Algebra 18.06 video lectures. On Amazon, some reviewers give his books low marks for lack of rigour and informality, but they're missing the point. He tries to help the student develop intuition so that matrices "come alive". He's obviously focused on applied math and scientific computing. His style is exactly what you want when you've just read a research paper and you want to go and write some code to apply the results. If you want to get very abstract, then not so much. On the other hand, a more abstract presentation won't help you to actually make things, except perhaps proofs.

I've been interested in cloth and hair simulation for a long time, but I'm hitting my math limits trying to implement recent papers, like this one. I can read them just fine, but when I sit down to design some code, I realize I haven't really got the math in a form I can use. Then there's the Finite Element stuff. I really want to learn it for simulation, but it's out of my league, for now.

There are lots of game-development references on how to write a simple cloth simulator using a mass-spring model with linear springs. You can do it in an afternoon. Usually these use explicit numerical integration methods to solve the system's differential equations. The problem is that the quality is really low, and the systems aren't that stable. I.e. they have an annoying tendency to get twisted or even blow apart. Numerically solving equations on a computer is different from solving them on paper. You have to worry about stability, accuracy, and precision.

So, most high-quality simulators for movies and non-real-time rendering seem to use implicit or semi-implicit numerical integration methods. The problem is that instead of just stepping the system, you need to formulate and solve some matrix equations. I was introduced to these ideas while working at Pseudo Interactive (RIP), and I started reading Baraff and Witkin's Large Steps in Cloth Simulation.

Now, in game development, I use matrices and quaternions pretty much every day. The problem is I never actually took a linear algebra course. I was using matrices for computer graphics in high-school so I just assumed I knew all about linear algebra. I started at University of Toronto, in the Engineering Science program, but switched to Computer Engineering after the first term, after getting spooked about marks. Seeing your 90's go to 60's will do that. Unfortunately, Eng. Sci. offered linear algebra in second term, while Comp. Eng. offered it in the first term. So ... no linear algebra for me.

I've been slowly working through Strang's Linear Algebra. I think I first heard about it from Chris Hecker's Bibliography for Rigid Body Dynamics. Whew -- I just realized I read that for the first time around 1998. I'm getting old! Anyway, Strang's book's alright, but it really shines as a companion to his incredible video lectures. Progress has been slow, because I'm determined to do every exercise in the book. I'm not sure that this is the most efficient approach, but my rationale is that I want to learn it properly, so that I don't have to re-learn it.

On a bit of a tangent, I'm really interested in how to increase the rate at which I can learn, understand, and retain new skills. The problem is that I'm spending a lot of time re-learning things that I paid time and money to learn before. So, retention is key. To that end, I've been trying to do every problem, and also to use spaced-repetition memory software; I'm using Mnemosyne. With my various interests and responsibilities, it's slow going.

Anyway, let's get to the point. You can imagine how stoked I am now that I've found these two Strang courses on OCW (complete with videos). You can also download them for free from the iTunes Store. Sweet! Add another couple of years' worth of work to the queue...