Matrix exponentiation (early view)

I've enjoyed all your videos, but I especially appreciated this one. Because it gave a completely different intuition around matrix exponential than what I've though about! Imagine an adjacency matrix of a graph. Then the exp of that matrix represents a way of exploring that graph out to infinity, with each step weighted inversely by the factorial of which step it is. The fact that the eigenvectors contain all the information necessary to explore the graph to infinity is pretty mindblowing to me. Edit: More specifically, I mean that each successive power of that matrix represents "number of ways any given vertex can reach another". The Taylor series of exp then gives an interesting metric of "reachability" between any two vertices, with a particular weighting. The matrix log would also give an interesting metric (weighted by the inverse of the step rather than its factorial) except that it doesn't work for most adjacency matrices while exp does.

Hi Nicky, thanks for the feedback! I did end up removing that mini-history section, but perhaps it could be recalled again for some future video more dedicated to the history of e. I went back and forth for a while about whether the series definition should be at the start or a bit later. Ultimately, I decided that having it clear from the get-go that what we're talking about is a well-defined operation has value. It offers a nice backdrop to lurk in the viewer's mind once we do get to motivating it, maybe even trying to draw the connection between the motivation building up and the series at the start. The actual path of discovering math is more painful, with lots more doubt, which isn't necessarily the feeling I wanted to evoke. I'm set in stone that it was the right decision, but that's sort of the reasoning behind why I landed there.

3blue1brown

Good point, in the updated version I tried to be more explicit about tying back to that matrix. I see what you mean about referencing quaternions/QM, but personally, I think there's some value in referencing things beyond the scope of a video. I trust the viewers are intelligent enough to know those comments aren't meant to be fully understood for the uninitiated, but it does give a nice hint about how there's more treasure to be found for those willing to keep digging, and it adds something of interest for those viewers who do happen to be in the know.

3blue1brown

Thanks! Ultimately that whole sectioned ended up removed, which I suppose is one way to address the typo :)

3blue1brown

Hi Marco, thanks for the feedback. It's a good point about the typical misconception, wanting x^y to be "x-like". I ultimately decided there was enough going on in this video it didn't quite make sense to talk about. As to obviating the need for the polynomial, do you mean instead talking about the limit definition? Or not talking about the definition at all? To me, at least, seeing that sum, working with it, and seeing the rotation matrix pop out in a way that can be verified intuitively through other means was a big part of what ties things together here.

3blue1brown

Great point!

3blue1brown

Hi Konstantin, I did indeed end up cutting the section around 15:40, thanks for the feedback. I think for more of the details on operators relevant to S.E., I'll just wait to do a full video more on that topic. That where there's time to also motivate things like what are the Pauli matrices, etc.

3blue1brown

That’s actually a really good point about the gendered nature of the example, but it’s likely because the example was lifted directly from the cited Steven Strogatz article (from 2009), including the names of the characters. I doubt it was intentional by Grant or Steven, but I agree it’s worth changing nevertheless. I think Grant can make everybody happy by changing them to gender-neutral names, like Blake and Sam. It might be funny to add a blurb like “we used to call the characters Romeo and Juliet, but we live in a more progressive time now”, but that would likely be pushing it. Edit: Just saw that the video is already out. Gave my feedback too late. Whoops!

Andrew Alvarez

I’m left with some questions, one mathematical and one not. Mathematically, I’m left wondering what makes these 2D vector fields different from the ones I saw in into calc. Why did I see vector fields there but not matrix exponentiation? Second, I found the framing of the Romeo and Juliet example distracting - I found the gendered nature of the example kind of sexist (the rules for Romeo and juliet’s functions follow some dangerous stereotypes about gender and heterosexual relationships) and honestly I was too distracted by that to enjoy the video. I love your content but this video didn’t totally do it for me.

This is great! I was already familiar with exponentiating matrices, yet I still learnt new things (specifically, the proof that (d/dt) e^Mt = M e^Mt ... to my shame I've just always taken that on faith) and got new perspectives on old things. (like the e^Mt visualization showing how the vector flow changes the *basis vectors*) I especially love the slide, "Textbook progression vs Discovery progression", with "specific problems first". For that reason, I think it may be worth putting the Romeo/Juliet bit earlier. It may even be worth putting the *one dimensional* case even earlier than that, to illustrate why we even care about e^whatever in differential eq solutions in the first place. (Or at least mentioning this motivation *before* the formal Taylor definition, if re-organizing the whole video's too much surgery at this point) It's already spread throughout the video, but I think it may be worth having one slide, somewhere, that *explicitly* writes out the analogy: AS x' = rx → x(t) = e^(rt) * x(0) SO IS v' = Mv → v(t) = e^(Mt) * v(0) I struggled to understand the "why" behind matrix exponentiation, until I was told *that* analogy. In that light, it's a beautiful (and convenient) solving tool. As for the "exponentials recap" section, you could always add a "skip to this timestamp" for folks already familiar with it. I did appreciate the part, though, where you clarify that exponential curves *only apply* when x' = rx holds. Too often I've seen someone post some chart (usually a price chart) and say "look at that it's exponential, it's going TO THE MOON" but of course there's no reason to assume its growth is proportional to itself, so it doesn't. Excited to see the final video! :) P.S: I adored the mini-history lesson. I had no idea it *wasn't* Euler who created Euler's number. And that "e" *wasn't* named after himself. Mighty convenient for his brand, tho.

Nicky Case

that book is awesome!! I feel like it makes sense that when there are a lot of bunnies, the foxes population should grow, and when there are a lot of foxes, the bunnies population should shrink (very similar to how Romeo and Juliet behave) However, the predator-prey equations you linked to don’t seem to match up, since no choice of the alpha, beta, gamma, delta parameters would reduce to the Romeo and Juliet diffeq. One of the differences is that since bunnies make more bunnies, there is a component of dx/dt that depends on x alone (and similarly for foxes, dy/dt would have a piece that depends on y alone). Then there’s the cross-terms. As foxes eat bunnies, the growth rate of foxes should increase and the bunnies should decrease. Maybe you can think of some reason that these terms depend on the product of the two populations... I’m drawing a bit of blank. But it’s mainly these cross terms that make the predator-prey system fundamentally different from the Romeo-Juliet system.

I really like this video! One thing I found kind of unsatisfying is that you never explained the [0 -pi pi 0] matrix again after you explained the rotation stuff, and the fact that your animations keep bringing up quaternions or other interesting vector spaces might be cool-looking, but it has the potential to confuse a lot of people, and is kind of distracting from the main point.

Re 20:54-21:04: The magnitude of the position is the radius on the circle around the origin. The magnitude of the velocity is the "speed", in units of distance per second. Since 1 rad/s means "1 radius per unit of time", this is our angular speed, since we move on the circle at the speed of 1 (length of radius) per second.

At 14:38 or so, the speech bubble says "This function is about about e". Probably want to remove one of the abouts.

There's a bit of a sound glitch around the 23:48 mark

Jeffrey Samuelson

Dear Grant, great work as always! I agree that the general e-introduction is indeed a bit long, and might be better covered by giving pointers to your earlier videos. This of course depends on the target audience: a more self-contained video works better for a stand-alone class, whereas for on-line surfers, it's easy (and a more thorough learning experience) to suspend this video and go through the preliminary material first. One point I would make (even) more explicit at the start is the misperception of e^M as "e to the matrixth". This feels alien because we have learned that x^y means multiplying x by itself y times, and so we feel that y must be "number-like". How could it be a matrix? Also, we feel that whatever the result of x^y is, it should be "x-like" as it is x times x times x ... . Why would it be "y-like"? So, when we see e^M, as we think of e as a number - isn't it 2.71828... after all? - we feel that e^M should give us a number. The intuition to get to is that e^M does not have the "shape" of e but the "shape" of M. Now, just as we should think of e^bt as (e^b)^t = k^t when working with compound interest or population growth, with k the factor by which to keep multiplying "what you have", we should think of e^Mt as (e^M)^t = L^t, where L is a matrix[1]. So e^Mt is L^t is just repeated matrix multiplication, with L the transformation to keep applying to "what you have". I'm aware that you point this out formally in the section on e and exp(), but (for me) the enlightening intuition was that e^X or exp(X) are functions that return "X-like" things, not e-like things. This may also obviate the need for going into the infinite sum expansion of e. It detracts from the line of the narrative, and you have covered that comprehensively elsewhere, if I remember well. [1] And, as L and M are both matrices, then we could ask: what transformation produces L from M, and what would this look like geometrically?

2:50 Well, you could say that exp(x) in the real domain is already an "exotic" generalization of the "sensible" definition (x is a positive integer -> multiply e repeatedly x times). Edit 1: Also, I just remembered my favorite math textbook from my degree (weird combination of physics, electronics, and communications engineering): Differential Equations with Applications and Historical Notes, by George F. Simmons. I think it doesn't cover the matrix exp method, but possibly of your interest: I just re-read chapter 4, and I am still blown away by the derivation of the properties of sin x and cos x without using any trigonometry. Edit 2: are the "love equations" a particular case of the predator-prey equations? See: https://en.wikipedia.org/wiki/Lotka%E2%80%93Volterra_equations

jp42

Was wondering why you did not start with $e^{a+bi}$. It might give better intuition for matrix powers?

Reginald Carey

The formula at 18:37 could be written as as exp(J t) = cos(t) + sin(t) J, to make it clear it's really Euler's formula.

I'm used to my mind being blown somewhere around the midway mark; here I had that moment quite early on :D Great stuff Grant, and very interesting!

Boudewijn Redeker

I think it's quite good. @15:40 if you like you could cut a little bit and leave it just saying that the complete solution is a family of functions (a linear combination of all solutions), where the constants are defined by the initial conditions. Obviously, if you decide to do so, you'd phrase it much more nicely. Either way you would want to keep the allusion that the exponent can be also thought of as an operator (the point you make at 16:40). You could even decide to make that point again after the actual example with the matrix, since it did show explicitly how a hermitian matrix produces a unitary transformation by exponentiation. @23:20, when you talk about the Shrodinger's equation, you could tie it back to your concrete matrix example, by saying that this is what you just showed - the exponent of the y Pauli matrix.

I found the section on exponentiation helpful, but I do struggle with the overall concept; my exposure to matrix algebra is nearly 30 years old at this point :)

This was a fun video and satisfying to watch :) I especially loved the attention you gave to the role of discovery in math (how it relates to invention and how it differs from typical exposition). As far as the geometric solution goes, I think your gut is right to want to move it earlier to set up our expectations for how the analytic solution should look. I think you could also intuitively point out _why_ the matrix [[0, -1], [1, 0]] produces a vector that is clockwise perpendicular to our position vector when it acts on our position vector. The way I was doing it in my head was by using some test points: imagine we are at (3,4): since x = 3 (Juliet is positive) then y (Romeo) must be increasing. On the other hand, since y = 4 (Romeo is positive), then x (Juliet) must be decreasing. I don't think you would even need to get mired in the relative magnitudes very much to get a good sense for why the position vector would rotate in a clockwise manner, but it might make sense to use the matrix to calculate the exact magnitude and direction of our velocity at each of the test points to verify that it is indeed perpendicular. I think this whole bit would fit nicely around 9:20. Thinking about relative magnitudes brings me to one question that I had about the geometric solution. From 20:54 - 21:04, you explain that since the magnitude of the velocity is equal to the magnitude of our position, you would expect our angular velocity about the origin to be exactly 1rad/(unit time). I'm drawing a bit of a blank as to why we would expect that other than just waving my hands a bit and saying "well it's the most natural thing so why not?" Can you help me with that intuition at all? (Thank you!!) I don't have much to say about the background on e and ordinary differentiation other than that I think your instinct is right on this one as well. I have always seen exponential functions written with the initial condition in front [like this: x(t) = x_0 e^(rt) ] which suggests that x_0 is acting on the exponential, scaling the function e^(rt) up or down or reflecting it across the x-axis, etc. When you reversed the order, putting the exponential e^(rt) in front, I had a deeply satisfying "ah-ha" moment, because (as you clearly narrated) we can appropriately view the exponential e^(rt) as some time-dependent operator on the initial conditions, and writing in the form x(t) = e^(rt) x_0 is highly suggestive of that perspective. Since this view is the one that generalizes naturally to our matrices, I think it is very natural (and, once again, deeply satisfying to see) to include it in the video. I would be sad for all the people who only watched the finished version if you chose to cut this bit out, since they would have missed out on such a nice perspective on how we are truly generalizing the more familiar and intuitive single variable mathematics. Okay, I guess I did have a lot to say about this after all - I'm so sorry if my response is excessively long, I just don't have a ton of spare time to edit it down :( Anyways, I hope you have a great evening and please let me know if you have any questions about how I articulated something!

Hi Grant. This is a great video I had hoped you would produce some day :-) Regarding your concern about repeating basic properties of the exponential: I don't think it is too much. History is always fine, and for most people it is reassuring to see new things grounded in what you already know. At worst, some people will jump a little forward, but no one would quit the video for that. Here my list of spontaneously typed comments while watching: 1:25 a little trivial but ok. 2:00 beautiful visualization 2:17 hint on sin/cos ? 3:00 nice philosophical turn 3:24 excellent 6:00-6:25 like that very much 6:40 wonderful analogy. Will show that in my next math class. 15:10 Yes! That's what I wanted to hear. 18:00 great visualization. 21:09 beautiful. 23:20-23:45 wonderful 24:05 Wow. You manage to link the two examples 25:32-25:45 this is the only part I fould slightly repetitive (to around 16:00) (but there are reasons to justify that) For a follow-up video, my personal favorite would be unit quaternions/antisymmetric matrices, as you migth understand from https://www.youtube.com/watch?v=_2K-iRQuhkc :-) Best, Alexander

Grant, thank you for creating and sharing this video. Even in the first 10 minutes you make the relation of the exponential function to rotation matrices and linear systems of ODEs light up, which is nice priming for the end. I'm struck by the richness of matrix exponentiation. What a fascinating topic!

I don't think the e foundations part is too long. It's probably not news to most viewers but I do think it bears repeating at this point. However, with the pace set up there, I found the jump to the closed form solution of the matrix exp function a bit jarring. Maybe add another step there to make it flow better?

Christian Leichsenring

Ah, good catch; I had that as a stand-in with a note to find something better, but evidently hadn’t gotten to it before posting!

3blue1brown

Excellent video, as usual. Thanks for all the work you put into this. Since you asked about the introduction to e: I like the analogy with the real case, and thinking about that helped me better understand applying the exponential function to a matrix, but I agree that it could, and probably should, be shortened a bit. In particular, I think you could excise some of the history, as well as the details about being able to substitute any other base for e, as well as the connections between exponential growth and the pandemic.

J. Dmitri Gallow

Excellent exposition of the fundamental beauty of mathematics! Wow!!

Nick Seidenman

Just beautiful. I probably overlooked a few minor things, but the general construction of the video is very nice. From my point of view no fundamental rearrangements necessary.

Lionel Pöffel

In other words, I think the background section is too long for some viewers, so either provide means to skip it or release it in a separate video. Also, I think it would be good to include an actual meaty example at the end.

Edith Dubiner

Really great video, thank you. Looking forward to see the proof that this really ks the solution

Noam Ta Shma

I agree with your instinct that the coverage of one-dimensional exponentiation feels like too big a detour. Frankly there are great videos about this aplenty (probably even some 3b1b videos I'm forgetting), and if someone is shaky on "ordinary" exponentiation then they're probably going to struggle a lot with this video anyway. I generally appreciate that your videos are fairly self-contained, without some multi-step curriculum of prereqs .... but it seems to me that in this case 12:00 through 16:40 could be excised, and the viewer would better be served by an up-front blurb saying "go check out these other videos if you need a refresher on what ordinary exponentiation is and how it works..."

Minor corrections: At 11:20 in the video you have somewhat of a graden path sentence - it made me think you are going to say something like "But whatever functions are really used are ...". Probably best to put a comma after "whatever". Also, in 14:30 you wrote "about" twice.

Noam Ta Shma

Great video! I think moving Romeo and Juliet section would be a good idea. While the video might feel a bit long, nothing out of the history section seems extraneous. Maybe moving the R+J section will fix that anyways. Altogether enjoyed the video though!

Well done gently carrying people who vaguely remember e^x to the point where they have an understanding of what e^A would be.However, people who already have a vague idea of what e^A is, might get bored/frustrated with the pace. Perhaps include section links in the description (when it is ready) and mention at the beginning what would be covered and the option to skip to relevant sections?

Edith Dubiner

Wow, what a great video! You give a wonderful explanation of matrix exponentiation and why it is useful. I already knew most of this stuff, but I was illuminated by your example of how rotations of the wavefunction play a role in quantum mechanics. Just one very minor criticism though. I don't believe the picture you showed of Goldbach was really of him. I had the same problem when looking for a picture of Goldbach, when all I could find was the picture you showed, which clearly could not have been of him because it appears to be a photograph, which didn't exist during his lifetime, as well as his clothes, which weren't worn during his lifetime either. But in any case, I really enjoyed this video as I have all of the 3Blue1Brown videos I've seen so far. Keep up the good work!

David Terr

Great video! Really looking forward to this entire series!

Great video! It might be nice to add information about logarithms and maybe using set theory to explain how orthogonal (zero) properties are when "the barber shaves himself." In other words, orthogonal phenomena as the canvas that doesn't warp the content it fixates. Thanks.

14:41 "about about e"? Not sure what the speech bubble is saying there.

Rosuav

Minor comment (I am watching it now): at around 9:33, isn't x-dot and y-dot more appropriate for the derivative of x and y w/r/t time? I know they are both function of time and thus a "prime" means basically the first derivative w/r/t the variable (time), but some might get confused.

Alipasha Sadri

I think the section giving background on e and ordinary exponentiation is a perfect length, at least for someone like me who had not heard of the matrix case before. So getting a refresher on ordinary exponentiation was very handy for understanding the matrix case, so that I didn't have to try and remember on my own while trying to learn a new thing!

Hmm, that's probably a good idea. Maybe I liked seeing the relative size of each term, and how the swell of the chart maxes out over the relevant exponent, but that is of course beside the point. I'll play around with a 1d version :)

3blue1brown

Hi Grant! Why do you use a 2D presentation (the bar graph) for the terms of the Taylor series of exp at around minute 2:00? Wouldn’t a 1D presentation be more intuitive, as you could represent the sum by putting the terms one after another?

Coincidentally, I've been staring at https://onlinelibrary.wiley.com/doi/pdf/10.1002/9780470694626.app2 for months now and dreading having to implement it... :-) (solid mechanics)

Luke D