1 00:00:01,550 --> 00:00:03,920 The following content is provided under a Creative 2 00:00:03,920 --> 00:00:05,310 Commons license. 3 00:00:05,310 --> 00:00:07,520 Your support will help MIT OpenCourseWare 4 00:00:07,520 --> 00:00:11,610 continue to offer high quality educational resources for free. 5 00:00:11,610 --> 00:00:14,180 To make a donation, or to view additional materials 6 00:00:14,180 --> 00:00:18,140 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:18,140 --> 00:00:19,026 at ocw.mit.edu. 8 00:00:22,510 --> 00:00:27,340 GILBERT STRANG: So last time was orthogonal matrices-- 9 00:00:27,340 --> 00:00:32,995 Q. And this time is symmetric matrices, S. 10 00:00:32,995 --> 00:00:37,620 So we're really talking about the best matrices of all. 11 00:00:37,620 --> 00:00:43,620 Well, I'll start with any square matrix and about eigenvectors. 12 00:00:43,620 --> 00:00:47,130 But you've heard of eigenvectors more than once-- 13 00:00:47,130 --> 00:00:48,540 more than twice-- 14 00:00:48,540 --> 00:00:50,580 more than 10 times, probably. 15 00:00:50,580 --> 00:00:51,690 OK. 16 00:00:51,690 --> 00:00:54,270 So eigenvectors. 17 00:00:54,270 --> 00:00:58,440 And then, let's be sure we know why they're useful, 18 00:00:58,440 --> 00:01:02,820 and maybe compute one or two. 19 00:01:02,820 --> 00:01:05,910 But then we'll move to symmetric matrices 20 00:01:05,910 --> 00:01:08,260 and what is special about those. 21 00:01:08,260 --> 00:01:12,060 And then, even more special and more important 22 00:01:12,060 --> 00:01:16,350 will be positive definite symmetric matrices-- 23 00:01:16,350 --> 00:01:20,280 so that when I say, positive definite, I mean symmetric. 24 00:01:20,280 --> 00:01:26,850 So start with A. Next comes S. Then come the special S-- 25 00:01:26,850 --> 00:01:29,190 special symmetric matrices that have 26 00:01:29,190 --> 00:01:31,950 this extra positive definite property. 27 00:01:31,950 --> 00:01:33,270 OK. 28 00:01:33,270 --> 00:01:39,360 So start with A. So an eigenvector-- 29 00:01:39,360 --> 00:01:44,470 if I multiply A by x, I get some vector. 30 00:01:44,470 --> 00:01:51,010 And sometimes, if x is especially chosen well, 31 00:01:51,010 --> 00:01:55,988 Ax comes out in the same direction as x. 32 00:01:55,988 --> 00:02:01,830 Ax comes out some number times x. 33 00:02:01,830 --> 00:02:04,530 So there are-- normally, there would be, 34 00:02:04,530 --> 00:02:06,120 for an n by n matrix-- 35 00:02:06,120 --> 00:02:13,040 so let's say A is n by n today. 36 00:02:13,040 --> 00:02:16,910 Normally, if we live right, there 37 00:02:16,910 --> 00:02:21,260 will be n different independent vectors-- 38 00:02:21,260 --> 00:02:26,740 x eigenvectors-- that have this special property. 39 00:02:26,740 --> 00:02:32,680 And we can compute them by hand if n is 2 or 3-- 40 00:02:32,680 --> 00:02:35,140 2, mostly. 41 00:02:35,140 --> 00:02:38,920 But the computation of the x's and the lambdas-- 42 00:02:38,920 --> 00:02:46,230 so this is for i equal 1 up to n, 43 00:02:46,230 --> 00:02:49,920 if I use this sort of math shorthand-- 44 00:02:49,920 --> 00:02:53,880 that I have n of these almost always. 45 00:02:53,880 --> 00:02:59,190 And my first question is, what are they good for? 46 00:02:59,190 --> 00:03:05,840 Why does course after course introduce eigenvectors? 47 00:03:05,840 --> 00:03:12,440 And to me the key property is seen by looking at A squared. 48 00:03:12,440 --> 00:03:13,910 So let me look at A squared. 49 00:03:17,430 --> 00:03:20,690 So it's another n by n matrix. 50 00:03:20,690 --> 00:03:24,140 And we would ask, suppose we know these guys? 51 00:03:24,140 --> 00:03:27,890 Suppose we've found those somehow. 52 00:03:27,890 --> 00:03:30,410 What about A squared? 53 00:03:30,410 --> 00:03:35,020 Is x an eigenvector of A squared also? 54 00:03:35,020 --> 00:03:39,310 Well, the way to find out is to multiply A squared by x, 55 00:03:39,310 --> 00:03:41,390 and see what happens. 56 00:03:41,390 --> 00:03:43,640 Do you see what's going to happen here? 57 00:03:43,640 --> 00:03:50,960 This is A times Ax, which is A times-- 58 00:03:50,960 --> 00:03:54,370 Ax is lambda x-- 59 00:03:54,370 --> 00:03:58,150 and now what do I do now? 60 00:03:58,150 --> 00:04:01,150 Because I'm shooting for the answer yes. 61 00:04:01,150 --> 00:04:04,300 X is an eigenvector of A squared also. 62 00:04:04,300 --> 00:04:05,250 So what do I do? 63 00:04:05,250 --> 00:04:09,010 That number-- that lambda is just a number. 64 00:04:09,010 --> 00:04:10,930 I can put it anywhere I like. 65 00:04:10,930 --> 00:04:13,210 So I can put it out front. 66 00:04:13,210 --> 00:04:16,170 And then I have Ax, which is? 67 00:04:16,170 --> 00:04:17,395 AUDIENCE: Lambda x. 68 00:04:17,395 --> 00:04:18,000 GILBERT STRANG: Lambda x. 69 00:04:18,000 --> 00:04:18,500 Thanks. 70 00:04:18,500 --> 00:04:19,800 So I have another lambda x. 71 00:04:19,800 --> 00:04:21,200 So there's lambda squared x. 72 00:04:21,200 --> 00:04:24,870 So I learned the crucial thing here-- 73 00:04:24,870 --> 00:04:28,270 that x is also an eigenvector of A squared, 74 00:04:28,270 --> 00:04:33,700 and the eigenvalue is lambda squared. 75 00:04:33,700 --> 00:04:36,710 And of course, I can keep going. 76 00:04:36,710 --> 00:04:39,025 So A to the nth-- 77 00:04:39,025 --> 00:04:42,580 x is lambda to the nth x. 78 00:04:42,580 --> 00:04:46,660 We have found the right vectors for that particular matrix A. 79 00:04:46,660 --> 00:04:49,970 What about A inverse x? 80 00:04:49,970 --> 00:04:53,230 That will be-- if everything is good-- 81 00:04:53,230 --> 00:04:57,040 1 over lambda x. 82 00:04:57,040 --> 00:04:58,750 Well, yeah. 83 00:04:58,750 --> 00:05:01,420 So anytime I write 1 over lambda, 84 00:05:01,420 --> 00:05:07,810 my mind says, you gotta make some comment 85 00:05:07,810 --> 00:05:11,680 on the special case where it doesn't work, which is? 86 00:05:11,680 --> 00:05:13,405 AUDIENCE: Lambda is not equal to 0. 87 00:05:13,405 --> 00:05:14,280 GILBERT STRANG: Yeah. 88 00:05:14,280 --> 00:05:17,720 If lambda is not 0, I'm golden. 89 00:05:17,720 --> 00:05:22,590 If lambda is 0, it doesn't look good. 90 00:05:22,590 --> 00:05:26,137 And what's happening if lambda is 0? 91 00:05:26,137 --> 00:05:27,470 AUDIENCE: A inverse [INAUDIBLE]. 92 00:05:27,470 --> 00:05:29,428 GILBERT STRANG: A doesn't even have an inverse. 93 00:05:29,428 --> 00:05:32,010 If lambda was 0-- 94 00:05:32,010 --> 00:05:33,620 which it could be-- 95 00:05:33,620 --> 00:05:36,710 no rule against it. 96 00:05:36,710 --> 00:05:40,520 If lambda was 0, this would say, A times the eigenvector 97 00:05:40,520 --> 00:05:42,860 is 0 times the eigenvector. 98 00:05:42,860 --> 00:05:44,900 So that would tell me that the eigenvector 99 00:05:44,900 --> 00:05:47,750 is in the null space. 100 00:05:47,750 --> 00:05:51,040 It would tell me the matrix A isn't invertible. 101 00:05:51,040 --> 00:05:55,220 It's taking some vector x to 0. 102 00:05:55,220 --> 00:05:58,550 And so everything clicks. 103 00:05:58,550 --> 00:06:01,760 This works when it should work. 104 00:06:01,760 --> 00:06:05,950 And if we have other fun-- any function of the matrix, 105 00:06:05,950 --> 00:06:08,590 we could define the exponential of a matrix. 106 00:06:08,590 --> 00:06:10,570 18.03 would do that. 107 00:06:10,570 --> 00:06:15,340 Let's just write it down, as if we know what it means. 108 00:06:15,340 --> 00:06:17,740 Does it have the same eigenvector? 109 00:06:17,740 --> 00:06:18,820 Well, sure. 110 00:06:18,820 --> 00:06:20,330 Because e to the At-- 111 00:06:20,330 --> 00:06:22,900 the exponential of a matrix-- 112 00:06:22,900 --> 00:06:25,120 if I see e to the something-- 113 00:06:25,120 --> 00:06:27,610 I think of that long, infinite series 114 00:06:27,610 --> 00:06:29,980 that gives the exponential. 115 00:06:29,980 --> 00:06:34,720 Those-- all the terms in that series have powers of A. 116 00:06:34,720 --> 00:06:36,250 So everything is working. 117 00:06:36,250 --> 00:06:38,430 Every term in that series-- 118 00:06:38,430 --> 00:06:39,830 x is an eigenvector. 119 00:06:39,830 --> 00:06:41,830 And when I put it all together, I 120 00:06:41,830 --> 00:06:47,740 learn that the eigenvalue is e to the lambda t. 121 00:06:47,740 --> 00:06:54,670 That's just a typical and successful work use. 122 00:06:54,670 --> 00:06:55,660 OK. 123 00:06:55,660 --> 00:06:58,720 So that's eigenvectors and eigenvalues, 124 00:06:58,720 --> 00:07:02,810 and we'll find some in a minute. 125 00:07:05,800 --> 00:07:07,965 Now, so I'm claiming that this-- 126 00:07:11,720 --> 00:07:16,120 that from this first thing-- which was just about certain 127 00:07:16,120 --> 00:07:17,950 vectors are special-- 128 00:07:17,950 --> 00:07:21,490 now we're beginning to see why they're useful. 129 00:07:21,490 --> 00:07:22,720 So special is good. 130 00:07:22,720 --> 00:07:25,060 Useful is even better. 131 00:07:25,060 --> 00:07:37,810 So let me take any vector, say v. And OK, 132 00:07:37,810 --> 00:07:39,910 what do I want to do? 133 00:07:39,910 --> 00:07:42,190 I want to use eigenvectors. 134 00:07:42,190 --> 00:07:45,280 This v is probably not an eigenvector. 135 00:07:45,280 --> 00:07:49,330 But I'm supposing that I've got n of them. 136 00:07:49,330 --> 00:07:51,400 You and I are agreed that there are 137 00:07:51,400 --> 00:07:54,760 some matrices for which there are not 138 00:07:54,760 --> 00:07:57,290 a full set of eigenvectors. 139 00:07:57,290 --> 00:08:02,480 That's really the main sort of annoying point 140 00:08:02,480 --> 00:08:05,440 in the whole subject of linear algebra, 141 00:08:05,440 --> 00:08:08,870 is some matrices don't have enough eigenvectors. 142 00:08:08,870 --> 00:08:14,280 But almost all do, and let's go forward 143 00:08:14,280 --> 00:08:16,730 assuming our matrix has. 144 00:08:16,730 --> 00:08:17,820 OK. 145 00:08:17,820 --> 00:08:22,200 So if I've got n independent eigenvectors, that's a basis. 146 00:08:22,200 --> 00:08:26,250 I can write any vector v as a combination 147 00:08:26,250 --> 00:08:30,620 of those eigenvectors. 148 00:08:30,620 --> 00:08:33,250 Right. 149 00:08:33,250 --> 00:08:40,090 And then I can find out what A to any power. 150 00:08:40,090 --> 00:08:42,820 So that's the point. 151 00:08:42,820 --> 00:08:49,290 This is going to be the simple and reason why 152 00:08:49,290 --> 00:08:50,520 we like to have-- 153 00:08:50,520 --> 00:08:52,840 we like to know the eigenvectors. 154 00:08:52,840 --> 00:08:56,470 Because if I choose those as my basis vectors, 155 00:08:56,470 --> 00:08:58,750 v is a combination of them. 156 00:08:58,750 --> 00:09:04,030 Now if I multiply by A, or A squared, or A to the k power, 157 00:09:04,030 --> 00:09:05,890 then it's linear. 158 00:09:05,890 --> 00:09:08,950 So I can multiply each one by A to the k. 159 00:09:08,950 --> 00:09:15,300 And what do I get if I multiply that guy by A to the kth power? 160 00:09:15,300 --> 00:09:16,240 OK. 161 00:09:16,240 --> 00:09:18,580 Well, I'm just going to use-- or, here 162 00:09:18,580 --> 00:09:21,122 I said n, but let me say k. 163 00:09:21,122 --> 00:09:23,260 Because n-- I'm sorry. 164 00:09:23,260 --> 00:09:26,210 I'm using n for the size of the matrix. 165 00:09:26,210 --> 00:09:31,870 So I better use k for the typical case here. 166 00:09:31,870 --> 00:09:34,460 So what do I get? 167 00:09:34,460 --> 00:09:39,140 Just help me through this and we're happy. 168 00:09:39,140 --> 00:09:44,330 So what happens when I multiply that by A to the k? 169 00:09:44,330 --> 00:09:47,720 It's an eigenvector, remember, so when I 170 00:09:47,720 --> 00:09:49,670 multiply by A to the k, I get? 171 00:09:49,670 --> 00:09:50,360 AUDIENCE: C1. 172 00:09:50,360 --> 00:09:51,610 GILBERT STRANG: C1. 173 00:09:51,610 --> 00:09:53,350 That's just a number. 174 00:09:53,350 --> 00:09:56,840 And A to the k times that eigenvector gives? 175 00:09:56,840 --> 00:09:57,650 AUDIENCE: Lambda 1. 176 00:09:57,650 --> 00:10:00,653 GILBERT STRANG: Lambda 1 to the k times the eigenvector. 177 00:10:03,970 --> 00:10:05,730 Right? 178 00:10:05,730 --> 00:10:07,170 That's the whole point. 179 00:10:07,170 --> 00:10:11,340 And linearity says keep going. 180 00:10:11,340 --> 00:10:18,270 Cn, lambda n to the kth power, Xn. 181 00:10:18,270 --> 00:10:22,080 In other words, I can take-- 182 00:10:22,080 --> 00:10:24,820 I can apply any power of a matrix. 183 00:10:24,820 --> 00:10:27,250 I can apply the exponential of a matrix. 184 00:10:27,250 --> 00:10:33,380 I can do anything quickly, because I've 185 00:10:33,380 --> 00:10:35,210 got the eigenvector. 186 00:10:35,210 --> 00:10:38,630 So really, I'm saying the first use 187 00:10:38,630 --> 00:10:41,900 for eigenvectors-- maybe the principle use for which they 188 00:10:41,900 --> 00:10:47,660 were invented-- is to be able to solve difference equations. 189 00:10:47,660 --> 00:10:51,560 So if I call that Vk-- 190 00:10:51,560 --> 00:10:55,790 the kth power-- then the equation I'm solving here 191 00:10:55,790 --> 00:11:01,530 is a one step difference equation. 192 00:11:01,530 --> 00:11:03,790 This is my difference equation. 193 00:11:03,790 --> 00:11:06,140 And if I wanted to use exponentials, 194 00:11:06,140 --> 00:11:12,090 the equation I would be solving would be dv, dt equal Av. 195 00:11:17,050 --> 00:11:29,820 Solution to discrete steps, or continuous time evolution comes 196 00:11:29,820 --> 00:11:32,290 is trivial, if I know the eigenvectors. 197 00:11:32,290 --> 00:11:35,560 Because here is the solution to this one. 198 00:11:35,560 --> 00:11:40,325 And the solution to this one is the same thing, C1, e 199 00:11:40,325 --> 00:11:43,480 to the lambda, 1, t, x1. 200 00:11:43,480 --> 00:11:49,180 Is that what you were expecting for the solution here? 201 00:11:49,180 --> 00:11:50,830 Because if I takes the derivative, 202 00:11:50,830 --> 00:11:52,810 it brings down a lambda. 203 00:11:52,810 --> 00:11:56,680 If I multiply by A, it brings down a lambda-- 204 00:11:56,680 --> 00:11:59,020 so, plus the other guys. 205 00:12:03,520 --> 00:12:04,020 OK. 206 00:12:09,600 --> 00:12:15,110 Not news, but important to remember what eigenvectors 207 00:12:15,110 --> 00:12:17,280 are for in the first place. 208 00:12:17,280 --> 00:12:17,780 Good. 209 00:12:21,960 --> 00:12:22,620 Yeah. 210 00:12:22,620 --> 00:12:24,220 Let me move ahead. 211 00:12:24,220 --> 00:12:34,270 Oh-- one matrix fact is about something 212 00:12:34,270 --> 00:12:35,800 called similar matrices. 213 00:12:35,800 --> 00:12:38,680 So I have on my matrix A. Then I have 214 00:12:38,680 --> 00:12:41,980 the idea of what it means to be similar to A, 215 00:12:41,980 --> 00:12:56,740 so B is similar to A. What does that mean? 216 00:12:56,740 --> 00:12:59,410 So here's what it means, first of all. 217 00:12:59,410 --> 00:13:04,450 It means that B can be found from A, by-- 218 00:13:04,450 --> 00:13:07,310 this is the key operation here-- 219 00:13:07,310 --> 00:13:10,930 multiplying by a matrix M, and its inverse-- 220 00:13:10,930 --> 00:13:12,940 M inverse AM. 221 00:13:12,940 --> 00:13:18,550 When I see two matrices, B and A, 222 00:13:18,550 --> 00:13:24,620 that are connected by that kind of a change, 223 00:13:24,620 --> 00:13:28,160 M could be any invertible matrix. 224 00:13:28,160 --> 00:13:34,410 Then I would say B was similar to A. And that changed-- 225 00:13:34,410 --> 00:13:38,920 that appearance of AM is pretty natural. 226 00:13:38,920 --> 00:13:43,770 If I change variables here by M, then I get-- 227 00:13:43,770 --> 00:13:47,850 that similar matrix will show up. 228 00:13:47,850 --> 00:13:49,110 So what's the key factor? 229 00:13:49,110 --> 00:13:53,760 Do you remember the key fact about similar matrices? 230 00:13:53,760 --> 00:13:56,972 If B and A are connected like that-- 231 00:13:56,972 --> 00:13:58,680 AUDIENCE: They have the same eigenvalues. 232 00:13:58,680 --> 00:14:01,380 GILBERT STRANG: They have the same eigenvalues. 233 00:14:01,380 --> 00:14:04,170 So this is just a useful point to remember. 234 00:14:04,170 --> 00:14:13,320 So I'll-- this is like one fact in the discussion 235 00:14:13,320 --> 00:14:15,780 of eigenvalues and eigenvectors. 236 00:14:15,780 --> 00:14:28,600 So similar matrices, same eigenvalues. 237 00:14:37,140 --> 00:14:38,100 Yeah. 238 00:14:38,100 --> 00:14:42,810 So in some way in the eigenvalue, eigenvector world, 239 00:14:42,810 --> 00:14:45,090 they're in this-- they belong together. 240 00:14:49,390 --> 00:14:54,503 They're connected by this relation that just turns out 241 00:14:54,503 --> 00:14:55,420 to be the right thing. 242 00:14:58,120 --> 00:15:03,820 Actually, that is-- it gives us a clue of how eigenvalues 243 00:15:03,820 --> 00:15:05,950 are actually computed. 244 00:15:05,950 --> 00:15:10,510 Well, they're actually computed by typing eig of A, 245 00:15:10,510 --> 00:15:14,080 with parentheses around A. That's how they're-- 246 00:15:14,080 --> 00:15:17,890 in real life. 247 00:15:17,890 --> 00:15:22,000 But what happens when you type eig of A? 248 00:15:22,000 --> 00:15:24,010 Well, you could say the eigenvalue shows up 249 00:15:24,010 --> 00:15:26,080 on the screen. 250 00:15:26,080 --> 00:15:28,660 But something had to happen in there. 251 00:15:28,660 --> 00:15:33,250 And what happened was that MATLAB-- 252 00:15:33,250 --> 00:15:40,540 or whoever-- took that matrix A, started using good choices 253 00:15:40,540 --> 00:15:41,140 of m-- 254 00:15:43,860 --> 00:15:44,610 better and better. 255 00:15:47,170 --> 00:15:50,550 Took a bunch of steps with different m's. 256 00:15:50,550 --> 00:15:53,860 Because if I do another m, I still have a similar matrix, 257 00:15:53,860 --> 00:15:54,360 right? 258 00:15:54,360 --> 00:16:00,300 If I take B and do a different m2 to B-- 259 00:16:00,300 --> 00:16:02,220 so I get something similar to B, then 260 00:16:02,220 --> 00:16:04,230 that's also similar to A. I've got 261 00:16:04,230 --> 00:16:07,020 a whole family of similar things there. 262 00:16:07,020 --> 00:16:13,410 And what does MATLAB do with all these m's, m1 and m2 and m3 263 00:16:13,410 --> 00:16:14,580 and so on? 264 00:16:14,580 --> 00:16:22,140 It brings the matrix to a triangular matrix. 265 00:16:22,140 --> 00:16:24,915 It gets the eigenvalues showing up on the diagonal. 266 00:16:27,620 --> 00:16:33,110 It's just tremendously-- it was an inspiration when that-- 267 00:16:33,110 --> 00:16:35,990 when the good choice of m appeared. 268 00:16:35,990 --> 00:16:38,060 And let me just say-- 269 00:16:38,060 --> 00:16:41,120 because I'm going on to symmetric matrices-- 270 00:16:41,120 --> 00:16:47,360 that for a symmetric matrices, everything is sort of clean. 271 00:16:47,360 --> 00:16:51,770 You not only go to a triangular matrix, 272 00:16:51,770 --> 00:16:54,380 you go toward a diagonal matrix. 273 00:16:54,380 --> 00:16:57,380 They off-- you choose m's that make 274 00:16:57,380 --> 00:17:00,710 the off diagonal stuff smaller and smaller and smaller. 275 00:17:00,710 --> 00:17:03,230 And the eigenvalues are not changing. 276 00:17:03,230 --> 00:17:08,800 So there, shooting up on the diagonal, are the eigenvalues. 277 00:17:08,800 --> 00:17:13,329 So I guess I should verify that fact, 278 00:17:13,329 --> 00:17:16,329 that similar matrices have the same eigenvalues. 279 00:17:16,329 --> 00:17:19,300 Can we-- there can't be much to show. 280 00:17:19,300 --> 00:17:24,710 There can't be much in the proof because that's all I know. 281 00:17:24,710 --> 00:17:27,400 And I want to know its eigenvalues and eigenvectors. 282 00:17:27,400 --> 00:17:32,920 So let me say, suppose m inverse Am has the eigenvector 283 00:17:32,920 --> 00:17:35,125 y and the eigenvalue of lambda. 284 00:17:41,360 --> 00:17:44,610 And I want to show-- 285 00:17:44,610 --> 00:17:48,810 do I want to show that y is an eigenvector also, of A itself? 286 00:17:48,810 --> 00:17:50,010 No. 287 00:17:50,010 --> 00:17:52,140 Eigenvectors are changing. 288 00:17:52,140 --> 00:17:56,610 Do I want to show that lambda is an eigenvalue of A itself? 289 00:17:56,610 --> 00:17:57,150 Yes. 290 00:17:57,150 --> 00:17:58,560 That's my point. 291 00:17:58,560 --> 00:18:00,656 So can we see that? 292 00:18:00,656 --> 00:18:01,650 Ha. 293 00:18:01,650 --> 00:18:05,250 Can I see that lambda is an eigenvector? 294 00:18:05,250 --> 00:18:07,440 There's not a lot to do here. 295 00:18:07,440 --> 00:18:10,200 I mean, if I can't do it soon, I'm never going to do it, 296 00:18:10,200 --> 00:18:12,300 because-- 297 00:18:12,300 --> 00:18:13,845 so what am I going to do? 298 00:18:13,845 --> 00:18:15,930 AUDIENCE: Define the vector x equals my-- 299 00:18:15,930 --> 00:18:17,530 GILBERT STRANG: Yeah, I could. 300 00:18:17,530 --> 00:18:19,320 Yeah. 301 00:18:19,320 --> 00:18:24,120 X is-- m-y is going to be a key, and I can see m-y coming. 302 00:18:24,120 --> 00:18:26,550 Just-- when I see m inverse over there, 303 00:18:26,550 --> 00:18:28,345 what am I going to do with the darn thing? 304 00:18:28,345 --> 00:18:29,220 AUDIENCE: [INAUDIBLE] 305 00:18:29,220 --> 00:18:31,500 GILBERT STRANG: I'm going to put it on the other side. 306 00:18:31,500 --> 00:18:34,020 I'm going to multiply that equation by m. 307 00:18:34,020 --> 00:18:37,560 So I'll have-- that will put the m over here. 308 00:18:37,560 --> 00:18:44,630 And I'll have A-M-y equals lambda My, right? 309 00:18:48,680 --> 00:18:50,520 And is that telling me what I want to know? 310 00:18:50,520 --> 00:18:51,790 Yes. 311 00:18:51,790 --> 00:18:54,560 That's saying that My-- 312 00:18:54,560 --> 00:18:58,340 that you wisely suggested to give a name x to-- 313 00:18:58,340 --> 00:19:01,025 is lambda times My. 314 00:19:01,025 --> 00:19:02,285 Do you see that? 315 00:19:02,285 --> 00:19:06,650 That the eigenvalue lambda didn't change. 316 00:19:06,650 --> 00:19:08,970 The eigenvector did change. 317 00:19:08,970 --> 00:19:11,480 It changed from y to My. 318 00:19:11,480 --> 00:19:13,350 That's the x. 319 00:19:13,350 --> 00:19:14,930 The eigenvector of x. 320 00:19:14,930 --> 00:19:18,530 This is lambda x. 321 00:19:18,530 --> 00:19:19,910 Yeah. 322 00:19:19,910 --> 00:19:24,110 So that's the role of M. It just gives you a different basis 323 00:19:24,110 --> 00:19:25,610 for eigenvectors. 324 00:19:25,610 --> 00:19:28,010 But it does not change eigenvalues. 325 00:19:28,010 --> 00:19:28,880 Right. 326 00:19:28,880 --> 00:19:30,250 Yeah. 327 00:19:30,250 --> 00:19:31,540 OK. 328 00:19:31,540 --> 00:19:35,380 So those are similar matrices. 329 00:19:35,380 --> 00:19:37,180 Yeah, some other good things happen. 330 00:19:37,180 --> 00:19:39,190 A lot of people don't know-- in fact, 331 00:19:39,190 --> 00:19:42,610 I wasn't very conscious of the fact 332 00:19:42,610 --> 00:19:48,500 that A times B has the same eigenvalues as B times A. Well, 333 00:19:48,500 --> 00:19:51,130 I should maybe write that down. 334 00:19:51,130 --> 00:19:58,190 AB has the same eigenvalues-- 335 00:19:58,190 --> 00:20:00,300 the same non-zero ones-- 336 00:20:00,300 --> 00:20:01,720 you'll see. 337 00:20:01,720 --> 00:20:07,240 I have to-- as BA. 338 00:20:07,240 --> 00:20:10,690 This is any A and B same size. 339 00:20:10,690 --> 00:20:13,390 I'm not talking similar matrices here. 340 00:20:13,390 --> 00:20:19,270 I'm talking any two A and B. Yeah. 341 00:20:19,270 --> 00:20:23,140 So that's a good thing that happens. 342 00:20:23,140 --> 00:20:29,880 Now could we see y? 343 00:20:29,880 --> 00:20:35,240 And then I'm going to be really pretty happy with basic fact 344 00:20:35,240 --> 00:20:37,870 about eigenvalues. 345 00:20:37,870 --> 00:20:41,070 So if I want to show that two things have 346 00:20:41,070 --> 00:20:45,760 the same eigenvalues, what do you propose? 347 00:20:45,760 --> 00:20:49,660 Show that they are similar. 348 00:20:49,660 --> 00:20:51,440 I already said, if they are similar. 349 00:20:51,440 --> 00:20:53,540 So is there an m? 350 00:20:53,540 --> 00:20:58,370 Is there an m that will connect this matrix? 351 00:20:58,370 --> 00:21:05,820 So is there an m that will multiply this matrix that way? 352 00:21:05,820 --> 00:21:07,880 So that would be similar to AB. 353 00:21:07,880 --> 00:21:09,950 And can I produce BA then? 354 00:21:16,770 --> 00:21:19,110 So I'll just put the word want up here. 355 00:21:22,960 --> 00:21:28,000 I want-- if I have that, then I'm 356 00:21:28,000 --> 00:21:32,140 done, because that's saying that those two matrices, AB and BA, 357 00:21:32,140 --> 00:21:33,160 are similar. 358 00:21:33,160 --> 00:21:36,930 And I know that then they have the same eigenvalues. 359 00:21:36,930 --> 00:21:41,425 So what should m be? 360 00:21:41,425 --> 00:21:49,640 M should be-- so what is M here? 361 00:21:49,640 --> 00:21:50,870 I want that to be true. 362 00:21:54,700 --> 00:21:57,890 Should M be B? 363 00:21:57,890 --> 00:21:58,660 Yeah. 364 00:21:58,660 --> 00:22:01,020 M equal B. Boy. 365 00:22:01,020 --> 00:22:05,710 Not the most hidden fact here. 366 00:22:05,710 --> 00:22:11,110 Take M equal B. 367 00:22:11,110 --> 00:22:14,950 So then I have B times A, times BB inverse-- 368 00:22:14,950 --> 00:22:16,130 which is the identity. 369 00:22:16,130 --> 00:22:18,130 So I have B times A. Yes. 370 00:22:18,130 --> 00:22:19,690 OK. 371 00:22:19,690 --> 00:22:23,440 So AB and BA are fine. 372 00:22:23,440 --> 00:22:28,240 Now, what do you think about this question? 373 00:22:28,240 --> 00:22:31,610 Are the eigenvalues-- I now know that AB and BA 374 00:22:31,610 --> 00:22:33,230 have the same eigenvalues. 375 00:22:33,230 --> 00:22:40,730 And the reason I had to be careful about non-zero is that 376 00:22:40,730 --> 00:22:45,065 if I had zero eigenvalues, then-- 377 00:22:45,065 --> 00:22:46,055 AUDIENCE: [INAUDIBLE] 378 00:22:46,055 --> 00:22:46,930 GILBERT STRANG: Yeah. 379 00:22:46,930 --> 00:22:49,370 I can't count on those inverses. 380 00:22:49,370 --> 00:22:50,380 Right. 381 00:22:50,380 --> 00:22:51,570 Right. 382 00:22:51,570 --> 00:22:56,100 So that's why I put it in that little qualifier. 383 00:22:56,100 --> 00:22:59,190 But now I want to ask this question. 384 00:22:59,190 --> 00:23:02,070 If I know the eigenvalues of A-- 385 00:23:02,070 --> 00:23:05,570 separately, by itself, A-- and of B-- 386 00:23:05,570 --> 00:23:09,850 now I'm talking about any two matrices, A and B. 387 00:23:09,850 --> 00:23:12,855 If I have two matrices, A-- 388 00:23:12,855 --> 00:23:15,580 I have a matrix A and a matrix B. 389 00:23:15,580 --> 00:23:19,170 And I know their eigenvalues and their eigenvalues. 390 00:23:19,170 --> 00:23:21,790 What about AB? 391 00:23:21,790 --> 00:23:25,510 A times B. Can I multiply the eigenvalues of A times 392 00:23:25,510 --> 00:23:27,640 the eigenvalues of B? 393 00:23:27,640 --> 00:23:28,540 Don't do it. 394 00:23:28,540 --> 00:23:29,080 Right. 395 00:23:29,080 --> 00:23:29,730 Yes. 396 00:23:29,730 --> 00:23:30,230 Right. 397 00:23:30,230 --> 00:23:33,340 The eigenvalues of A times the eigenvalues of B 398 00:23:33,340 --> 00:23:35,620 could be damn near anything. 399 00:23:35,620 --> 00:23:36,680 Right. 400 00:23:36,680 --> 00:23:40,890 They're not connected to the eigenvalues of AB specially. 401 00:23:40,890 --> 00:23:45,800 And maybe something could be discovered, but not much. 402 00:23:45,800 --> 00:23:50,920 And similarly, for A plus B. So yeah. 403 00:23:50,920 --> 00:23:54,670 So let me just write down this point. 404 00:23:54,670 --> 00:24:00,100 Eigenvalues of A plus B are generally not 405 00:24:00,100 --> 00:24:08,830 eigenvalues of A plus eigenvalues of B. 406 00:24:08,830 --> 00:24:09,670 Generally not. 407 00:24:09,670 --> 00:24:12,200 Just-- there is no reason. 408 00:24:12,200 --> 00:24:15,580 And the reason that that's-- 409 00:24:15,580 --> 00:24:20,260 I get that no answer is, that the eigenvectors can 410 00:24:20,260 --> 00:24:21,040 be all different. 411 00:24:21,040 --> 00:24:23,770 If the eigenvectors for A are totally 412 00:24:23,770 --> 00:24:26,300 different from the eigenvectors for B, 413 00:24:26,300 --> 00:24:30,160 then A plus B will have probably some other, totally different 414 00:24:30,160 --> 00:24:34,360 eigenvectors, and there's nothing happening there. 415 00:24:38,420 --> 00:24:43,650 That's sort of thoughts about eigenvalues in general. 416 00:24:43,650 --> 00:24:50,230 And I could-- there'd be a whole section on eigenvectors, 417 00:24:50,230 --> 00:24:53,830 but I'm really interested in eigenvectors 418 00:24:53,830 --> 00:24:56,450 of symmetric matrices. 419 00:24:56,450 --> 00:25:02,550 So I'm going to move on to that topic. 420 00:25:02,550 --> 00:25:06,410 So now, having talked about any matrix A, 421 00:25:06,410 --> 00:25:09,710 I'm going to specialize to symmetric matrices, 422 00:25:09,710 --> 00:25:12,020 see what's special about the eigenvalues 423 00:25:12,020 --> 00:25:14,790 there, what's special about eigenvectors there. 424 00:25:14,790 --> 00:25:17,720 And I think we've already said it in class. 425 00:25:17,720 --> 00:25:20,070 So let me-- let me ask you to tell me 426 00:25:20,070 --> 00:25:21,590 about it-- tell me again. 427 00:25:21,590 --> 00:25:26,270 So I'll call that matrix S now, as a reminder 428 00:25:26,270 --> 00:25:29,970 always that I'm talking here about symmetric matrices. 429 00:25:29,970 --> 00:25:33,710 So what do I-- what are the key facts to know? 430 00:25:33,710 --> 00:25:44,360 Eigenvalues are real numbers, if the matrix is. 431 00:25:44,360 --> 00:25:48,840 I'm thinking of real symmetric matrices. 432 00:25:48,840 --> 00:25:51,060 Of course, other real matrices could 433 00:25:51,060 --> 00:25:55,400 have imaginary eigenvalues. 434 00:25:55,400 --> 00:25:57,620 Other real matrices-- so just-- 435 00:25:57,620 --> 00:26:00,860 let's just think for a moment. 436 00:26:00,860 --> 00:26:01,360 Yeah. 437 00:26:01,360 --> 00:26:02,620 Maybe I'll just put it here. 438 00:26:02,620 --> 00:26:09,760 Can I back up, before I keep going with symmetric matrices? 439 00:26:09,760 --> 00:26:15,610 So you take a matrix like that. 440 00:26:20,150 --> 00:26:20,850 Q, yeah. 441 00:26:20,850 --> 00:26:25,500 That would be a Q. But it's not specially a Q. Maybe 442 00:26:25,500 --> 00:26:28,320 the most remarkable thing about that matrix 443 00:26:28,320 --> 00:26:31,080 is that it's anti-symmetric. 444 00:26:31,080 --> 00:26:33,630 So I'll call it A. Right. 445 00:26:33,630 --> 00:26:37,942 If I transpose that matrix, what do I get? 446 00:26:37,942 --> 00:26:38,900 AUDIENCE: The negative. 447 00:26:38,900 --> 00:26:40,108 GILBERT STRANG: The negative. 448 00:26:40,108 --> 00:26:42,400 So that's like anti-symmetric. 449 00:26:42,400 --> 00:26:45,430 And I claim that an anti-symmetric matrix 450 00:26:45,430 --> 00:26:47,830 has imaginary eigenvalues. 451 00:26:47,830 --> 00:26:51,010 So that's a 90 degree rotation. 452 00:26:54,330 --> 00:26:57,040 And you might say, what could be simpler than that? 453 00:26:57,040 --> 00:27:01,050 A 90 degree rotation-- that's not a weird matrix. 454 00:27:01,050 --> 00:27:03,740 But from the point of view of eigenvectors, 455 00:27:03,740 --> 00:27:07,620 something a little odd has to happen, right? 456 00:27:07,620 --> 00:27:11,010 Because if I have a 90 degree rotation-- 457 00:27:11,010 --> 00:27:12,720 if I take a vector x-- 458 00:27:12,720 --> 00:27:18,250 any vector x-- could it possibly be an eigenvector? 459 00:27:18,250 --> 00:27:20,910 Well, apply A to it. 460 00:27:20,910 --> 00:27:24,510 You'd be off in this direction, Ax. 461 00:27:24,510 --> 00:27:30,390 And there is no way that Ax can be a multiple of x. 462 00:27:30,390 --> 00:27:34,950 So there's no real eigenvector for that anti-symmetric matrix, 463 00:27:34,950 --> 00:27:38,710 or any anti-symmetric matrix. 464 00:27:38,710 --> 00:27:43,480 So you see that when we say that the eigenvalues 465 00:27:43,480 --> 00:27:46,090 of a symmetric matrix are real, we're 466 00:27:46,090 --> 00:27:48,180 saying that this couldn't happen-- 467 00:27:48,180 --> 00:27:51,160 that this couldn't happen if A were symmetric. 468 00:27:51,160 --> 00:27:54,010 And here, it's the very opposite, it's anti-symmetric. 469 00:27:56,880 --> 00:27:59,820 Well, while that's on the board, you might say, wait a minute. 470 00:27:59,820 --> 00:28:02,160 How could that have any eigenvector whatsoever? 471 00:28:05,950 --> 00:28:09,300 So what is an eigenvector of that matrix A? 472 00:28:09,300 --> 00:28:13,080 How do you find the eigenvectors of A? 473 00:28:13,080 --> 00:28:19,850 When they're 2 by 2, that's a calculation we know how to do. 474 00:28:19,850 --> 00:28:21,950 You remember the steps there? 475 00:28:21,950 --> 00:28:26,690 I'm looking for Ax equal lambda x. 476 00:28:26,690 --> 00:28:30,020 So right now I'm looking for both lambda and x. 477 00:28:30,020 --> 00:28:31,050 I've got 2. 478 00:28:31,050 --> 00:28:35,900 It's not linear, but I'm going to bring this over to this side 479 00:28:35,900 --> 00:28:39,405 and write it as A minus lambda I, x equals 0. 480 00:28:43,263 --> 00:28:44,680 And then I'm going to look at that 481 00:28:44,680 --> 00:28:48,850 and say, wow, A minus lambda I must be not invertible, 482 00:28:48,850 --> 00:28:52,690 b because it's got this x in its null space. 483 00:28:52,690 --> 00:28:56,680 So the determinant of this matrix must be 0. 484 00:28:59,490 --> 00:29:06,300 I couldn't have a null space unless the determinant is 0. 485 00:29:06,300 --> 00:29:12,590 And then when I look at A minus lambda I, for this A, 486 00:29:12,590 --> 00:29:20,350 I've got minus lambdas, minus A-- 487 00:29:20,350 --> 00:29:21,820 oh, A is just the 1. 488 00:29:21,820 --> 00:29:23,700 And that's minus 1. 489 00:29:23,700 --> 00:29:26,250 I'm going to take the determinant. 490 00:29:26,250 --> 00:29:29,100 And what am I going to get for the determinant? 491 00:29:29,100 --> 00:29:30,732 Lambda squared-- 492 00:29:30,732 --> 00:29:31,722 AUDIENCE: Plus 1. 493 00:29:31,722 --> 00:29:32,680 GILBERT STRANG: Plus 1. 494 00:29:36,620 --> 00:29:38,156 And I set that to 0. 495 00:29:44,220 --> 00:29:48,020 So I'm just following all the rules, 496 00:29:48,020 --> 00:29:51,322 but it's showing me that the lambda-- 497 00:29:51,322 --> 00:29:55,150 the two lambdas-- there are two lambdas here-- 498 00:29:55,150 --> 00:29:58,300 but they're not real, because that equation, the roots 499 00:29:58,300 --> 00:29:59,770 are i and minus i. 500 00:30:02,660 --> 00:30:04,250 So those are the eigenvalues. 501 00:30:07,060 --> 00:30:08,710 And they have the nice-- 502 00:30:08,710 --> 00:30:10,180 they have all the-- 503 00:30:10,180 --> 00:30:11,530 well, they are the eigenvalues. 504 00:30:11,530 --> 00:30:12,520 No doubt about it. 505 00:30:15,560 --> 00:30:20,360 With 2 by 2 there are two quick checks that tell you, yeah, 506 00:30:20,360 --> 00:30:22,790 you did a calculation right. 507 00:30:22,790 --> 00:30:31,970 If I add up the two eigenvalues in this-- 508 00:30:31,970 --> 00:30:34,550 if I add up the two eigenvalues for any matrix, 509 00:30:34,550 --> 00:30:37,310 and I'm going to do it for this one-- 510 00:30:37,310 --> 00:30:38,420 I get what answer? 511 00:30:38,420 --> 00:30:39,500 AUDIENCE: The trace? 512 00:30:39,500 --> 00:30:43,430 GILBERT STRANG: I get the same answer from the adding-- 513 00:30:43,430 --> 00:30:48,410 add the lambdas gives me the same answer 514 00:30:48,410 --> 00:30:56,690 as add the diagonal of the matrix-- 515 00:30:56,690 --> 00:31:03,130 which I'm calling A. So if I add the diagonal I get 0 and 0. 516 00:31:03,130 --> 00:31:03,910 So it's 0 plus 0. 517 00:31:07,790 --> 00:31:11,165 And this number adding the diagonal is called the trace. 518 00:31:13,790 --> 00:31:19,130 And we'll see it again because it's so simple. 519 00:31:19,130 --> 00:31:22,910 Just adding the diagonal entries gives you 520 00:31:22,910 --> 00:31:25,370 a key bit of information. 521 00:31:25,370 --> 00:31:27,230 When you add down the diagonal it 522 00:31:27,230 --> 00:31:29,960 tells you the sum of the eigenvalue-- some 523 00:31:29,960 --> 00:31:32,670 of the lambdas. 524 00:31:32,670 --> 00:31:35,970 Doesn't tell you each lambda separately, 525 00:31:35,970 --> 00:31:37,890 but it tells you the sum. 526 00:31:37,890 --> 00:31:41,380 So it tells you one fact by doing one thing. 527 00:31:41,380 --> 00:31:42,210 Yeah. 528 00:31:42,210 --> 00:31:45,170 That's pretty handy. 529 00:31:45,170 --> 00:31:49,010 Gives you a quick check if you've-- 530 00:31:49,010 --> 00:31:50,840 when you compute this determinant 531 00:31:50,840 --> 00:31:54,414 and solve for lambda-- 532 00:31:54,414 --> 00:32:03,440 the thing you-- this is a way to compute eigenvalues by hand. 533 00:32:03,440 --> 00:32:05,120 You could make a mistake, because it's 534 00:32:05,120 --> 00:32:10,770 a quadratic formula for 2 by 2, but you can 535 00:32:10,770 --> 00:32:13,270 check by adding the two roots. 536 00:32:13,270 --> 00:32:17,640 Do you get the same as the trace 0 plus 0? 537 00:32:20,420 --> 00:32:25,370 Well, there's one other check, equally quick, for 2 by 2, 538 00:32:25,370 --> 00:32:26,690 so 2 by 2s-- 539 00:32:26,690 --> 00:32:28,940 you really get them right. 540 00:32:28,940 --> 00:32:31,070 What's the other check to-- 541 00:32:31,070 --> 00:32:33,860 we add the eigenvalues, we get the trace. 542 00:32:33,860 --> 00:32:34,970 AUDIENCE: [INAUDIBLE] 543 00:32:34,970 --> 00:32:37,010 GILBERT STRANG: We multiply the eigenvalues. 544 00:32:37,010 --> 00:32:45,640 So we take-- so now multiply the lambdas. 545 00:32:45,640 --> 00:32:50,600 So then I get i times minus i. 546 00:32:50,600 --> 00:32:54,550 And that should equal-- let's-- don't look yet. 547 00:32:54,550 --> 00:32:58,060 What should it equal if I multiply the eigenvalues 548 00:32:58,060 --> 00:32:59,870 I should get the? 549 00:32:59,870 --> 00:33:00,810 AUDIENCE: Determinant. 550 00:33:00,810 --> 00:33:02,700 GILBERT STRANG: Determinant, right. 551 00:33:02,700 --> 00:33:12,680 Of A. So that's two handy checks. 552 00:33:12,680 --> 00:33:14,900 Add the eigenvalues-- for any size-- 553 00:33:14,900 --> 00:33:18,230 3 by 3, 4 by 4-- but it's only two checks. 554 00:33:18,230 --> 00:33:21,080 So for 2 by 2, it's kind of, you've got it. 555 00:33:21,080 --> 00:33:23,300 3 by 3, 4 by 4-- you could still have 556 00:33:23,300 --> 00:33:29,630 made an error and the two checks could potentially still work. 557 00:33:29,630 --> 00:33:30,920 Let's just check it out here. 558 00:33:30,920 --> 00:33:32,570 What's i times minus i? 559 00:33:35,530 --> 00:33:36,030 AUDIENCE: 1. 560 00:33:36,030 --> 00:33:37,230 GILBERT STRANG: 1. 561 00:33:37,230 --> 00:33:40,290 Because it's minus i squared, and that's plus 1. 562 00:33:40,290 --> 00:33:44,950 And the determinant of that matrix is 0 minus-- 563 00:33:44,950 --> 00:33:45,450 is 1. 564 00:33:45,450 --> 00:33:47,170 Yeah. 565 00:33:47,170 --> 00:33:47,820 OK. 566 00:33:47,820 --> 00:33:49,150 So we got 1. 567 00:33:49,150 --> 00:33:49,650 Good. 568 00:33:53,610 --> 00:33:56,340 Those are really the key fact about eigenvalues. 569 00:33:59,010 --> 00:34:02,550 But of course they're not-- it's not 570 00:34:02,550 --> 00:34:06,360 as simple as solving Ax equal B to find them, 571 00:34:06,360 --> 00:34:13,980 but if you follow through on this idea of similar matrices, 572 00:34:13,980 --> 00:34:18,510 and sort of chop down the off diagonal part, then 573 00:34:18,510 --> 00:34:22,900 sure enough, the eigenvalue's gotta show up. 574 00:34:22,900 --> 00:34:24,790 OK. 575 00:34:24,790 --> 00:34:25,449 Symmetric. 576 00:34:27,989 --> 00:34:28,905 Symmetric matrices. 577 00:34:33,830 --> 00:34:38,830 So now we're going to have symmetric, 578 00:34:38,830 --> 00:34:42,639 and then we'll have the special, even better than symmetric, 579 00:34:42,639 --> 00:34:45,690 is symmetric positive definite. 580 00:34:45,690 --> 00:34:46,190 OK. 581 00:34:46,190 --> 00:34:56,330 Symmetric-- you told me the main facts are the eigenvalues real, 582 00:34:56,330 --> 00:35:02,040 the eigenvectors orthogonal. 583 00:35:08,030 --> 00:35:10,280 And I guess, actually-- 584 00:35:10,280 --> 00:35:11,420 yeah. 585 00:35:11,420 --> 00:35:17,380 So I want to put those into math symbols instead of words. 586 00:35:22,480 --> 00:35:26,500 So yeah. 587 00:35:26,500 --> 00:35:31,270 I guess-- shall I just jump in? 588 00:35:31,270 --> 00:35:36,830 And the other thing hidden there-- but very important is-- 589 00:35:36,830 --> 00:35:39,790 there's a full set of eigenvectors, 590 00:35:39,790 --> 00:35:42,400 even if some eigenvalues happen to be repeated, 591 00:35:42,400 --> 00:35:44,650 like the identity matrix. 592 00:35:44,650 --> 00:35:47,940 It's still got plenty of eigenvectors. 593 00:35:47,940 --> 00:35:51,150 So that's a added point that I've not made there. 594 00:35:51,150 --> 00:35:54,720 And I could prove those two statements, 595 00:35:54,720 --> 00:35:59,998 but why don't I ask you to accept them and go onward? 596 00:36:02,810 --> 00:36:05,240 What are we going to do with them? 597 00:36:05,240 --> 00:36:05,740 OK. 598 00:36:11,600 --> 00:36:13,430 Can you just-- let's have an example. 599 00:36:16,550 --> 00:36:18,690 Let me put an example here. 600 00:36:18,690 --> 00:36:22,910 Suppose S-- now I'm calling it S-- 601 00:36:22,910 --> 00:36:26,610 is 0s, 1 and 1. 602 00:36:26,610 --> 00:36:29,960 So that's symmetric. 603 00:36:29,960 --> 00:36:33,080 What are its eigenvalues? 604 00:36:33,080 --> 00:36:37,140 What are the eigenvalues of that symmetric matrix, S? 605 00:36:37,140 --> 00:36:38,370 AUDIENCE: Plus and minus 1. 606 00:36:38,370 --> 00:36:40,878 GILBERT STRANG: Plus and minus 1. 607 00:36:40,878 --> 00:36:43,660 Well, if you propose two eigenvalues, 608 00:36:43,660 --> 00:36:46,730 I'll write them down, 1 and minus 1. 609 00:36:46,730 --> 00:36:49,727 And then what will I do to check them? 610 00:36:49,727 --> 00:36:51,060 AUDIENCE: Trace and determinant. 611 00:36:51,060 --> 00:36:53,810 GILBERT STRANG: Trace and determinant. 612 00:36:53,810 --> 00:36:54,330 OK. 613 00:36:54,330 --> 00:36:57,600 So are they-- is it true that the eigenvalues 614 00:36:57,600 --> 00:37:01,600 are 1 and minus 1? 615 00:37:01,600 --> 00:37:02,370 OK. 616 00:37:02,370 --> 00:37:04,020 How do I check the trace? 617 00:37:04,020 --> 00:37:07,640 What is the trace of that matrix? 618 00:37:07,640 --> 00:37:08,350 0. 619 00:37:08,350 --> 00:37:10,630 And what's the sum of the eigenvalues-- 620 00:37:10,630 --> 00:37:11,450 0. 621 00:37:11,450 --> 00:37:12,570 Good. 622 00:37:12,570 --> 00:37:13,710 What about determinant? 623 00:37:13,710 --> 00:37:15,380 What's the determinant of S? 624 00:37:15,380 --> 00:37:16,290 AUDIENCE: Minus 1. 625 00:37:16,290 --> 00:37:17,290 GILBERT STRANG: Minus 1. 626 00:37:17,290 --> 00:37:19,290 The product of the eigenvalues-- minus 1. 627 00:37:19,290 --> 00:37:20,590 So we've got it. 628 00:37:20,590 --> 00:37:21,480 OK. 629 00:37:21,480 --> 00:37:24,820 What are the eigenvectors? 630 00:37:24,820 --> 00:37:29,070 What vector can you multiply by and it 631 00:37:29,070 --> 00:37:32,010 doesn't change direction-- in fact, doesn't change at all? 632 00:37:32,010 --> 00:37:35,735 I'm looking for the eigenvector that's a steady state? 633 00:37:35,735 --> 00:37:37,180 AUDIENCE: 0, 1? 634 00:37:37,180 --> 00:37:38,115 GILBERT STRANG: 0, 1? 635 00:37:38,115 --> 00:37:41,090 AUDIENCE: 1, 1. 636 00:37:41,090 --> 00:37:42,780 GILBERT STRANG: I think it's 1, 1. 637 00:37:42,780 --> 00:37:43,280 Yeah. 638 00:37:43,280 --> 00:37:45,380 So here is the lambdas. 639 00:37:45,380 --> 00:37:47,420 And then the eigenvectors are-- 640 00:37:47,420 --> 00:37:48,800 I think 1, 1. 641 00:37:51,670 --> 00:37:52,760 Is that right? 642 00:37:52,760 --> 00:37:53,260 Yeah. 643 00:37:53,260 --> 00:37:54,190 Sure. 644 00:37:54,190 --> 00:37:57,010 S is just a permutation here. 645 00:37:57,010 --> 00:37:59,230 It's just exchanging the two entries. 646 00:37:59,230 --> 00:38:01,770 So 1 and 1 won't change. 647 00:38:01,770 --> 00:38:04,378 And what's the other eigenvector? 648 00:38:04,378 --> 00:38:05,940 AUDIENCE: Minus 1? 649 00:38:05,940 --> 00:38:07,190 GILBERT STRANG: 1 and minus 1. 650 00:38:15,220 --> 00:38:19,090 And then, I'm thinking-- remembering about this similar 651 00:38:19,090 --> 00:38:20,140 stuff-- 652 00:38:20,140 --> 00:38:27,610 I'm thinking that S is similar to a matrix that 653 00:38:27,610 --> 00:38:29,710 just shows the eigenvalues. 654 00:38:29,710 --> 00:38:31,690 So S is similar to-- 655 00:38:31,690 --> 00:38:34,410 I'm going to put in an M-- 656 00:38:34,410 --> 00:38:37,960 well, I'm going to connect S-- that matrix-- 657 00:38:37,960 --> 00:38:45,160 with the eigenvalue matrix, which has the eigenvalues. 658 00:38:45,160 --> 00:38:48,430 So here is my-- 659 00:38:50,950 --> 00:38:53,770 everybody calls that matrix capital lambda, 660 00:38:53,770 --> 00:38:57,550 because everybody calls the eigenvalues little lambda. 661 00:38:57,550 --> 00:39:02,330 So the matrix that has them is called capital lambda. 662 00:39:02,330 --> 00:39:06,610 And I-- my claim is that these guys are similar-- 663 00:39:06,610 --> 00:39:09,650 that this matrix, S, that you're seeing up there-- 664 00:39:09,650 --> 00:39:12,890 I believe there is an M I believe 665 00:39:12,890 --> 00:39:15,650 there is an M. So that S-- 666 00:39:15,650 --> 00:39:17,510 what did I put in here? 667 00:39:17,510 --> 00:39:19,370 So I'm following this pattern. 668 00:39:19,370 --> 00:39:24,280 I believe that there would be an M and an M inverse, 669 00:39:24,280 --> 00:39:27,590 so that this would mean that. 670 00:39:27,590 --> 00:39:29,870 And that's nice. 671 00:39:29,870 --> 00:39:33,530 First of all, it would confirm that the eigenvalues 672 00:39:33,530 --> 00:39:38,210 stay the same, which was certain to happen. 673 00:39:38,210 --> 00:39:43,720 And then it would also mean that I had got a diagonal matrix. 674 00:39:43,720 --> 00:39:45,520 And of course, that's a natural goal-- 675 00:39:45,520 --> 00:39:46,945 to get a diagonal matrix. 676 00:39:49,540 --> 00:39:52,210 So we might hope that the M that gets us there 677 00:39:52,210 --> 00:39:57,510 is like an important matrix. 678 00:39:57,510 --> 00:39:59,520 So do you see what I'm doing here? 679 00:39:59,520 --> 00:40:04,350 It comes under the heading of diagonalizing a matrix. 680 00:40:04,350 --> 00:40:08,840 I start with a matrix, S. I find it's eigenvalues. 681 00:40:08,840 --> 00:40:11,420 They go on into lambda. 682 00:40:11,420 --> 00:40:19,980 And I believe I can find an M, so that I see they're similar. 683 00:40:19,980 --> 00:40:23,660 They have the same eigenvalues, 1 and minus 1, both sides. 684 00:40:23,660 --> 00:40:27,650 So only remaining question is, what's M? 685 00:40:27,650 --> 00:40:33,262 What's the matrix that diagonalizes S? 686 00:40:33,262 --> 00:40:35,745 The-- what have we got left to use? 687 00:40:35,745 --> 00:40:36,870 AUDIENCE: The eigenvectors. 688 00:40:36,870 --> 00:40:39,140 GILBERT STRANG: The eigenvectors. 689 00:40:39,140 --> 00:40:43,010 The matrix that-- so, can I put the M over there? 690 00:40:45,530 --> 00:40:46,460 Yeah. 691 00:40:46,460 --> 00:40:48,590 I'll put-- that M inverse is going 692 00:40:48,590 --> 00:40:52,180 to go over to the other side. 693 00:40:52,180 --> 00:40:52,680 Oh. 694 00:40:52,680 --> 00:40:54,800 It goes here, doesn't it? 695 00:40:54,800 --> 00:40:56,240 I was worried there. 696 00:40:56,240 --> 00:40:58,630 It didn't look good, but yeah. 697 00:40:58,630 --> 00:41:02,192 So this is all going to be right, if-- 698 00:41:07,990 --> 00:41:09,460 this is what I'd like to have-- 699 00:41:09,460 --> 00:41:11,980 SM equal M lambda. 700 00:41:11,980 --> 00:41:14,810 SM equal M lambda. 701 00:41:14,810 --> 00:41:17,120 That's diagonalizing a matrix. 702 00:41:17,120 --> 00:41:22,390 That's finding the M using the eigenvectors. 703 00:41:22,390 --> 00:41:25,850 That produces a similar matrix lambda, 704 00:41:25,850 --> 00:41:27,270 which has the eigenvalues. 705 00:41:27,270 --> 00:41:35,810 That's the great fact about diagonalizing. 706 00:41:35,810 --> 00:41:38,160 That's how you use-- that's another way to say, 707 00:41:38,160 --> 00:41:40,610 this is how the eigenvectors pay off. 708 00:41:40,610 --> 00:41:43,880 You put them into M. You take the similar matrix 709 00:41:43,880 --> 00:41:45,830 and it's nice and diagonal. 710 00:41:45,830 --> 00:41:47,970 And do you see that this will happen? 711 00:41:47,970 --> 00:41:51,410 S times-- so M has the first eigenvector 712 00:41:51,410 --> 00:41:53,870 and the second eigenvector. 713 00:41:53,870 --> 00:41:59,690 And I believe that first eigenvector times the second-- 714 00:41:59,690 --> 00:42:03,660 and the second eigenvector-- that's M again, on this side. 715 00:42:03,660 --> 00:42:08,375 Let me just write in 1, 0, 0, minus 1. 716 00:42:13,110 --> 00:42:17,620 I believe is has got to be confirming that we've 717 00:42:17,620 --> 00:42:19,510 done the thing right-- 718 00:42:19,510 --> 00:42:22,030 confirming that the eigenvectors work here. 719 00:42:24,550 --> 00:42:27,410 Please make sense out of that last line. 720 00:42:30,060 --> 00:42:33,180 When you see that last line, what do I 721 00:42:33,180 --> 00:42:35,460 mean to make sense out of it? 722 00:42:35,460 --> 00:42:37,860 I want to see that that's true. 723 00:42:37,860 --> 00:42:39,180 How do I see that-- 724 00:42:39,180 --> 00:42:40,770 how do I do this-- 725 00:42:40,770 --> 00:42:43,050 so what's the left side and what's the right side? 726 00:42:47,850 --> 00:42:50,910 So what-- if I multiply S by a couple 727 00:42:50,910 --> 00:42:54,013 of columns, what's the answer? 728 00:42:54,013 --> 00:42:55,615 AUDIENCE: Sx1 and Sx2. 729 00:42:55,615 --> 00:42:56,920 GILBERT STRANG: Sx1 and Sx2. 730 00:42:56,920 --> 00:42:59,590 That's the beauty of matrix multiplication. 731 00:42:59,590 --> 00:43:02,090 If I multiply a matrix by another matrix, 732 00:43:02,090 --> 00:43:05,400 I can do it a column at a time. 733 00:43:05,400 --> 00:43:07,690 There are four great ways to multiply matrices, 734 00:43:07,690 --> 00:43:10,900 so this is another one-- 735 00:43:10,900 --> 00:43:12,010 a column at a time. 736 00:43:12,010 --> 00:43:16,810 So this left hand side is Sx1, Sx2. 737 00:43:16,810 --> 00:43:19,810 I just do each column. 738 00:43:19,810 --> 00:43:23,030 And what about the right hand side? 739 00:43:23,030 --> 00:43:25,404 I can do that multiplication. 740 00:43:25,404 --> 00:43:26,730 AUDIENCE: X1 minus x2. 741 00:43:26,730 --> 00:43:29,860 GILBERT STRANG: X1 minus x2 did somebody say? 742 00:43:29,860 --> 00:43:31,840 Death. 743 00:43:31,840 --> 00:43:32,340 No. 744 00:43:32,340 --> 00:43:33,520 I don't want-- Oh, x1-- 745 00:43:33,520 --> 00:43:34,020 sorry. 746 00:43:34,020 --> 00:43:35,360 You said it right. 747 00:43:35,360 --> 00:43:36,330 OK. 748 00:43:36,330 --> 00:43:39,120 When you said x1 minus x2, I was subtracting. 749 00:43:39,120 --> 00:43:42,660 But you meant that that's-- the first column is x1, 750 00:43:42,660 --> 00:43:44,820 and the second column is minus x2. 751 00:43:44,820 --> 00:43:45,758 Correct. 752 00:43:45,758 --> 00:43:46,734 Sorry about that. 753 00:43:50,640 --> 00:43:53,170 And did we come out right? 754 00:43:53,170 --> 00:43:54,310 Yes. 755 00:43:54,310 --> 00:43:56,140 Of course, now I compare. 756 00:43:56,140 --> 00:43:59,990 Sx1 is lambda one x1. 757 00:43:59,990 --> 00:44:02,860 Sx2 is lambda two x2. 758 00:44:02,860 --> 00:44:03,520 And I'm golden. 759 00:44:08,040 --> 00:44:12,310 So what was the point of this board? 760 00:44:12,310 --> 00:44:15,640 What did we learn? 761 00:44:15,640 --> 00:44:20,750 We learned-- well, we kind of expected that the original S 762 00:44:20,750 --> 00:44:26,690 would be similar to the lambdas, because the eigenvalues match. 763 00:44:26,690 --> 00:44:28,790 S has eigenvalues lambda. 764 00:44:28,790 --> 00:44:30,860 And this diagonal matrix certainly 765 00:44:30,860 --> 00:44:33,260 has eigenvalues 1n minus 1. 766 00:44:33,260 --> 00:44:35,210 A diagonal matrix-- the eigenvalues 767 00:44:35,210 --> 00:44:37,050 are right in front of you. 768 00:44:37,050 --> 00:44:38,280 So they're similar. 769 00:44:38,280 --> 00:44:40,560 S is similar to the lambda. 770 00:44:40,560 --> 00:44:44,310 And there should be an M. And then somebody suggested, maybe 771 00:44:44,310 --> 00:44:46,260 the M is the eigenvectors. 772 00:44:46,260 --> 00:44:48,550 And that's the right answer. 773 00:44:48,550 --> 00:44:53,070 So finally, let me write that conclusion here-- 774 00:44:55,730 --> 00:44:59,650 which isn't just for symmetric matrices. 775 00:44:59,650 --> 00:45:04,710 So maybe I should put it for matrix A. 776 00:45:04,710 --> 00:45:12,480 So if it has lambdas and eigenvectors, 777 00:45:12,480 --> 00:45:22,300 and the claim is that A times the eigenvector matrix 778 00:45:22,300 --> 00:45:28,793 is the eigenvector matrix times the eigenvalues. 779 00:45:34,220 --> 00:45:39,750 And I would shorten that to Ax equals x lambda. 780 00:45:42,310 --> 00:45:44,200 And I could rewrite that, and then I'll 781 00:45:44,200 --> 00:45:49,270 slow down, as A equal x lambda x inverse. 782 00:45:58,120 --> 00:46:00,040 Really, this is bringing it all together 783 00:46:00,040 --> 00:46:02,740 in a simple, small formula. 784 00:46:02,740 --> 00:46:07,360 It's telling us that A is similar to lambda. 785 00:46:07,360 --> 00:46:10,360 It's telling us the matrix M, that does the job-- 786 00:46:10,360 --> 00:46:13,390 it's a matrix of eigenvectors. 787 00:46:13,390 --> 00:46:21,580 And so it's like a shorthand way to write the main fact 788 00:46:21,580 --> 00:46:24,390 about eigenvalues and eigenvectors. 789 00:46:24,390 --> 00:46:26,470 What about A squared? 790 00:46:26,470 --> 00:46:28,840 Can I go back to the very first-- 791 00:46:28,840 --> 00:46:31,510 I see time is close to the end here. 792 00:46:31,510 --> 00:46:33,850 What about A squared? 793 00:46:33,850 --> 00:46:36,810 What are the eigenvectors of A squared? 794 00:46:36,810 --> 00:46:39,410 What are the eigenvalues of A squared? 795 00:46:39,410 --> 00:46:42,700 That's like the whole point of eigenvalues. 796 00:46:42,700 --> 00:46:45,160 Well, or I could just square that stupid thing. 797 00:46:45,160 --> 00:46:50,140 X lambda, x inverse, x lambda, x inverse. 798 00:46:50,140 --> 00:46:52,600 And what have I got? 799 00:46:52,600 --> 00:46:54,730 X inverse, x in the middle is-- 800 00:46:54,730 --> 00:46:55,570 AUDIENCE: Identity. 801 00:46:55,570 --> 00:46:58,220 GILBERT STRANG: Identity. 802 00:46:58,220 --> 00:47:03,010 So I have x, lambda squared, x inverse. 803 00:47:03,010 --> 00:47:06,910 And to me and to you that says, the eigenvalues 804 00:47:06,910 --> 00:47:07,810 have been squared. 805 00:47:07,810 --> 00:47:10,840 The eigenvectors didn't change. 806 00:47:10,840 --> 00:47:11,740 Yeah. 807 00:47:11,740 --> 00:47:12,300 OK. 808 00:47:12,300 --> 00:47:15,250 And now finally, last breath is, what 809 00:47:15,250 --> 00:47:18,280 if the matrix is symmetric? 810 00:47:18,280 --> 00:47:19,870 Then we have different letters. 811 00:47:19,870 --> 00:47:23,850 That's the only-- that's the significant change. 812 00:47:23,850 --> 00:47:30,310 The eigenvector matrix is now an orthogonal matrix. 813 00:47:30,310 --> 00:47:34,850 I'm coming back to the key fact of what makes symmetric-- 814 00:47:34,850 --> 00:47:35,930 how do I read-- 815 00:47:35,930 --> 00:47:39,350 how do I see symmetric helping me in the eigenvector 816 00:47:39,350 --> 00:47:41,180 and eigenvalue world? 817 00:47:41,180 --> 00:47:46,960 Well, it tells me that the eigenvectors are orthogonal. 818 00:47:46,960 --> 00:47:53,140 So the x is Q. The eigenvalues are real. 819 00:47:53,140 --> 00:47:56,880 And the eigenvectors is x inverse. 820 00:47:56,880 --> 00:48:00,910 But now I'm going to make those eigenvectors unit vectors. 821 00:48:00,910 --> 00:48:02,230 I'm going to normalize it. 822 00:48:02,230 --> 00:48:04,330 So I'm really allowing-- 823 00:48:04,330 --> 00:48:08,020 I have an orthogonal matrix Q. So I have a different way 824 00:48:08,020 --> 00:48:10,780 to write this, and this is the end of the-- 825 00:48:10,780 --> 00:48:12,370 today's class. 826 00:48:12,370 --> 00:48:15,430 Q lambda. 827 00:48:15,430 --> 00:48:18,355 And what can you tell me about Q inverse? 828 00:48:18,355 --> 00:48:19,480 AUDIENCE: It's Q transpose. 829 00:48:19,480 --> 00:48:20,630 GILBERT STRANG: It's Q transpose. 830 00:48:20,630 --> 00:48:21,130 Thanks. 831 00:48:21,130 --> 00:48:23,240 So that was the last lecture. 832 00:48:23,240 --> 00:48:27,490 So now the orthogonal lecture is coming up 833 00:48:27,490 --> 00:48:31,600 at the last second of the symmetric matrices lecture. 834 00:48:31,600 --> 00:48:35,380 And this has the name spectral theorem, 835 00:48:35,380 --> 00:48:36,730 which I'll just put there. 836 00:48:40,560 --> 00:48:46,640 And the whole point is that it tells you 837 00:48:46,640 --> 00:48:50,430 what every symmetric matrix looks like-- 838 00:48:50,430 --> 00:48:54,965 orthogonal eigenvectors, real eigenvalues.