WEBVTT 1 00:00:01.739 --> 00:00:08.070 Good. 2 00:00:19.588 --> 00:00:30.120 You can hear me in case, sir. Okay. 1st at the start. 3 00:00:30.120 --> 00:00:33.630 In cases problems. 4 00:00:36.210 --> 00:00:41.189 Silence. 5 00:00:41.189 --> 00:00:47.219 Okay, so I just gave you my phone number here and. 6 00:00:48.600 --> 00:00:55.979 If there's a problem in class, and you cannot get hold of me like everything muted you cannot talk and so on then. 7 00:00:57.210 --> 00:01:05.189 You can call me and, of course, that means I might get a spam call during the class, but we have to work on the probabilities of this. Good. 8 00:01:05.189 --> 00:01:13.019 So, what I want to do is for the 1st class, I have a blog online and. 9 00:01:13.019 --> 00:01:20.280 I'll show it silence. 10 00:01:25.620 --> 00:01:34.079 Okay, so in theory you're seeing my screen now. Let me just show you at the start. How to. 11 00:01:34.079 --> 00:01:47.579 Find things if you search for something like RPI, you have a chance of getting my home page, depending on your search engine. That is. 12 00:01:47.579 --> 00:01:52.049 And then you go to teaching and then arrow. 13 00:01:52.049 --> 00:01:55.530 So, there's any questions with that that's how you can. 14 00:01:55.530 --> 00:02:01.290 How you can find stuff also what I'd like to do 1st, is. 15 00:02:01.290 --> 00:02:06.780 Going to keep this window open with chatting. 16 00:02:06.780 --> 00:02:10.740 So, in case people have questions you are. 17 00:02:11.879 --> 00:02:15.389 We are seeing the screen Thank you. So. 18 00:02:15.389 --> 00:02:26.939 If you have chat questions, there is a theory that I will see them right away. You're also welcome since this is a small class to speak up and so on. 19 00:02:26.939 --> 00:02:30.360 Silence. 20 00:02:31.830 --> 00:02:39.780 Okay also have it, so. 21 00:02:43.314 --> 00:02:43.555 So, 22 00:02:43.555 --> 00:02:44.935 it's the point of this course, 23 00:02:45.955 --> 00:02:58.884 my point is to give you hands on experience with parallel computing and to give you some theory and some tools that you can use in your research and use in other projects. 24 00:02:59.129 --> 00:03:02.729 And so target audience as well. 25 00:03:02.729 --> 00:03:06.449 And I see everyone in the class so far are. 26 00:03:06.449 --> 00:03:10.590 Seniors grads and so on now. 27 00:03:10.590 --> 00:03:17.550 I am not and I'm not trying to compete with the computer science. Course you can. 28 00:03:17.550 --> 00:03:27.930 Profitably take both courses. The computer science course goes more into supercomputer and so on and MTI and so on. 29 00:03:27.930 --> 00:03:32.039 I am teaching mostly tools that are. 30 00:03:34.050 --> 00:03:39.810 There are tools that you can usefully use you can afford maybe and. 31 00:03:40.889 --> 00:03:44.430 And use in your, you know, things that you can buy. 32 00:03:44.430 --> 00:03:56.759 If you can't afford a blue Jean, for example, so that's a unique part of my course I'm more down on teaching tools and I teach the theory and general principals in many cases. 33 00:03:56.759 --> 00:04:03.180 By way of teaching you the tools. So it's in that sense is a bottom up course. 34 00:04:04.379 --> 00:04:07.560 I'm going to teach you several different types of hardware here. 35 00:04:08.819 --> 00:04:16.649 And 1st, I'll start with the multi core Intel Z on. 36 00:04:16.649 --> 00:04:19.980 So that because, you know. 37 00:04:19.980 --> 00:04:23.879 Almost every machines are are multi core. 38 00:04:23.879 --> 00:04:27.538 And my phone is my phone is multi core. 39 00:04:27.538 --> 00:04:31.019 My laptop here is a duel 6 core. 40 00:04:31.019 --> 00:04:35.428 Dion, if you want to see just for the fun of it. 41 00:04:35.428 --> 00:04:41.968 And, okay, this is my laptop. 42 00:04:41.968 --> 00:04:48.538 Dual 6 core up here under 128 gigabytes of memory. 43 00:04:48.538 --> 00:04:56.098 So, it's a nice laptop. I have a machine in my lab parallel to, you. 44 00:04:56.098 --> 00:05:02.579 Dual 12 core system, so we'll start with teaching you that because it's easiest machine available. 45 00:05:02.579 --> 00:05:06.329 Then we'll move on to NVIDIA. 46 00:05:06.329 --> 00:05:11.999 And show you all, even the machine on my laptop, just. 47 00:05:11.999 --> 00:05:15.718 You know, I'm talking about affordable machines here. 48 00:05:15.718 --> 00:05:21.509 And so my laptop here. 49 00:05:21.509 --> 00:05:25.738 My this is the GPU on my laptop. 50 00:05:25.738 --> 00:05:29.218 It's 16 gigabytes of memory on. 51 00:05:29.218 --> 00:05:32.548 On the GPU with 3000 course. 52 00:05:32.548 --> 00:05:35.699 Not bad for a laptop, I think, you know. 53 00:05:35.699 --> 00:05:40.228 If I can boast a little it's a very good laptop. 54 00:05:40.228 --> 00:05:45.509 Okay, so and then parallel is. 55 00:05:45.509 --> 00:05:49.588 Is actually the best video gpo at the time? I. 56 00:05:49.588 --> 00:05:55.019 Time I bought it about a year ago, so we'll talk to you, but you can get. 57 00:05:55.019 --> 00:06:01.348 Good NVIDIA cards for a few 100 bucks so, or you can pay a few 1000 bucks and get excellent ones. 58 00:06:01.348 --> 00:06:06.028 And, of course, the 3rd of all PCs basically have and then them. 59 00:06:07.288 --> 00:06:19.163 And then we'll move on to quantum computing. I taught a pure quantum computing course in the fall, but I noticed at least as of yesterday that no, 1, taking my parallel course was also in the quantum computing course. 60 00:06:19.553 --> 00:06:23.394 So, I'll spend about a month, this spring on quantum computing census. 61 00:06:24.028 --> 00:06:28.829 Apparently new to all of you and it's new. It's important. 62 00:06:28.829 --> 00:06:36.569 And it could be useful in a few years. So that's 3 types of hardware. Each types of hardware has strengths and it has weaknesses. 63 00:06:36.569 --> 00:06:39.869 Just give you an initial thing. 64 00:06:39.869 --> 00:06:44.129 And video of thousands, of course, but 1. 65 00:06:44.129 --> 00:06:48.658 And video core is about 5% as powerful as 1 Z on core. 66 00:06:48.658 --> 00:06:53.278 Quantum computing is potentially very powerful, but. 67 00:06:53.278 --> 00:06:57.778 Not useful not yet. Maybe in a few years. 68 00:06:57.778 --> 00:07:02.369 Software, I'm going to teach you. 69 00:07:02.369 --> 00:07:08.968 Not every detail I'm going to introduce you to several different software packages. 70 00:07:08.968 --> 00:07:13.588 So, where these are useful for generally. 71 00:07:13.588 --> 00:07:20.819 If you have your algorithm is parallelizable, these are the tools that help you paralyze it. 72 00:07:20.819 --> 00:07:26.788 So, open ACC is a standard, which means. 73 00:07:26.788 --> 00:07:40.228 Many vendors are using it and it's also it's a living standard. It has it's being extended C plus plus compilers have it. So it is some extent some small extensions. 74 00:07:40.228 --> 00:07:50.309 To the C plus plus language so that the compiler and extensions under the hood did the compiler is of course, to try and compile your programs that will execute in parallel. 75 00:07:51.418 --> 00:08:01.468 Then there is, there's some extension there's a parallel version of the standard template library. S. T. L. that's called. 76 00:08:01.468 --> 00:08:05.459 Thrust sponsored by NVIDIA. 77 00:08:05.459 --> 00:08:08.668 And it says a functional programming. 78 00:08:08.668 --> 00:08:12.988 Ideas actually, and that is that will. 79 00:08:12.988 --> 00:08:17.848 We'll see introduction to that somehow way to do parallel. 80 00:08:17.848 --> 00:08:25.228 Parallelize your algorithm Matlab and Mathematica parallel tools in them. 81 00:08:25.228 --> 00:08:29.608 So, what may see a little of that, depending on time in people's interest. 82 00:08:29.608 --> 00:08:35.729 Mathematica is a symbolic programming package. It works with formulas. 83 00:08:35.729 --> 00:08:41.369 Not just numbers for specifically programming on. 84 00:08:41.369 --> 00:08:50.908 And videos, their low level language is called and that's the GPU analog to an assembly language on. 85 00:08:50.908 --> 00:09:00.028 And then tell me on, let's say, so, get you down at the low level programming on the CUDA cores, the different types of memory hierarchy and so on. 86 00:09:00.028 --> 00:09:04.408 And finally in video just recently has. 87 00:09:04.408 --> 00:09:08.428 A parallel C plus plus compiler that. 88 00:09:08.428 --> 00:09:14.729 Again, or it tries to embed thing after compiler do things automatically if. 89 00:09:14.729 --> 00:09:21.839 Your algorithm is parallelizable so it is to some extent obsoleting. Some of the other things. 90 00:09:23.158 --> 00:09:27.538 Now, these techniques that I'll be showing your applicable to. 91 00:09:27.538 --> 00:09:31.229 Larger and larger parallel machines, I mean. 92 00:09:31.229 --> 00:09:36.629 These things can be used on an affordable machine you could use them on my laptop, for example. 93 00:09:36.629 --> 00:09:41.548 But you need to also use them on a super computer. I just want to give you an example of here. 94 00:09:45.328 --> 00:09:48.629 So this is the. 95 00:09:51.178 --> 00:09:56.698 To look at the list here, summarize the the fastest 500 known computers in the world. 96 00:09:56.698 --> 00:10:00.989 Of course, there's very there might be very fast computers. 97 00:10:00.989 --> 00:10:11.129 That we don't know about and not just military computers. Houston, the oil companies might have very fast computers that they would not be talking about. For example. 98 00:10:11.129 --> 00:10:17.009 So, we look at number 2 here has and video. 99 00:10:17.009 --> 00:10:21.568 Voltage so number 3 has invidious the number 1. 100 00:10:21.568 --> 00:10:27.688 Does not, but so invidious are and the 2nd and 3rd. 101 00:10:27.688 --> 00:10:32.219 Fastest computers and the fastest known computers. 102 00:10:34.859 --> 00:10:40.528 Okay, and also we'll get into various other, you know. 103 00:10:40.528 --> 00:10:47.458 Effective program learn about resource limitations and so on prerequisite I put in chaos. 104 00:10:47.458 --> 00:10:54.568 Or whatever I forget what I put in, but permission of instructor also works. You got to have some minimal knowledge that C plus plus. 105 00:10:54.568 --> 00:10:57.808 The background about me. 106 00:10:57.808 --> 00:11:02.278 Ph. D. from Harvard batch master PT from Harvard. 107 00:11:02.278 --> 00:11:05.879 bachelor's from Toronto and. 108 00:11:05.879 --> 00:11:11.308 Well, you can call and way to get. Hold of me. Now, I'm working online this semester. 109 00:11:11.308 --> 00:11:17.698 So, I haven't been into my office in quite a long time. If you fall in my office number. However, it forwards. 110 00:11:17.698 --> 00:11:26.369 If you phone, you can yeah, I got many email addresses. This works if you send from some unusual email address. 111 00:11:26.369 --> 00:11:30.269 Then it helpful if you use your real name in the. 112 00:11:30.269 --> 00:11:35.489 In the front line, and maybe even put a hashtag in there. So. 113 00:11:35.489 --> 00:11:40.678 My Web site, which links to the course, which links to the course blog. 114 00:11:40.678 --> 00:11:46.318 And so you can Google me and I'll try to stay around if people have questions. So. 115 00:11:46.318 --> 00:11:51.089 Okay, and this, I'm going off the home page here. 116 00:11:51.089 --> 00:11:54.629 For the textbook I'm going to. 117 00:11:54.629 --> 00:12:04.558 Well, for different parts of the course, recommend a few things, they'll be optional. They'll be from free to 50 box or buy. So for could have, for example, there are. 118 00:12:04.558 --> 00:12:11.129 The some books here, you can also go to invidious free documentation online. 119 00:12:11.129 --> 00:12:16.019 Event and there's some free tutorial courses online the. 120 00:12:16.019 --> 00:12:19.408 Advantage of the boxes they might have bought tutorial information. 121 00:12:19.408 --> 00:12:24.119 But it's a problem I found with the books is that. 122 00:12:24.119 --> 00:12:32.428 This is a field it's growing it's growing very fast and some, especially for tools like, could a, for example. 123 00:12:32.428 --> 00:12:35.548 And I've noticed that the textbooks. 124 00:12:35.548 --> 00:12:42.119 Even if they've been updated lately, have not been updated enough to include the newest ideas. 125 00:12:42.119 --> 00:12:45.298 So that you're learning. 126 00:12:45.298 --> 00:12:49.708 So this is actually just instead of divine text box is here. 127 00:12:49.708 --> 00:12:53.339 They're going to teach you techniques that are obsolete. 128 00:12:53.339 --> 00:13:00.149 I'll give an example now with C plus plus plus plus for a few years says, had. 129 00:13:00.149 --> 00:13:04.078 Does in a little in line anonymous functions. 130 00:13:04.078 --> 00:13:07.798 And they're incredibly useful and. 131 00:13:07.798 --> 00:13:12.538 Especially in parallel programming and. 132 00:13:12.538 --> 00:13:16.528 A lot of the examples don't cover them. 133 00:13:16.528 --> 00:13:23.339 Another example of something that's fairly new is that Nvidia now? Has. 134 00:13:23.339 --> 00:13:28.198 Demand paging now has effectively virtual memory with demand paycheck. 135 00:13:28.198 --> 00:13:31.438 So, in the past. 136 00:13:31.438 --> 00:13:39.509 You'd have to explicitly copy a race between the hosts. That would be the Intel probably and the device that would be the. 137 00:13:39.509 --> 00:13:46.168 And there'd be subroutines that would copy a race back and forth and you'd have to explicitly do that. 138 00:13:46.168 --> 00:13:50.249 Now, for a couple of years now. 139 00:13:50.249 --> 00:13:55.379 You've been able to allocate an array so it's in. 140 00:13:55.379 --> 00:13:59.519 What's called, like, unified managed memory and. 141 00:13:59.519 --> 00:14:04.168 It'll get copied back and forth between the host and the device as needed. 142 00:14:05.548 --> 00:14:10.349 So you don't have to explicitly copy things now now. 143 00:14:10.349 --> 00:14:18.538 There can be a performance penalty because, you know how your program operates more than. 144 00:14:18.538 --> 00:14:23.999 You know, the operating system knows how your program operates. 145 00:14:23.999 --> 00:14:27.389 So you can perhaps do it better. 146 00:14:27.389 --> 00:14:31.918 However, that. 147 00:14:31.918 --> 00:14:36.359 You know, it's taking your time so, the question is, which is more important. 148 00:14:36.359 --> 00:14:41.068 I had a paper published some years ago, actually, where we beat the. 149 00:14:41.068 --> 00:14:44.729 The UNIX, the coupon to virtual memory. 150 00:14:44.729 --> 00:14:53.519 Manager for working with some trained data sets and line of sight, getting visibility on terrain when we're paging the train in and out, because we knew. 151 00:14:53.519 --> 00:14:59.458 How we were going to be accessing the terrain better than the virtual memory manager could infer. 152 00:15:00.354 --> 00:15:09.953 But still use the virtual memory manager, a lot of free stuff on the web. I'll give you links to that. I'll be talking about some Lawrence Livermore, National Lab stuff. 153 00:15:10.464 --> 00:15:16.374 The, the national labs I call them the energy Labs, because they do very energetic experiments. 154 00:15:16.678 --> 00:15:20.519 Make a ton range sometimes. 155 00:15:20.519 --> 00:15:24.389 Thought for a few decades, but that's why they were formed. 156 00:15:24.389 --> 00:15:28.408 And so the government Labs, the national labs. 157 00:15:28.408 --> 00:15:32.339 Lawrence Livermore, Sandia, and so on. 158 00:15:32.339 --> 00:15:40.649 Are very interested in Super computing because they like to simulate large Physic systems and so on. So we're going to use. 159 00:15:40.649 --> 00:15:46.649 Some of their tutorials computers I'm using. 160 00:15:46.649 --> 00:15:51.269 Okay, got a lab machine I'll give you all accounts on SS H into. 161 00:15:51.269 --> 00:15:57.688 So, I do a 14 core Z on quarter terabyte of main memory. 162 00:15:58.769 --> 00:16:04.288 And a nice GPU here. 163 00:16:04.288 --> 00:16:08.639 The GPU is 48 gigabytes of memory. 164 00:16:08.639 --> 00:16:17.938 Reasonable speed and reasonable number of course so real time re, tracing if you wanted to do that on a 2nd, slower machine. So. 165 00:16:19.828 --> 00:16:26.969 I can think through if you want to and see about it here. 166 00:16:26.969 --> 00:16:37.019 So, it's got other smaller stuff. 167 00:16:37.019 --> 00:16:42.269 And I'll have a, I'll give you accounts on that and website on that. 168 00:16:42.269 --> 00:16:46.349 I may use some stuff on Amazon. 169 00:16:46.349 --> 00:16:51.269 Later in the course for parallel computing, perhaps. 170 00:16:51.269 --> 00:16:54.269 I mean, for quantum computing, I'm sorry. 171 00:16:54.269 --> 00:17:00.389 Because Amazon actually has quantum computers available for a low cost price. 172 00:17:02.969 --> 00:17:06.898 Great scope. 173 00:17:06.898 --> 00:17:10.919 Give you a, joining code and I'll enter most of you. Webex. 174 00:17:10.919 --> 00:17:14.009 Um, I'm using playing now. 175 00:17:14.009 --> 00:17:17.068 Uh, may use the Webex teams later. 176 00:17:17.068 --> 00:17:20.699 Because it integrates in various things. 177 00:17:20.699 --> 00:17:24.419 The problem was, is, I couldn't get screen sharing working on it. 178 00:17:24.419 --> 00:17:33.179 And other tools, grades, homeworks, and in class presentations as being a small class, going to try to get. 179 00:17:33.179 --> 00:17:38.519 You are participating in class starting with next week actually. 180 00:17:38.519 --> 00:17:45.419 So homework, so you can do homeworks and team if you want grades, submit stuff in teams. 181 00:17:47.038 --> 00:17:51.209 Term projects or no exams term project. 182 00:17:51.209 --> 00:17:54.358 That's part of the course you can work. 183 00:17:54.358 --> 00:17:57.689 On teams of any size that you're like. 184 00:17:57.689 --> 00:18:03.269 You may combine this with another course, but you have to tell me that you're doing it and tell them. 185 00:18:03.269 --> 00:18:09.328 If your grad student combined it combined it with your research, you can build on previous work if you want. 186 00:18:09.328 --> 00:18:13.348 Just have to talk about it, have to have permission. 187 00:18:13.348 --> 00:18:17.038 Like, if it's on gate hub, automatically have permission, I guess. 188 00:18:17.038 --> 00:18:21.179 If there's anything on my website, then you've got permission. So. 189 00:18:21.179 --> 00:18:26.909 So, you can do something that's at least vaguely related to parallel computing implemented, demonstrate it document it and. 190 00:18:26.909 --> 00:18:32.669 Give a talk on it in class so various things. How many lines of code. 191 00:18:32.669 --> 00:18:39.808 That's hard to say I mean, I've got a little program test points and a Polygon. 192 00:18:39.808 --> 00:18:46.409 I wrote it some time ago and and see, it's 8 lines of code. 193 00:18:46.409 --> 00:18:51.749 To get it, or I can hear. Oops, Scott fix that. 1. 194 00:18:51.749 --> 00:18:56.368 Um, but it's a point that sometimes. 195 00:18:56.368 --> 00:19:01.169 Lines of Congress doesn't isn't that meaningful? And there's a story about. 196 00:19:01.169 --> 00:19:05.669 A few decades ago when Bill Gates was just starting, he was actually friendly with IBM at a point. 197 00:19:05.669 --> 00:19:13.528 And he was working with them, helping them, right? Parts of always 2, I think, which is an operating system. 198 00:19:13.528 --> 00:19:18.118 In fact, he took some of their code and rewrote it to be shorter. 199 00:19:18.118 --> 00:19:21.598 Which cars hurt that team he was working with because. 200 00:19:21.598 --> 00:19:29.939 And metric management metric was lines of code they produced and it just, you know, they just produced negative lines of code because of him. 201 00:19:31.378 --> 00:19:36.659 No thing so post an early warning system if we have to. 202 00:19:36.659 --> 00:19:39.989 Academic integrity you've seen this before. 203 00:19:39.989 --> 00:19:44.759 Giving you 0T and irrelevant lowers it give you Sarah on the relevant thing. 204 00:19:46.588 --> 00:19:52.679 Most read something in this course is when somebody submitted something to this course that they submit it to another course. 205 00:19:52.679 --> 00:19:56.818 And didn't say it, so so. 206 00:19:56.818 --> 00:19:59.848 I found out about it that assignment got 0T. So. 207 00:19:59.848 --> 00:20:03.749 Has got some other rules, we'll just assume that they are. 208 00:20:04.979 --> 00:20:11.818 There it's included here by reference and also I like the topic I created this course. 209 00:20:11.818 --> 00:20:16.318 I asked to create this course I created it and. 210 00:20:18.058 --> 00:20:21.239 So, I welcome feedback to improve it. 211 00:20:21.239 --> 00:20:26.489 And you're also if you don't want to talk to me directly. 212 00:20:26.489 --> 00:20:33.088 James losing undergrad head John. when's the department head? 213 00:20:33.088 --> 00:20:37.108 And so on any questions. 214 00:20:37.108 --> 00:20:43.888 There no. 215 00:20:43.888 --> 00:20:46.979 We're still here. Okay. 216 00:20:49.078 --> 00:20:52.679 You can still hear me, I guess. Yeah. Okay. 217 00:20:52.679 --> 00:20:58.499 So, it's going to class 1 stuff. 218 00:20:58.499 --> 00:21:02.759 Oh, now here I got stuff tag class homework syllabus. 219 00:21:09.449 --> 00:21:14.608 Ok, so I would like you to talk now for. 220 00:21:14.608 --> 00:21:21.058 Managed or 2, we can attach it to go down the class list. So, tell me about yourself. 221 00:21:21.058 --> 00:21:24.659 And if any particular interests and so on, then. 222 00:21:32.278 --> 00:21:35.878 Ben, would you like to tell me about yourself? 223 00:21:35.878 --> 00:21:38.939 Tell the class sure. 224 00:21:38.939 --> 00:21:42.358 My name's Ben, I'm a senior, um. 225 00:21:42.358 --> 00:21:46.288 Majoring in computer. 226 00:21:46.288 --> 00:21:50.098 Computer systems at engineering. 227 00:21:50.098 --> 00:21:54.419 I find interesting I've been interested in hardware and. 228 00:21:54.419 --> 00:21:58.409 Programming for a while, except for the course. 229 00:21:58.409 --> 00:22:05.489 No, thank you. So, what you find to do next year, get a job, go to grad school see what the market looks like. 230 00:22:05.489 --> 00:22:12.838 I'd like to get a job. I'm still working on that. Okay. Focusing a bit more on software. 231 00:22:12.838 --> 00:22:16.798 I'm looking to get into software. 232 00:22:16.798 --> 00:22:20.548 Software Engineering yeah, we'll see. 233 00:22:20.548 --> 00:22:24.898 No, thank you. So, Blaine, what are you doing? 234 00:22:24.898 --> 00:22:32.489 Hi, I'm a senior CSC and E student and I'm here because. 235 00:22:32.489 --> 00:22:44.038 I did I had internships over the past 2 summers where I did a lot of work in Python, using existing libraries that were built on top of CUDA and I just really wanted to understand. 236 00:22:44.038 --> 00:22:50.459 How any of that actually worked okay. 237 00:22:50.459 --> 00:23:03.659 Thank you. So, what do you plan to do next year? I don't know. Sort of split between. I've applied to a couple of graduate programs, or I've also been looking at jobs and sort of trying to figure out where I'm going to go there. 238 00:23:04.798 --> 00:23:09.179 Thank you Connor tell us about yourself. 239 00:23:09.179 --> 00:23:23.249 Yes, my name is Connor Duncan. I may dual. I currently for the last year year, and a half. I've been working for a defense contractor as an embedded software engineer. 240 00:23:23.249 --> 00:23:32.338 So, that's really where the interest in paralyzed computing, um, has come because embedded platforms um. 241 00:23:32.338 --> 00:23:37.439 No, speed is 1 thing, but it also resources and everything else. So. 242 00:23:37.439 --> 00:23:45.028 Just looking to dive deeper in and understand more about how it works and how it can be applied specifically and embedded platforms. 243 00:23:47.159 --> 00:23:50.578 Thank you. Are you also taking the computer science parallel? Course. 244 00:23:50.578 --> 00:23:54.298 I am not actually, I'm. 245 00:23:54.298 --> 00:23:57.929 Yeah, at this point, I'm just taking the course. 246 00:23:57.929 --> 00:24:05.338 Okay, thank you. Oh, can you tell us anything about the hardware you're working on or not? 247 00:24:06.773 --> 00:24:21.682 Yeah, I can so, um, a lot of armed processors I work on a lot of armed processors, but we're also starting to explore some of the risk 5 options. There's have started to explode a little bit in the embedded. 248 00:24:22.019 --> 00:24:28.648 World, and there are a few companies that are starting to develop the risk 5 processors that are actually. 249 00:24:28.648 --> 00:24:32.699 Starting to come up to speed with some of the arm stuff. So. 250 00:24:34.558 --> 00:24:38.939 Thank you. Let's see, Dan. 251 00:24:38.939 --> 00:24:42.868 Tell us about yourself. Hello? 252 00:24:42.868 --> 00:24:46.318 Why can you hear me? Yeah. Okay. Um. 253 00:24:46.318 --> 00:24:50.818 I'm a grad student and computer science. 254 00:24:50.818 --> 00:24:54.449 And I'm focusing on graphics. 255 00:24:54.449 --> 00:25:01.919 Or trying to so I thought it'd be a good idea to learn some parallel stuff. 256 00:25:03.749 --> 00:25:10.469 Yeah, I'm a 1st year. Oh, thank you. So, graphics are you working? Professor? Cutlery? 257 00:25:10.469 --> 00:25:13.888 Yes, what sort of project. 258 00:25:13.888 --> 00:25:20.969 I'm not sure yet I want to try and discuss that this semester. Yeah Yeah. 259 00:25:22.288 --> 00:25:34.499 Taking the CS file. Of course, I'm not I thought about it, but I thought this would be more, you know, we're friendly with each other. 260 00:25:34.499 --> 00:25:41.969 Isaac, how are you doing? I'm doing well, I'm a graduate student in the, um. 261 00:25:41.969 --> 00:25:46.348 The aerospace engineering department and, um. 262 00:25:46.348 --> 00:25:51.959 The lap I'm working in does a lot of I'm parallel computing for food dynamic simulations. 263 00:25:51.959 --> 00:25:59.699 And right now the parallelized part of that is, um, kind of black box to me. So that's why I'm taking the course. 264 00:26:00.868 --> 00:26:07.259 Thanks so what sort of software do you use there? Some high level cft packages or something. 265 00:26:07.259 --> 00:26:11.909 Well, the lab that we develop our own. 266 00:26:11.909 --> 00:26:15.568 Code so finite elements solver. 267 00:26:15.568 --> 00:26:20.489 Yeah, but we work with a lot of external laps too so. 268 00:26:21.509 --> 00:26:26.128 Does any Mark shepherd stuff? Oh, yes, definitely. 269 00:26:26.128 --> 00:26:29.398 And good stuff. Yeah, thanks. 270 00:26:29.398 --> 00:26:34.169 Jack tell us about yourself. 271 00:26:34.169 --> 00:26:38.489 Um, I'm a senior computer systems engineer. 272 00:26:38.489 --> 00:26:43.648 At home, I would consider myself a computer enthusiast. I like to play with. 273 00:26:43.648 --> 00:26:47.338 Networking and servers and stuff like that. Um. 274 00:26:47.338 --> 00:26:52.769 For the past year, I worked for a small company out of New York City. I was a. 275 00:26:52.769 --> 00:26:59.128 Technology consultant, and I had worked on a project there that gave me some exposure to some. 276 00:26:59.128 --> 00:27:02.848 A sync libraries and JavaScript, so. 277 00:27:02.848 --> 00:27:07.618 I figured I'd take this class and maybe get a little bit lower level experience. 278 00:27:07.618 --> 00:27:17.878 With parallel programming. Okay, thank you. We're not going to any Java script here, so you'll get a yeah, we're not overlapping with stuff. You already know. 279 00:27:18.989 --> 00:27:24.058 Okay, cool Joseph. How are you? 280 00:27:24.058 --> 00:27:31.138 I'm good how I'm good and I'm a senior computer systems engineering, computer science student. 281 00:27:31.138 --> 00:27:40.648 I won't take the course because I had more interest in parallel computing reading and stuff like that. It as kind of from me for a loop in the past and figured another course and that. 282 00:27:40.648 --> 00:27:48.989 Wouldn't hurt me as far as post graduation plans I have secured employment and that's what I plan to be doing. 283 00:27:51.538 --> 00:27:59.038 Can you tell us about it or is it still confidential? It's not confidential on the software engineer at math works upon graduation. 284 00:28:00.719 --> 00:28:05.249 Oh, mass works. Okay. So we could ask your questions. 285 00:28:05.249 --> 00:28:14.909 I know how to make Matt lab I know the code from that lab. I don't know how to make Matlab how to use math lab. Is that a better way to say? Okay. 286 00:28:14.909 --> 00:28:25.138 My entire job, there was getting other things to run in that lab because 1 of the lesser known features is you can pull libraries from, like, say Python, Internet lab. 287 00:28:25.138 --> 00:28:34.919 Yeah, I did a student students some years ago he called could a code from inside Matlab using the C interface. 288 00:28:34.919 --> 00:28:40.199 So, you can do that that yeah, so it makes a lot faster. Yeah. 289 00:28:43.409 --> 00:28:49.888 The end, there's a lot of overhead at the interface, but then once we got on to you, it was so fast to pay for the overhead. So. 290 00:28:51.118 --> 00:28:54.298 Justin. 291 00:28:54.298 --> 00:29:00.419 Hey, so I'm a junior electrical and computer, uh, engineering student. 292 00:29:00.419 --> 00:29:05.368 Over the last summer at a CO op some. 293 00:29:05.368 --> 00:29:10.108 Basic programming and C sharp so a bit more curious about parallel programming. Now. 294 00:29:12.959 --> 00:29:18.269 Great. So you can compare then see shark with C plus plus yeah. Yeah. A little bit. Yeah. 295 00:29:18.269 --> 00:29:31.648 Oh, thanks for including your photo. That's an old photo. Well, still, hey, you other people here's a suggestion maybe. So we could get a sense what you look like. I mean, I can look it up 1 by 1 in your. 296 00:29:31.648 --> 00:29:35.999 You know, and the class list, but it's nice if you put a photo up that other people can see you. 297 00:29:35.999 --> 00:29:39.148 Like, you need to do. 298 00:29:39.148 --> 00:29:45.388 Mark, how are you? Good. Thanks. 299 00:29:45.388 --> 00:29:49.169 So, I'm a senior student. 300 00:29:49.169 --> 00:29:56.159 And I believe I had the same. I'm on the same place that Justin was right now. Actually. 301 00:29:56.159 --> 00:30:02.368 Um, and it's for a CO op I'm doing a parallel call. 302 00:30:02.368 --> 00:30:08.219 And it's an embedded Linux system that I work with. So. 303 00:30:08.219 --> 00:30:13.078 I want to learn more about, you know, paralyzing and. 304 00:30:13.078 --> 00:30:18.088 Because I think it was, it might have been, um. 305 00:30:18.088 --> 00:30:23.398 Joseph effort forgot who mentioned how like an embedded system, you know, our resources are important and. 306 00:30:23.398 --> 00:30:27.778 Paralyzing is important. 307 00:30:27.778 --> 00:30:33.239 And I don't have too much experience with it. So I'm here to learn. 308 00:30:34.499 --> 00:30:39.358 Okay, thank you. And. 309 00:30:40.223 --> 00:30:53.963 How are you? Yeah, I'm good. Thank you. So, our senior told major, and I had been doing something related to machine learning and deep learning for about 2 years, mostly research. 310 00:30:54.324 --> 00:30:58.913 And so most of the deep learning techniques are related to kudos. 311 00:30:58.913 --> 00:31:10.913 So, I want to learn from our programming course, like, how I can use CUDA to support the machine learning process. 312 00:31:11.159 --> 00:31:14.578 Oh, thank you. 313 00:31:14.578 --> 00:31:18.838 So, does anyone have any general questions before. 314 00:31:19.919 --> 00:31:24.328 And move on. Okay now, let's. 315 00:31:26.459 --> 00:31:33.628 Okay, so okay, so, as you see, I've got to chat window up. 316 00:31:33.628 --> 00:31:36.719 Just in case you have questions, you can also speak up. 317 00:31:38.878 --> 00:31:46.949 The syllabus I have a tutorial here from Lawrence Livermore, national labs. 318 00:31:46.949 --> 00:31:50.608 And go through that some quickly. 319 00:31:51.719 --> 00:31:57.058 It's familiar to some of you and not to not to others. 320 00:31:58.648 --> 00:32:03.929 Okay, Eva. Okay. 321 00:32:05.278 --> 00:32:08.429 It's it's low level at the start, but. 322 00:32:08.429 --> 00:32:15.598 I said sequential machine. This is fairly low level stuff. 323 00:32:15.598 --> 00:32:20.939 The level stuff here. 324 00:32:22.558 --> 00:32:28.409 And again, most of you seeing this, but just in case that. 325 00:32:28.409 --> 00:32:37.588 Already court, Intel cars are super scaler in 1 cycle. They're doing many different things and. 326 00:32:37.588 --> 00:32:42.419 So, I won't go into detail. Most of you already know this. 327 00:32:42.419 --> 00:32:49.499 So this is a blue Jean IBM has actually canceled the blue jeans project. They're going to use. 328 00:32:49.499 --> 00:32:53.159 What they learned in that another in the next generation of stuff, but. 329 00:32:53.159 --> 00:32:56.848 Things in parallel. 330 00:32:56.848 --> 00:33:04.229 Get the Lawrence Livermore cluster or clubs or anything it's got hierarchies here. 331 00:33:04.229 --> 00:33:08.909 That will cause and that seemed interesting. There. I'm going through things quickly. 332 00:33:08.909 --> 00:33:18.808 And the point here is that I made before well, with the NVIDIA, is that. 333 00:33:18.808 --> 00:33:25.048 We got these, we get the high performance by in many cases clustering together stuff. That's. 334 00:33:27.058 --> 00:33:35.038 Well, the cluster connection office is trivial, but clustering together off the shelf components to some extent. 335 00:33:35.038 --> 00:33:45.719 Why parallel computing is useful is going problems are trying to solve maybe massively parallel. 336 00:33:45.719 --> 00:33:54.148 Analyzing sorts of climate and whatever so the, it's nice when the computer model matches. 337 00:33:54.148 --> 00:33:58.259 What the problem that you're working on, there's a match there, you're going to get. 338 00:33:58.259 --> 00:34:02.249 It's more it's cleaner, it's more efficient. So these are. 339 00:34:02.249 --> 00:34:06.689 Parallel sorts of problems you're solving on many different elements or something. 340 00:34:07.979 --> 00:34:15.148 Other points so are the parallel machine stuff can be faster. 341 00:34:15.148 --> 00:34:19.378 Stuff can be bigger and so I'm going to this fast because it's fairly. 342 00:34:19.378 --> 00:34:22.498 Obvious here, it gets deeper later, so. 343 00:34:22.498 --> 00:34:27.568 Performance goes up now I'm like that. 344 00:34:28.739 --> 00:34:35.878 Yeah, you can browse through it, so I'm just scanning through it here. You can. 345 00:34:38.369 --> 00:34:48.418 Um, some history stuff. 346 00:34:48.418 --> 00:34:57.148 Is that you may or may not have seen as a buzzword that's worth looking at called the volume and architecture. 347 00:34:57.148 --> 00:35:02.219 This is a classical stored program computer. 348 00:35:02.219 --> 00:35:06.088 You've got instructions and data so what was new. 349 00:35:07.259 --> 00:35:11.518 Well, I mean, Alan Turing, for example, you know, he's gotten a lot of press. 350 00:35:11.518 --> 00:35:16.228 1 of the things that he and not just him, but some magicians. 351 00:35:16.228 --> 00:35:26.159 Get irritation instead, is that before that there was special purpose computers if people were thinking of, you might have a computer that. 352 00:35:27.268 --> 00:35:31.349 Factored numbers or whatever, and different special purpose computers. 353 00:35:33.414 --> 00:35:46.103 And they even had what Turing had, he had an abstract model for a computer called a turning machine, which is a theoretical formal way to describe how you would have like a stored program computer. Perhaps you'd have. 354 00:35:46.528 --> 00:35:51.958 Some data and the turning machine thing, the data would be on a 8 and you have a program. 355 00:35:51.958 --> 00:36:04.858 And you have a program calendar, like, point to somewhere on the take and it'd be rules you'd read what was there, you'd follow some rules, you may overwrite what's on the tape and then move left to right? It's. 356 00:36:04.858 --> 00:36:08.789 It was just a theoretical way to describe a particular computer. 357 00:36:08.789 --> 00:36:14.009 And with particular computer, it was would be the program, which said what you get at each step. 358 00:36:14.009 --> 00:36:23.728 Now, what turning dead is, he invented a concept called a universal Turing machine and this turning machine could emulate. 359 00:36:23.728 --> 00:36:31.438 Every other Turing machine, so part of the data for the universal turning machine was that. 360 00:36:31.438 --> 00:36:38.699 What would be the description of a program for any other turning machine? See, all you needed was 1 computer. You did not need different computers. 361 00:36:38.699 --> 00:36:44.579 For each application, you could write a program and so. 362 00:36:44.579 --> 00:36:52.079 The theoretical contribution of this was the concept that you could write a program that would describe how any computer. 363 00:36:52.079 --> 00:36:55.708 Would operate and this was. 364 00:36:55.708 --> 00:37:00.509 So that was his contribution, all the other contributions working on code breaking and so on. 365 00:37:00.509 --> 00:37:04.108 So, Nyman then was also. 366 00:37:05.309 --> 00:37:08.759 I'm describing this concept here that. 367 00:37:08.759 --> 00:37:14.759 You would have a general computer, and it could do anything within resource limitations. 368 00:37:14.759 --> 00:37:18.449 You go to a program which tells it what to do. So. 369 00:37:18.449 --> 00:37:23.938 And you'd have your breakdown, like you have, and Coke or something here. 370 00:37:23.938 --> 00:37:32.579 Seeing sorry your data so, and you've seen that 1 before. 371 00:37:32.579 --> 00:37:37.498 Okay, and the relevance of this. 372 00:37:37.498 --> 00:37:42.838 To parallel computing is it's the same idea. Just in parallel. 373 00:37:44.188 --> 00:37:51.838 Now, okay, this might be starting to get, you know, here's the, your sequential computers are just. 374 00:37:52.889 --> 00:37:59.429 Single instruction, single data stream my laptop, which is a dual 6 core Z on. 375 00:37:59.429 --> 00:38:06.628 Would be multiple instruction, single, a multiple data streams each. 376 00:38:06.628 --> 00:38:09.778 Each corps is running a separate instruction set. 377 00:38:09.778 --> 00:38:13.409 The CUDA cores let's get to later. 378 00:38:13.409 --> 00:38:19.170 Our single instruction, multiple data stream. So what happens with a could a car. 379 00:38:19.170 --> 00:38:22.889 Is that 1 instruction is. 380 00:38:22.889 --> 00:38:26.940 Applied to 32 data streams in parallel. 381 00:38:26.940 --> 00:38:30.179 So 1 instruction is called. 382 00:38:30.179 --> 00:38:37.980 Out of the program where wherever the program counter is pointing, and it gets applied to the data. 383 00:38:37.980 --> 00:38:41.670 32 sets of data in parallel. 384 00:38:41.670 --> 00:38:52.769 This would be called 132 threads would be called what's a warp? Actually is if you even cloth. So the could is single instruction, multiple data stream. 385 00:38:52.769 --> 00:39:01.260 And that if you can work ahead of me, what does this apply. 386 00:39:01.260 --> 00:39:04.559 Well, a smaller fraction. 387 00:39:04.559 --> 00:39:08.340 Chip is dedicated to instruction. 388 00:39:08.340 --> 00:39:11.789 Be coating and so on and you have to have. 389 00:39:11.789 --> 00:39:16.050 32 sets of data that want the same thing done to them. 390 00:39:16.050 --> 00:39:23.190 Which is multiple instructions single data stream would be if you wanted to do several things to the same type of data. 391 00:39:23.190 --> 00:39:26.219 I don't know anyone that actually does that. 392 00:39:26.219 --> 00:39:32.789 Okay, single instructions single data stream. 393 00:39:32.789 --> 00:39:36.210 Do it. 394 00:39:36.210 --> 00:39:40.889 Okay. 395 00:39:42.599 --> 00:39:47.489 Okay, and. 396 00:39:49.019 --> 00:39:54.150 Vulnerable instruction single data. Very unusual. 397 00:39:54.150 --> 00:40:00.960 Well, the example would be code breaking actually try several cryptography algorithms on the same data. Shame. 398 00:40:00.960 --> 00:40:04.139 Monday would be the Z on and so on. 399 00:40:04.139 --> 00:40:08.130 And you will also send wheels some 70. 400 00:40:08.130 --> 00:40:11.369 Some components of the machine. 401 00:40:17.489 --> 00:40:25.679 Terminology here, I'm going to go through quickly again. I'm just previewing this because I expect you to look at it later. 402 00:40:25.679 --> 00:40:34.110 And the terminology is also not standardized, so that's why I'll go through skip through it. 403 00:40:34.110 --> 00:40:43.949 The task this terminology is not standardized either. So that's why I'm skipping through it fast. I'll give you. 404 00:40:43.949 --> 00:40:54.539 Examples I find you've seen before shared memory, if you also think about the hardware implications of this. 405 00:40:54.539 --> 00:40:59.969 For example, of course, of access to the same memory, then you have to have some way. 406 00:40:59.969 --> 00:41:05.670 To ensure that the cores have a consistent view of the memory if the car has a cash. 407 00:41:05.670 --> 00:41:15.360 Then the other car won't see the cash for example. So, how do you handle that latest version of C plus plus allows the different course to have inconsistent views of the memory? In fact. 408 00:41:15.360 --> 00:41:20.130 Various other things yeah. 409 00:41:20.130 --> 00:41:24.059 You can grab that yourself. 410 00:41:24.059 --> 00:41:30.989 May want to serialize things that's 2 course of inconsistent views of the same. 411 00:41:30.989 --> 00:41:35.429 Memory at some point, you want things to become consistent so. 412 00:41:37.889 --> 00:41:52.110 Parallel overhead I've observed with some parallel tools if I, if I run the program with 2 threads instead of 1 thread, it takes more real time, but 2 threads, and it does with 1 threat. 413 00:41:52.110 --> 00:41:56.639 Go figure, so. 414 00:41:56.639 --> 00:42:03.030 It gets crazy because the parallel overhead was more than the speed up. 415 00:42:04.320 --> 00:42:08.369 Hello. 416 00:42:08.369 --> 00:42:13.889 There is embarrassingly parallel. 417 00:42:13.889 --> 00:42:17.400 Okay, there's some terminology here. 418 00:42:17.400 --> 00:42:20.519 Which is interesting is. 419 00:42:21.539 --> 00:42:26.579 When you get a machine, which is larger. 420 00:42:26.579 --> 00:42:32.610 You might want to use it on the same old problem and do the same old problem faster. 421 00:42:32.610 --> 00:42:36.840 Except that may not work, because the same old problem may not. 422 00:42:36.840 --> 00:42:46.739 Be parallelizable to a greater extent. So they often talk about when they talk about parallel speed up, they talk about doing a bigger problem on the bigger machine. 423 00:42:46.739 --> 00:42:50.099 And that's how they get talk about the scalability. So. 424 00:42:51.840 --> 00:42:56.429 There's an abdul's law here and what this says that. 425 00:42:56.429 --> 00:43:00.389 If let's say half of your program. 426 00:43:00.389 --> 00:43:03.840 Can be done in parallel and half. 427 00:43:03.840 --> 00:43:10.829 Cannot then you'll never get the program more than twice as fast even depending how many course you'll have. That's that. 428 00:43:10.829 --> 00:43:15.750 So, the speed up depends on what fraction of the program. 429 00:43:15.750 --> 00:43:22.889 Can be parallelized, if you can't paralyze any of the program, then. 430 00:43:22.889 --> 00:43:35.909 You get no speed up just makes common sense. So, and depending on what fraction of the program can be done in parallel, the more speed up, you'll get. 431 00:43:35.909 --> 00:43:44.159 So, 90% of the progress parallelizable, you'll never get more than a factor of 10 speed up because. 432 00:43:44.159 --> 00:43:47.400 As a serial part, I'll never get any past ourselves. 433 00:43:47.400 --> 00:43:54.210 Tell us what to work on so, and. 434 00:43:57.539 --> 00:44:01.289 But here's the point I mentioned before that. 435 00:44:02.460 --> 00:44:11.280 1 way people measure the performance of a new parallel machine is that you can do a larger problem. 436 00:44:11.280 --> 00:44:23.760 In parallel, so the old problem may not be more parallelizable because it's got this essentially cereal part, but you into the larger, you can do a larger problem faster. So this is. 437 00:44:24.869 --> 00:44:30.659 Document increase the problem size, so. 438 00:44:30.659 --> 00:44:34.139 Yeah. Okay. Complexity. 439 00:44:36.750 --> 00:44:40.949 Firewall is in his heart yeah. More detail. 440 00:44:40.949 --> 00:44:47.909 For working on a serious software project. 441 00:44:47.909 --> 00:44:53.010 And we look at the relative costs up to a release. 442 00:44:53.010 --> 00:44:57.119 This all of this. 443 00:44:57.119 --> 00:45:01.800 Might be half and then maintenance would cost as much as everything before it. 444 00:45:03.090 --> 00:45:09.329 Portability issues there is a big attempt. 445 00:45:09.329 --> 00:45:14.820 To try to get parallel tools, which are portable to different classes of hardware. 446 00:45:14.820 --> 00:45:18.510 So the things with open ACC. 447 00:45:18.510 --> 00:45:23.250 The idea is that and other things here. 448 00:45:23.250 --> 00:45:27.420 Is to try to have it, so you don't have to totally rewrite your program. 449 00:45:28.469 --> 00:45:34.710 If you want to use a different platform, so this is a problem with things such as kudo for example. 450 00:45:34.710 --> 00:45:38.760 That gets you low level access. 451 00:45:38.760 --> 00:45:42.239 To the NVIDIA. 452 00:45:42.239 --> 00:45:45.269 But it's not going to work on any other platform. 453 00:45:45.269 --> 00:45:49.769 So, there's your trade off. 454 00:45:49.769 --> 00:45:53.250 If you use higher level tools. 455 00:45:53.250 --> 00:45:58.110 They won't give you the performance that could have does however, they will. 456 00:45:59.340 --> 00:46:03.119 You don't have to totally rewrite the program to work on a new platform. 457 00:46:03.119 --> 00:46:06.960 So this is a lot of effort going on to things like this. 458 00:46:06.960 --> 00:46:11.309 So. 459 00:46:11.309 --> 00:46:16.800 Here's a point another point here I'm hitting the high points was parallel. 460 00:46:16.800 --> 00:46:22.949 Programming the usual metric is wall clock time elapsed time. 461 00:46:22.949 --> 00:46:26.219 Teach you time is not so important. 462 00:46:26.219 --> 00:46:31.650 So, if I'm using my laptop and I can run something, let's say. 463 00:46:31.650 --> 00:46:40.800 5 times faster I don't care that it may take a lot more total CPU time. I care about wall clock time. 464 00:46:40.800 --> 00:46:45.690 Now, there's a couple of reasons for this. 465 00:46:45.690 --> 00:46:54.570 Number 1 is some applications like weather prediction. Wall clock time is the important thing you want to. 466 00:46:54.570 --> 00:46:58.199 Predict tomorrow's weather before tomorrow arrives. 467 00:46:58.199 --> 00:47:02.130 And extra time. 468 00:47:02.130 --> 00:47:07.739 I mean, the CPA is sitting there whether or not, you use it, so you. 469 00:47:07.739 --> 00:47:14.489 If I have my laptop, the 6 cores, they're there, whether or not, I use them. So. 470 00:47:14.489 --> 00:47:22.050 If you might say, if I'm wasting them using more cfu time, it's not so important. Maybe I could be running something else. That's a fair criticism. 471 00:47:22.050 --> 00:47:25.739 But it's also difficult to measure CPU time. 472 00:47:25.739 --> 00:47:33.960 And some ways of paralyzing actually, just burn CPU time a lot. So, what people do is, they look at a lapsed time. 473 00:47:33.960 --> 00:47:40.800 O'clock on, going in parallel could increase memory use because. 474 00:47:40.800 --> 00:47:47.250 And so on this is the point I mentioned here decrease in performance for. 475 00:47:47.250 --> 00:47:52.860 Not just short riding because it's the overhead and setting up the parallel programming and stuff. 476 00:47:56.789 --> 00:48:00.449 I mean, this point here can be surprising. Okay scalability. 477 00:48:00.449 --> 00:48:07.050 I've mentioned it before. Okay. 478 00:48:07.050 --> 00:48:11.400 So, how do you measure performance as your machine gets bigger? 479 00:48:11.400 --> 00:48:22.679 The strong scaling would be to sit, you run the same problem on the bigger machine and it gets faster. So if the machine is twice as powerful, your program runs and half the time. 480 00:48:22.679 --> 00:48:27.989 Ain't going to happen, but this would be perfect. Scaling would be. 481 00:48:29.340 --> 00:48:34.110 P processes it runs in 1 of the time. That's not going to happen. 482 00:48:34.110 --> 00:48:40.469 Oh, really unusual. Can't say won't ever happen. You get the concept. 483 00:48:40.469 --> 00:48:43.650 Now, weak scaling is that. 484 00:48:43.650 --> 00:48:48.510 As we add more processors, we make the problem bigger so the. 485 00:48:48.510 --> 00:48:52.050 Work per processor stays the same. 486 00:48:54.329 --> 00:49:00.329 So, the goal is to run the same problem around bigger problems in the same amount of time. So perfect week scaling. 487 00:49:00.329 --> 00:49:05.190 Would be if you have P processors in a bigger program, it takes the same time. 488 00:49:05.190 --> 00:49:09.539 Yeah, well that may not happen either because. 489 00:49:09.539 --> 00:49:13.800 Larger process, if something has to communicate eventually and so on. 490 00:49:16.320 --> 00:49:22.469 And the point is, yeah, something may scale to some. This is motherhood something scale. So a certain point it. 491 00:49:22.469 --> 00:49:26.280 The obvious hardware factors at play an issue and so on. 492 00:49:29.550 --> 00:49:34.469 Some points here. 493 00:49:34.469 --> 00:49:39.449 General characteristics with shared memory. 494 00:49:39.449 --> 00:49:44.280 That we've got different types of shared memory, the uniform thing like. 495 00:49:45.989 --> 00:49:50.880 All the processes of equal access, more or less to all the banks of memory. 496 00:49:53.429 --> 00:49:58.920 Uniform thing, and so you have some. 497 00:49:58.920 --> 00:50:02.730 You have to some way to ensure the different to have. 498 00:50:02.730 --> 00:50:12.690 Inconsistent few now, it's not going to be entirely uniformed. Perhaps, for example, talk about the Intel Z on. 499 00:50:12.690 --> 00:50:16.590 Like, in a parallel machine it'll have access to and also. 500 00:50:16.590 --> 00:50:23.280 Like, my laptop, which is not an obviously far different from the parallel machine in my lab. 501 00:50:25.019 --> 00:50:28.980 The memory is in fact, physically attached to 1. 502 00:50:28.980 --> 00:50:38.760 Some core some of course, or other CPU cores. So it, in fact is a little faster for some CPU some course to get at it. 503 00:50:38.760 --> 00:50:42.960 And, in fact, with Linux, the. 504 00:50:42.960 --> 00:50:48.570 Which, of course, the page of memory is closer to. 505 00:50:48.570 --> 00:50:53.190 Depends on which core 1st references in fact. 506 00:50:55.019 --> 00:50:59.849 On the non uniform thing, we're starting approach MTI and so on. 507 00:50:59.849 --> 00:51:03.900 Or seriously, differences and speeds between. 508 00:51:03.900 --> 00:51:07.380 Let's see few accesses the memory. 509 00:51:09.780 --> 00:51:13.170 And again, cash, coherent issues. 510 00:51:13.170 --> 00:51:18.960 And advantages and disadvantages. Now, 1 reason I'd like. 511 00:51:20.550 --> 00:51:24.570 Talk about multi core on programming. 512 00:51:24.570 --> 00:51:32.400 Is that the cores all have access to the large global memory? 513 00:51:32.400 --> 00:51:37.440 Fairly quickly was cashes involved problem with the GPU. 514 00:51:37.440 --> 00:51:41.789 Well, 1st, there's much less memory on the GPU card. 515 00:51:41.789 --> 00:51:46.260 And if you have, like, a gaming very, very much less memory. 516 00:51:46.260 --> 00:51:51.150 And the 2nd problem is what is the hierarchy of memory. 517 00:51:51.150 --> 00:51:54.539 The large global memory on the. 518 00:51:54.539 --> 00:52:03.539 gpo is very slow to access. It has a high bandwidth, but it has a large latency, which will get to later. 519 00:52:03.539 --> 00:52:10.500 So advantage of the programming with a large school address, based on the. 520 00:52:10.500 --> 00:52:13.800 Cpu the host of the device. 521 00:52:15.420 --> 00:52:18.929 Um, some scalability problems and so on, but. 522 00:52:20.849 --> 00:52:26.909 Distributed memory, I am going to go light on distributed memory and I'll tell you why. 523 00:52:26.909 --> 00:52:30.900 In my opinion, a lot. 524 00:52:30.900 --> 00:52:39.000 Of jobs that people want to run, do not require distributed memory. 525 00:52:39.000 --> 00:52:46.559 Like I said, my laptop here, he could scarcely my laptop here has 128 gigabytes of real. 526 00:52:46.559 --> 00:52:50.489 Ever correcting DRAM so. 527 00:52:50.489 --> 00:52:58.559 I don't need to go virtual. I can run a pretty big problem in real memory. I don't need to go virtual. 528 00:52:58.559 --> 00:53:03.570 And the parallel machine in my lab has doubled that 256. 529 00:53:03.570 --> 00:53:06.599 Gigabytes of RAM D RAM. 530 00:53:06.599 --> 00:53:17.849 You don't need to go distributed. Well, so you have a problem. That's too big for that. Well, you could buy more memory. You can buy workstations now, which have, I think. 531 00:53:17.849 --> 00:53:21.210 2 terabytes of DRAM. 532 00:53:21.210 --> 00:53:26.789 No paging you don't need to go distributed so I get into disagreements with people about this. 533 00:53:26.789 --> 00:53:34.469 If you have to go distribute it, you have to go distributed, but you'd be amazed how big a problem you can run in real memory. 534 00:53:34.469 --> 00:53:37.679 You can debate me on this, but. 535 00:53:37.679 --> 00:53:44.159 This is actually why I figure that MTI is not needed a lot of the time. 536 00:53:47.250 --> 00:53:51.630 If you're using your local memory, you don't need npi just adds overhead. So. 537 00:53:54.000 --> 00:53:57.719 It also means. 538 00:53:57.719 --> 00:54:06.360 Never page, because given the memory on my laptop, the cost to page that memory in and out once with just takes so long. 539 00:54:06.360 --> 00:54:13.050 The only point for virtual memory in a case, like, this is to have 1 address space. 540 00:54:13.050 --> 00:54:17.940 Or different sets of resources. So, for example. 541 00:54:17.940 --> 00:54:24.690 Well, the CPO and the gpo together would be in 1 address space 49 with a 49. 542 00:54:24.690 --> 00:54:31.079 Virtual address and they'd be paging between CPO and the GP be paging between the host and the device. 543 00:54:31.079 --> 00:54:40.739 But not paging within the host itself, because he can't afford it. It's, it takes too long. And you got the, because the memory got bigger faster than. 544 00:54:40.739 --> 00:54:46.949 The bandwidth to the memory got February between the memory and the disc got bigger even though this is a fast as deep. 545 00:54:46.949 --> 00:54:52.530 So, okay, so that's this. 546 00:54:52.530 --> 00:54:56.190 My opinion on distributed memory here you usually don't need it. 547 00:54:56.190 --> 00:55:01.409 But debate me if you want and. 548 00:55:03.150 --> 00:55:08.280 There's no concept well, not completely. Sometimes areas. 549 00:55:10.769 --> 00:55:14.820 other points saw here's the another one point here . 550 00:55:14.820 --> 00:55:22.050 Programmers still have to worry about some of the synchronization. The tools don't do it all the time. Now. 551 00:55:24.570 --> 00:55:28.980 And you may have gotten the theme that. 552 00:55:28.980 --> 00:55:38.219 I am an advocate of lots of local memory. In fact, this was 1 of the thing problems with the blue jeans, is that each processor had too little memory. 553 00:55:38.219 --> 00:55:42.059 You got how much it had, but. 554 00:55:42.059 --> 00:55:46.170 It wasn't enough, so, in fact, 1 way of programming, the blue Jane. 555 00:55:46.170 --> 00:55:57.630 The processes were like, in Paris with their memory. So that is like, half the processors and each process there would use the memory of its unused twin also. So double the effective memory for each processor. 556 00:55:57.630 --> 00:56:05.730 So that, yes, so I think this is a design mistake to use to have too little memory for processor. 557 00:56:06.750 --> 00:56:12.210 And, in fact, in videos, they would agree with that. 558 00:56:12.210 --> 00:56:20.099 Because, like I said, the GPU card I've got in my lab machine is 48 gigabytes on the GPU card. 559 00:56:20.099 --> 00:56:24.449 So, video seeing the same thing about going to more. 560 00:56:24.449 --> 00:56:36.420 Local memory, in fact, and videos got a tool. They have an idea. They can't I can't do this on Z on, but you could do this. If the host was in IBM. 561 00:56:36.420 --> 00:56:42.389 Or station, you could put several of the cards on the machine. 562 00:56:42.389 --> 00:56:46.829 On the IBM workstation and have a very fast. 563 00:56:46.829 --> 00:56:51.150 Bus between them and would effectively combine the memories so you could get. 564 00:56:52.739 --> 00:56:56.940 Effectively a couple of 100 gigabytes of memory available to the. 565 00:56:59.429 --> 00:57:05.730 Disadvantages yeah programming problems. 566 00:57:07.440 --> 00:57:13.320 And obviously you get your optimum by combining all the techniques in the hierarchy. 567 00:57:13.320 --> 00:57:18.989 So, the general design lesson okay. 568 00:57:20.969 --> 00:57:25.050 I want to before I continue here, I want to go back. 569 00:57:25.050 --> 00:57:29.039 Take a breath I want to go back to what I'm talking about in class here. 570 00:57:29.039 --> 00:57:38.280 And then I'll switch back to that to just summarize some of the points. I could be writing stuff down on. I have a hover cam here. 571 00:57:38.280 --> 00:57:43.860 And I could be writing stuff down on a pad, but it's easier for you to read it. If. 572 00:57:43.860 --> 00:57:50.789 Um, I type it in in advance, so Here's some of my cake. Some of these, maybe a little bigger here. 573 00:57:51.929 --> 00:57:58.289 So, parallel computing has been around for a very long. I graduated 2. 574 00:57:58.289 --> 00:58:03.090 In parallel computing, about 30 parallel geometry about 30 years ago. 575 00:58:03.090 --> 00:58:07.590 And then didn't find parallel computing. Interesting at that point. 576 00:58:07.590 --> 00:58:13.710 Um, because it wasn't that much faster than serial computers and clock speech were incredibly stable. 577 00:58:13.710 --> 00:58:19.380 And so I lost interest in it for. 578 00:58:19.380 --> 00:58:24.900 But a few years and got interested in it some years ago, started this course. And so on. 579 00:58:24.900 --> 00:58:28.500 But parallel computing people have been thinking about it for a very long time. 580 00:58:28.500 --> 00:58:33.059 But, of course, the reason that the. 581 00:58:34.469 --> 00:58:38.550 Clock, excuse me clock speed are not increasing. Now. 582 00:58:39.690 --> 00:58:44.880 Of course, is that a bit of memories like. 583 00:58:44.880 --> 00:58:52.590 Or just like a capacitor and you charging and discharging it. So you've got your RC time constant. 584 00:58:52.590 --> 00:58:56.670 And if you want to make your time constant less, you make the. 585 00:58:56.670 --> 00:59:06.329 Past or less, so the resistance last courses limits on the integrated circuit to how much you can change these. Actually but if your capacity is less than. 586 00:59:06.329 --> 00:59:09.449 Discharge as fast, and has to be recharged more often. 587 00:59:09.449 --> 00:59:17.070 For example, and also the gates are using power when they're switching. 588 00:59:17.070 --> 00:59:23.190 So, as you increase the clock, speed, decreased the clock time. Of course, the power consumption goes up. 589 00:59:23.190 --> 00:59:29.219 And as I said, how much you can increase it so the clock speech are not really increasing. 590 00:59:29.219 --> 00:59:34.800 I mean, the next big increase in processor power will be quantum computers. 591 00:59:34.800 --> 00:59:41.010 If they get them actually useful, so you can increase clock speed. You do things in parallel. 592 00:59:41.010 --> 00:59:45.000 In video I mentioned. 593 00:59:46.559 --> 00:59:52.679 I mentioned in video before, since they're the leader, I guess, in parallel computing now as well as in. 594 00:59:52.679 --> 00:59:59.519 In graphics and the history of that, and this is very relevant to our API. 595 00:59:59.519 --> 01:00:02.820 Because we've got Curtis. 596 01:00:02.820 --> 01:00:06.119 He was an RPI grad from. 597 01:00:06.119 --> 01:00:11.039 I don't know what about 35 years ago give or take. 598 01:00:11.039 --> 01:00:14.309 And he. 599 01:00:14.309 --> 01:00:20.610 He went to work designing graphics hardware for IBM, IBM graphics card at 1. 600 01:00:20.610 --> 01:00:25.679 Then he went to Sun Microsystems sun does not exist any more. 601 01:00:25.679 --> 01:00:33.599 They folded into Africa. Oh, the database company starts with an. 602 01:00:33.599 --> 01:00:38.579 They but Sun Microsystems. 603 01:00:38.579 --> 01:00:42.300 Was founded about. 604 01:00:42.300 --> 01:00:53.519 Oh, 40 years ago, I guess by some Stanford University, and in where they were cheap Unix workstations, they used commodity components. 605 01:00:53.519 --> 01:00:57.780 And they provided a lot of power. 606 01:00:57.780 --> 01:01:03.329 low price so at one point university computer science departments had . 607 01:01:03.329 --> 01:01:11.670 All heads on workstations. In fact, I bought the 1st and 2nd sudden workstation in the engineering center at and. 608 01:01:11.670 --> 01:01:18.750 So so, Curtis cream was 1 of the people at Sun doing. 609 01:01:18.750 --> 01:01:22.829 Doing their graphics hardware after that at IBM. 610 01:01:22.829 --> 01:01:31.920 And then he quit son, and he was 1 of the founders of invidia. Well, there were 3 founders. He was a technical. 611 01:01:31.920 --> 01:01:37.230 The technical person later on, he sold out and moved on. 612 01:01:37.230 --> 01:01:41.789 But so, Curtis you said was. 613 01:01:41.789 --> 01:01:47.610 Help to make and video what it is today. So now, Curtis is. 614 01:01:47.610 --> 01:01:55.289 On our board of trustees, and he's the 2nd person on the board of trustees. I forget what his exact title is. 615 01:01:55.289 --> 01:01:58.769 And he's also the biggest. 616 01:01:58.769 --> 01:02:02.789 He's also the reputed donor of the of impact. 617 01:02:02.789 --> 01:02:09.000 And he's the largest donor in our history by far. In fact. 618 01:02:09.000 --> 01:02:14.760 So so Curtis is connection to NVIDIA. 619 01:02:14.760 --> 01:02:21.599 Is it the largest donor ever to our API was the technical person founding NVIDIA. 620 01:02:21.599 --> 01:02:26.670 Any case talk about NVIDIA they. 621 01:02:26.670 --> 01:02:35.400 Started designing accelerators for game and graphics salary to do gaming because the did not do graphics very well. 622 01:02:36.420 --> 01:02:40.139 And the thing is with. 623 01:02:40.139 --> 01:02:49.949 If you're doing graphics, you want to process, say, all the pixels in parallel, you want to process all the graphic elements and parallel the triangles. Let's say. 624 01:02:49.949 --> 01:02:55.949 So, you're processing the triangles they don't affect each other unless you're doing Ray Tracy. 625 01:02:55.949 --> 01:02:59.099 Which they, they weren't doing at that point so that. 626 01:03:00.300 --> 01:03:08.849 Parallelism was obvious was very natural language, so you're rotating in graphics. You're rotating an object. 627 01:03:08.849 --> 01:03:21.539 So, each for attacks gets a 3 by 3 matrix or 4 by forfeiting homogeneous matrix supply to all the vertices can be rotated in parallel. It's just natural to do things in parallel or you have a frame buffer. 628 01:03:21.539 --> 01:03:28.380 You're updating the frame buffer. That's buffer frame buffer together. Let's say it was a new object. 629 01:03:28.380 --> 01:03:36.119 And the pixel's going to be updated in parallel parallel is just natural for the graphics accelerators. 630 01:03:36.119 --> 01:03:50.219 So, they did graphics very well, but they were specialized to do only graphics very well. There were special purpose, parallel processors, but what they did, they did really well. 631 01:03:52.199 --> 01:04:00.570 So, people who were not playing games are doing the graphics, noticed how powerful the invidia Co processors were. 632 01:04:00.570 --> 01:04:06.360 And tried to start using them for their other applications that were not graphics applications. 633 01:04:07.679 --> 01:04:12.510 Um, so example, it was a mess because, for example. 634 01:04:12.510 --> 01:04:20.880 The only general memory in the NVIDIA Co processes was the texture memory. So you'd use the texture memory to store and array. For example. 635 01:04:20.880 --> 01:04:24.510 Oh, so they had to really, you know. 636 01:04:24.510 --> 01:04:31.800 Use a lot of tricks, but if they could make it work, they could get better performance solving their own problems. 637 01:04:31.800 --> 01:04:41.730 On the video on the video. In fact, they invented a field called JP JP, a general purpose programming sort of. 638 01:04:41.730 --> 01:04:45.780 Okay, and video is watching this happening. 639 01:04:45.780 --> 01:04:50.039 And it didn't expect us to happen, I think, but they saw it happening. 640 01:04:50.039 --> 01:04:54.030 And they saw a new market. 641 01:04:54.030 --> 01:05:04.440 You know, they wanted, they're smart. Hardworking people wanted to make money. They saw what their customers were trying to do with their product, wasn't what they'd expected, but hey, it's a market. 642 01:05:04.440 --> 01:05:10.079 And they added more capability to the to serve this market. 643 01:05:10.079 --> 01:05:18.599 Example of things the added I triple E standard double precision floats, which takes a lot of hardware to do right? 644 01:05:18.599 --> 01:05:31.769 But they thought it was worth it, they added it well, they also segmented their market so they had the gamer level, the G force level then on top of that at the Quadro market. 645 01:05:31.769 --> 01:05:37.289 And they added this high performance to the segmented market, the Quadro market that provided. 646 01:05:37.289 --> 01:05:45.269 More performance at a much higher price and the champion on top of that, then they've got basically the compute server. Margaret, listening to that graphics. 647 01:05:45.269 --> 01:05:50.519 That's the point I'm mentioning there, is that the highest and video cards. 648 01:05:50.519 --> 01:05:58.110 Boards whatever they don't even have video output he cannot do graphics because because their customers. 649 01:05:58.110 --> 01:06:01.230 Don't want to do graphics. I just want to do a Super computing. 650 01:06:02.250 --> 01:06:06.539 And video out takes up space, so that's sort of a history of video. 651 01:06:08.550 --> 01:06:13.380 Now, talk a little about cores here, so. 652 01:06:14.820 --> 01:06:22.679 The Intel CPU so we talk so. 653 01:06:22.679 --> 01:06:26.369 We talked to say my laptop here since I'm sitting in front of it. 654 01:06:26.369 --> 01:06:29.760 I got dual 6 core. Excuse me? 655 01:06:29.760 --> 01:06:34.289 Cpo such 12 cores so whereas the. 656 01:06:35.699 --> 01:06:43.260 Gpu has 3000 cars or whatever it was just see what it is. 657 01:06:43.260 --> 01:06:51.989 You do 3072 cores. 658 01:06:55.349 --> 01:06:59.760 But the point is that. 659 01:06:59.760 --> 01:07:05.369 It's not such a big performance difference for the following reason. 660 01:07:05.369 --> 01:07:14.730 1st, the host, the CPU is faster clock speed. If I go back over here. 661 01:07:14.730 --> 01:07:20.610 Gpu Max clock rate is 1.4 gigahertz. 662 01:07:20.610 --> 01:07:31.409 And by the way, this is really this is a really fast typically invidia GPU clock rates are under 1 gigahertz seminar megahertz or something. 663 01:07:31.409 --> 01:07:37.170 Okay, so the have fast o'clock rates by a factor of several 2nd point. 664 01:07:37.170 --> 01:07:47.789 Is that in 1 cycle the entails are super scaler. So they can do perhaps a floating ad and separately a floating multiplied and separately and entered your ad and separately. Some. 665 01:07:47.789 --> 01:07:50.940 Comparison all of these 4 or 5. 666 01:07:50.940 --> 01:07:57.449 Things in 1 cycle, whereas the invidia in 1 cycle may not even do 1 thing. So. 667 01:07:57.449 --> 01:08:03.360 A double precision on the invidia corps may require more than 1 cycle. Perhaps, depending. 668 01:08:03.360 --> 01:08:11.369 So, I asked to make that an Intel Core is 20 times as fast as and video core give or take. 669 01:08:11.369 --> 01:08:17.220 So the 3000 CUDA cores on my laptop or about equivalent to 150. 670 01:08:17.220 --> 01:08:20.520 Intel course 58 bad. 671 01:08:20.520 --> 01:08:24.180 Okay, but, I mean, better than 12, but so. 672 01:08:25.890 --> 01:08:31.890 Shared memory advantage and disadvantages. 673 01:08:33.119 --> 01:08:39.239 Big advantage is well, it's easy to reference. Everyone can get at the shared memory. 674 01:08:39.239 --> 01:08:45.659 The disadvantage is implementing it in hardware. The idea doesn't scale up. 675 01:08:45.659 --> 01:08:49.920 Because you need a bigger and bigger problems with cache consistency. 676 01:08:49.920 --> 01:08:54.569 And Crossfire switches or whatever. 677 01:08:56.310 --> 01:09:02.489 Point see, here, this is what I mentioned before kudos low level, it's assembly programming. 678 01:09:02.489 --> 01:09:07.649 Well, not exactly but close enough to assembly programming for the GPU. 679 01:09:07.649 --> 01:09:15.659 Open empty open ACC, their higher level. So you may not get the same performance but what you call is portability. 680 01:09:15.659 --> 01:09:18.750 And that's worth it perhaps. 681 01:09:18.750 --> 01:09:23.159 And finally. 682 01:09:23.159 --> 01:09:28.529 Another point here is that in many problems. 683 01:09:28.529 --> 01:09:32.850 It will take you more time to get the data into the GPU. 684 01:09:32.850 --> 01:09:35.880 Or she then it will take the process that. 685 01:09:35.880 --> 01:09:42.210 Is surprisingly often the rate leveraging cost is the old time not the computation time. 686 01:09:42.210 --> 01:09:46.529 And so this is. 687 01:09:47.819 --> 01:09:52.859 You know, so this affects your algorithm optimizations and so on. 688 01:09:52.859 --> 01:09:59.550 So, actually, algorithms people like running in parallel tend to be 1 where the computation is super linear. Like. 689 01:09:59.550 --> 01:10:03.180 Multiplying matrices. 690 01:10:03.180 --> 01:10:09.119 So 2 end by end matrices. 691 01:10:09.119 --> 01:10:15.630 And square data, but then may take. 692 01:10:15.630 --> 01:10:23.069 Cubic time the number of elements or something so super linear. So matrix complication is something people like to do in parallel. 693 01:10:24.449 --> 01:10:29.159 I'll continue more of this later scanning on. 694 01:10:29.159 --> 01:10:33.869 You can take a bottom up to the end points and I'll continue to the Lawrence Livermore thing later. 695 01:10:33.869 --> 01:10:39.689 But 1 thing I want to do is. 696 01:10:39.689 --> 01:10:45.119 Mention homework 1, I want you to do some talking next week. 697 01:10:46.710 --> 01:10:51.060 So, I'd like you to pick a topic from the list and talk about it for. 698 01:10:51.060 --> 01:10:58.979 Every 10 minutes, you can call teams so I say 2 topics, because if somebody else picks the same topics here. 699 01:10:58.979 --> 01:11:04.409 So, your assignment side to read some of the Lawrence Livermore stuff. 700 01:11:04.409 --> 01:11:12.270 And pick 2 topics on this list and give a, and I'll get back to you right away. Okay. So I won't make you wait and. 701 01:11:12.270 --> 01:11:19.109 Or he can form a team and give a talk on this next week. So on this so next Monday. 702 01:11:19.109 --> 01:11:26.159 You get to talk instead of me. So what I did today just to review is. 703 01:11:26.159 --> 01:11:30.149 Introduce you to the blog, went to the syllabus. 704 01:11:30.149 --> 01:11:33.149 And did a. 705 01:11:34.319 --> 01:11:40.109 A start of the Lawrence Livermore thing, and I'll put a note here actually. 706 01:11:40.109 --> 01:11:43.619 Remind you how far I got and also did. 707 01:11:43.619 --> 01:11:47.789 A good chunk of what I mentioned. 708 01:11:48.840 --> 01:11:57.420 Your class 1 and so on not to the end of this, but a good chunk on this. So. 709 01:11:57.420 --> 01:12:02.220 I got as far as here and I'll. 710 01:12:02.220 --> 01:12:06.510 I'll mark that, so, you know, okay. 711 01:12:06.510 --> 01:12:10.050 Any questions I'd like to talk about. 712 01:12:11.699 --> 01:12:16.680 I mean, I have Webex teams meeting space. 713 01:12:16.680 --> 01:12:20.460 As a point place to post questions also. 714 01:12:20.460 --> 01:12:27.029 If anyone would like, and if I can figure out how to get screen sharing, working properly. 715 01:12:27.029 --> 01:12:32.819 Will switch over and use that, but. 716 01:12:32.819 --> 01:12:38.520 Meanwhile. 717 01:12:46.170 --> 01:12:50.909 And I had this available so much, I don't need it. So. 718 01:12:50.909 --> 01:13:03.899 Silence. 719 01:13:07.079 --> 01:13:21.689 No questions silence. 720 01:13:25.050 --> 01:13:28.319 Silence. 721 01:13:33.180 --> 01:13:40.529 Okay, actually, yeah, I guess I have a question. Okay I'm listening. 722 01:13:40.529 --> 01:13:49.800 For homework on 1, should it be more of a formal Carlton teach Jim or just kind of a conversation. 723 01:13:49.800 --> 01:14:02.010 Conversations are fine. Okay. You're also free to copy shamelessly from the various websites. You know, this is so take a topic and introduce the rest of the class to that topic. 724 01:14:02.010 --> 01:14:05.850 You're perfectly fine to use to use their material. 725 01:14:07.710 --> 01:14:12.630 So, wonderful, thank you. So, informal. 726 01:14:12.630 --> 01:14:16.859 Silence. 727 01:14:19.409 --> 01:14:23.159 Silence. 728 01:14:24.869 --> 01:14:28.439 Just put on sure. 729 01:14:37.229 --> 01:14:40.260 Anything else. 730 01:14:43.170 --> 01:14:50.699 Okay. 731 01:14:53.789 --> 01:14:55.680 Hello.