ECSE-4740 Applied Parallel Computing for Engineers, my spring parallel computing course, is an intro course that does not require any other parallel computing course as a prerequisite. As stated on the course web site, the prerequisite is ECSE-2660 CANOS or equivalent, basic C++. The course will use Linux, which students will be expected to know or learn on their own.

RPI article about a completed project that I was part of:

A Game-Changing Approach: Using the X-Box Kinect as a Sensor to Conduct Centrifuge Research. Team of Rensselaer Researchers Develop New Visualization Method to Evaluate Erosion Quantity and Pattern, November 8, 2016, by Jessica Otitigbe.

Another RPI article, on my student Viana Gomes de Magalhães: The Winning Algorithm, Oct 17, 2016, by Mary Martialay.

Note on updating web pages:

1. You may change a web page after it has been displayed. Editing elements and even adding or deleting them is allowed.
2. One of the pick programs changes the displayed text in a div to show what you picked.
3. Mathjax, the package I use to render LaTeX math in a web page, edits the displayed page to replace the math source with the rendered math.
4. A standard way to make things appear and disappear is to include at the start all the elements that will be wanted later. Then make them visible or invisible. The tool on my homepage that makes sections expand and collapse does that.
5. Every time something is changed on the page, the locations of everything on the page have to be recomputed. With a slow web browser, like old versions of IE, you can watch this.

12_2 Hierarchical Modeling 2.

12_3 Graphical Objects and Scene Graphs.

12_4 Graphical Objects and Scene Graphs 2.

12_5 Rendering overview.

At this point we've learned enough WebGL. The course now switches to learn the fundamental graphics algorithms used in the rasterizer stage of the pipeline.

13_1 Clipping.

11_3 Agent based models.

Where I was last week: 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL 2016)

1. I gave the keynote talk at the Third ACM SIGSPATIAL 2016 PhD Symposium.
2. The team of Salles Magalhaes, Wenli Li, Marcus Andrade (Federal University of Vicosa, Brazil) and me came 2nd in the GISCUP programming contest. Here are our talk and paper.

I've mentioned DARPA several times; they funded research into computer graphics starting in the 1970s. So here's some enrichment material on DARPA and autonomous vehicles. This shows why the US is the best in the world at R&D.

Autonomous vehicles make a marvelous example of a successful engineering project. Many different parts all have to work to make the total project succeed. They include laser rangefinding, image recognition, a road database accurate to the specific lane, and GPS navigation. This is also a model for government - university - industry interaction.

DARPA (The Defense Advanced Research Projects Agency) started this concept with a contest paying several $million in prizes. (DARPA started connecting computers in different cities with phone lines in 1968. This was the ARPAnet. They funded computer graphics in the 1970s and some early steps in virtual reality in the 1980s.)

In the 1st contest, the best vehicle failed after about 10 miles. Five vehicles completed the 130 mile course in the 2nd contest. The winning project leader was Sebastian Thrun, a Stanford CS prof. He quit and moved to Google, who has now been funding this for several years.

Finally, Dr Tony Tether, Director of DARPA when this happened, is an RPI BS EE grad. (He went to Stanford for his PhD).

More programs from Chapter 7.

I'm not covering all the gruesome details of these programs, but just hitting highlights.

1. particleDiffusion: buffer ping ponging of 50 particles initially placed randomly and then moving randomly with their previous positions diffused as a texture.

   It renders to a texture instead of to a color buffer. Then it uses the texture.

   There are better descriptions on the web. Search for 'webgl render texture'. I'll work with the textbook code. Jumping back and forth is confusing, partly because they might use different utility files.

   There are often better descriptions on the web than in the textbook. I've considered running a course only with web material. What do you think?

   https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API has good documentation.

2. pickcube etc. This prints the name of the name that you pick.

   1. This draws a label for each face into a separate buffer.
   2. Then it indexes into it with the coordinates of the mouse.

Stereo viewing. There are a number of techniques for stereo viewing of movies and videos, dating back to the 19th century.

1. Make the left image red and the right one blue. Use red-blue glasses.

   This decreases the color experience, but the glasses are cheap.

2. Polarize the left and right images differently. Movie theatres do this.

   The glasses are cheap. There is full color. The projection process is more complicated (= expensive).

3. The glasses have shutters that block and clear the left and right eyes in turn. In sync, the TV or projector shows the left or right image.

   The projector is cheaper. The glasses are more expensive.

4. Use a display with a parallax barrier.

   Only one person can view at a time. No glasses are needed.

5. Use the fact that your eyes see bright objects faster than they see dim objects. The glasses have a simple grey filter over one eye. Beer companies have used this for commercials.

   No special display is needed. This works only with moving objects.

12_1 Hierarchical Modeling 1.

Tomorrow, Prof Cutler will tell us about her spring course. CSCI-4530-01 Advanced Computer Graphics.

Aliasing and anti-

1. The underlying image intensity, as a function of x, is a signal, f(x).
2. When the objects are small, say when they are far away, f(x) is changing fast.
3. To display the image, the system evaluates f(x) at each pixel. That is, f(x) is sampled at x=0,1,2,3,...
4. If f(x), when Fourier transformed, has frequencies higher than 1/2 (cycle per pixel), then that sampling is too coarse to capture the signal. See the Nyquist sampling theorem.
5. When this hi-freq signal is sampled at too low a frequency, then the result computed for the frame buffer will have visual problems.
6. It's not just that you won't see the hi frequencies. That's obvious.
7. Worse, you will see fake low frequency signals that were never in the original scene. They are called '''aliases''' of the hi-freq signals.
8. These artifacts may jump out at you, because of the Mach band effect.
9. Aliasing can even cause (in NTSC) rapid intensity changes to cause fake colors and vv.
10. Aliasing can occur with time signals, like a movie of a spoked wagon wheel.
11. This is like a strobe effect.
12. The solution is to filter out the hi frequencies before sampling, or sample with a convolution filter instead of sampling at a point. That's called '''anti-aliasing'''.
13. OpenGl solutions:
    1. Mipmaps.
    2. Compute scene on a higher-resolution frame buffer and average down.
    3. Consider pixels to be squares not points. Compute the fraction of each pixel covered by each object, like a line. Lines have to have finite width.
14. Refs:
    1. http://en.wikipedia.org/wiki/Aliasing
    2. http://en.wikipedia.org/wiki/Clear_Type
    3. http://en.wikipedia.org/wiki/Wagon-wheel_effect
    4. http://en.wikipedia.org/wiki/Spatial_anti-aliasing (The H Freeman referenced worked at RPI for 10 years).
    5. http://en.wikipedia.org/wiki/Mipmap
    6. http://en.wikipedia.org/wiki/Jaggies

Videos - military applications of graphics

US Military's Futuristic Augmented Reality Battlefield - Augmented Immersive Team Trainer (AITT)

Daqri's Smart Helmet Hands On

HoloLens Review: Microsoft's Version of Augmented Reality

US Military's Futuristic Augmented Reality Battlefield - Augmented Immersive Team Trainer (AITT)

'''Modeling and simulation''' is a standard term.

10_5 Rendering the Mandelbrot Set.

Shows the power of GPU programming.

The program is in Chapter 10.

11_1 Framebuffer objects.

11_2 Render to texture.

11_3 Agent based models.

If my order here looks a little chaotic, it's because the slides don't exactly align with the programs.

More programs from Chapter 7.

1. Several of the programs don't display on my laptop, so I won't show them. They include bumpmap and render.

2. Cubet: enabling and disabling the depth buffer and drawing translucent objects.

3. particleDiffusion: buffer ping ponging of 50 particles initially placed randomly and then moving randomly with their previous positions diffused as a texture.

   It renders to a texture instead of to a color buffer. Then it uses the texture.

   There are better descriptions on the web. Search for 'webgl render texture'. I'll work with the textbook code. Jumping back and forth is confusing, partly because they might use different utility files.

   There are often better descriptions on the web than in the textbook. I've considered running a course only with web material. What do you think?

   https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API has good documentation.

Reminder of the Dreamworks visit:

Monday, Nov 7
8pm-9pm
Sage 3303
Company Overview
Nicole Dial, University Relations

Tuesday, Nov 8
10a-11a
PDI Studio @ Sage 2211
Technical Direction at DWA
Kathryn Skorpil, Lead Technical Director

If you're looking for masters projects or independent study courses for the spring and next year, then I might be available. Let's talk.

I would expect you to do all the work, including teaching yourself, then attend a weekly or semiweekly meeting and write me reports on a blog. You would work yourself as hard as if you were in a regular course run by Prof. xxxxxx.

ECSE-4740 Engineering Parallel Computing is available for students wanting access to state-of-the-art parallel computing facilities. You will learn things useful in your work on other topics.

My parallel computer (dual 14-core Intel Xeon, Intel Xeon Phi, Nvidia GTX 1080, 256GB memory) is also available for any RPI-related work.

Sikorsky is hiring. Dan Flickinger, bdbabnbibeblb.bmb.bfblbibcbkbibnbgbebrb@blbmbcbob.bcbobmb (remove all the b's), former post-doc of Jeff Trinkle, writes:

I'm working with the perception team at Sikorsky, part of the autonomy team. We will soon have openings in our group. I currently have a proof-of-concept system together that processes point cloud streams into a polygonal world model, with the intentions of rapidly growing it into a high performance aircraft perception system capable of supporting landing zone selection, obstacle avoidance, and human interface tasks. So we're definitely looking for people interested in meshing, texturing, and object recognition. People interested in image processing, high performance computing, and sensor fusion may also apply.

https://www.wired.com/2016/11/darpa-alias-autonomous-aircraft-aurora-sikorsky/

Also, we definitely have internships available for the Summer. Talk to David Glowny, a sophomore in the CS department. He did an internship last Summer with our group, and helped build a great world model object visualization system for our ground control interface.

Wed is a regular lecture.

Iclicker questions.

10_1 Reflection and Environment Maps.

10_2 Bump Maps.

Perturb the surface normals to emulate small details (bumps).

10_3 Compositing and Blending.

Some of this material was removed from the current OpenGL, and some of the remaining stuff hasn't been included in WebGL (yet?). So, you have to learn just the high-level stuff, like what this is and why it's interesting.

1. Compositing, using partial transparancy, creates complex scenes by blending simple components.
   1. It's hard to do accurately in the pipeline.
   2. Drawing objects back-to-front into the color buffer is a good heuristic.
   3. Clamping and accuracy are concerns.
2. Fog effects provide a depth cue that helps the viewer understand the image.
3. Anti-aliasing improves the image quality when there are small objects.

10_4 Imaging Applications.

1. The GPU is so powerful that people figure ways to use it to do image processing things that it wasn't intended to.
2. That started the field of GPGPU.
3. Nvidia watched this happening and added new features to the GPU, like double-precision IEEE-standard floating point computations, to make this easier.
4. The top line of Nvidia GPUs is designed for scientific computation and doesn't even have video outputs.
5. The 2nd fastest known supercomputer has 15000 Nvidia GPUs.

10_5 Rendering the Mandelbrot Set.

Show the power of GPU programming.

Videos - commercial applications of graphics

1. Hydraulic Fracture Animation
2. Deepwater Horizon Blowout Animation www.deepdowndesign.com
3. On-board fire-fighting training simulator made for the Dutch navy
4. Ship accident SMS

3D Object Manipulation in a Single Photograph using Stock 3D Models (Siggraph 2014).

From here Photo-editing software restricts the control of objects in a photograph to the 2D image plane. We present a method that enables users to perform the full range of 3D manipulations, including scaling, rotation, translation, and nonrigid deformations, to an object in a photograph. As 3D manipulations often reveal parts of the object that are hidden in the original photograph, our approach uses publicly available 3D models to guide the completion of the geometry and appearance of the revealed areas of the object. The completion process leverages the structure and symmetry in the stock 3D model to factor out the effects of illumination, and to complete the appearance of the object. We demonstrate our system by producing object manipulations that would be impossible in traditional 2D photo-editing programs, such as turning a car over, making a paper-crane flap its wings, or manipulating airplanes in a historical photograph to change its story.

Iclicker questions.

Today's big new idea: Textures.

1. Textures started as a way to paint images onto polygons to simulate surface details. They add per-pixel surface details without raising the geometric complexity of a scene.
2. That morphed into a general array data format with fast I/O.
3. If you read a texture with indices that are fractions, the hardware interpolates a value, using one of several algorithms. This is called '''sampling'''. E.g., reading T[1.1,2] returns something like .9*T[1,2]+.1*T[2,2].
4. Textures involve many coordinate systems:
   1. (x,y,z,w) - world.
   2. (u,v) - parameters on one polygon
   3. (s,t) - location in a texture.
5. Aliasing is also important.

9_1 Buffers.

Ignore anything marked old or deprecated.

Not a lot of content in this file.

9_2 Bitblt.

9_3 Texture mapping.

Start of a big topic.

9_4 Texture mapping.

9_5 WebGL Texture mapping I.

9_6 WebGL Texture mapping II.

Texture mapping programs.

This Wed will be a regular lecture.

There will be no classes next week.

Spring course to consider: ECSE-4740 Applied Parallel Computing for Engineers. Its instructor is excellent.

RPI has a long history in Computer Graphics. E.g. Prof Mike Wozny, Head of ECSE, was one of the founders of IEEE Computer Graphics and Applications, which printed the article by Maureen Stone that I presented last time.

Iclicker questions.

8_2 Lighting in WebGL.

shadedSphere1, 2 etc programs.

8_3 Polygonal Shading.

8_4 Per Vertex and Per Fragment Shading.

8_E3 Marching Squares.

Will be omitted.

Computing surface normals.

1. For a curved surface, the normal vector at a point on the surface is the cross product of two tangent vectors at that point. They must not be parallel to each other.
2. If it's a parametric surface, partial derivatives are tangent vectors.
3. A mesh is a common way to approximate a complicated surface.
4. For a mesh of flat (planar) pieces (facets):
   1. Find the normal to each facet.
   2. Average the normals of the facets around each vertex to get a normal vector at each vertex.
   3. Apply Phong (or Gouraud) shading from those vertex normals.

Estimated grades to date will be uploaded to LMS. It was computed as follows:

1. The homeworks were scaled to have the same weight, added, and made a total weight of 48%.
2. The midterm exam was weighted at 48%.
3. Each iclicker class was given 1 if at least one question was answered. All the iclickers together were weighted at 4%.
4. The class average was 76%. That would earn a B+.

Handwritten notes from Wed.

I added notes to yesterday's posting giving the big ideas from the slide sets.

Iclicker questions.

New slides:

1. 7_4 Shadows.

   Big idea:

   1. If there is one light source, we can compute visibility with it as the viewer.
   2. Somehow mark the visible (i.e., lit) and hidden (i.e., shadowed) portions of the objects.
   3. Then recompute visibility from the real viewer and shade the visible objects depending on whether they are lit or shadowed.
   4. This works for a small number of lights.

7_5 Lighting and shading I.

Big big topic.

This Woman sees 100 times more colors than the average person.

Phong lighting model: The total light at a pixel is the sum of

1. Incoming ambient light times ambient reflectivity of the material at the pixel,
2. Incoming diffuse light times diffuse reflectivity times a factor for the light source being low on the horizon,
3. Incoming specular light times specular reflectivity times a factor for the eye not being aligned to the reflection vector, with an exponent for the material shininess,
4. Light emitted by the material.

That is not intended to be completely physical, but to give the programmer lots of parameters to tweak.

In OpenGL you can do several possible levels of shading. Pick one of the following choices. Going down the list makes the shading better but costlier.

1. Shade the whole polygon to be the color that you specified for one of the vertices.
2. Bilinearly shade the polygon, triangle by triangle, from the colors you specified for its vertices.
3. Use the Phong lighting model to compute the color of each vertex from that vertex's normal. Bilinearly interpolate that color over the polygon. That is called Gouraud shading.
4. Bilinearly interpolate a surface normal at each pixel from normals that you specified at each vertex. Then normalize the length of each interpolated normal vector. Evaluate the Phong lighting model at each pixel from the interpolated normal. That is called Phong shading.
5. Maureen Stone: Representing Colors as 3 Numbers (enrichment)
6. Why do primary schools teach that the primary colors are Red Blue Yellow?

Summary of the new part of shadedCube:

1. var nBuffer = gl.createBuffer();

   Reserve a buffer id.

2. gl.bindBuffer( gl.ARRAY_BUFFER, nBuffer );

   1. Create that buffer as a buffer of data items, one per vertex.
   2. Make it the current buffer for future buffer operations.

3. gl.bufferData( gl.ARRAY_BUFFER, flatten(normalsArray), gl.STATIC_DRAW );

   Write a array of normals, flattened to remove metadata, into the current buffer.

4. var vNormal = gl.getAttribLocation( program, "vNormal" );

   Get the address of the shader (GPU) variable named "vNormal".

5. gl.vertexAttribPointer( vNormal, 3, gl.FLOAT, false, 0, 0 );

   Declare that the current buffer contains 3 floats per vertex.

6. gl.enableVertexAttribArray( vNormal );

   Enable the array for use.

7. (in the shader) attribute vec3 vNormal;

   Declare the variable in the vertex shader that will receive each row of the javascript array as each vertex is processed.

8. The whole process is repeated with the vertex positions.

   Note that the variable with vertex positions is not hardwired here. You pass in whatever data you want, and your shader program uses it as you want.

8_1 Lighting and shading 2.