Project 5: Paul (San) Jewell

README.md

@@ -3,25 +3,103 @@ WebGL Forward+ and Clustered Deferred Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Paul (San) Jewell
+  * [LinkedIn](https://www.linkedin.com/in/paul-jewell-2aba7379), [work website](
+    https://www.biociphers.org/paul-jewell-lab-member), [personal website](https://gitlab.com/inklabapp), [twitter](https://twitter.com/inklabapp), etc.
+* Tested on: (TODO) Linux pop-os 5.11.0-7614-generic, i7-9750H CPU @ 2.60GHz 32GB, GeForce GTX 1650 Mobile / Max-Q 4GB
 
-### Live Online
+[comment]: <> (### Live Online)
 
-[![](img/thumb.png)](http://TODO.github.io/Project5-WebGL-Forward-Plus-and-Clustered-Deferred)
+[comment]: <> ([![]&#40;img/thumb.png&#41;]&#40;http://TODO.github.io/Project5-WebGL-Forward-Plus-and-Clustered-Deferred&#41;)
 
 ### Demo Video/GIF
 
-[![](img/video.png)](TODO)
+|       |  |
+| ----------- | ----------- |
+| ![](images/forward1.gif)      | ![](images/buggy2.gif)       |
+| ![](images/toon1.gif)   | ![](images/debug_frust.png)        |
 
 ### (TODO: Your README)
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+This project involved many components. Including implementing Clustered lighting, Deferred shading,
+optimizations and visual enhancements in a new language and framework. 
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+The method I used for the longest part of the project, the clustered lighting, was based off of this article:
+http://www.aortiz.me/2018/12/21/CG.html#part-2? ; Unfortunately, because seemingly the method described had not been
+used my any of my mentors, I was unable to receive much feedback, and thus, while I can demonstrate the core
+crux of the algorithm, it is not in a completed state at the time of submission. 
+
+### Clustering approach
+
+While the above document was extremely helpful in general, it was marginally difficult to adapt to my use case. In the 
+base code that we are provided, We are instructed to compute the lights-per-cluster algorithm on the CPU (javascript), 
+whereas in the article author's approach, all code was done in parallel on the GPU, and some rendering aspects relevant
+to adapting it to this project were left unmentioned. (specifically, how to apply the lights to the scene after calculation)
+Because I'd heard that compute shaders has very limited support in webGL, I went ahead with trying to adapt the program
+to follow the project template structure. 
+
+The other main difference is that this article used exponential Z-slicing instead of linear. This ia actually a great idea
+in general and I was excited to try it. Here is a screenshot from my debugging to give you an idea of how the exponential
+frustums look:
+
+![](images/debug_frust.png)
+
+(The camera would be directly behind the smallest red group, representing the near plane)
+
+The beauty of this approach is that the general "baseline" of this shape can be calculated only once with the same FOV 
+camera, and then checked each frame against lights, a much simpler calculation, by applying the camera/world transformation.
+(this can be seen in the assembleClusters and updateClusters in base.js). The method I used for detecting which lights were 
+in each cluster involved using three.js Box3 and Sphere classes, making one for each AABB in the mini-frustum, and using 
+three.js collision detection algorithm to check for overlap. 
+
+### Toon shader
+
+After the bulk of time spent on the more technical aspects of clustered shading, this fun visual effect was a nice distraction.
+
+![](images/toon1.gif)
+
+In this example I implemented it over the original forward shader to avoid any interactions with other performances changes
+I may or may not have been making with my deferred and cluster attempts. The shader only introduces minimal operations at the 
+very end of the shading pipeline and practically has no noticeable effects on performance.
+
+The toon shader uses a few basic rules with respect to lighting and camera position to effectively reduce the number of colors
+in the palette over one surface of an object, which makes the rendering have a slightly more 'traditional art' feel, 
+less photorealistic. 
+
+I did not attempt it, but the feature could be likely improved past my quick write up by pre-computing the result of
+the components used in the branching statements and storing them in a compressed g-buffer for the deferred pass. 
+
+### Performance improvement attempt
+
+I attempted to use two G buffers instead of three to store the needed information, by encoding the normal values
+into the last two unused positions (.w) of the color and position g-buffers. While theoretically using fewer G-buffers
+should have an impact, I was not able to spot a discernible difference in performance using this approach. 
+
+
+### Performance Analysis
+
+Even with the troublesome rendering bugs, I'm still able to get an accurate judgement of the differences in performance 
+in the core of the implementations. In the work I completed, which produces the same overall results, for both Clustered-Deferred
+and Forward+. In the case of my implementation, almost all of the core algorithm is the same. Both primarily use clusters
+to reduce the number of lights applied to each object rendered for each screen pixel. The deferred method additionally separated
+the pass for calculating positions and normals into a texture buffer instead of calculating them in the main fragment. 
+
+..Possibly because of additional memory bandwidth, but more likely because none of the optimizations I did were on the 
+deferred component, I notice a significant performance difference between Forward+ and Clustered Deferred. My Forward
+plus implementation is highly optimized as described earlier, and gets from 55-60FPS on my machine, while the clustered
+implementation only reaches an average of 32FPS. Clearly it's may not be as useful for the tested environment, all other
+things considered. 
+
+### Unfortunate conclusions
+
+Before, I'd mentioned that the clustered algorithm was incomplete. Despote many iterations I was unable to isolate the 
+issue in time. I was able to isolate that my x and y slices were behaving properly, as in the left shot when the number
+or Z slices is set to zero, however, when using the depth slicing with exponential components, I encountered frequent bugs.
+I plan to resolve this issue once the resources become available. 
+
+|       |  |
+| ----------- | ----------- |
+| ![](images/buggy1.gif)      | ![](images/buggy2.gif)       |
 
 
 ### Credits
@@ -31,3 +109,6 @@ to implementation work. Complete the implementation early to leave time!
 * [webgl-debug](https://github.com/KhronosGroup/WebGLDeveloperTools) by Khronos Group Inc.
 * [glMatrix](https://github.com/toji/gl-matrix) by [@toji](https://github.com/toji) and contributors
 * [minimal-gltf-loader](https://github.com/shrekshao/minimal-gltf-loader) by [@shrekshao](https://github.com/shrekshao)
+* [A Primer On Efficient Rendering Algorithms & Clustered Shading.] (http://www.aortiz.me/2018/12/21/CG.html#part-2?)
+* [Wikibooks: Toon shading] (https://en.wikibooks.org/wiki/GLSL_Programming/Unity/Toon_Shading)
+

src/main.js

@@ -1,16 +1,20 @@
-import { makeRenderLoop, camera, cameraControls, gui, gl } from './init';
+import {makeRenderLoop, camera, cameraControls, gui, gl, canvas} from './init';
 import ForwardRenderer from './renderers/forward';
 import ForwardPlusRenderer from './renderers/forwardPlus';
 import ClusteredDeferredRenderer from './renderers/clusteredDeferred';
 import Scene from './scene';
 import Wireframe from './wireframe';
+import {Box3, Face3, BufferGeometry, Vector3, Vector4, Vector2} from "three";
 
 const FORWARD = 'Forward';
 const FORWARD_PLUS = 'Forward+';
 const CLUSTERED = 'Clustered Deferred';
+const numXslice = 10;
+const numYslice = 10;
+const numZslice = 10;
 
 const params = {
-  renderer: FORWARD_PLUS,
+  renderer: FORWARD,
   _renderer: null,
 };
 
@@ -22,10 +26,10 @@ function setRenderer(renderer) {
       params._renderer = new ForwardRenderer();
       break;
     case FORWARD_PLUS:
-      params._renderer = new ForwardPlusRenderer(15, 15, 15);
+      params._renderer = new ForwardPlusRenderer(numXslice, numYslice, numZslice);
       break;
     case CLUSTERED:
-      params._renderer = new ClusteredDeferredRenderer(15, 15, 15);
+      params._renderer = new ClusteredDeferredRenderer(numXslice, numYslice, numZslice);
       break;
   }
 }
@@ -39,29 +43,59 @@ scene.loadGLTF('models/sponza/sponza.gltf');
 // It lets you draw arbitrary lines in the scene.
 // This may be helpful for visualizing your frustum clusters so you can make
 // sure that they are in the right place.
-const wireframe = new Wireframe();
+let wireframe = new Wireframe();
+
+// var segmentStart = [-14.0, 0.0, -6.0];
+// var segmentEnd = [14.0, 20.0, 6.0];
+// var segmentColor = [1.0, 0.0, 0.0];
+// wireframe.addLineSegment(segmentStart, segmentEnd, segmentColor);
+// wireframe.addLineSegment([-14.0, 1.0, -6.0], [14.0, 21.0, 6.0], [0.0, 1.0, 0.0]);
+
+
+
+
+
+
+
+
+//wireframe.addLineSegment([0, 0, 0], [4, 4, 4], [1.0, 1.0, 1.0])
+
+
+
 
-var segmentStart = [-14.0, 0.0, -6.0];
-var segmentEnd = [14.0, 20.0, 6.0];
-var segmentColor = [1.0, 0.0, 0.0];
-wireframe.addLineSegment(segmentStart, segmentEnd, segmentColor);
-wireframe.addLineSegment([-14.0, 1.0, -6.0], [14.0, 21.0, 6.0], [0.0, 1.0, 0.0]);
 
 camera.position.set(-10, 8, 0);
 cameraControls.target.set(0, 2, 0);
+
+//camera.position.set(0, 0, 0);
+//cameraControls.target.set(0, 0, -camera.near);
 gl.enable(gl.DEPTH_TEST);
 
+
+let n = 0;
+
 function render() {
-  scene.update();  
+  scene.update();
+  //n += 1;
   params._renderer.render(camera, scene);
 
   // LOOK: Render wireframe "in front" of everything else.
   // If you would like the wireframe to render behind and in front
   // of objects based on relative depths in the scene, comment out /
   //the gl.disable(gl.DEPTH_TEST) and gl.enable(gl.DEPTH_TEST) lines.
+
+
+  //console.log(camera.matrixWorldInverse.elements);
+  //console.log(camera.position, camera.up)
+
   gl.disable(gl.DEPTH_TEST);
   wireframe.render(camera);
   gl.enable(gl.DEPTH_TEST);
+
+  // if(n < 10 || true){
+  //renderLines(camera);
+  //
+  // }
 }
 
 makeRenderLoop(render)();