Project 5: Ziad Ben Hadj-Alouane

README.md

@@ -3,29 +3,70 @@ WebGL Clustered and Forward+ Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Ziad Ben Hadj-Alouane
+  * [LinkedIn](https://www.linkedin.com/in/ziadbha/), [personal website](https://www.seas.upenn.edu/~ziadb/)
+* Tested on: Google Chrome Version 70.0.3538.77 (WebGL), Windows 10, i7-8750H @ 2.20GHz, 16GB, GTX 1060
 
-### Live Online
+# Video Demo
+  [<img src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/thumb.jpg">](https://www.youtube.com/watch?v=J1Pvi4GN62o)
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+# Deferred Shading Intro
+This project showcases an implementation of Forward shading, with extensions: Forward+ and Clustered/Deferred shading.
+  * For **Forward** shading, we supply the graphics card the geometry data, which is then projected, broken into vertices, and split into fragments. Each fragment then gets the final lighting treatment before they are passed onto the screen.
+  * For **Forward+** shading, we do the same thing except that we break our viewing frustum into pieces, and compute the lights that overlap these pieces. As such, we can determine which section (i.e cluster) a fragment is in, and iterate over a select few number of lights.
+  * For **Clustered** shading (deferred shading), we do the same thing except that the rendering is deferred a little bit until all of the geometries have passed down many stages. The final image is then obtained by doing lighting calculations at the end. This essentially requires more passes on the scene.
+
+This project is fully implemented with Javascript and WebGL. See a live demo above.
 
-### Demo Video/GIF
+## Implementing the Cluster Datastructure
+To implement clustering, we must be able to break up our viewing frustum into sectors in every dimension (X, Y, and Z). Since the Z dimension is simple (from near clip to far clip), it is simple to test intersections of spheres with it. However, for the X and Y dimensions, our view grows in a cone shape. The image below helps visualize the test we do for each frustum section and each spherical light.
 
-[![](img/video.png)](TODO)
+<p align="center"><img width="700" height="500" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/test.png"/></p>
 
-### (TODO: Your README)
+We essentially represent each section of the frustum by a right triangle with sides 1 and d, d being the distance from 0. We then represent it with a normalized vector (the light blue one). Now if we dot that vector with the position vector of the sphere (the brown one), it will result in measuring the red distance outlined in the image above. We compare that red distance with the radius to get a sense of where the sphere is with respect to our frustum.
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+## Scene
+### Sponza
+<p align="center"><img width="1000" height="500" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/top.gif"/></p>
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+#### Views
+| Depth View | Normals View | Non-Debug View |
+| ------------- | ----------- | ----------- |
+| <p align="center"><img width="200" height="150" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/debug_z.png"/> </p>| <p align="center"><img width="200" height="150" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/debug_norm.jpg/"></p> | <p align="center"><img width="200" height="150" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/debug_none.jpg/"></p> |
 
+#### Blinn-Phong Effect
+<p align="center"><img width="700" height="400" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/blinn.jpg"/></p>
 
-### Credits
+Blinn-Phong is a shading effect achieved by adding a specular component to the albedo color:
+
+~~~~
+vec3 halfDirection = lightDirection + viewDirection
+float angle = dot(halfDirection, normal);
+float spec = pow(angle, exponent);
+albedo += spec;
+~~~~
+
+Adding Blinn-Phong virtually has no performance impact. It is an extra 3 instructions per lighting computation.
+
+# Performance
+## Forward vs. Forward+ vs. Clustered/Deferred
+The graph below shows performance differences for the different shading techniques. Overall, the more lights we have, the better Clustering is. However, if we have a low number of lights (say 100), then clustering does more work than needed, hurting performance.
+<p align="center"><img width="700" height="400" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/performance_diff.png"/></p>
+
+This is easily explained by the lost benefit of creating the clustering data-structure in the forward+ cases: the less lights you have, the less iterations you would have done anyways.
+
+## Packing Normals in the Position and Color G-Buffers
+To reduce the amount of G-Buffers we use, I pack the normals' x and y values in the w coordinate of the position and color vectors. I then retrieve the z coordinate since the normal vector is normalized. The sign value is not lost, since I also multiply the Red color channel by -1 if the z coordinate is negative. There are still some artifacts as showcased below:
 
+| With Packing | Without Packing |
+| ------------- | ----------- |
+| <p align="center"><img width="400" height="300" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/packed.png"/> </p>| <p align="center"><img width="400" height="300" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/unpacked.png/"></p> |
+
+As for performance, packing clearly wins because we do less global memory reads (at the expense of slightly more computation)
+<p align="center"><img width="700" height="400" src="https://github.com/ziedbha/Project5-WebGL-Clustered-Deferred-Forward-Plus/blob/master/imgs/performance_packing.png"/></p>
+
+
+### Credits
 * [Three.js](https://github.com/mrdoob/three.js) by [@mrdoob](https://github.com/mrdoob) and contributors
 * [stats.js](https://github.com/mrdoob/stats.js) by [@mrdoob](https://github.com/mrdoob) and contributors
 * [webgl-debug](https://github.com/KhronosGroup/WebGLDeveloperTools) by Khronos Group Inc.

src/renderers/base.js

@@ -1,30 +1,203 @@
 import TextureBuffer from './textureBuffer';
+import {vec4, vec3} from 'gl-matrix';
+import {NUM_LIGHTS} from "../scene";
 
 export const MAX_LIGHTS_PER_CLUSTER = 100;
 
+/**
+ * Converts an angle in degrees to radians
+ */
+function toRadian(deg) {
+    return deg * Math.PI / 180;
+}
+
+/**
+ * Returns an array containing the cosine and sine of a right triangle with sides 1 and opposite
+ */
+function getNormalForTriangle(opposite) {
+    let hypothenuse = Math.sqrt(1 + opposite * opposite);
+    return [1 / hypothenuse, opposite / hypothenuse];
+}
+
+/**
+ * Explained in detail in the README. Does frustum/sphere intersection test by approximating the frustum section
+ * as a right triangle with sides 1 and currentAxisDistance
+ */
+function getDotForFrustumCheck(currentAxisDistance, camSpaceLightPos, axis) {
+    // get the info for the frustum vector
+    let normalCoords = getNormalForTriangle(currentAxisDistance);
+
+    // flip the vector by around the cosine axis
+    let normal;
+    if (axis === "x") {
+        // X plane
+        normal = vec3.fromValues(normalCoords[0], 0, -normalCoords[1]);
+    } else {
+        // Y plane
+        normal = vec3.fromValues(0, normalCoords[0], -normalCoords[1]);
+    }
+
+
+    // return the dot product of the frustum vector with the light position vector
+    return vec3.dot(camSpaceLightPos, normal);
+}
+
 export default class BaseRenderer {
-  constructor(xSlices, ySlices, zSlices) {
-    // Create a texture to store cluster data. Each cluster stores the number of lights followed by the light indices
-    this._clusterTexture = new TextureBuffer(xSlices * ySlices * zSlices, MAX_LIGHTS_PER_CLUSTER + 1);
-    this._xSlices = xSlices;
-    this._ySlices = ySlices;
-    this._zSlices = zSlices;
-  }
-
-  updateClusters(camera, viewMatrix, scene) {
-    // TODO: Update the cluster texture with the count and indices of the lights in each cluster
-    // This will take some time. The math is nontrivial...
-
-    for (let z = 0; z < this._zSlices; ++z) {
-      for (let y = 0; y < this._ySlices; ++y) {
-        for (let x = 0; x < this._xSlices; ++x) {
-          let i = x + y * this._xSlices + z * this._xSlices * this._ySlices;
-          // Reset the light count to 0 for every cluster
-          this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = 0;
-        }
-      }
+    constructor(xSlices, ySlices, zSlices) {
+        // Create a texture to store cluster data. Each cluster stores the number of lights followed by the light indices
+        this._clusterTexture = new TextureBuffer(xSlices * ySlices * zSlices, MAX_LIGHTS_PER_CLUSTER + 1);
+        this._xSlices = xSlices;
+        this._ySlices = ySlices;
+        this._zSlices = zSlices;
     }
 
-    this._clusterTexture.update();
-  }
+    updateClusters(camera, viewMatrix, scene) {
+        /** Init light counter to 0 **/
+        for (let z = 0; z < this._zSlices; ++z) {
+            for (let y = 0; y < this._ySlices; ++y) {
+                for (let x = 0; x < this._xSlices; ++x) {
+                    let i = x + y * this._xSlices + z * this._xSlices * this._ySlices;
+                    this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = 0;
+                }
+            }
+        }
+
+        /** Get Frustum Info ***/
+        // tangent of half of vertical fov gives us the half the vertical camera frustum
+        let verticalStretch = 2.0 * Math.tan(toRadian(camera.fov / 2));
+        let xSliceLength = verticalStretch * camera.aspect / this._xSlices;
+        let ySliceLength = verticalStretch / this._ySlices;
+        let depthStretch = camera.far - camera.near;
+        let zSliceLength = depthStretch / this._zSlices;
+
+        /** For every light, compute the clusters**/
+        for (let lightIdx = 0; lightIdx < NUM_LIGHTS; lightIdx++) {
+            // get light info: radius and world position
+            let lightRad = scene.lights[lightIdx].radius;
+            let lightPos = vec4.create();
+            lightPos[0] = scene.lights[lightIdx].position[0];
+            lightPos[1] = scene.lights[lightIdx].position[1];
+            lightPos[2] = scene.lights[lightIdx].position[2];
+            lightPos[3] = 1;
+
+            // transform world position to camera space. Cam is positioned at 0 origin in this space
+            lightPos = vec4.transformMat4(lightPos, lightPos, viewMatrix);
+            lightPos[2] = -lightPos[2];
+
+            // vec3 version of light pos, adjusted for -z
+            let lightPos3 = vec3.fromValues(lightPos[0], lightPos[1], lightPos[2]);
+
+            /*************** X DIMENSION ********************/
+            // For the X & Y dimensions, we use a trigonometric method of computing
+            // sphere + frustum section intersection. See README for details
+            let minX = this._xSlices;
+            let maxX = this._xSlices;
+            for (let i = 0; i < this._xSlices; i++) {
+                // start at -half of X axis
+                let dot = getDotForFrustumCheck(i * xSliceLength - ((verticalStretch / 2.0) * camera.aspect),
+                    lightPos3, "x");
+                if (dot < lightRad) {
+                    // if a section of the frustum is inside the sphere, then take previous section
+                    minX = i - 1; // clamp this later just in case == -1
+                    break;
+                }
+            }
+            if (minX >= this._xSlices) {
+                continue;
+            }
+
+            for (let i = minX + 1; i < this._xSlices; i++) {
+                let dot = getDotForFrustumCheck(i * xSliceLength - ((verticalStretch / 2.0) * camera.aspect),
+                    lightPos3, "x");
+                if (dot > lightRad) {
+                    // if a section of the frustum becomes outside the sphere, then we have it
+                    maxX = i;
+                    break;
+                }
+            }
+
+            /*************** Y DIMENSION ********************/
+            let minY = this._ySlices;
+            let maxY = this._ySlices;
+            for (let i = 0; i < this._ySlices; i++) {
+                // start at -half of Y axis
+                let dot = getDotForFrustumCheck(i * ySliceLength - (verticalStretch / 2.0), lightPos3, "y");
+                if (dot < lightRad) {
+                    minY = i - 1;
+                    break;
+                }
+            }
+            if (minY >= this._ySlices) {
+                continue;
+            }
+
+            for (let i = minY + 1; i < this._ySlices; i++) {
+                // start at -half of Y axis
+                let dot = getDotForFrustumCheck(i * ySliceLength - (verticalStretch / 2.0), lightPos3, "y");
+                if (dot > lightRad) {
+                    maxY = i;
+                    break;
+                }
+            }
+
+            /*************** Z DIMENSION ********************/
+            // Z dimension is purely linear (from near clip to far clip)
+            let minZ = this._zSlices;
+            let maxZ = -1;
+            for (let i = 0; i < this._zSlices; i++) {
+                let currZ = i * zSliceLength + camera.near;
+                if (currZ > lightPos[2] - lightRad) {
+                    minZ = i - 1; // take the previous one because we still need to be < pos - rad
+                    break;
+                }
+            }
+            if (minZ >= this._zSlices) {
+                continue;
+            }
+
+            for (let i = minZ + 1; i < this._zSlices; i++) {
+                let currZ = i * zSliceLength + camera.near;
+                if (currZ > lightPos[2] + lightRad) {
+                    maxZ = i; // take the current one because it has the most recent change
+                    break;
+                }
+            }
+            if (maxZ <= -1) {
+                continue;
+            }
+
+            /********* CLAMPING THE BOUNDS ************/
+            // clamp cluster bounds
+            maxZ = Math.min(maxZ, this._zSlices - 1);
+            maxY = Math.min(maxY, this._ySlices - 1);
+            maxX = Math.min(maxX, this._xSlices - 1);
+
+            minZ = Math.max(minZ, 0);
+            minY = Math.max(minY, 0);
+            minX = Math.max(minX, 0);
+
+            /************ UPDATING AFFECTED CLUSTERS ************/
+            // iterate over the cluster bounds and fill the buffers
+            // optimized loop goes z y then x
+            for (let z = minZ; z <= maxZ; z++) {
+                for (let y = minY; y <= maxY; y++) {
+                    for (let x = minX; x <= maxX; x++) {
+                        // get current num lights, check if == num lights, and update if not
+                        let clusterIdx = x + y * this._xSlices + z * this._ySlices * this._xSlices;
+                        let nbLights = this._clusterTexture.buffer[this._clusterTexture.bufferIndex(clusterIdx, 0)];
+                        if (nbLights >= MAX_LIGHTS_PER_CLUSTER) {
+                            break;
+                        }
+                        this._clusterTexture.buffer[this._clusterTexture.bufferIndex(clusterIdx, 0)] = nbLights + 1;
+
+                        // update to store light index
+                        let row = Math.floor((nbLights + 1) / 4);
+                        let component = nbLights + 1 - row * 4;
+                        this._clusterTexture.buffer[this._clusterTexture.bufferIndex(clusterIdx, row) + component] = lightIdx;
+                    }
+                }
+            }
+        }
+        this._clusterTexture.update();
+    }
 }