Project 5: Alexander Chan

README.md

@@ -3,25 +3,39 @@ WebGL Clustered and Forward+ Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Alexander Chan
+* Tested on: **Google Chrome 70.0** on
+  Windows 10 Version 1803, i7-5820k @ 3.70 GHz 16GB, GTX 1080 @ 1620 MHz 8GB (Personal Computer)
+
 
 ### Live Online
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+[![](img/all.gif)](http://ascn.github.io/Project5-WebGL-Deferred-Shading)
 
 ### Demo Video/GIF
 
-[![](img/video.png)](TODO)
+![](img/1000.gif)
+1000 lights with deferred clustered shading at 60 fps.
+
+### Analysis
+
+![](img/performance.png)
+
+| Number of lights | Forward     | Forward+    | Deferred (3 G buffers) | Deferred (2 G buffers) |
+|------------------|-------------|-------------|------------------------|------------------------|
+| 100              | 16.66666667 | 16.66666667 | 16.66666667            | 16.66666667            |
+| 500              | 29.41176471 | 16.66666667 | 16.66666667            | 16.66666667            |
+| 1000             | 58.82352941 | 23.25581395 | 16.66666667            | 16.66666667            |
+| 2000             | 111.1111111 | 41.66666667 | 16.66666667            | 16.66666667            |
+| 4000             | 333.3333333 | 83.33333333 | 29.41176471            | 28.57142857            |
+
+At 100 lights, I was able to hit the FPS cap for all the different pipelines (even forward rendering). However, increasing the number of lights to 500 revealed that the vanilla forward pipeline had much worse performance, as the frametime almost doubled, while there was no change for the other 3 tested.
 
-### (TODO: Your README)
+As the number of lights increased, the deferred pipelines stayed at the FPS cap, while the forward+ pipeline started to fall at 1000 lights. This is interesting as the logic behind the deferred and forward+ fragment shaders is exactly the same. The performance increase is likely due to the fact that the fragment shader itself is quite expensive (as we iterate over the number of lights in this cluster, which can be very high), and the deferred pipeline guarantees that this expensive shader is run only once per pixel. The forward+ pipeline may be running the fragment shader on fragments that are not drawn to the screen.
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+The optimized deferred pipeline only uses 2 gbuffers instead of 3, encoding a 3 component normal in 2, and reconstructing it in the fragment shader. Another compression technique would be to store only the depth in the gbuffer, instead of the position, and reconstruct the world space position in the fragment shader.
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+There is a slight improvement, but not that noticeable (only 1 fewer ms per frame). However, when doing more complex shading, such as PBR shading with more fragment attributes (such as metallic, specular, roughness, velocity for motion blur, etc) it may have a much larger improvement packing all of the attributes into the smallest amount of data possible. This simple shading with a color and a normal does not benefit as much.
 
 
 ### Credits

src/init.js

@@ -1,5 +1,5 @@
 // TODO: Change this to enable / disable debug mode
-export const DEBUG = true && process.env.NODE_ENV === 'development';
+export const DEBUG = false && process.env.NODE_ENV === 'development';
 
 import DAT from 'dat.gui';
 import WebGLDebug from 'webgl-debug';

src/renderers/base.js

@@ -1,6 +1,8 @@
 import TextureBuffer from './textureBuffer';
+import { NUM_LIGHTS } from '../scene';
+import { mat4, vec4, vec3 } from 'gl-matrix';
 
-export const MAX_LIGHTS_PER_CLUSTER = 100;
+export const MAX_LIGHTS_PER_CLUSTER = 2000;
 
 export default class BaseRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -25,6 +27,66 @@ export default class BaseRenderer {
       }
     }
 
+    var halfHeight = Math.tan(camera.fov / 2.0 * (Math.PI/180.0));
+    var halfWidth = halfHeight * camera.aspect;
+
+    for (let i = 0; i < NUM_LIGHTS; ++i) {
+      // get light radius and position
+      var r = scene.lights[i].radius;
+      var lightPos = vec4.fromValues(scene.lights[i].position[0],
+                                     scene.lights[i].position[1],
+                                     scene.lights[i].position[2], 1.0);
+      vec4.transformMat4(lightPos, lightPos, viewMatrix);
+      lightPos[2] *= -1.0;
+
+      var xHalf = halfWidth * lightPos[2];
+      var xStride = (xHalf * 2.0) / this._xSlices;
+
+      var yHalf = halfHeight * lightPos[2];
+      var yStride = (yHalf * 2.0) / this._ySlices;
+
+      var zHeight = (camera.far - camera.near);
+      var zStride = (zHeight) / this._zSlices;
+
+      // get start and end for each cluster dimension
+      var xStart = Math.floor((lightPos[0] - r + xHalf) / xStride) - 1;
+      var xEnd   = Math.floor((lightPos[0] + r + xHalf) / xStride) + 1;
+      var yStart = Math.floor((lightPos[1] - r + yHalf) / yStride);
+      var yEnd   = Math.floor((lightPos[1] + r + yHalf) / yStride);
+      var zStart = Math.floor((lightPos[2] - r) / zStride);
+      var zEnd   = Math.floor((lightPos[2] + r) / zStride);
+
+      // clamp in case
+      var clamp = function(val, min, max) {
+        return val < min ? min : val > max ? max : val;
+      };
+      xStart = clamp(xStart, 0, this._xSlices - 1);
+      xEnd = clamp(xEnd, 0, this._xSlices - 1);
+      yStart = clamp(yStart, 0, this._ySlices - 1);
+      yEnd = clamp(yEnd, 0, this._ySlices - 1);
+      zStart = clamp(zStart, 0, this._zSlices - 1);
+      zEnd = clamp(zEnd, 0, this._zSlices - 1);
+
+      for (let z = zStart; z <= zEnd; ++z) {
+        for (let y = yStart; y <= yEnd; ++y) {
+          for (let x = xStart; x <= xEnd; ++x) {
+            let idx = x + y * this._xSlices + z * this._xSlices * this._ySlices;
+            var lightIdx = this._clusterTexture.bufferIndex(idx, 0);
+            if (this._clusterTexture.buffer[lightIdx] < MAX_LIGHTS_PER_CLUSTER) {
+              var numLights = this._clusterTexture.buffer[lightIdx] + 1;
+              var col = Math.floor(numLights / 4);
+              var row = Math.floor(numLights % 4);
+              this._clusterTexture.buffer[lightIdx] = numLights;
+              this._clusterTexture.buffer[this._clusterTexture.bufferIndex(idx, col) + row] = i;
+            }
+          }
+        }
+      }
+    }
+
     this._clusterTexture.update();
   }
-}
+}
+Number.prototype.clamp = function(min, max) {
+  return Math.min(Math.max(this, min), max);
+};

src/renderers/clustered.js

@@ -8,8 +8,9 @@ import QuadVertSource from '../shaders/quad.vert.glsl';
 import fsSource from '../shaders/deferred.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
 import BaseRenderer from './base';
+import { MAX_LIGHTS_PER_CLUSTER } from './base';
 
-export const NUM_GBUFFERS = 4;
+export const NUM_GBUFFERS = 2;
 
 export default class ClusteredRenderer extends BaseRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -28,8 +29,15 @@ export default class ClusteredRenderer extends BaseRenderer {
     this._progShade = loadShaderProgram(QuadVertSource, fsSource({
       numLights: NUM_LIGHTS,
       numGBuffers: NUM_GBUFFERS,
+      maxLightsPerCluster: MAX_LIGHTS_PER_CLUSTER,
+      xSlices: xSlices,
+      ySlices: ySlices,
+      zSlices: zSlices
     }), {
-      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_gbuffers[3]'],
+      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]',
+                 'u_lightbuffer', 'u_clusterbuffer',
+                 'u_viewMatrix', 'u_screenWidth', 'u_screenHeight',
+                 'u_near', 'u_far'],
       attribs: ['a_uv'],
     });
 
@@ -153,10 +161,25 @@ export default class ClusteredRenderer extends BaseRenderer {
     // Use this shader program
     gl.useProgram(this._progShade.glShaderProgram);
 
+    gl.uniformMatrix4fv(this._progShade.u_viewMatrix, false, this._viewMatrix);
+
+    // Set the light texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE2);
+    gl.bindTexture(gl.TEXTURE_2D, this._lightTexture.glTexture);
+    gl.uniform1i(this._progShade.u_lightbuffer, 2);
+     // Set the cluster texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE3);
+    gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
+    gl.uniform1i(this._progShade.u_clusterbuffer, 3);
+
     // TODO: Bind any other shader inputs
+    gl.uniform1i(this._progShade.u_screenWidth, canvas.width);
+    gl.uniform1i(this._progShade.u_screenHeight, canvas.height);
+    gl.uniform1f(this._progShade.u_near, camera.near);
+    gl.uniform1f(this._progShade.u_far, camera.far);
 
     // Bind g-buffers
-    const firstGBufferBinding = 0; // You may have to change this if you use other texture slots
+    const firstGBufferBinding = 4; // You may have to change this if you use other texture slots
     for (let i = 0; i < NUM_GBUFFERS; i++) {
       gl.activeTexture(gl[`TEXTURE${i + firstGBufferBinding}`]);
       gl.bindTexture(gl.TEXTURE_2D, this._gbuffers[i]);

src/renderers/forwardPlus.js

@@ -6,6 +6,7 @@ import vsSource from '../shaders/forwardPlus.vert.glsl';
 import fsSource from '../shaders/forwardPlus.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
 import BaseRenderer from './base';
+import { MAX_LIGHTS_PER_CLUSTER } from './base';
 
 export default class ForwardPlusRenderer extends BaseRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -16,8 +17,15 @@ export default class ForwardPlusRenderer extends BaseRenderer {
 
     this._shaderProgram = loadShaderProgram(vsSource, fsSource({
       numLights: NUM_LIGHTS,
+      maxLightsPerCluster: MAX_LIGHTS_PER_CLUSTER,
+      xSlices: xSlices,
+      ySlices: ySlices,
+      zSlices: zSlices
     }), {
-      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer'],
+      uniforms: ['u_viewProjectionMatrix', 'u_colmap',
+                 'u_normap', 'u_lightbuffer', 'u_clusterbuffer',
+                 'u_viewMatrix', 'u_screenWidth', 'u_screenHeight',
+                 'u_near', 'u_far'],
       attribs: ['a_position', 'a_normal', 'a_uv'],
     });
 
@@ -64,6 +72,7 @@ export default class ForwardPlusRenderer extends BaseRenderer {
 
     // Upload the camera matrix
     gl.uniformMatrix4fv(this._shaderProgram.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
+    gl.uniformMatrix4fv(this._shaderProgram.u_viewMatrix, false, this._viewMatrix);
 
     // Set the light texture as a uniform input to the shader
     gl.activeTexture(gl.TEXTURE2);
@@ -76,6 +85,10 @@ export default class ForwardPlusRenderer extends BaseRenderer {
     gl.uniform1i(this._shaderProgram.u_clusterbuffer, 3);
 
     // TODO: Bind any other shader inputs
+    gl.uniform1i(this._shaderProgram.u_screenWidth, canvas.width);
+    gl.uniform1i(this._shaderProgram.u_screenHeight, canvas.height);
+    gl.uniform1f(this._shaderProgram.u_near, camera.near);
+    gl.uniform1f(this._shaderProgram.u_far, camera.far);
 
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
     scene.draw(this._shaderProgram);

src/scene.js

@@ -8,7 +8,7 @@ export const LIGHT_RADIUS = 5.0;
 export const LIGHT_DT = -0.03;
 
 // TODO: This controls the number of lights
-export const NUM_LIGHTS = 100;
+export const NUM_LIGHTS = 2000;
 
 class Scene {
   constructor() {

src/shaders/deferred.frag.glsl.js

@@ -4,17 +4,130 @@ export default function(params) {
   precision highp float;
 
   uniform sampler2D u_gbuffers[${params.numGBuffers}];
-
+  uniform sampler2D u_lightbuffer;
+  uniform sampler2D u_clusterbuffer;
+
+  uniform mat4 u_viewMatrix;
+  uniform int u_screenWidth;
+  uniform int u_screenHeight;
+  uniform float u_near;
+  uniform float u_far;
+
   varying vec2 v_uv;
 
+  struct Light {
+    vec3 position;
+    float radius;
+    vec3 color;
+  };
+
+  float ExtractFloat(sampler2D texture, int textureWidth, int textureHeight, int index, int component) {
+    float u = float(index + 1) / float(textureWidth + 1);
+    int pixel = component / 4;
+    float v = float(pixel + 1) / float(textureHeight + 1);
+    vec4 texel = texture2D(texture, vec2(u, v));
+    int pixelComponent = component - pixel * 4;
+    if (pixelComponent == 0) {
+      return texel[0];
+    } else if (pixelComponent == 1) {
+      return texel[1];
+    } else if (pixelComponent == 2) {
+      return texel[2];
+    } else if (pixelComponent == 3) {
+      return texel[3];
+    }
+  }
+
+  Light UnpackLight(int index) {
+    Light light;
+    float u = float(index + 1) / float(${params.numLights + 1});
+    vec4 v1 = texture2D(u_lightbuffer, vec2(u, 0.3));
+    vec4 v2 = texture2D(u_lightbuffer, vec2(u, 0.6));
+    light.position = v1.xyz;
+
+    // LOOK: This extracts the 4th float (radius) of the (index)th light in the buffer
+    // Note that this is just an example implementation to extract one float.
+    // There are more efficient ways if you need adjacent values
+    light.radius = ExtractFloat(u_lightbuffer, ${params.numLights}, 2, index, 3);
+
+    light.color = v2.rgb;
+    return light;
+  }
+
+  // Cubic approximation of gaussian curve so we falloff to exactly 0 at the light radius
+  float cubicGaussian(float h) {
+    if (h < 1.0) {
+      return 0.25 * pow(2.0 - h, 3.0) - pow(1.0 - h, 3.0);
+    } else if (h < 2.0) {
+      return 0.25 * pow(2.0 - h, 3.0);
+    } else {
+      return 0.0;
+    }
+  }
+
+  vec3 decodeNormal(vec2 f) {
+    f = f * 2.0 - 1.0;
+    vec3 n = vec3(f.x, f.y, 1.0 - abs(f.x) - abs(f.y));
+    float t = clamp(-n.z, 0.0, 1.0);
+    n.x += n.x >= 0.0 ? -t : t;
+    n.y += n.y >= 0.0 ? -t : t;
+    return normalize(n);
+  }
+
   void main() {
     // TODO: extract data from g buffers and do lighting
-    // vec4 gb0 = texture2D(u_gbuffers[0], v_uv);
-    // vec4 gb1 = texture2D(u_gbuffers[1], v_uv);
-    // vec4 gb2 = texture2D(u_gbuffers[2], v_uv);
+
+    vec4 gb0 = texture2D(u_gbuffers[0], v_uv);
+    vec4 gb1 = texture2D(u_gbuffers[1], v_uv);
+    //vec4 gb2 = texture2D(u_gbuffers[2], v_uv);
+
+    // OPTIMIZED:
+    vec3 v_position = gb0.rgb;
+    vec3 albedo = vec3(gb0.w, gb1.xy);
+    vec3 normal = decodeNormal(gb1.zw);
+
+    // NOT OPTIMIZED:
+    // vec3 v_position = gb0.rgb;
+    // vec3 albedo = gb1.rgb;
+    // vec3 normal = gb2.rgb;
+
     // vec4 gb3 = texture2D(u_gbuffers[3], v_uv);
 
-    gl_FragColor = vec4(v_uv, 0.0, 1.0);
+    vec4 viewPos = u_viewMatrix * vec4(v_position, 1.0);
+    viewPos.z = -viewPos.z;
+
+    float xStride = float(u_screenWidth) / float(${params.xSlices});
+    float yStride = float(u_screenHeight) / float(${params.ySlices});
+    float zStride = float(u_far - u_near) / float(${params.zSlices});
+
+    int xCluster = int(gl_FragCoord.x / xStride);
+    int yCluster = int(gl_FragCoord.y / yStride);
+    int zCluster = int((viewPos.z - u_near) / zStride);
+
+    int clusterIdx = xCluster + yCluster * ${params.xSlices} + zCluster * ${params.xSlices} * ${params.ySlices};
+    int numClusters = ${params.xSlices} * ${params.ySlices} * ${params.zSlices};
+    float u = float(clusterIdx + 1) / float(numClusters + 1);
+    int texHeight = int(ceil(float(${params.maxLightsPerCluster} + 1) / 4.0));
+    int numLights = int(texture2D(u_clusterbuffer, vec2(u, 0)).r);
+
+    vec3 fragColor = vec3(0.0);
+
+    for (int i = 0; i < ${params.numLights}; ++i) {
+      if (i >= numLights) { break; }
+      Light light = UnpackLight(int(ExtractFloat(u_clusterbuffer, numClusters, texHeight, clusterIdx, i + 1)));
+      float lightDistance = distance(light.position, v_position);
+      vec3 L = (light.position - v_position) / lightDistance;
+
+      float lightIntensity = cubicGaussian(2.0 * lightDistance / light.radius);
+      float lambertTerm = max(dot(L, normal), 0.0);
+
+      fragColor += albedo * lambertTerm * light.color * vec3(lightIntensity);
+    }
+
+    const vec3 ambientLight = vec3(0.025);
+    fragColor += albedo * ambientLight;
+
+    gl_FragColor = vec4(fragColor, 1.0);
   }
   `;
 }