Image-Based Lighting

One can do complex environment lighting by precomputing lookup tables.

Recall: The Light Loop

vec3 color = vec3(0.0);
vec3 n = get_normal();
vec3 v = to_eye();
for (uint i = 0; i < 3; ++i) {
vec3 l = LIGHTS[i].to_light();
vec3 e = LIGHTS[i].energy();
//f depends on material
color += f(n,l,v)*e;
}
gl_FragColor = color;
$color \equiv \sum_{l=0}^2 f(\omega_l, v) e_l$

Recall that our current game code runs a loop over all lights.

Problem: Many Lights

vec3 color = vec3(0.0);
vec3 n = get_normal();
vec3 v = to_eye();
for (uint i = 0; i < 1000; ++i) {
vec3 l = LIGHTS[i].to_light();
vec3 e = LIGHTS[i].energy();
//f depends on material
color += f(n,l,v)*e;
}
gl_FragColor = color;
$color \equiv \sum_{l=0}^{999} f(\omega_l, v) e_l$

This model runs into trouble when there are many lights.

Unfortunately, lots of lights is the common case when doing realistic lighting, since anything you can see is a source of light.

A Simplification(?)

Sum over lights: $\sum_l f(\omega_l, v) e_l$

$$\;\;\rightarrow\;\;$$

Integral over directions: $\int_\omega f(\omega, v) e_\omega$

Instead of summing lights, we could integrate over a lighting environment which records the incoming light from every direction.

$\int_\omega f(\omega, v) e_\omega =$

$\langle$

$,$

$\rangle$

This doesn't seem simpler.
But if we expand $$f$$, there might yet be hope...

$\int_\omega f_{\textrm{mirror}}(\omega, v) e_\omega = e_{r} =$

$[r]$

E.g. for a mirror, only depends in $$r$$ (reflection direction).

$\int_\omega f_{\textrm{lambert}}(\omega, v) e_\omega = \int_\omega (n\cdot\omega) e_\omega =$

$\langle$

$,$

$\rangle =$

$[n]$

E.g. for Lambertian diffuse shading, only depends on $$n$$.
(Don't read the math too closely on this slide.)

$\int_\omega f_{\textrm{spec}}(\omega, v) e_\omega =$

$\langle$

$,$

$\rangle \approx$

$[r]$

E.g. for a glossy surface, only depends on reflection direction + roughness.
Standard hack: use mip-maps for roughness levels.

Strategy Overview

$\int_\omega f_{\textrm{mat}}(\omega, v) e_\omega = c_0\textrm{diffuse}(n) + c_1\textrm{specular}(r,roughness)$

So, for a generic physically-inspired material, we can decompose this integral into a diffuse lookup based on the face normal direction and a specular lookup based on the reflected view direction.

Code Details

Asset Pipeline: compute specular, diffuse, etc...

Distribution: $$2^{E} \frac{(R+0.5,G+0.5,B+0,5)}{256}$$ color storage

Runtime: cubemap lookups; tone-mapping

One More Fix

$$\;\;\rightarrow\;\;$$

So far, we've treated the lighting environment like it is infinitely far away.
This neglects parallax effects between objects. We can fix this to some extent with light probes -- lighting environments stored throughout our game world.

Summary

• You can simulate complex lighting with lookup tables.
• For physically-motivated materials, you'll generally use one lookup table each for the various parts of your lighting.
• In order to deal with different lighting at different locations in a scene, your game can store light probes.
• Your code can compute the lookup tables off-line or on-line.
• Aside: it is possible to pre-compute self-occlusion and self-interreflection and store this with an object -- see work on "precomputed radiance transfer".