# from headlight beams to glowing billboards: tons of lights in realtime

## lighting #1 – hybrid lights

by Nicholas Francis
Jun 10, 2013

This is the first of a series of blog posts on rendering in Static Sky. We yanked out Unity’s lighting system and replaced it with our own, super fast one that’s specialised for rendering tons of bumped dynamic lights at 60FPS. This post will talk about the hybrid lighting model that we’re using.

We want to do noir, so what does that actually mean? It means a high-contrast world, with lots of light sources. Typically we’d be doing nighttime in the city. This means headlight beams, billboards lighting up the world, somebody being lit by a streetlight as they pass under it.

In other words, lots of small light sources, most of which change – so there’s very little we can bake.

We’re a small shop. This means that we simply do not have time for bake steps – hence we didn’t want to go lightmapped (waiting to bake is such a creativity killer). It’s important for us to remain nimble, so this was seen as a key feature.

## Basic Idea

The solution we settled on is essentially a mix between vertex and fragment lighting. The trick is that for each vertex, we look a the lights affecting the object, and figure out an ‘average light’. It sends this interpolated (or hybrid) into the fragment shader, where we shade it just like we would any other pixel-driven light.

This lighting system mainly runs in the vertex shader – The first step is to find out which lights affect which vertex:

In the image above, the left image is the shaded results, the right one shows the per-vertex influence of each light. It fades to black at the range of the light. Basically, the vertex shader figures out a color and light direction per vertex.

This works really well, but what happens when the lights begin to overlap? Basically we need to average them out.

In the areas where the lights overlap, It’s not possible to approximate with one light – so we need to approximate somehow. What I do is that while I’m adding up to build the hybrid light, I also calculate how much light is coming in from the lights – assuming there’s no bump $$\underline {\vec{n} \cdot \vec{l}}$$. This gives me an idea for how much Then I subtract the hybrid light from the total lighting, to get a per-vertex ambient color that corrects the errors from the averaging. It’s not perfect and loses some bump, but I’d rather that than have the combined lighting be too dark in non-predicatble ways.

To sum up: This doesn’t work that well when lights overlap (they tend to blend together in a blur), but for many small pointlights hitting large objects, it works really well. See below:

Left: The sphere works well as the lights comes from opposite sides. Middle: Hybrid lights on a bad day. Right: Unity built-in. The image in the middle has lost quite a bit of bump definition in the region between the lights

Let’s take a look at how we implement this.

Note: This is inspired by a technique that you can read about here. Unfortunately, that process is patented, so while we really liked the idea, we had to come up with a completely different approach for implementation.

## Implementation

I split the calculations in 2 parts:

• First we need to figure out the direction of our hybrid light, by averaging the incoming light
• Once we have the direction, we need to figure out the color of light coming from that direction.
• We want to stuff the remainder into our per-vertex ambient term.

### Hybrid light direction

In this step we go over all our lights and figure out how important they are. The goal is to figure the direction where most of the illumination is coming from. This means we care about attenuation, light brightness, and especially how much the light faces the vertex normal ($$\underline{\vec{l} \cdot \vec{n}}$$). When calculating facing, it’s worth noting that the most interesting bump happens at an angle to the main surface, so I use wraparound to calculate it ( $$\vec{n} \cdot \vec{l} * .5 + .5$$ ). So we get:

$$Weight_L = Atten_L * Intensity_L * n \cdot l_L * .5 + .5$$

For $$Importance_L$$, I add up the the lights RGB values, and multiply with a fudge factor. You could use the greyscale value of the light, but I figured e.g. a highly saturated blue light was as important as a green light. It’s worth playing around with this formula in a real scenario (esp. once you get specular in). This is what I’m using for now.

We multiply the weights by the normalized light directions, so the hybrid light direction becomes:

$$\vec{hybrid} = \sum_{L=1}^n \vec{l}_L * Weight_L$$

### Figuring out the color

Now we have the hybrid light direction, it’s time to figure out how well each source light maps to it. Basically we do

$$hybridColor = \sum_{L=1}^n \underline{\vec{l}_L\cdot \vec{hybrid}} * color_L$$

The net effect of this is that only lights that actually ended up getting represented by our hybrid light direction contribute. The others are lost – we need to fix that.

### Ambient correction

We want to calculate how much light we need to add back in. What we do a standard diffuse calculation for each vertex, then subtract the hybrid light:
$$ambientColor = (\sum_{L=1}^n \underline{\vec{lightDir_L} \cdot n} * atten_L * color_L ) – hybridColor$$

We add this in as a per-vertex ambient in the pixel shader. The effect of this is that when we have hopeless case for hybrid lighting (different colored lights coming from different directions), we still get the correct brightness in the scene – the bumps just fade away.

## That’s it

This outlines the key concept for the hybrid lighting. In the future, I’ll look at improving the light’s falloff curves & how we get the lights into the system. In the meantime, Please ask away in the comments, and I’ll do whatever I can to answer.

Happy coding

by Sébastien Lagarde
June 12, 2013

Hey,

I use a similar idea but at pixel level instead of vertex and with Spherical harmonic (but at the end the calculation is similar). I describe the technique here if interested:
http://seblagarde.wordpress.com/2011/10/09/dive-in-sh-buffer-idea/

cons are: no shadow, wrong specular…

Cheers

by Nicholas Francis
June 13, 2013

Hey Sébastien

I’ve read your blog with great interest (I even snatched your material chart and integrated some of it into our authoring tools).

I also have the specular issue (I’ll get to that in a later blog post). In general I found it to not be too bad once you’ve got a suitably noisy specular.

I’d really like to play around with your cubemap reflections as well, but I have to move on and actually make a game 😉