discussion on attribute locations

[mattdesl]

leaving this here for future discussion.

The problem: let's say we have a utility mesh like gl-sphere and it expects attributes in the order [ position, colors, normals ]. However, we want to give the user the ability to render it with a custom shader -- but their shader might have attributes in the order [position, normals, colors].

The solution should not need to re-compile shaders since it's expensive and doesn't allow for shader re-use.

One idea is to use string aliases when defining the attributes of a VAO, and have the user pass a shader to vao.bind(). Then the VAO always knows which attribute locations to bind, assuming the gl-shader has them listed in the correct order.

[mikolalysenko]

I think we could also just have gl-shader cache the different program objects. That way, you would only pay a 1-time cost associated with recompiling the shader under a different permutation of attributes. This could also reuse the shader objects, so the overhead would be relatively lower than compiling a new program object from scratch.

[mattdesl]

I think there are a few issues with this:

1. Recompiling is slow. Generally I think we should avoid re-compiling "under the hood" as it causes janks and hitches that are hard to track and fix. This is what ThreeJS does; and it's a huge pain because you need to force a render frame just to get a shader compiled.
2. You might end up with an exponential amount of shaders. If you have 3 vertex attributes, in edge cases you could end up with 6 compiled shaders. Now imagine you have 20 shaders, each with 16 attributes.
3. Even if we can re-use the compiled shaders, edge cases will perform worse since there are many more shader switches involved. Imagine a typical scene graph composed of a lot of meshes, not all of them using the same vertex attribute layout. If we cache compiled versions, we might end up with hundreds of shader switches per frame even though they all use the same shader source.

Most game engines order their scene by shaders first. Allowing many meshes to take advantage of the same shader (and by that I mean same GL shader object, not gl-shader object) is the first step to efficient rendering.

[mikolalysenko]

1. The recompiling would be eager, and you would only hit jank on the first frame when the shader is loaded up. But once the app is loaded, everything should be fine.
2. At a coarse analysis, this would seem to be an issue. In the worst case, you could end up with O(n!) shaders, assuming that your app really does exhaust all permutations of attributes. However, this would be pretty difficult to actually achieve in practice, since creating that many shaders would also require drawing at least O(n!) different objects, each with a different ordering of attributes. As a result, I don't think it is really worth worrying about.
3. It would be up to the programmer to make sure that all their objects are using vertices in the same format. As you would have to rebind the shader anyway after doing a vertex attribute switch, it would be pretty obvious what is going on. Leaving this problem up to the application to define and stick to some convention seems reasonable to me.

On the topic of shader first rendering, I wholeheartedly agree and think that the current system enables applications to take advantage of this quite nicely, assuming that they are designed with it in mind.

[mattdesl]

1. The jank could happen at any time, whenever a new object is added to your scene graph that has a different vertex layout, or when you are rendering the same object with a fresh shader. The only way to avoid this is to force a render of all objects up-front (with all variations of shaders attached) without actually rendering it to screen. This is similar to the horrible hacks that we've had to employ on past ThreeJS projects to ensure that shaders and buffers are not being updated during gameplay.
2. You are right that we probably won't get all permutations. More likely is that some shaders will have 2-4 program objects under the hood, which is still not ideal.
3. Users don't want to think about vertex attribute layout when they require('gl-sphere'). Example scenario: if gl-sphere and gl-cube have a different layout, not only will every shader double when applied to them, but you will never be able to render both with the same shader object.

To me this is a massive problem and red flag for a game engine. And it prevents you from ordering your objects shader-first since the actual GL shader objects are muddled under a caching layer. So in edge cases, even if your gl-sphere and gl-cube meshes are sorted by their gl-shader object, you might end up with rendering that looks like this in terms of shader switches: ABABABAB.

There is also the subject of uniforms. Let's say you do shader.uniforms.proj = mat4, does that trigger a gl.uniform call to each cached shader? Uniforms might be relatively cheap, but there's no point in doubling or tripling your uniform uploads per frame if you can avoid it.

Further; it becomes more complicated when somebody wants to use the raw webgl api alongside stackgl. Using gl.getUniform(shader.handle, foo) etc could lead to tricky problems. It would have to be shader.handleForLayout( attribLayout ) or something.

[mikolalysenko]

• 1. The jank would only happen if you were dynamically shifting attributes around all the time, which would only happen if the user was doing something crazy like shuffling them around.
• 3. Another way to think about this though is that maybe it doesn't make sense to create custom prebaked objects like gl-sphere or gl-cube. Instead there could just be generic modules that generate the geometry as say indexed face lists or position attributes, then it would be up to the applications to package that up into some vertex format that they would consume. (See for example how the bunny or stanford-dragon modules work). That way everyone could use their own application/shader specific vertex format which would be tuned to the problem that they are trying to solve, rather than trying to over engineer some one-size fits all solution for all cases

Obviously shader switching is bad, and the current system in no way promotes this or encourages it. It is up to the application to handle these details (as there are practically infinite ways to set this up).

Regarding uniforms, setting the uniform value shouldn't change the value for all shaders. If you do change a location using shader.attributes.blah.location = x, the current behavior is that it will trigger a relink and clear out all the uniforms. Obviously doing this in a render loop is bad, and this behavior is discouraged.

Maybe the solution would be to just make it more clear in the documentation that changing attribute locations triggers relinking?

The current strategy for promoting reusability in graphics code is that users would still control vertex formats, buffers, and geometry and they would still write custom shader code. However, there would be plenty of tools to make this process as streamlined as possible, with reusable glsl subroutines (via glslify) and common geometry processing libraries via ndarray and others.

[mattdesl]

I agree sphere/cube should be in the form of cells/positions/normals, but I also think most users will want a higher-level wrapper so they don't need to manually set up buffers. Hooking up buffers and shaders manually is where things will crap out if you aren't really conscious of attribute locations (and who in their right mind wants to worry about those).

I ran into this in development of gl-quad-batcher because I was using the same VAO with various different shaders, some of which I had no control over and thus couldn't predict their vertex layout. In this case the "jank" (just a single frame of shader recompilation) could happen at any point, i.e. when a sprite with a custom shader enters the scene for the first time. Since shader recompilation is fairly slow in WebGL/ANGLE I think we should not be designing around it.

So maybe the best approach is to have another module like gl-shader-vao (or a better name) which handles the attribute mappings when you call vao.bind(shader). This way gl-vao and gl-shader will stay the same and have no ties to each other.

Then modules like gl-geometry could be changed to use this instead of gl-vao to also avoid shader recompilation.

[mattdesl]

I just ran into a tricky situation where the same shader compiled in Chrome with [ position, uv ], and in FF/Safari as [ uv, position ]... 😢 The end result was that nothing was drawn correctly. Nasty stuff.

[mikolalysenko]

@mattdesl maybe the solution there is to make gl-shader-core sort the attributes lexicographically unless otherwise specified, so that there is no ambiguity.

[nickdesaulniers]

@mattdesl , maybe I misunderstand the problem, but couldn't you guys use reflection over the shader program to get a mapping of uniform & attributes to locations? See: https://github.com/nickdesaulniers/webgl-shader-loader/blob/master/webGLShaderLoader.js#L123-L143

This uses gl.getProgramParameter with gl.ACTIVE_ATTRIBUTES and gl.ACTIVE_UNIFORMS to get the number of attributes and uniforms, then packs an object where the keys are the attribute or uniform's identifier and the values are the location. It gets the identifier using gl.getActiveAttrib and gl.getActiveUniform and the locations using gl.getAttribLocation and gl.getUniformLocation.

That way, it doesn't matter what browser compiles your shader, you get webgl implementation non-specific shader locations.