SPIR-V Builder: Do not emit invalid SPIR-V code if the instruction lenngth exceeds the allowed length, log an error and return empty SPIR-V if error count isn't zero.

[mbeck-nv]

• every SPIR-V instruction has a length field that is limited to 16 bit, allowing a maximum of 65535 words
• however, several instruction are created from GLSL entities that have looser restrictions
• a "typical case" when this might happen is a constant array that is too long
• detect this case and send error messages to a logger if the user has set one and stop, do not depend on optional validation pass

[arcady-lunarg]

Where would a user be able to come across this error that it shouldn't be treated as an internal compiler error? I would hope that the relevant GLSL extensions that can lead down this path allow for an implementation-defined limit, and if so, we should be catching that in the frontend rather that letting it get this far. That said, I think it's not a bad idea to have a check in the backend to make sure that we're not emitting invalid SPIR-V.

[mbeck-nv]

Currently it is rather simple to trigger this problem. Just declare an constant array of 65536 float3 for instance. Due to the way the OpConstant is defined, the maximum supported length is 65555 - 3, and the original code just clip it without checks to 16bit. That said, I don't like this solution. A compiler should not have the concept of "user defined logger" imho. However, as I don't see an infrastructure to producing errors in the Spir-V builder, this is better then nothing. Moreover I doubt such a clip can be detected in general in a validator, but havn't looked in the Spir-V tools yet, so I might be wrong. The other cases are rather esoteric. The bug can for instance be also triggered if one defines a struct with more the 65532 fields and create a struct constant of it... Handling it in the FE sounds more logical, but is really a backend restriction.

[mbeck-nv]

PS: I agree that with Vulkan in mind such big constants are rather seldom and more typically handed through SSBO or something similar. Nevertheless, the original case comes from a lookup table in some user written code...