[CUDA] Why is the compiler invoked to generate a PTX and cubin instead of simply a fatbin

[stgeke]

[kris-rowe]

• Since fat binaries can contain multiple cubins for multiple architectures, it is likely that a single cubin is smaller than a fat binary.
• Conversely, using fat binaries could potentially reduce the need to recompile cached kernels; however, we will need to introduce smarter logic into the jitting process to leverage this.
• There are also users who are looking to deploy their compiled kernels with their application without including the OKL source. This could also benefit them.

I will table this for discussion at the next OCCA TAF meeting.

[tcew]

iirc occa hashes on the compiler flags as well as source code. If you go with fat binary then as long as the command line arches agree with the cache then should things just work ?

[stgeke]

Right in this case the device driver will not invoke the JIT compiler, so no caching required. Obviously there can be still caching issues coming from OCCA but that's a different topic.

[tcew]

Since you brought it up, what is the latest status on OCCA caching on Summit. Is it very reliable ? and do you just default to warm up runs to populate the caches ?

[kris-rowe]

Sorry, just noticing now that we default to ptx generation. Changing this to either fatbin or cubin shouldn't be an issue.

As Tim pointed out, we add the arch flag and hash against the arch version, so using the -cubin flag likely makes the most sense.

Alternatively, to be more general, we could stop hashing against the arch, use -fatbin, and either stop adding the -arch flag—supposing that users will do this themselves—or add the -arch all flag.

[stgeke]

We have something robust now but we had to introduce several tweaks. No warm-up run is required anymore. I can share what we do in the next TAF meeting if that's of interest.