guts manifest -c singularity workarounds

[fredjaya]

Hi @vsoch, wondering if you have any tips on how guts can be used for singularity images?

@Mitchob and I are using shpc to automate building image versions that don't exist in the default shpc registry. These are usually older builds prior to when the GitHub actions were set up.

Main issue with the older versions is that the aliases don't reflect the latest registries, as expected. We'd like to somehow adapt guts for images from /cvmfs/singularity.galaxyproject.org, and docker will not be in the production environment.

A use case i'm currently to get running is star-fusion v1.0.0. More details of current testing in this draft PR if it helps.

Thanks!

[vsoch]

We would need a SingularityContainer class akin to the DockerContainer class that exposes the same function signatures, and I think that would do it. Knowing Singularity images, the issue we might run into is with respect to speed to do the same operations, but it is worth a test! If you are interested in taking a shot, the main entrypoint creates a ManifestGenerator that is going to call extract on that class. That happens here:

guts/container_guts/main/client.py

Line 102 in 5bb9f61

@ensure_container

Let me know what you think! It's a very solid use case for the tool. Another option is that we figure out a way to ask shpc to build a docker image that isn't in its registry by default, and during that process, also have a means to discover binaries of interest. I likely don't have bandwidth to work on this, but I am glad to support you (or someone on your team) to walk through the thinking and steps.

[fredjaya]

Amazing, thank you! Definitely leaning towards adding a SingularityContainer class.

I'll reach out again if this is something that we can take on.