feat(pod): unprivileged allow_fuse() via FUSE device plugin + volume examples#1205
Merged
Conversation
…efault allow_fuse() previously defaulted to privileged=True (a /dev/fuse hostPath + privileged container), and its privileged=False path used a hostPath + AppArmor-unconfined — which still fails at runtime: a hostPath surfaces the device node but the devices cgroup denies open() with EPERM, so an unprivileged container cannot use it. Default allow_fuse() now uses the FUSE device-plugin path instead: it requests the smarter-devices/fuse extended resource (so kubelet injects /dev/fuse into the container's devices-cgroup allowlist) plus CAP_SYS_ADMIN for mount(2) — no privileged, no hostPath. This composes with allow_nested_sandboxing() and is what actually works on a real cluster running a FUSE device plugin (the Union dataplane chart ships an opt-in fuseDevicePlugin DaemonSet). allow_fuse(privileged=True) is retained as a legacy escape hatch (hostPath + privileged) for clusters without a device plugin. Validated end-to-end on a real Union dataplane (dogfood): an unprivileged pod with exactly this template mounted a flyteplugins-union JuiceFS Volume with IO. Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
kumare3
reviewed
Jun 13, 2026
| _stamp_capability(pt, "fuse") | ||
| return pt | ||
|
|
||
| # Legacy escape hatch (privileged=True): /dev/fuse hostPath + privileged, for |
Contributor
There was a problem hiding this comment.
is this weird that this will fallback on randomly, i think we should cleanly separate 2 methods.
if privileged
_fallback_to_legacy
else:
enable_fuse_device
Contributor
Author
There was a problem hiding this comment.
Good call — split into two clearly separated helpers in de82835c: _apply_fuse_device_plugin(primary) (default, unprivileged) and _apply_fuse_privileged(pt, primary) (legacy hostPath + privileged escape hatch). _apply_fuse just dispatches on the flag; the shared CAP_SYS_ADMIN grant + capability annotation stamp run once after. No inline fallback.
…ged helpers Per review: dispatch cleanly to _apply_fuse_device_plugin (default) vs _apply_fuse_privileged (legacy escape hatch) instead of an inline fallback; CAP_SYS_ADMIN + capability stamp are shared across both. Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
…fuse() Adds the flyteplugins-union Volume examples (volume_example, volume_cold_fork, volume_recover, bench) and aligns gcsfuse_example. They request unprivileged FUSE via pod_template=flyte.PodTemplate().allow_fuse() — the device-plugin grant this PR makes the default — instead of a privileged container. Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
EngHabu
changed the title
_apply_fuse_privileged mutates in place and is annotated -> None; remove the leftover 'return pt' that mypy flagged (the dispatcher _apply_fuse returns pt). Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
plugins/redis/uv.lock had drifted from its pyproject resolution on main (`uv lock --check` failure, unrelated to this PR's changes). Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
kumare3
approved these changes
Jun 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two things, together:
PodTemplate.allow_fuse()now grants unprivileged FUSE via a device plugin by default —smarter-devices/fuseresource request +CAP_SYS_ADMIN, no privileged container, no/dev/fusehostPath.allow_fuse(privileged=True)is kept as a legacy escape hatch (hostPath + privileged) for clusters without a device plugin. The two paths are cleanly separated (_apply_fuse_device_plugin/_apply_fuse_privileged).Volumeexamples (examples/volumes/) updated to usepod_template=flyte.PodTemplate().allow_fuse()instead of the old privileged/enable_fuse_mountapproach.Why
The old
allow_fuse(privileged=False)path used a/dev/fusehostPath, which doesn't actually work: a hostPath surfaces the device node but the container's devices cgroup still deniesopen()withEPERM. Onlyprivileged(or a device plugin) grants access. The device-plugin path is the real unprivileged solution — requestingsmarter-devices/fusemakes kubelet inject/dev/fuseinto the devices-cgroup allowlist;CAP_SYS_ADMINcoversmount(2). Composes withallow_nested_sandboxing().The cluster must run a FUSE device plugin advertising
smarter-devices/fuse— the Union dataplane chart ships an opt-infuseDevicePluginDaemonSet (unionai/helm-charts#443, unionai/cloud#16489).Validated end-to-end
On a real Union dataplane (dogfood, device plugin deployed): an unprivileged task pod built from
PodTemplate().allow_fuse()(CAP_SYS_ADMIN +smarter-devices/fuse, not privileged, no hostPath) opened/dev/fuseand mounted an S3-backed flyteplugins-unionVolumewith a read/write round-trip.CapEff=a82425fb(SYS_ADMIN only).Tests
TestAllowFuse(device-plugin default) +TestAllowFusePrivileged(legacy hostPath escape hatch) + updated sandboxing-composition tests — 54 passed.