Skip to content

feat(apps): fuse/parameter serde fixes for app pods + userspace volume-serve example#1157

Open
kumare3 wants to merge 1 commit into
haytham/volume-examplesfrom
ketan/app-volume-serve
Open

feat(apps): fuse/parameter serde fixes for app pods + userspace volume-serve example#1157
kumare3 wants to merge 1 commit into
haytham/volume-examplesfrom
ketan/app-volume-serve

Conversation

@kumare3

@kumare3 kumare3 commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Makes Volumes usable from Flyte apps. Three SDK fixes found by deploying a real app against demo.hosted.unionai.cloud, plus a new example that serves a sealed ROVolume from a scale-from-zero app with zero privileges.

SDK fixes (src/flyte)

  1. app_serde: wrong primary container name for FUSE apps. The enable_fuse_mount pod augmentation built the template with primary container "primary"; the apps backend requires "app" (enforced in _get_k8s_pod). Deploys failed with Primary container name must be 'app'.

  2. app_serde: pod-templated apps never materialized RunOutput parameters. get_proto_container received parameter_overrides, but _get_k8s_pod/_serialized_pod_spec did not — so the raw RunOutput JSON spec was baked into the fserve --parameters blob and reached on_startup un-materialized (ValidationError: Field required when the app tried to parse it). Threaded parameter_overrides through the pod path.

  3. _pod: new privileged_only= mode for pod_template_with_fuse_mount. Knative's admission webhook rejects hostPath volumes (unsupported before Serving 1.17, feature-gated after) and gates capabilities.add; privileged alone implies device access + all caps, so app pods now emit only that. Caveat discovered en route: Knative silently drops privileged on every version, so kernel FUSE mounts still don't work in apps — which motivates the example below.

New example: examples/volumes/volume_browser_app.py

  • repo-snapshot.snapshot_repo downloads the flyte-sdk GitHub tarball into a fresh Volume (FUSE-mounted task) and seals it; returns (ROVolume, json_spec) so the typed volume keeps UI lineage while the app picks up the JSON via a RunOutput string parameter (getter=(1,)).
  • serve_volume_webdav() — prototype for a future ROVolume.serve(): the same juicefs engine mount() uses (index download + metadata materialization via plugin internals), exposed over localhost WebDAV (juicefs webdav --read-only --no-bgjob) instead of through /dev/fuse. No privileges, no PodTemplate, runs on stock Knative — only the kernel mount(2) step is skipped.
  • FastAPI file browser over a stdlib WebDAV client (PROPFIND Depth:1 + GET; file-vs-dir decided from the self entry's resourcetype since PROPFIND on a file still 207s).
  • Scaling(replicas=(0, 1), scaledown_after=300) — scale-from-zero; each cold start is one index download + a subprocess spawn.

Test plan

  • tests/flyte/app/ (121 tests) pass
  • WebDAV client + serve flow validated locally against a file://-backed juicefs volume (dir listing, file read, 404, read-only rejects writes)
  • End-to-end on demo.hosted.unionai.cloud: snapshot run succeeded; app deployed unprivileged on Knative 1.16 (no feature flags); /browse/, deep listings, and file reads all serve from the sealed volume

🤖 Generated with Claude Code

…serve example

Three SDK fixes that make Volumes usable from Flyte apps, plus a new
example that prototypes serving a sealed ROVolume from a scale-from-zero
app with zero privileges.

SDK fixes (found by deploying the example against a real cluster):

- app_serde: the enable_fuse_mount pod augmentation built the pod
  template with primary container "primary"; the apps backend requires
  "app" (enforced in _get_k8s_pod), so deploys failed with "Primary
  container name must be 'app'".

- app_serde: pod-templated apps never materialized RunOutput/delayed
  parameters — get_proto_container received parameter_overrides but
  _get_k8s_pod/_serialized_pod_spec did not, so the raw RunOutput JSON
  was baked into the fserve --parameters blob and reached on_startup
  un-materialized. Threaded parameter_overrides through the pod path.

- _pod: new privileged_only mode for pod_template_with_fuse_mount.
  Knative's admission webhook rejects hostPath volumes (unsupported
  before Serving 1.17, feature-gated after) and gates capabilities.add;
  privileged alone implies device access + all caps, so app pods emit
  only that. (Knative drops `privileged` silently on every version, so
  kernel FUSE mounts still don't work in apps — hence the example below.)

New example (examples/volumes/volume_browser_app.py):

- a task snapshots the flyte-sdk repo into a Volume and seals it; the
  sealed ROVolume crosses into the app as its JSON spec via a RunOutput
  string parameter (getter=(1,))
- serve_volume_webdav(): prototype of a future ROVolume.serve() — the
  same juicefs engine mount() uses (index download + metadata
  materialization via plugin internals), exposed over localhost WebDAV
  (`juicefs webdav --read-only`) instead of through /dev/fuse. No
  privileges, no PodTemplate, runs on stock Knative.
- FastAPI file browser over a stdlib WebDAV client (PROPFIND + GET),
  Scaling(replicas=(0, 1)) with a 5-minute scaledown.

Verified end-to-end on demo.hosted.unionai.cloud: snapshot task ran,
app deployed unprivileged, browse/file/404 paths all serve correctly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant