Skip to content

fix(function-autoscaler): re-evaluate metric source after first LLM request #1601

Description

@borao

Describe the bug

In autoscaler 1.21.0, a running LLM function with no prior requests has no worker or per-function gateway series. The autoscaler selects ControlPlane and caches it for one hour, so later LLM traffic remains invisible and the function cannot scale correctly.

Steps or code to reproduce bug

  1. Deploy a new LLM function with minInstances: 1.
  2. Wait 2–3 minutes without sending requests.
  3. Send sustained LLM traffic.
  4. Observe that the autoscaler continues using ControlPlane metrics and reports 0% utilization.

Expected behavior

Absence-based fallback is re-evaluated after gateway metrics appear, and missing utilization data cannot trigger scale-to-zero.

Additional context


By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageIssue or PR awaiting maintainer triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions