Proposal: Add Bulk Blob Download Links API

[njucjc]

Summary

Add GET /v2/<name>/manifests/<reference>/blobs to return pre-signed download URLs for all blobs (config + layers) of a single manifest in one request, with an optional expiry parameter.

Motivation

The Current Workflow

Today, downloading a complete image requires:

1. GET /v2/<name>/manifests/<reference>
2. N+1 calls to GET /v2/<name>/blobs/<digest> for config and each layer

Production Pain: LB Connection Exhaustion and CPU Saturation

In large-scale image pull scenarios — such as CI/CD platforms launching hundreds of concurrent builds, or data migration tools replicating thousands of images — we consistently observe two failure modes:

1. Frontend load balancer connection exhaustion: Each blob request is an independent HTTP request. With C concurrent clients pulling images averaging N=15 layers, the registry frontend receives C × (N+1) connections. Even with HTTP/2 multiplexing, the connection pool and request queue on the LB quickly saturate, causing 502/504 errors and cascading retries that worsen the overload.

2. Registry CPU saturation: Each blobs/<digest> request triggers:

   • Authentication and authorization (token validation, permission checks).
   • Repository and blob digest resolution (database/cache lookups).
   • Pre-signed URL generation (HMAC or object-storage signature computation).

   Signature generation is CPU-bound and does not benefit from horizontal scaling without shared caching. When thousands of clients request blobs concurrently, the registry's CPU is consumed by these per-request overheads, not by data transfer (which is offloaded to object storage).

Why This Matters

Metric	Existing API	Proposed API
Requests per image to registry frontend	N+1	1
Registry CPU overhead per image	N+1 × auth + signature	1 × batched auth + signature
LB connection pressure	Scales with C × (N+1)	Scales with C
Link expiration control	None (backend-defined)	Explicit expiry parameter

This API is not intended to replace the standard container runtime pull path. It targets bulk operations: image migration, offline export, and CI platforms that pull complete images at scale.

Detailed Design

Endpoint

GET /v2/<name>/manifests/<reference>/blobs?expiry=<seconds>

Parameters

Parameter	Type	Required	Default	Description
name	string	yes	—	Repository name
reference	string	yes	—	Tag or digest
expiry	integer	no	3600	Link validity in seconds (60–604800)

Success Response: 200 OK

{
  "manifestDigest": "sha256:abc...",
  "expiresAt": "2026-04-28T12:00:00Z",
  "totalSize": 104857600,
  "blobs": [
    {
      "digest": "sha256:cfg...",
      "size": 1024,
      "mediaType": "application/vnd.oci.image.config.v1+json",
      "kind": "config",
      "downloadUrl": "https://..."
    },
    {
      "digest": "sha256:lyr...",
      "size": 52428800,
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "kind": "layer",
      "downloadUrl": "https://..."
    }
  ]
}

Error Responses

Status	Condition
401	Authentication failed
403	No pull permission
404	Repository or reference not found (same response for both to avoid information leakage)
400	Invalid expiry value
415	Reference resolves to a manifest list (multi-arch) instead of a single manifest. See discussion below.
500	Internal error during signature generation

References

• OCI Distribution Specification

[sudo-bmitch]

Please do not use an LLM to submit overly verbose issues. Do you have an existing implementation showing that this works? Do you have the metrics showing the performance improvement in general use cases, including mirroring a repository where the base image is reused for a majority of the copied images? In /v2/<name>/manifests/<reference>/blobs, what stops <reference> from being set to manifests which would pull the tag blobs?

[njucjc]

@sudo-bmitch Thank you for your prompt reply! The issue we are encountering is that during large-scale image pulling scenarios, the connection to the registry's front-end load balancer often becomes saturated, or the registry's CPU usage reaches maximum capacity. This proposal is a design sketch (generated by LLM) for the problem that I am meeting, but I don't yet have the implementation or metrics to prove it. I'm submitting it to gauge whether the others see the same bottleneck, and if so, whether the design direction is worth refining with real data.