Container for multi-pass and randomized subset transform loops

[C-you-know]

🚀 The feature

I propose adding a new meta-transform container to the v2 ecosystem. This container would accept a list of child transforms and execute a randomly sampled subset of them over a configurable number of successive passes, with randomized parameters applied at each step.

The container could take configuration arguments to dynamically shuffle and repeat operations from a pool of provided transforms.

Motivation, pitch

I am working on low-level image restoration and super-resolution pipelines. To train my models, I need to implement high-order degradation pipelines, similar to the double-pass loops pioneered by Real-ESRGAN in 2021. This process requires shuffling the execution order of operations and repeating a sequence of transformations across multiple successive loops to mimic complex, layered artifacts.

Currently, torchvision.transforms.v2 does not seem to have a container to execute this natively. While RandomOrder exists, it appears to enforce a single pass where every provided transform must run exactly once. I have not found a straightforward way to use it for dynamic multi-pass loops, subset sampling, or repeated execution paths.

Because of this limitation, it feels completely necessary to bypass the transform ecosystem for these workflows. Even when using wrap_dataset_for_transforms_v2 to get structured tv_tensors, I might have to manually unpack the data, handcode custom execution loops, and manage randomization myself. A native container could cleanly solve this and keep these workflows entirely within the v2 pipeline.

Alternatives

Nesting Existing Containers: I considered manually stacking combinations of RandomOrder, RandomChoice, and Compose. However this seems to create rigid blocks that may not easily handle dynamic multi-pass subsets or varying parameter ranges across different loop passes.

Custom Transforms: Writing a massive and monolithic custom transform wrapper for the entire pipeline, but this defeats the purpose of modular, reusable transform objects.

External Libraries: Moving away from Torchvision to third-party augmentation packages which fragments the codebase.

Additional context

This addition would provide a reusable structural utility for workflows involving complex data synthesis.

from torchvision.transforms import v2

pipeline = v2.RandomSequence(
    [
        v2.GaussianBlur(kernel_size=(7, 21)),
        v2.GaussianBlur(kernel_size=(3, 11)),
        v2.JPEGCompression(quality=(30, 95)),
        v2.GaussianNoise(sigma=(0.0, 0.2))
    ],
    num_passes=2,
    num_ops_per_pass=2
)

If this fits the roadmap of the v2 framework, I would be happy to implement this container and open a pull request for it.

[NicolasHug]

Hi @C-you-know ,
This seems almost exactly like Compose([RandomChoice(...)] * (num_passes * num_ops_per_pass)) ? For the intended purpose, they should be equivalent?

[C-you-know]

Thanks for the suggestion! That works well for uniform chains but it doesn't quite cover high-order pipelines for a couple of reasons.

The main issue is that RandomChoice samples with replacement at every single step meaning it can easily end up repeating the exact same operation back-to-back. in a pass-based loop, we need to sample a unique subset of operations per pass without immediate duplicates like picking two distinct operations from the pool in Pass 1 and then a different subset in Pass 2.

High-order degradation pipelines like Real-ESRGAN often need to scale their parameter ranges based on the current pass. for example, you might want heavy degradation in Pass 1 and then shift to milder degradation ranges in Pass 2. Managing those kinds of shifts gets pretty messy to track inside a Compose sequence whereas a dedicated container keeps that logic clean

[NicolasHug]

we need to sample a unique subset of operations per pass without immediate duplicates like picking two distinct operations from the pool in Pass 1 and then a different subset in Pass 2.

How would you specify that in your proposed interface? I don't see how this behavior can be currently enforced by just specifying v2.RandomSequence([...], num_passes=2, num_ops_per_pass=2) unless it's a built-in convention of v2.RandomSequence that two passes must never yield the same set of transforms - but then it's not uniformly random anymore.

for example, you might want heavy degradation in Pass 1 and then shift to milder degradation ranges in Pass 2

I also don't see how you can enforce that from the suggested interface

[C-you-know]

I realise my previous explanation was a bit unclear. Please, let me clarify. The semantics were clear in my head but the message didn't express them precisely enough.

we need to sample a unique subset of operations per pass without immediate duplicates like picking two distinct operations from the pool in Pass 1 and then a different subset in Pass 2.

I meant to communicate 

from torchvision.transforms import v2 

pipeline = v2.RandomSequence( 
    [ 
        v2.GaussianBlur(kernel_size=(7, 21)), 
        v2.GaussianBlur(kernel_size=(3, 11)), 
        v2.JPEGCompression(quality=(30, 95)), 
        v2.GaussianNoise(sigma=(0.0, 0.2)) 
    ], 
    num_passes=2, 
    num_ops_per_pass=2 
) 

For the first pass, the result may be [Blur(7,21), Noise] or the other five possibilities. For the second pass, the result might be [Noise, Compression] or any other possible combination. RandomSequence will sample from the same list of transforms without replacement. The transforms picked in the first pass won't affect second pass. Both are sampled independently. The same transform may be present in result of first pass and second pass, but the same transform shouldn't be picked twice in a single pass. For eg. [Blur(7,21), Blur(7,21)] is invalid. The output transforms of the pass will be applied sequentially.

I also don't see how you can enforce that from the suggested interface

Yes. The current architecture alone won't handle mild/heavy in a single object. I was thinking of using two transforms objects, then putting them inside a single Compose.

t1 = v2.RandomSequence([Heavy_Blur, Heavy_Noise, Compression], num_passes = 1, num_ops_per_pass=2)
t2 = v2.RandomSequence([Mild_Blur, Mild_Noise, ColorJitter], num_passes = 1, num_ops_per_pass=2)
t_finale = v2.Compose([t1, t2])

But while writing this, I realised there's a cleaner way to handle it natively.

pipeline = v2.RandomSequence(
    [
        [
            v2.GaussianBlur(kernel_size=(11, 31), sigma=(2.0, 6.0)),
            v2.JPEGCompression(quality=(10, 50)),
            v2.GaussianNoise(sigma=(0.10, 0.30)),
        ],
        [
            v2.GaussianBlur(kernel_size=(3, 7), sigma=(0.2, 1.2)),
            v2.JPEGCompression(quality=(70, 95)),
            v2.GaussianNoise(sigma=(0.0, 0.08)),
            v2.ColorJitter(
                brightness=0.15,
                contrast=0.15,
                saturation=0.15,
                hue=0.05,
            ),
        ],
    ],
    num_ops_per_pass=2,
)

I will pass a list of lists (List[List[Transform]]) which contains transforms (as shown above) which I want to be applied at a particular pass. Each list contains a number of transforms. The ith list will be sampled for the i+1th pass.
num_passes becomes redundant here; it will be the number of lists passed inside the list.

num_ops_per_pass could be modified to be a list in case number of ops chosen per pass should differ. Eg: [2, 1]. 2 ops for the first pass. Only 1 op for second pass.

Example output for the above snippet:

list[0] -- [Blur, Compression]
list[1] -- [Compression, Noise]

The two can coexist via input overloading. A flat list with num_passes handles the symmetric case; a list of lists handles per-pass pool, parameter control, and variable transforms per pass, with num_passes inferred from length.

[C-you-know]

Hi @NicolasHug, just following up. I am curious to hear your thoughts on this approach. I am happy to refine the design if you think this aligns with the v2 roadmap!