YOLO-family models: slow compile that grows dramatically with input size

[itikhono]

Cold compile of YOLO-family detection models is slow and grows dramatically with input size (batch / image size). Analysis done using YOLO26m, fp16 on MI350/R9700, default migraphx options
The behavior is the same for different yolo models and GPUs. I think all conv-based models affected, not only yolo.

Release 7.2.1 (the issue is still present in the develop branch):

Also, in recent versions (starting with 7.2.3), a new regression was introduced that led to an increase in the values listed above without any improvement in performance for the YOLO family of models:

Two independent, removable costs dominate it.

Cause 1. Inputs re-generated per candidate - Covered by this ticket, PR - #4971

Each of ~1339 candidates calls generate_argument() (host PRNG) + H2D copy: the same buffers rebuilt ~1339×. Same in 7.2.1 and develop → baseline cost, not a regression. Grows fast with the candidate input size.

Cause 2. bundle 1→10 (PR #4486) - A separate ticket to track it #4979

#4486 Improve tuning for splitk
Each candidate is now timed with 10×20 = 200 GPU runs instead of 20 — for all candidates, not just split-K. Shipped in ROCm 7.2.3.

GPU part 7.2.1 vs develop (the same in 7.2.3, 7.2.4 Releases)

Batch	GPU runs 7.2.1 (s)	GPU runs develop (s)	factor
1	4.5	32.9	7.3×
2	6.3	54.2	8.6×
4	10.9	94.4	8.7×
8	20.2	178.8	8.9×
16	38.3	347.8	9.1×
32	76.4	685.9	9.0×

Fix

1. Build tuning inputs on the GPU (splitmix64 algo) instead of host generate_argument() + H2D ---> [4970] Generate tuning inputs on GPU via splitmix64 device RNG #4971
2. Revert bundle 10→1 (or gate to split-K) ---> not part of this ticket (new ticket [regression] YOLO-family models: slow compile that grows dramatically with input size #4979), but validated together

Result

1. MI350

2. R9700

Perf testing for YOLO-family models (MI350):

Used migraphx-driver perf, no actual diff detected, the results are quite noisy

Model	Fixed, img/s	Develop (before), img/s	Δ
yolov8m	1255.7	1242.5	+1.1%
yolov9m	1177.1	1111.2	+5.9%
yolov10m	1337.1	1303.4	+2.6%
yolo11m	1583.1	1564.2	+1.2%
yolo12m	1341.4	1350.1	−0.6%
yolo26m	1415.0	1407.2	+0.6%