Samsung Exynos AI LiteCore - Support yolo26 - #22960
Jiseong-oh wants to merge 6 commits into
Conversation
silu which is followed by split operator is not properly annotated to have qparam. detect activation function by backward propagation search Co-Authored-By: Jintech Noh <jintech.noh@samsung.com> Co-Authored-By: Jingya Zhang <jingya.zhang@samsung.com> Signed-off-by: Jiseong Oh <jiseong.oh@samsung.com>
split_with_sizes_copy's getitem users may only consume part of the op's output, so propagate quantization parameters per-output instead of assuming every output is used, and support having output branches with differing quant params. Co-Authored-By: Jintech Noh <jintech.noh@samsung.com> Signed-off-by: Jiseong Oh <jiseong.oh@samsung.com>
Regarding remainder op, decompose it into the equivalent div/floor/mul/sub sequence before lowering and Wire the new pass into EnnPassManager. Co-Authored-By: Jingya Zhang <jingya.zhang@samsung.com> Signed-off-by: Jiseong Oh <jiseong.oh@samsung.com>
Register the four new NodeVisitors in builders/__init__.py, and add aten.floor_divide.default to EnnPartitioner.ops_to_not_decompose so the new op_floor_divide visitor actually sees the op instead of its decomposition. Co-Authored-By: Jingya Zhang <jingya.zhang@samsung.com> Signed-off-by: Jiseong Oh <jiseong.oh@samsung.com>
Refactor op_topk.py's output/dim handling and simplify op_index.py. Co-Authored-By: Jingya Zhang <jingya.zhang@samsung.com> Signed-off-by: Jiseong Oh <jiseong.oh@samsung.com>
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22960
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 9d35e34 with merge base dba8a83 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
Wire EnnPassManager's transform_for_export_pass into to_edge_transform_and_lower_to_enn, drop the now-redundant DecomposeScaledDotProductAttention call from quantize_module, and rewrite examples/samsung/utils.py's save_tensors to recursively walk arbitrary nested tensor structures instead of a flat list. Add the yolo26 model test and validation example script, and loosen test_add's atol for the new coverage. Co-Authored-By: Jingya Zhang <jingya.zhang@samsung.com> Signed-off-by: Jiseong Oh <jiseong.oh@samsung.com>
c9195e5 to
9d35e34
Compare
psiddh
left a comment
There was a problem hiding this comment.
Automated review of the yolo26 enablement changes. I checked the branch out into a worktree and verified several of these by execution rather than inspection (noted inline).
Highest priority: the op_upsample_nearest2d.py scale-factor change reads out_shape[0]/out_shape[1] (N and C) instead of [-2]/[-1]; I reproduced [0.0625, 0.25] instead of [2.0, 2.0] using the module already in test_upsample_nearest2d.py. Close behind: op_max_dim.py's guard is and-chained and drops the indices output in the only state RemoveGetItemPass can leave it in, and op_split_with_sizes_copy.py's copied_indices bookkeeping only lines up when getitem users happen to be in ascending order.
Worth noting on the positive side: removing DecomposeScaledDotProductAttention from quantize_module is correct — prepare_pt2e already runs it via EnnQuantizer.transform_for_annotation → transform_for_annotation_pass. And routing transform_for_export_pass through to_edge_transform_and_lower_to_enn fixes a real inconsistency with samsung_tester.py.
One process note: this adds five op builders and a new pass with no new tests under backends/samsung/test/ops/. An op-level test would have caught the upsample bug directly.
This review was generated by an AI reviewer (Claude Code). Findings are offered as starting points — please verify each one against your own understanding of the ENN backend before acting.
| scale_factor = [ | ||
| output_size[0] * 1.0 / in_shape[-2], | ||
| output_size[1] * 1.0 / in_shape[-1], | ||
| out_shape[0] * 1.0 / in_shape[-2], |
There was a problem hiding this comment.
Wrong axes on out_shape — scale factor is off for every size=-style upsample.
out_shape = get_shape(node)
scale_factor = [
out_shape[0] * 1.0 / in_shape[-2],
out_shape[1] * 1.0 / in_shape[-1],
]get_shape returns the full tensor shape, so for NCHW out_shape[0] is N and out_shape[1] is C. The output_size this replaced was a 2-element [H, W] list, which is why indices [0]/[1] used to be correct.
Reproduced with the module already in backends/samsung/test/ops/test_upsample_nearest2d.py:
args: (x, [32, 32], None)
in_shape: [1, 4, 16, 16] out_shape: [1, 4, 32, 32]
this PR : [0.0625, 0.25]
expected: [2.0, 2.0]
args[2] (scale_factors) is None whenever size= is used, so the override on line 40 does not rescue it. This is the common path.
Fix: out_shape[-2] / out_shape[-1].
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| input_id = self.define_tensor(input, enn_graph, vals_to_ids) | ||
|
|
||
| users = list(node.users.keys()) | ||
| if ( |
There was a problem hiding this comment.
The guard is and-chained, and in the only reachable case the indices output is silently dropped.
users = list(node.users.keys())
if (
len(users) != 1
and users[0].target.__name__ == "getitem"
and users[0].args[1] != 0
):
return FalseTwo issues that compound:
-
To bail out on anything other than "exactly one user, a
getitem(0)" these clauses needorwith negated comparisons. As written,len(users) == 1short-circuits the guard toFalseno matter what that user is — andusers[0].target.__name__raisesAttributeErrorfor a non-call_functionuser (e.g.output, whose target is the string"output"). -
RemoveGetItemPassruns before serialization (_passes/enn_pass_manager.py:79) and rewrites single-consumeraten.max.dimintoexir_ops.edge.aten.amax.default, raising for any count other than 1 or 2 (backends/transforms/remove_getitem_op.py:30-46). So this visitor only ever seesmax.dimwith exactly two users — values and indices. It then defines onlyoutput_idx=0, registers onlyvals_to_ids[users[0]], and emits a single-outputReduceMax; the indicesgetitemnever gets a tensor id. And ifusers[0]is the indicesgetitem, all three clauses are true and the build fails.
Either emit both outputs the way op_topk.py now does, or return False when the indices output is live.
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| for user in node.users.keys(): | ||
| if user.target.__name__ == "getitem" and len(user.args) > 1: | ||
| copied_idx = user.args[1] | ||
| copied_indices.append(copied_idx) |
There was a problem hiding this comment.
copied_indices is appended unconditionally, so copied_indices[idx] only lines up by accident.
copied_indices.append(copied_idx) # runs for every (output_idx, user) pair
if copied_idx == output_idx:
output_id = self.define_tensor(user, enn_graph, vals_to_ids)
all_output_tensors.append(output_id)copied_indices ends up with len(args[1]) * num_getitem_users entries while all_output_tensors has num_getitem_users. Line 53's copied_indices[idx] therefore reads the first N entries — getitem indices in node.users iteration order — whereas all_output_tensors[idx] is ordered by ascending output_idx.
Those orders coincide only when the getitems were created in ascending index order. With x = s[2] written before y = s[1], copied_indices[0] is 2 while all_output_tensors[0] is index 1's tensor, and the emitted STRIDEDSLICE bounds are swapped.
Fix: move the append inside the if copied_idx == output_idx: branch.
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| "begin": begin, | ||
| "end": end, | ||
| "strides": strides, | ||
| "shrink_axis_mask": pow(2, axis), |
There was a problem hiding this comment.
Two smaller things in this block:
- Line 52,
end = in_shapealiases the list fromget_shape;end[axis] = ...mutates it in place, and every iteration'sparams["end"]is the same object. Harmless today becausedefine_opcopies eagerly, butend = list(in_shape)costs nothing. "shrink_axis_mask": pow(2, axis)— asplit_with_sizes_copyoutput keeps the split axis with sizepoints[i]; shrinking it would drop the dimension entirely. Is that what ENN'sSTRIDEDSLICEexpects here, or should this be0?
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| if idx == 0: | ||
| val_id = self.define_tensor(user, enn_graph, vals_to_ids) | ||
| vals_to_ids[user] = val_id | ||
| outputs.append(val_id) |
There was a problem hiding this comment.
Output order follows node.users iteration order rather than the getitem index.
outputs is passed straight to define_op(..., all_output_tensors, ...) alongside output="both". When both getitems exist but getitem(1) was created first — e.g. t = torch.topk(x, k); i = t.indices; v = t.values — outputs comes out as [indices, values] and the two ENN outputs are transposed. The previous code appended value then index unconditionally, so the ordering was structural.
test_topk.py returns tuple(torch.topk(...)), which always produces ascending order, so it won't catch this.
Separately: a topk whose result tuple is consumed by a non-getitem user hits the continue on line 62, leaving outputs empty while output_type stays "both" — a zero-output op.
Fix: collect into {idx: tensor_id} and emit by index rather than by append order.
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| pt_model, example_args, compile_specs=compile_specs | ||
| ) | ||
|
|
||
| edge = edge_prog.to_backend(EnnPartitioner(compile_specs)) |
There was a problem hiding this comment.
The model is lowered twice.
edge_prog = to_edge_transform_and_lower_to_enn(pt_model, example_args, compile_specs=compile_specs)
edge = edge_prog.to_backend(EnnPartitioner(compile_specs))to_edge_transform_and_lower_to_enn (backends/samsung/utils/export_utils.py:78-82) already calls to_edge_transform_and_lower with EnnPartitioner, so this re-runs partitioning over an already-delegated program. Drop this line and call edge_prog.to_executorch(...) directly.
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| type=int, | ||
| ) | ||
| parser.add_argument( | ||
| "--dump", |
There was a problem hiding this comment.
Two argparse foot-guns in this block:
--dumpusestype=boolwithnargs="?", so--dump Falseevaluates tobool("False") == True.action="store_true"is what's meant.--input_dims(line 314) usestype=eval, which evaluates arbitrary input;ast.literal_evalornargs=2, type=intis safer. Relatedly, theargs.input_dims != [640, 640]check on line 264 rejects a tuple literal such as(640,640).
Also: line 159 constructs RuntimeExecutor(exec_prog, input_tensor) inside the per-batch loop, so the .pte is re-pushed to the device once per image (128× for coco128). And the usage examples on lines 28-31 say python test_yolo26.py, but the file is yolo26_validate.py.
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| @@ -0,0 +1,383 @@ | |||
| # Copyright (c) Intel Corporation | |||
There was a problem hiding this comment.
Copyright (c) Intel Corporation — presumably carried over from OpenVINO's export_and_validate.py, which this mirrors. Worth confirming that's the intended attribution for a file in examples/samsung/.
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| sub_op = ( | ||
| exir_ops.edge.aten.sub.Tensor | ||
| if is_edge | ||
| else torch.ops.aten.sub.Tensor |
There was a problem hiding this comment.
This branch is unreachable as the pass is currently wired.
Both const-materialisation branches are gated on is_x_scalar and is_edge / is_y_scalar and is_edge. The pass is registered only in transform_for_export_pass (_passes/enn_pass_manager.py:63), which runs on the pre-edge ATen graph, so is_edge is always False and create_const_node never fires. I confirmed the ATen path works fine without it — scalars pass through as raw Python values and floor_divide dispatches correctly, including for remainder.Scalar_Tensor.
Beyond the dead code: if the pass were moved to the edge stage, create_const_node would register_buffer and insert a get_attr into an ExportedProgram's graph without updating graph_signature or state_dict. torch.export lifts buffers to placeholders, so a bare get_attr isn't valid there.
Either drop the scalar/const machinery, or register the pass where it's needed and extend the helper to update the program signature.
(The decomposition itself checks out — I ran it standalone over Scalar, Scalar_Tensor, Tensor, and integer inputs; max abs error ≤ 2.4e-7.)
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
| .check_count({"torch.ops.higher_order.executorch_call_delegate": 1}) | ||
| .to_executorch() | ||
| .run_method_and_compare_outputs(inputs=inputs) | ||
| .run_method_and_compare_outputs(inputs=inputs, atol=0.003) |
There was a problem hiding this comment.
FP32 tolerance loosened 3× with no explanation.
The harness default is atol=1e-3 (backends/test/harness/tester.py:321). This is the FP32 path of a plain elementwise add, where near-exact agreement is the expectation, and the change isn't mentioned in the PR description.
If an FP32 add genuinely started needing 3e-3 during this work, that's worth understanding rather than absorbing — the annotate_2in1out change in quantizer/annotator.py is a candidate, though it shouldn't affect the FP32 path. What regressed?
Generated by an AI reviewer (Claude Code). Please verify before acting on it.
Summary
Test plan
python test_yolo26.py -c E9955 -m yolo26s -d /path/to/images -p A8W8 --validate coco128.yaml
cc @SS-JIA @digantdesai @kimishpatel