DAOS-19368 vos: callbacks' rcs should not produce iterator errors

[janekmi]

Steps for the author:

• Commit message follows the guidelines.
• Appropriate Features or Test-tag pragmas were used.
• Appropriate Functional Test Stages were run.
• At least two positive code reviews including at least one code owner from each category referenced in the PR.
• Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

• Gatekeeper requested (daos-gatekeeper added as a reviewer).

[github-actions]

Ticket title is 'Aurora daos_user: Observed Error message in log "DER_KEY2BIG(-2012): 'Key is too large'" while doing rebuild'
Status is 'In Progress'
Labels: 'ALCF'
https://daosio.atlassian.net/browse/DAOS-19368

[NiuYawei]

I think printing errors returned by callback is fine. The fundamental issue is that the callback shouldn't return error code when it isn't actually an error.

What if we fix the places in src/object/srv_enum.c that return -DER_KEY2BIG? Instead of returning an error code, they should return 1 and set a flag in ds_obj_enum_arg. Then ds_obj_enum_pack() can check that flag and return -DER_KEY2BIG to the caller.

[janekmi]

I very much like this idea. Error codes strictly reserved to internals. Callback can indicate either a failure or success but it has to pass the details on its own.

Here is a new draft: #18767