.NET: Python: .NET: Make skill resource guidance rule-only - #7664
.NET: Python: .NET: Make skill resource guidance rule-only#7664Sam (samrusani) wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
Removes fictional skill resource examples from Python and .NET prompts while preserving explicit resource discovery.
Changes:
- Replaces examples with equivalent rule-only guidance.
- Adds prompt regressions and explicit-path positive controls.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
python/packages/core/agent_framework/_skills.py |
Updates Python resource guidance. |
python/packages/core/tests/core/test_skills.py |
Adds Python regression coverage. |
dotnet/src/Microsoft.Agents.AI/Skills/AgentSkillsProvider.cs |
Updates .NET resource guidance. |
dotnet/tests/Microsoft.Agents.AI.UnitTests/AgentSkills/AgentSkillsProviderTests.cs |
Adds .NET regression coverage. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
|
Thanks for preparing this fix. We also tested a broader, domain-neutral revision of the skills prompt that preserves the rule-only resource guidance while making skill selection and follow-up orchestration more explicit. Sharing the exact tested prompt for further consideration: We evaluated this with 60 uncached runs across 12 multi-turn scenarios. Compared with our previous prompt, it preserved overall quality while improving resolution and follow-up routing:
The remaining failures were tied to application-specific skill content, eval rubrics and safety policy rather than generic skill orchestration. |
Motivation & Context
AgentSkillsProvidercurrently places fictional resource names in the defaultPython and .NET instructions. A model can mistake those examples for resources
that the loaded skill actually provides and make a guaranteed-to-fail
read_skill_resourcecall.This change keeps resource access grounded in the loaded skill while preserving
the existing resource discovery and tool APIs.
Fixes #7663
Description & Review Guide
matching rule-only guidance: read only explicitly referenced resources,
preserve the referenced path exactly, and never infer resource paths.
examples.
exactly when the loaded skill explicitly provides it.
names. Public APIs and valid resource listings are unchanged.
.NET and that the C# assertions match the existing skills-test conventions.
The focused Python regressions fail against the base prompt and pass with this
change. The complete Python
test_skills.pyfile and core syntax checks pass.The .NET tests were not run locally because the .NET CLI was unavailable; draft
CI is expected to provide that validation.
Related Issue
Fixes #7663
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.