fix(skills): prevent empty responses after load_skill#6226
Open
lwangverizon wants to merge 1 commit into
Open
Conversation
The skill system instruction's rule 2 told the model to follow loaded skill instructions "before replying to the user". Some models (notably Gemini) read this as license to treat the load_skill tool call as the entire turn and stop with no visible output, producing empty responses. This was most acute for tool-heavy skills, whose next correct action after load_skill is to call more tools rather than reply. Two changes to _build_skill_system_instruction: - Reword rule 2 to drop the "before replying" framing. - Add rule 7 stating that load_skill only retrieves instructions and does NOT complete the turn; the model must continue in the same turn (calling whatever tools the skill requires) and never end with an empty response right after loading a skill. Verified in a production environment: the empty-response rate for Gemini after skill loading dropped substantially with this guidance. Adds tests asserting the new prompt guarantees on both the default and prefixed system instruction.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(skills): prevent empty responses after
load_skillFixes #6225
Problem
When an agent uses the skill tools, ADK appends a skill system instruction to the
prompt (
_build_skill_system_instructioninsrc/google/adk/tools/skill_toolset.py). With Gemini models, we observed asignificant rate of empty model responses right after a
load_skillcall — themodel loads the skill and ends its turn with no visible output.
We saw this as a spike in empty responses in a Verizon production environment running
ADK + Gemini, most acutely for tool-heavy skills whose next correct action after
loading is to call more tools rather than reply.
Root cause
Rule 2 told the model to follow loaded instructions "before replying to the user".
Gemini reads this as license to treat the
load_skilltool call as the whole turnand stop. Nothing in the instruction states that
load_skillis non-terminal.Change
_build_skill_system_instructiononly — covers bothDEFAULT_SKILL_SYSTEM_INSTRUCTIONand the per-toolset prefixed variant:
Reword rule 2 to drop the "before replying" framing:
Add rule 7 (continue-after-load guard):
No signature or behavior change; rules stay contiguous and the
prefixsubstitutionis preserved (rule 7 uses
{p}load_skilllike the other rules).Result
After deploying this guidance in production, the empty-response rate for Gemini after
skill loading dropped substantially.
Tests
Adds focused tests in
tests/unittests/tools/test_skill_toolset.py:test_system_instruction_marks_load_skill_as_non_terminal— rule 7 present.test_system_instruction_rule_2_avoids_before_replying_framing— rule 2 no longersays "before replying".
test_prefixed_system_instruction_includes_continue_after_load_rule— prefixedvariant carries rule 7 with the prefix.
Existing tests reference
DEFAULT_SKILL_SYSTEM_INSTRUCTIONby constant (not hardcodedtext), so they are unaffected.
Backwards compatibility
Prompt-text-only change. No public API, signature, or default-behavior change.