fix: enforce expression size limit before source construction#1302
Open
thesmartshadow wants to merge 1 commit intogoogle:masterfrom
Open
fix: enforce expression size limit before source construction#1302thesmartshadow wants to merge 1 commit intogoogle:masterfrom
thesmartshadow wants to merge 1 commit intogoogle:masterfrom
Conversation
Enforce the configured ParserExpressionSizeLimit in Env.Compile() and Env.Parse() before calling common.NewTextSource(), preventing memory allocation proportional to the full input size for oversized expressions. Previously, oversized expressions were correctly rejected but only after the internal rune buffer had already been allocated. This allowed substantial memory allocation even when a strict size limit was configured. The fix adds an early utf8.RuneCountInString() check which is O(n) in CPU but avoids the additional memory allocation and GC pressure caused by eager source/rune-buffer construction. CWE-400: Uncontrolled Resource Consumption
thesmartshadow
force-pushed
the
fix/early-expression-size-limit-enforcement
branch
from
985387a to
9b5a353
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR enforces the configured
ParserExpressionSizeLimitinEnv.Compile()and
Env.Parse()before callingcommon.NewTextSource(), preventing memoryallocation proportional to the full input size for oversized expressions.
Problem
The current flow is:
Env.Compile(txt)→common.NewTextSource(txt)common.NewTextSource()→runes.NewBufferAndLineOffsets(contents)runes.newBuffer()eagerly allocates capacity proportional to the full inputbuf.Len() > p.expressionSizeCodePointLimitApplications configuring
ParserExpressionSizeLimit(1000)expect oversizedexpressions to be rejected cheaply, but the internal source/rune-buffer
materialization still occurs before enforcement.
Fix
Add an early
utf8.RuneCountInString(txt)check in bothCompile()andParse()before callingcommon.NewTextSource().Before / After
The test
TestExpressionSizeLimitEarlyEnforcementconfirms 0MiB allocationdelta for a 10MB oversized expression with a 1000 code point limit.