feat(awk): add restricted awk builtin by matt-dz · Pull Request #599 · DataDog/rshell

matt-dz · 2026-08-10T15:09:15Z

Summary

consolidates feat(awk): implement phase 1 core interpreter #244, feat(awk): implement phase 2 basics #240, feat(awk): implement phase 3 practical features #241, and feat(awk): implement phase 4 investigation features #245 into one AWK implementation based directly on current main
adds a restricted AWK parser/runtime with fields, arrays, control flow, functions, regex/text builtins, getline, and rshell-sandboxed command pipes
replaces the 826-file generated corpus and bespoke harness with 50 ordinary scenario tests plus focused runtime, resource, and security tests
keeps AWK expectations directly in the scenario YAML with no GNU awk oracle metadata or harness
validates nested command scripts and bounds aggregate output, pipe, redirection, split, and variable resources

matt-dz · 2026-08-10T15:33:37Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 72fce59a7a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-10T16:02:00Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0277fc43c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-10T16:28:35Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8be3d36aab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-10T16:58:12Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ebeecaa8dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-10T17:21:32Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd21ece8bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-10T18:10:13Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc8d45e657

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-10T18:32:36Z

@codex review

matt-dz · 2026-08-14T17:37:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfc1295f51

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T17:55:58Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e5016aa701

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T18:39:20Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc9d3ebe48

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T19:14:05Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4875120e4f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T19:34:46Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ee95a7f9ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T20:05:20Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76c5853bfb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T20:26:50Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8aebc73610

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T20:35:06Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d0db2b6dfa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T20:50:30Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd1f3aaf34

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

matt-dz · 2026-08-14T21:00:02Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 740298bd5f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-14T21:11:50Z

+	start := match[0] + 1
+	length := match[1] - match[0]


Preserve terminal empty-match positions for multibyte strings

Under the C.UTF-8 GNU awk reference, BEGIN { s="éa"; print match(s, /$/), RSTART, RLENGTH } prints 4 4 0, while converting the terminal byte offset to a rune offset here makes rshell print 3 3 0. The same incorrect position is stored in the third-argument capture metadata, so scripts using RSTART or a[0,"start"] to process end-of-string matches diverge on multibyte input.

AGENTS.md reference: AGENTS.md:L61-L62

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-14T21:11:50Z

+	if _, ok := rt.vars[name]; ok {
+		return fmt.Errorf("cannot use scalar %s as array", name)


Classify runtime scalar-array misuse as fatal

When a scalar is used as an array at runtime, such as BEGIN { x=1; print x[1] }, this unprefixed error is classified by errorResult as an ordinary failure and exits 1, whereas GNU awk treats the runtime type violation as fatal and exits 2. Array-to-scalar misuse follows the same path, so these errors should receive the fatal classification expected by callers that distinguish awk runtime failures by status.

AGENTS.md reference: AGENTS.md:L61-L62

Useful? React with 👍 / 👎.

feat(awk): add restricted awk builtin

833edd5

This was referenced Aug 10, 2026

feat(awk): implement phase 2 basics #240

Closed

feat(awk): implement phase 4 investigation features #245

Closed

feat(awk): implement phase 1 core interpreter #244

Closed

feat(awk): implement phase 3 practical features #241

Closed

test(awk): use standard scenario tests

72fce59

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread builtins/awk/eval.go

Comment thread tests/scenarios_test.go

fix(awk): bound function call depth

0277fc4

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread builtins/awk/eval.go Outdated

Comment thread builtins/awk/eval.go Outdated

fix(awk): honor ignorecase in string operations

8be3d36

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread builtins/awk/parser.go

fix(awk): bound parser nesting depth

ebeecaa

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread builtins/awk/awk.go

Comment thread builtins/awk/eval.go Outdated

Comment thread builtins/awk/printf.go Outdated

matt-dz added 2 commits August 10, 2026 13:20

fix(awk): preserve invalid bytes in substr

e1bffc9

fix(awk): ignore zero padding for text formats

bd21ece

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread builtins/awk/eval.go Outdated

Comment thread interp/runner_exec.go Outdated

matt-dz added 3 commits August 10, 2026 13:31

fix(awk): bound program file loading

298db31

fix(awk): bound gensub match storage

4c4f3a4

fix(interp): bound nested script execution

fc8d45e

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread interp/runner_exec.go

Comment thread builtins/awk/runtime.go Outdated

Comment thread builtins/awk/runtime.go Outdated

matt-dz added 3 commits August 10, 2026 14:30

fix(awk): expose only exported variables in environ

e5cb3c7

fix(awk): preserve invalid bytes in character split

53a956f

fix(awk): accept multibyte separators at record limit

a252776

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/parser.go Outdated

fix(awk): parse dynamic field operands

e5016aa

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/runtime.go Outdated

Comment thread builtins/awk/runtime.go

Comment thread builtins/awk/runtime.go

fix(awk): harden regex normalization

dc9d3eb

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/eval.go Outdated

Comment thread builtins/awk/runtime.go Outdated

fix(awk): address GNU compatibility gaps

4875120

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/runtime.go

Comment thread builtins/awk/runtime.go

Comment thread builtins/awk/eval.go Outdated

fix(awk): align GNU edge-case behavior

ee95a7f

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/parser.go

Comment thread builtins/awk/runtime.go Outdated

fix(awk): pin parser and Unicode edge cases

76c5853

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/eval.go Outdated

Comment thread builtins/awk/runtime.go Outdated

fix(awk): apply CONVFMT in string contexts

8aebc73

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/eval.go Outdated

Comment thread builtins/awk/runtime.go Outdated

fix(awk): complete CONVFMT conversions

d0db2b6

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/eval.go

Comment thread builtins/awk/runtime.go

This comment has been minimized.

Sign in to view

fix(awk): align separator and regex errors

cd1f3aa

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread builtins/awk/eval.go Outdated

fix(awk): convert redirection targets with CONVFMT

740298b

chatgpt-codex-connector Bot reviewed Aug 14, 2026

View reviewed changes

		if _, ok := rt.vars[name]; ok {
		return fmt.Errorf("cannot use scalar %s as array", name)

Uh oh!

Conversation

matt-dz commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

matt-dz commented Aug 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

matt-dz commented Aug 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

matt-dz commented Aug 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

matt-dz commented Aug 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matt-dz commented Aug 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

matt-dz commented Aug 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matt-dz commented Aug 10, 2026

Uh oh!

matt-dz commented Aug 14, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

matt-dz commented Aug 14, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matt-dz commented Aug 14, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

matt-dz commented Aug 10, 2026 •

edited

Loading