add a fuzz test for zig fmt #23793
add a fuzz test for zig fmt #23793
Conversation
|
Very nice! You know what else might be a nice property to test is that running fmt a second time does not change the output. Also, I think this test will be much more efficient once we have a smith. I don't know how far along @mlugg got on that, but that's a fun project. Basically take a fuzz input and convert it into a valid AST. |
|
My zig-smith has taken a back seat for now, but I also don't think it would be a great test of the parser. My goal with that project has been to emit programs which compile correctly and do interesting things in the context of incremental compilation, and which, crucially, are deterministic at runtime and do not trigger IB. Those are big restrictions, so the thing I was working on was unfortunately quite limited in terms of syntax. I would definitely be interested in something more generalized, but I'd have to figure out how to make it useful in the context of testing incremental compilation! |
Right but we're talking about testing the renderer. The renderer happens to also be a great way to test the parser, which is why they are related. But my point is that it would provide the ability to test the idempotency property of |
gooncreeper
changed the title
|
Just updated this PR to also test idempotency and some other stuff, and fixed most of the bugs it found. |
|
Thanks for the update, I have this open in a tab (among many...) |
This fuzz test checks several properties of zig fmt are upheld, namely idempotency, textual equivilence, and no trailing whitespace. All functions in the fuzz test have @disableInstrumentation for performance and since their branches are not interesting to the fuzzer.
Adds two new space modes maybe_space and comma_maybe_space which only render the space when the next token is not a multiline string literal. This is used in a lot of places to avoid a trailing space on lines caused by multiline string literals forcing newlines. Fixes places where the node tag is compared to be a multiline string literal to instead check if the first token is a multiline string literal line. This fixes cases like `\\ ++ text`. This commit also rewrites quite a bit of code to be clearer / deduplicated.
Adds a new function becomesMultilineExpr to determine this and extracts the logic from several functions.
renderSpace takes the length of the original lexeme, so passing the length of the new lexeme is not correct.
There were just too many bugs. This new implementation supports zig fmt: on/off. It also puts expressions with unicode characters on their own line, which avoids issues with aligning them.
gooncreeper
force-pushed
the
fmt-fuzzing
branch
from
0fa1673 to
7d24a43
Compare
3 participants
This fuzz test checks several properties of zig fmt are upheld, namely idempotency, textual equivilence, and no trailing whitespace.
All functions in the fuzz test have @disableInstrumentation for performance and since their branches are not interesting to the fuzzer.
Closes #10418
Closes #14654
Closes #15929
Closes #18088
Closes #20108
Closes #22778
Closes #23071
Closes #23722
Closes #24065