--modify-files crashes or produces wrong results

[greenfork]

I have encountered this strange behavior with rak, here is reproduction https://github.com/greenfork/rak-modify-interl

[greenfork]

Also versions

> rak --version
rak - provided by App::Rak 0.3.12, running Raku 6.d with Rakudo 2024.07.

[lizmat]

It appears App::Rak is tickling some underlying issue (possibly multiple ones) in MoarVM and/or NQP and/or Rakudo. Am investigating at the moment.

If you specify --degree, it should not happen (albeit at the cost of being slower).

[greenfork]

Yeah, --degree helps, thank you! Good luck investigating. Let me know when I can remove the repo with reproduction.

[lizmat]

You can remove that repo. Sadly I have plenty of other reproductions :-(

[lizmat]

I've just uploaded 0.3.13 which should hopefully fix your query, albeit at the expense of being less asynchronous for big files.

I don't fully understand yet how this happens, but this should be a lot stabler. Please let me know if this didn't fix it.

Also, today I uncovered an issue in Rakudo with the use of / foo /, aka running a regex on the topic in a multi-threaded manner. This could lead to strange disatch errors. Submodules of App::Rakhave been updated to make sure that that cannot happen anymore.

[greenfork]

I still see bogus results from the command and it crashes about 20% of the time. Command for reference:

rak --modify-files '{ .subst(/«next»/, "return") if .match(/«next»/)}' jobs/

One of my attempts got this stack trace, maybe it helps:

MoarVM oops: MVM_str_hash_fetch_nocheck called with a stale hashtable pointer
   at src/Perl6/Metamodel/MROBasedMethodDispatch.nqp:13  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/Metamodel.moarvm:find_method)
 from src/vm/moar/dispatchers.nqp:1122  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/BOOTSTRAP/v6c.moarvm:)
 from SETTING::src/core.c/Rakudo/SlippyIterator.rakumod:81  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:slip-all)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/E399776C8F1A3226E3347E20A6BDC43053862F3F (rak):1414  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/E3/E399776C8F1A3226E3347E20A6BDC43053862F3F:)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):193  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:run)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):1636  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):789  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
MoarVM oops: MVM_str_hash_fetch_nocheck called with a stale hashtable pointer
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:905  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:272  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:253  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:250  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:run-one)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:291  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Thread.rakumod:69  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:THREAD-ENTRY)

   at src/Perl6/Metamodel/MROBasedMethodDispatch.nqp:13  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/Metamodel.moarvm:find_method)
 from src/vm/moar/dispatchers.nqp:1122  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/BOOTSTRAP/v6c.moarvm:)
 from SETTING::src/core.c/Rakudo/SlippyIterator.rakumod:81  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:slip-all)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/E399776C8F1A3226E3347E20A6BDC43053862F3F (rak):1414  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/E3/E399776C8F1A3226E3347E20A6BDC43053862F3F:)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):193  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:run)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):1636  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):789  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:905  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:272  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:253  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:250  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:run-one)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:291  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Thread.rakumod:69  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:THREAD-ENTRY)

[lizmat]

Thanks! That is helpful! I will be afk most of today, so it will probably be tomorrow before I can look at it.

One note: '*.subst(/«next»/, "return") is functionally equivalent to '{ .subst(/«next»/, "return") if .match(/«next»/)}' as .subst will return the string unchanged if it couldn't do the substitution.

[greenfork]

That is a very nice trick, I haven't thought about it, thank you!

[lizmat]

OOC, did this happen right after you started this run, or had it been busy for a while already?

[greenfork]

I'd say it's rather when started. But it is hard to say because VM takes time to start, maybe it was in the middle. The replacement itself takes very little time