Update documentation using deterministic generation#8557
Open
rustyrussell wants to merge 31 commits into
Open
Conversation
ShahanaFarooqui
left a comment
Collaborator
There was a problem hiding this comment.
@rustyrussell The changes in ./tests/autogenerate-rpc-examples.py look good at first glance, but I am unable to test them locally due to missing test data files:
./tests/data/autogenerate-bitcoin-blocks.json
./tests/data/autogenerate-bitcoind-wallet.dat
Could you please add these files or provide instructions on how to generate them for local testing?
9fc3b5e to
c455c51
Compare
Collaborator
|
Now And |
9d0aae6 to
43fdc51
Compare
This allows us to override it for deterministic results. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
dev_override_randbytes() hashes argv0 into the CLN_DEV_ENTROPY_SEED stream, so every plugin and subdaemon gets a distinct seed. But lightningd execs them with absolute paths, so the "deterministic" stream silently depended on where the source tree was checked out: canned blocks generated in one directory would not replay in another (funding tx output order and nlocktime fuzz diverge, so txids change). This is why check-doc-examples passed locally but failed in CI. Hash only the basename: binaries still get distinct seeds, but the stream no longer depends on the checkout path.
Trace/span ids now come from a plain counter when CLN_DEV_ENTROPY_SEED is active, so tracing (which CI's HAVE_USDT=1 build enables by default) no longer shifts the deterministic RNG stream that PSBT serial ids and locktime fuzz are drawn from.
Previous commit stopped trace ids consuming the deterministic randbytes stream, but tracing still touched it once: span_key_hash() calls siphash_seed(), whose lazy init draws 16 bytes from the stream. Since randbytes call sites are assigned their offset blocks in first-use order, that single draw shifts every later draw in the process by 1000 when tracing is active (HAVE_USDT builds, as in CI): keysend preimages and invoicerequest payer keys then differ from a build with tracing disabled, showing up as an 11-file doc-examples diff. Use a cheap multiplicative hash for span keys when the deterministic override is active, so tracing touches the RNG stream zero times.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We temporarily add a flake8 exception so it doesn't complain about all the now-unused variables. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…hannel example. Now order has changed, the first one is insufficient; this is more robust. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Unless REGENERATE_BLOCKCHAIN is true, in which case we make them. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes our transaction generation (nlocktime) consistent, as well as our htlcs for payment. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Move the elements config writing into an ElementsD.set_port() override. This makes many of our information outputs consistent. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Don't mangle bookkeeper events. 2. Use l3, not l1, for listtransactions: it's more stable. 3. Use l2, not l1, for listpeerchannels and listchannels. 3. Use l2, not l2, for listfunds. (l1 has a lot of variation). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And hand in CLN_NEXT_VERSION to replace the version string. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…s_settled, wait_for_wallet_txs
… request in Rewriter
…smtool getsecret`
…stabilise the response And flake fixes
…wProxy mocks These tests built their RawProxy against a hardcoded bitcoin.conf in the node's datadir. That only worked on liquid-regtest by accident: BitcoinD.__init__ used to write bitcoin.conf even when the node was actually an ElementsD. Now that config writing is deferred to set_port() and ElementsD only writes elements.conf, the mocks found no config and no .cookie, so their RPC calls failed and bcli's getblock timed out, leaving lightningd stuck during restart. Use bitcoind.conf_file, which points at the config the running daemon actually uses (elements.conf on liquid-regtest, bitcoin.conf on egtest).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
a8eafc8 to
00dc521
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #8556Merged!Now we do major surgery on autogenerate-rpc-examples.py. Finally we run it to produce the canned blocks and wallet for future runs, and re-enable it in CI.