Conversation
t-bast
force-pushed
the
audit-db-rework
branch
from
057acbb to
a093279
Compare
t-bast
force-pushed
the
audit-db-rework
branch
from
a093279 to
a50fbbb
Compare
t-bast
force-pushed
the
audit-db-rework
branch
from
a50fbbb to
33fd5cb
Compare
We improve the `channel_events` table in the `AuditDb` by: - adding the `funding_txid` from each event - adding the `channel_type` from each event - using hex instead of blobs in sqlite - providing APIs to list these events - adding indexes on `channel_id` and `remote_node_id` The added data isn't available in past events. We decide to simply rename the older tables and start fresh: previous data will not be available from the API, but can still be queried directly in SQL if necessary.
We improve the `transactions_published` and `transactions_confirmed` tables in the `AuditDb` by: - adding more details about mining fees and feerates - adding input and output counts - using hex instead of blobs in sqlite - providing APIs to list transactions by `remote_node_id` - adding indexes for listing APIs The added data isn't available in past events. We decide to simply rename the older tables and start fresh: previous data will not be available from the API, but can still be queried directly in SQL if necessary.
We improve the `sent` and `received` payment tables in the `AuditDb` by: - adding `start_at` / `settled_at` timestamps to measure duration - adding `remote_node_id` for every channel used - using hex instead of blobs in sqlite - reordering columns and renaming them wherever useful The added data isn't available in past events. We decide to simply rename the older tables and start fresh: previous data will not be available from the API, but can still be queried directly in SQL if necessary.
We improve the `relayed` and `relayed_trampoline` payment tables in the `AuditDb` by: - unifying the way we handle multi-part payments, by having one row per payment, whether incoming or outgoing, and reconciling based on type and direction when listing events - adding `remote_node_id` for every channel used - using hex instead of blobs in sqlite - renaming columns for better clarify The added data isn't available in past events. We decide to simply rename the older tables and start fresh: previous data will not be available from the API, but can still be queried directly in SQL if necessary.
We change the relay statistics we compute, to more accurately reflect nodes that are good or bad peers and inform liquidity allocation decisions.
We now track liquidity purchases as well in transaction events, as their fees must be taken into account when evaluating the overall fees earned from a node. We also add the number of on-chain transactions made with a given peer to its relay stats.
pm47
reviewed
Mar 26, 2026
Member
There was a problem hiding this comment.
How about adding some temporary code preventing boot, to allow adjusting the DB model before the next release? (already in place)
This would probably be the right time to do optimizations such as indexing based on sequences instead of long identifiers. I'm not sure the trade-off is worth it, in terms of additonal JOINs at read time.
eclair-core/src/main/scala/fr/acinq/eclair/db/sqlite/SqliteAuditDb.scala
Show resolved
Hide resolved
pm47
approved these changes
Mar 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR, we make a collection of changes to all tables of the
AuditDb. The main change is that it is way more relevant to track statistics for peer nodes instead of individual channels, so we want to track thenode_idassociated with each event. We also track more data about transactions we make and relayed payments, to more easily score peers based on the fees we're earning vs the fees we're paying (for on-chain transactions or for liquidity purchases).Each commit should be reviewed individually, as they are scoped to different tables of the DB and will make the review easier.
Note that we cannot migrate existing data (since it is lacking information that we now need), so we simply rename older tables and create new ones. Past data will thus not be accessible through the APIs, but can be queried directly using SQL if necessary. It should be fine, since liquidity decisions should be taken based on relatively recent data (a few weeks) in order to be economically relevant (nodes that generated fees months ago but aren't generating any new fees since then are probably not good peers).
Fixes #2977