Agree with core on the LedgerCloseMeta V3 layout #1150

Parent: #1147.

The core team has drafted LedgerCloseMeta V3 (stellar/stellar-xdr branch lcm_v3, commit 8cf594ce). In the draft, each transaction gets one record that holds its envelope, result, base fee and meta, in apply order. The tx set and the outer transaction hash are gone.

This issue is where RPC and core agree on the final layout. #1151 builds on it. One record per transaction in ledger meta has the background.

What RPC needs from the layout

1. Enough of the tx set to check signatures and base fees

RPC v2 wants to check ledgers from untrusted sources, like the data lake, against trusted ledger hashes. The ledger header commits to the transactions through two hashes:

• txSetResultHash covers the results and every envelope field except the signatures.
• txSetHash is the only one that covers the signatures and the base fees.

The draft keeps every envelope and each transaction's base fee. But it drops how the tx set groups them: its phases, its components and their base fees, and the Soroban stages and clusters. Without that grouping, a reader can't rebuild the tx set and check txSetHash, so a changed signature or base fee goes unnoticed.

The history archive's tx sets would fill the gap, but that means a second download, a second source to depend on and a second hashing pass. The core sketch notes that the same gap stops replay-debug-meta from replaying V3 meta.

2. A size in front of each record

In the draft, ingestion can only find where a record ends by parsing it. So it reads the records one after another, on one core. A 4-byte length in front of each record would let ingestion jump between records and spread the work across cores.

Hot ingestion per ledger, at sac-9000 density:

3. Little before the first record

For a ledger over 4 MiB, #1151 stores the bytes before the first record next to a small index, and every getTransaction lookup on that ledger reads them. The draft's header and record count are fine there. Anything new, such as the tx set's grouping, fits better after the records.

4. No outer transaction hash

RPC recomputes each transaction's hash from its envelope, as the draft expects. Keeping the hash would make a compressed ledger about 11% bigger (2.83 MB instead of 2.54 MB). It would save 1.3 ms at p50 and nothing at p99.

How these were measured

On a c6id.8xlarge with local NVMe, ingesting the first 1,000 ledgers of sac-9000 as fast as it can, with two runs per layout. The ledgers still had diagnostic events. This is the only run that compares the layouts side by side. The sizes come from five of the same ledgers, compressed with zstd.

Suggested approach

A starting point for the discussion, from the prototype work:

• Keep the draft's record, with a 4-byte length in front of each one. Wrapping each record in opaque<> gives the same bytes.
• Put the tx set's grouping after the records, with each envelope replaced by its record's position. A reader puts the envelopes back and checks txSetHash. A prototype converter turned over a thousand pubnet ledgers into this form and back to identical bytes, and both hashes checked out. The grouping adds 0.32% to a pubnet ledger's compressed size.
• Say in a comment what a missing baseFee means.

Prototype readers in go-stellar-sdk and RPC read the draft with and without these changes. On 50 pubnet ledgers converted from V2, they return the same transactions, events and fees as V2. They are on local branches, not pushed yet.

An option to discuss: meta as a stream of records, header last

In the draft, as today, the ledger header comes first. Core can only compute the header at the end of close, so it can't write any meta until close is done. Instead, core could write each part of a ledger's meta as soon as it is final:

1. an open record, when the ledger is externalized;
2. one record per transaction, as in the draft;
3. one record per applied upgrade;
4. an eviction record, when keys were evicted;
5. a close record with the header.

Each record starts with its length. RPC could then ingest while core finishes closing, and core could write meta on a separate thread.

The numbers suggest it would help. With 9,000 transactions per ledger, core's close took 1,032 ms: 622 ms applying transactions and about 140 ms writing meta. That leaves roughly 410 ms of close after apply. Ingesting a ledger with a size per record took 82 to 83 ms at p99 in the table above, so ingestion would fit in that window (estimated). The length in front of each record also gives the size per record, and the tx set's grouping would be one more record type.

How these were measured: core's times are means from apply-load on the same c6id.8xlarge, over 150 ledgers of 9,000 synthetic Soroban transactions (core build 4eb83337), writing meta to /dev/null with diagnostic events on. Apply-load closes ledgers as fast as it can, not every 900 ms. The ingestion times are the ones in the table above.

typedef opaque LedgerMetaRecordBody<>;   // the XDR encoding of one body value

struct LedgerMetaRecord
{
    int type;                   // int, not enum: readers skip unknown types
    LedgerMetaRecordBody body;  // by the length word and lose nothing
};

const LEDGER_META_OPEN = 0;      // body: LedgerOpenMeta
const LEDGER_META_TX = 1;        // body: TransactionResultMetaV2, unchanged
const LEDGER_META_UPGRADE = 2;   // body: UpgradeEntryMetaV2
const LEDGER_META_EVICTION = 3;  // body: EvictionMeta, omitted when empty
const LEDGER_META_CLOSE = 4;     // body: LedgerCompletionMeta

struct LedgerOpenMeta
{
    ExtensionPoint ext;
    uint32    ledgerSeq;
    Hash      previousLedgerHash;
    TimePoint closeTime;
    uint32    ledgerVersion;     // the version transactions apply under
    uint32    txCount;
};

struct UpgradeEntryMetaV2
{
    ExtensionPoint ext;
    LedgerUpgrade upgrade;
    LedgerEntryChanges changes;
};

struct EvictionMeta
{
    ExtensionPoint ext;
    LedgerKey evictedKeys<>;
};

struct LedgerCompletionMeta
{
    ExtensionPoint ext;
    LedgerHeader ledgerHeader;   // its hash is derivable, so not carried
    uint64 totalByteSizeOfLiveSorobanState;
    int64  sorobanFeeWrite1KB;
};

Links

• Draft XDR: stellar/stellar-xdr lcm_v3 (8cf594ce).
• Core sketch: dmkozh/stellar-core lcm_v3 (cf2cb3c2). It emits V3 from protocol 29 when USE_LEDGER_CLOSE_META_V3 is set.
• Depends on this issue: RPC v2: use LCM V3 so getTransaction reads only part of a large ledger #1151.

🤖 Generated with Claude Code