Skip to content

feat(metrics): export per-level SST file counts to Prometheus - #189

Closed
philippem wants to merge 1 commit into
Blockstream:new-indexfrom
philippem:add-rocksdb-levels-exporter
Closed

philippem wants to merge 1 commit into
Blockstream:new-indexfrom
philippem:add-rocksdb-levels-exporter

Conversation

@philippem

Copy link
Copy Markdown
Collaborator

Adds rocksdb_num_files_at_level{db, level} gauge, polled every 5s via the existing stats exporter loop. Covers levels 0-6 (RocksDB default). Useful for monitoring L0 file accumulation during initial sync.

@philippem
philippem requested a review from RCasatta March 4, 2026 14:48
@EddieHouston

Copy link
Copy Markdown
Collaborator

lgtm

shesek
shesek previously approved these changes Mar 5, 2026

@shesek shesek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, two minor nits but I'd also be fine with it as-is.

Comment thread src/new_index/db.rs Outdated
let prop = format!("rocksdb.num-files-at-level{}", level);
let level_str = level.to_string();
if let Ok(Some(value)) = db_arc_levels.property_value(&prop) {
if let Ok(v) = value.parse::<f64>() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches the existing update_gauge() behavior, but its kind of strange that invalid values are silently discarded. Maybe log a warnings if parse::<f64> fails?

Comment thread src/new_index/db.rs Outdated
for level in 0..7u32 {
let prop = format!("rocksdb.num-files-at-level{}", level);
let level_str = level.to_string();
if let Ok(Some(value)) = db_arc_levels.property_value(&prop) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe handle the per-level gauges through update_gauge() too? It can take an additional_labels: &[&str] argument and concat() it with the default db name label.

This would reduce some duplication and make the extra db_arc_levels/label_levels clones unnecessary.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check again. claude made these changes, so may not be reasonable.

Comment thread src/new_index/db.rs Outdated
update_gauge(&db_metrics.block_cache_capacity, "rocksdb.block-cache-capacity");
update_gauge(&db_metrics.block_cache_usage, "rocksdb.block-cache-usage");
update_gauge(&db_metrics.block_cache_pinned_usage, "rocksdb.block-cache-pinned-usage");
for level in 0..7u32 {

@shesek shesek Mar 5, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could use a ROCKSDB_NUM_LEVELS const (with ..=)

Adds rocksdb_num_files_at_level{db, level} gauge, polled every 5s
via the existing stats exporter loop. Covers levels 0-6 (RocksDB default).
Useful for monitoring L0 file accumulation during initial sync.
@philippem
philippem force-pushed the add-rocksdb-levels-exporter branch from 9724742 to 14eefe8 Compare March 5, 2026 21:07
@philippem
philippem requested a review from shesek March 6, 2026 14:58
@philippem

Copy link
Copy Markdown
Collaborator Author

closing. will include simpler changes in another branch.

@philippem philippem closed this Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants