Skip to content

fix(wallet-toolbox): StorageIdb empty array filters match everything, like StorageKnex - #792

Merged
ty-everett merged 18 commits into
bsv-blockchain:mainfrom
shruggr:fix/idb-empty-array-filters
Oct 10, 2026
Merged

ty-everett merged 18 commits into
bsv-blockchain:mainfrom
shruggr:fix/idb-empty-array-filters

Conversation

@shruggr

@shruggr shruggr commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Program and scope

IndexedDB now treats empty certificate certifier/type arrays, output-tag ID arrays and transaction status arrays as omitted optional filters, matching Knex. Nonempty filters, partial predicates and user ownership are still enforced.

Includes the preceding merged David fixes. The main integration changed ancestry only; the qualified candidate tree is identical. Exact reviewed head: c59d6d812d6e704fe47401ee564d6382bf614f96.

Impact

  • Node and browser/client wallet implementations affected; included in the held 2.14.7 candidate. The mobile output is byte-identical to the preceding candidate because it does not include StorageIdb.
  • Persistence/user-boundary review: no schema migration, unrestricted cross-user query or authorization change. Empty optional filters do not remove partial/user predicates.
  • Changelog, affected package pages, release notes/migration ledger and generated API documentation updated. No public API or wire migration.

Verification

  • Node 24.18.0 / pnpm 10.33.2, frozen lifecycle-disabled install and audited tooling.
  • pnpm health:check, pnpm lint, pnpm format:check, pnpm build, pnpm typecheck, documentation build and governed wallet property suite: passed.
  • Focused query suite: 15 tests passed, each executed against IndexedDB and SQLite. Covers all four empty-array filters, omitted/nonempty filters, partial predicates and user ownership. The previous IndexedDB implementation fails the certificate, transaction and output-tag regressions.
  • Complete four-worker wallet coverage: 292 suites / 3,171 tests passed, 7 existing skips. Client coverage: 22 tests passed. Existing governed skips/timeouts unchanged.
  • Local patch gate: 95.83% (23/24 changed line/branch points) against the unchanged 90% requirement.
  • Node/client pack:check contracts passed: conditional/wildcard exports, declarations, source maps, publint, strict type resolution and clean consumers.
  • Real browser passed with governed budgets/composition: Vite 2,228,645 raw / 533,697 gzip / 406,978 Brotli bytes, 132 modules; esbuild 1,737,554 / 483,044 / 379,163 bytes.
  • git diff --check and owned documentation generation passed. No generated test report committed.
  • Complete incremental diff self-reviewed for correctness, security, storage/consumer compatibility, package artifacts, dependencies, documentation and operations.
  • All repository-owned exact-head checks passed, including coverage, consumers, platforms, conformance and aggregate/merge gates. Hosted patch gate: 95.83% (23/24). The advisory external codecov/patch report is 75.00% and failed its reporting target; it is explicitly informational in the existing CI workflow. No gate, skip, suppression or baseline was changed.

Hosted CI 38096012347, conformance 38096012335 and CodeQL 38096012339 passed; existing scope skips were validated by the merge gate. Sonar zero-findings and both Socket checks passed. The advisory coverage discrepancy is recorded above.

Security and dependencies

  • No new dependency, override, exception, dismissal, skipped test or workflow-permission change.
  • User/partial-predicate and restrictive nonempty-filter tests retained across both backends.
  • Exact-head CodeQL has no new alert; 0 open merge-ref alerts.
  • Exact-head Sonar has zero new findings and zero unreviewed hotspots.

Release and operations

  • Included in the unpublished 2.14.7 Node/browser candidate with explicit no-migration guidance.
  • No npm publication from this PR/workstation and no repository settings change.
  • Empty optional arrays now behave like omission; applications retain their user and partial predicates. The protected npm OIDC workflow will publish the final qualified three-package wallet candidate after the scoped David-fix sequence.

Completion evidence

  • Final-head required checks and review conversations complete; 0 threads.
  • Normal protected merge verified: 8da361f, 2026-10-10T23:53:42Z. Remote main tree exactly equals the qualified candidate.

Original implementation explanation

Fixes #791.

StorageIdb rejected every row when an array filter was present but empty: certifiers and types in filterCertificates, tagIds in filterOutputTagMaps, status in filterTransactions. StorageKnex and the other StorageIdb filters skip an empty array. listCertificates always passes certifiers: [], so a lookup by type returned nothing on IndexedDB.

Adds the .length > 0 guard to the four checks and two assertions to test/storage/idb/find.test.ts.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R53QzSzGv4LkfmxBd4URiD

shruggr and others added 6 commits October 3, 2026 14:24
…ry JSON carries it

binaryRequests encodes only Uint8Array values, but the toolbox built its
largest storage payloads as number[]: StorageClientBase.createAction set
inputBEEF from pruned.toBinary(), and the signer's internalizeAction passed
ab.toBinaryAtomic(txid). They travelled as JSON number arrays even with
binary negotiated, and a BEEF over ~100 KB exceeded the auth middleware's
100,000-value request limit.

Use the SDK's toUint8Array() / toUint8ArrayAtomic() for createAction and
no-send-expiry inputBEEF and internalizeAction AtomicBEEF, and return
listOutputs BEEF, createAction sourceTransaction and competingBeef as
Uint8Array from Knex storage. BEEF and AtomicBEEF already allow Uint8Array,
and the server's byte validation accepts it. Legacy peers that have not
negotiated binary still receive number arrays.
BRC-177 Prefunding step 2 requires the anchor to be large enough both to
fund the protected action and to be reclaimed, net of a reclaim fee. The
anchor was sized for the protected action only, so value-neutral actions
(e.g. a 1-sat token transfer) were refused because the anchor could not
pay for its own reclaim.

Size the anchor as the larger of the protected action's need and the
reclaim floor (reclaim fee plus minimum reclaim output). The protected
action carries no change, so the funding planner now runs with a zero
change cap and pays any surplus as fee. generateChangeSdk accepts
maxChangeOutputs: 0 to mean "create no change".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R53QzSzGv4LkfmxBd4URiD
Replace the "maxChangeOutputs: 0 means no change" overload with a
dedicated GenerateChangeSdkParams.surplusToFee flag. When set, the
planner creates no change output and pays any surplus beyond the
required fee as fee. maxChangeOutputs keeps its original rule (minimum
1, default 8); the BRC-177 protected branch goes back to a cap of 1 and
sets surplusToFee. The protected funding plan is the only setter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R53QzSzGv4LkfmxBd4URiD
Compute the change-output cap as before, then zero it when surplusToFee
is set. No behaviour change (SonarCloud typescript:S3358).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R53QzSzGv4LkfmxBd4URiD
… JSON

StorageServer converts the byte columns of findOutputsAuth and
findProvenTxReqs results when the client negotiated binary JSON, as it
already does for getSyncChunk. Clients convert them back to number[] in
validateEntities.

Restore competingBeef to number[] in createAction's WERR_REVIEW_ACTIONS.
Thrown errors are serialized with stringifyBRC100, so it never reached
the binary codec.
filterCertificates (certifiers, types), filterOutputTagMaps (tagIds) and
filterTransactions (status) rejected every row when the array was empty.
StorageKnex and the other StorageIdb filters skip an empty array. Guard
the four checks with `.length > 0`.

Fixes bsv-blockchain#791

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R53QzSzGv4LkfmxBd4URiD
@shruggr
shruggr marked this pull request as ready for review October 4, 2026 08:00
shruggr added a commit to shruggr/ts-stack that referenced this pull request Oct 4, 2026
Adds bsv-blockchain#792 (StorageIdb empty array filters) to the interim repack. Not for
upstream.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R53QzSzGv4LkfmxBd4URiD
shruggr added a commit to yours-org/yours-wallet that referenced this pull request Oct 4, 2026
StorageIdb empty array filters (bsv-blockchain/ts-stack#792), so
listCertificates by type works on local storage.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R53QzSzGv4LkfmxBd4URiD
@ty-everett
ty-everett marked this pull request as draft October 10, 2026 23:42
@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Oct 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...es/wallet/wallet-toolbox/src/storage/StorageIdb.ts 75.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@ty-everett
ty-everett marked this pull request as ready for review October 10, 2026 23:53
@ty-everett
ty-everett merged commit 8da361f into bsv-blockchain:main Oct 10, 2026
43 of 44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

wallet-toolbox: StorageIdb rejects every row when an array filter is empty; listCertificates by type returns nothing on IndexedDB

2 participants