-
Notifications
You must be signed in to change notification settings - Fork 9
docs: add ADR for local-first reads #702
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,85 @@ | ||||||
| # Architecture Decision Record: Local-first reads | ||||||
|
|
||||||
| Title: Local-first reads - return format for reads that render without waiting on the network | ||||||
|
|
||||||
| ## status | ||||||
|
|
||||||
| proposed | ||||||
|
|
||||||
| Updated on 05-08-2026 | ||||||
|
|
||||||
| ## contributors | ||||||
|
|
||||||
| - Main contributor(s): nogringo | ||||||
|
|
||||||
| - Reviewer(s): frnandu, 1leo | ||||||
|
|
||||||
| - Final decision made by: frnandu, 1leo, nogringo | ||||||
|
|
||||||
| ## Context and Problem Statement | ||||||
|
|
||||||
| Reads must render immediately from what is known locally, and refine when relays answer. The | ||||||
| current reads do neither: `getSingleNip51List(kind, forceRefresh:)` returns the cache and never | ||||||
| refreshes it, or skips the cache and blocks on the network. | ||||||
|
|
||||||
| A read must also never let one state degrade into another. "not known yet" and "cannot be read" | ||||||
| must not look like "does not exist", because that is what an app acts on to create the missing | ||||||
| data, and for a replaceable event that overwrites what was already there. | ||||||
|
|
||||||
| ## Main Proposal | ||||||
|
|
||||||
| ```dart | ||||||
| class NdkDataResponse<T> { | ||||||
| /// Emits a `cache` value first, then every newer `relays` value as it | ||||||
| /// arrives, even when it holds the same value as the cache. | ||||||
| /// Closes after EOSE or timeout. | ||||||
| final Stream<NdkValue<T>> stream; | ||||||
|
|
||||||
| /// The relay-confirmed value, so the last emitted one. | ||||||
| Future<T> get future; | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== repo files matching ADR/main candidates =="
git ls-files | rg '(^|/)(local-first-reads|user_relay_lists|.*ndk.*\.dart)$' | sed -n '1,200p'
echo
echo "== ADR target =="
if [ -f doc/library-development/ADRs/local-first-reads.md ]; then
nl -ba doc/library-development/ADRs/local-first-reads.md | sed -n '1,120p'
else
echo "ADR file not found"
fi
echo
echo "== user_relay_lists relevant lines =="
if [ -f packages/ndk/lib/domain_layer/usecases/user_relay_lists/user_relay_lists.dart ]; then
wc -l packages/ndk/lib/domain_layer/usecases/user_relay_lists/user_relay_lists.dart
nl -ba packages/ndk/lib/domain_layer/usecases/user_relay_lists/user_relay_lists.dart | sed -n '1,220p'
else
echo "user_relay_lists.dart not found"
fi
echo
echo "== Search for NdkValue future/stream definitions and replace/by version =="
rg -n "NdkValue|final Stream<NdkValue|Future<T> get future|createdAt|kind: NdkKind\.kind10013|kind10013|replace|latest|relays|relays values|newer|last emitted" -S .Repository: relaystr/ndk Length of output: 12652 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== ADR target with cat -A line numbers =="
if [ -f doc/library-development/ADRs/local-first-reads.md ]; then
cat -n doc/library-development/ADRs/local-first-reads.md | sed -n '1,120p'
else
echo "doc/library-development/ADRs/local-first-reads.md not found"
fi
echo
echo "== user_relay_lists.dart relevant lines =="
if [ -f packages/ndk/lib/domain_layer/usecases/user_relay_lists/user_relay_lists.dart ]; then
wc -l packages/ndk/lib/domain_layer/usecases/user_relay_lists/user_relay_lists.dart
cat -n packages/ndk/lib/domain_layer/usecases/user_relay_lists/user_relay_lists.dart | sed -n '1,240p'
else
echo "packages/ndk/lib/domain_layer/usecases/user_relay_lists/user_relay_lists.dart not found"
fi
echo
echo "== Search relevant identifiers =="
rg -n "NdkValue|final Stream<NdkValue|Future<T> get future|createdAt|kind10013|kind1221|kind30078|latest|latest_by|sortedBy|sortBy|last emitted|newer|last relay|last event|Replace|replace" -S . | sed -n '1,240p'Repository: relaystr/ndk Length of output: 39897 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== Locate replacement/selection helpers =="
rg -n "replaceable|latestStateOnly|dedupe|deduplicate|sort\\(.*createdAt|shouldReplace|isReplaceableKind|isParameterizedReplaceable" packages/ndk/lib packages/ndk/test -S | sed -n '1,240p'
echo
echo "== Candidate implementation files likely containing read response/event selection =="
git ls-files packages/ndk/lib | rg 'requests|cache_read|read|local|nip01|event_utils|relay|cache_manager|user_relay_list' | sed -n '1,200p'
echo
echo "== Read implementation slices around selected candidates =="
for f in $(git ls-files packages/ndk/lib | rg 'requests|cache_read|read|local|event_utils|nip01_utils|user_relay_lists' | sed -n '1,50p'); do
echo "--- $f ($(wc -l < "$f") lines) ---"
cat -n "$f" | sed -n '1,260p'
doneRepository: relaystr/ndk Length of output: 50368 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== Replaceable winner selection implementations =="
for f in \
packages/ndk/lib/shared/nips/nip01/nip01_event.dart \
packages/ndk/lib/data_layer/entities/event_cache_records.dart \
packages/ndk/lib/data_layer/cache_manager/ndk_extensions.dart \
packages/ndk/lib/data_layer/cache_manager/mem_cache_manager.dart \
packages/ndk/lib/data_layer/cache_manager/sembast_cache_manager.dart \
packages/ndk/lib/domain_layer/usecases/users/users.dart \
packages/ndk/lib/domain_layer/usecases/metadata/metadata.dart \
packages/ndk/lib/domain_layer/usecases/lists/lists.dart
do
if [ -f "$f" ]; then
echo "--- $f ($(wc -l < "$f") lines) ---"
rg -n "created_at|createdAt|id|replaceableWinners|latestStateOnly|_shouldReplace|isReplaceableKind|_isParameterized|visible.winners|sort|first" "$f" -A 6 -B 4 | sed -n '1,220p'
fi
done
echo
echo "== Events model selection helpers =="
if [ -f packages/ndk/lib/shared/nips/nip01/nip01_event.dart ]; then
cat -n packages/ndk/lib/shared/nips/nip01/nip01_event.dart | sed -n '200,380p'
fi
if [ -f packages/ndk/lib/data_layer/cache_manager/ndk_extensions.dart ]; then
cat -n packages/ndk/lib/data_layer/cache_manager/ndk_extensions.dart | sed -n '1,220p'
fi
echo
echo "== Local first replaceable tests relevant slices =="
cat -n packages/ndk/test/usecases/local_first/local_first_test.dart | sed -n '220,370p'
cat -n packages/ndk/test/usecases/local_first/local_first_test.dart | sed -n '620,670p'Repository: relaystr/ndk Length of output: 15642 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== Event entity implementation slices =="
for f in \
packages/ndk/lib/shared/nips/nip01/nip01_event.dart \
packages/ndk/lib/data_layer/entities/nip_01_event_model.dart \
packages/ndk/lib/data_layer/entities/event_cache_records.dart
do
if [ -f "$f" ]; then
echo "--- $f ($(wc -l < "$f") lines) ---"
rg -n "class Nip01Event|getTags|getDtag|createdAt|id|operator <:|compareTo|sort|shouldReplace|replaceable|visibleWinners|_buildReplaceableConflictKey|_replaceableConflictKey" "$f" -A 12 -B 4 | sed -n '1,260p'
fi
done
echo
echo "== Replaceable comparison helper tests =="
cat -n packages/ndk/test/usecases/local_first/local_first_subscription_test.dart | sed -n '150,290p'
echo
echo "== Static verifier: selection in ADR vs implementation patterns =="
python3 - <<'PY'
from pathlib import Path
import re
adr = Path('doc/library-development/ADRs/local-first-reads.md').read_text()
checks = {
"future defined as last emitted": re.search(r"The relay-confirmed value, so the last emitted one\.", adr) is not None,
"newer values mentioned": re.search(r"every newer `relays` value", adr) is not None,
}
for name, ok in checks.items():
print(f"{name}: {ok}")
files = {
"nip01_event.dart": Path('packages/ndk/lib/shared/nips/nip01/nip01_event.dart'),
"event_cache_records.dart": Path('packages/ndk/lib/data_layer/entities/event_cache_records.dart'),
"local_first_subscription_test.dart": Path('packages/ndk/test/usecases/local_first/local_first_subscription_test.dart'),
}
for name, path in files.items():
text = path.read_text()
print(f"{name} has operator '<': {bool(re.search(r'operator\s+<\s*\(', text, re.S))}")
print(f"{name} has createdAt/id comparison: {bool(re.search(r'returned\s+event\.createdAt|return.*event\.createdAt|return.*event\.id|createdAt.*id|lower id|lower id|tie|id <=|created_at|_compare|compare', text, re.S))}")
PYRepository: relaystr/ndk Length of output: 6753 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== Files existence and sizes =="
wc -l packages/ndk/lib/shared/nips/nip01/nip01_event.dart packages/ndk/lib/data_layer/entities/event_cache_records.dart packages/ndk/lib/domain_layer/usecases/local_first/local_first.dart 2>/dev/null || true
echo
echo "== Nip01 event implementation slices =="
cat -n packages/ndk/lib/shared/nips/nip01/nip01_event.dart | sed -n '1,280p'
cat -n packages/ndk/lib/shared/nips/nip01/nip01_event.dart | sed -n '280,500p'
echo
echo "== Event cache records implementation slices =="
cat -n packages/ndk/lib/data_layer/entities/event_cache_records.dart | sed -n '450,580p'
cat -n packages/ndk/lib/data_layer/entities/event_cache_records.dart | sed -n '580,710p'
echo
echo "== local-first implementation slices =="
cat -n packages/ndk/lib/domain_layer/usecases/local_first/local_first.dart | sed -n '1,260p'
cat -n packages/ndk/lib/domain_layer/usecases/local_first/local_first.dart | sed -n '260,520p'Repository: relaystr/ndk Length of output: 315 Define relay-version ordering before completing
🤖 Prompt for AI Agents |
||||||
| } | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n -C 8 \
'\b(NdkDataResponse|future|addError|onError|timeout|EOSE|eose|decrypt|10013)\b' \
packages/ndk doc \
--glob '*.dart' \
--glob '*.md'Repository: relaystr/ndk Length of output: 50369 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== target ADR =="
cat -n doc/library-development/ADRs/local-first-reads.md | sed -n '1,130p'
echo "== focused symbol search =="
rg -n -C 6 \
'\b(NdkDataResponse|future\s*:|Stream<|addError\(|onError:|timeout\(|EOSE|eose|decrypt|10013)\b' \
packages/ndk --glob '*.dart'Repository: relaystr/ndk Length of output: 50369 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== local-first-reads.md =="
cat -n doc/library-development/ADRs/local-first-reads.md | sed -n '1,130p'
echo "== NdkDataResponse candidates =="
rg -n -C 10 'class NdkDataResponse|abstract class NdkDataResponse|interface NdkDataResponse|NdkDataResponse' packages/ndk --glob '*.dart'
echo "== focused lines/regions around addError/onError/timeout/eose in packages/ndk/lib =="
for f in $(rg -l 'addError\(|onError:|timeout\(|eose\(|EOSE|class .*DataResponse|abstract class .*Response|Future<T> get future' packages/ndk/lib --glob '*.dart' | head -80); do
echo "--- $f"
rg -n -C 4 'addError\(|onError:|timeout\(|eose\(|EOSE|Future<T> get future|class .*Response|abstract class .*Response' "$f" | sed -n '1,220p'
doneRepository: relaystr/ndk Length of output: 3518 Define failure completion for The ADR reports cache decryption failures and no-reach cases as 🤖 Prompt for AI Agents |
||||||
|
|
||||||
| class NdkValue<T> { | ||||||
| final T value; | ||||||
| final DataOrigin origin; | ||||||
| } | ||||||
|
Comment on lines
+51
to
+54
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift Make the cache-miss state representable for every declared At Lines 42-45 and 50-58, the contract requires a cache emission when no cache exists, but Constrain reads that can miss the cache to nullable Also applies to: 50-58 🤖 Prompt for AI Agents |
||||||
|
|
||||||
| enum DataOrigin { cache, relays } | ||||||
| ``` | ||||||
|
|
||||||
| A `cache` value is always emitted first, even when nothing is cached. | ||||||
|
|
||||||
| `stream` is backed by a `BehaviorSubject`, so `stream` and `future` can both be consumed and a | ||||||
| listener attached late still receives the latest value. | ||||||
|
|
||||||
| `NdkValue` may later carry `createdAt`, `receivedAt` and `hasPendingWrites`. Only metadata that | ||||||
| makes sense for every read belongs there, anything specific to one read belongs in `T`. | ||||||
|
|
||||||
| When the value can be absent, `T` is nullable and the origin disambiguates `null`: | ||||||
|
|
||||||
| | emission | meaning | | ||||||
| | --- | --- | | ||||||
| | `(value, cache)` | local value, not confirmed | | ||||||
| | `(null, cache)` | nothing local yet, still loading | | ||||||
| | `(value, relays)` | confirmed value | | ||||||
| | `(null, relays)` | confirmed: nothing exists | | ||||||
|
|
||||||
| Two cases are reported as stream errors, never as `null`, since `(null, relays)` is what an | ||||||
| app acts on to create the missing data: | ||||||
|
|
||||||
| - the value exists but cannot be read, for instance it cannot be decrypted | ||||||
| - no relay was reachable, so absence cannot be concluded | ||||||
|
|
||||||
| ## First use: private relay list (NIP-37 kind 10013) | ||||||
|
|
||||||
| ```dart | ||||||
| // on Lists, requires a logged in account, no pubkey parameter | ||||||
| NdkDataResponse<List<String>?> getPrivateUserRelays({Duration timeout}); | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
if ! command -v dart >/dev/null 2>&1; then
echo "dart is required for this check" >&2
exit 2
fi
tmp_dir="$(mktemp -d)"
trap 'rm -rf "$tmp_dir"' EXIT
cat >"$tmp_dir/bad.dart" <<'DART'
class NdkDataResponse<T> {}
abstract class Lists {
NdkDataResponse<List<String>?> getPrivateUserRelays({Duration timeout});
}
DART
if dart analyze "$tmp_dir/bad.dart"; then
echo "The invalid signature was accepted unexpectedly" >&2
exit 1
fi
cat >"$tmp_dir/good.dart" <<'DART'
class NdkDataResponse<T> {}
abstract class Lists {
NdkDataResponse<List<String>?> getPrivateUserRelays({Duration? timeout});
}
DART
dart analyze "$tmp_dir/good.dart"Repository: relaystr/ndk Length of output: 182 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "Dart availability:"
command -v dart || true
dart --version 2>/tmp/dart-version 2>/tmp/dart-version-err || true
cat /tmp/dart-version 2>/dev/null || true
cat /tmp/dart-version-err 2>/dev/null || true
echo
echo "Target file location/context:"
rg -n "getPrivateUserRelays|LocalFirstReads|NdkDataResponse" -g '!*.lock' .Repository: relaystr/ndk Length of output: 448 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "File excerpt:"
sed -n '1,100p' doc/library-development/ADRs/local-first-reads.md | cat -n
echo
echo "Top-level Dart-related files:"
fd -i 'analysis_options|pubspec\.yaml|README|CONTRIBUTING' . -t f | sed 's#^\./##' | head -80
echo
echo "Optional named parameter references in ADR/documentations:"
rg -n '\{[^;]*Duration [^?]+\}|\(.*Duration .*=\)|{[^{}]*required\s+\w+\s+\w+}' .Repository: relaystr/ndk Length of output: 5521 🌐 Web query:
💡 Result: In Dart, null safety enforces that variables cannot hold null unless explicitly marked as nullable with a question mark (?) [1][2]. This impacts how optional parameters are defined [1]. Named parameters are optional by default unless they are marked with the 'required' keyword [3][2]. Because optional parameters have an implicit default value of null, they must either be nullable or have a non-null default value provided [3][1][4]. If you define a named parameter with a non-nullable type and do not provide a default value, you must mark it as 'required' [1][4][2]. This tells the compiler that the caller must always provide the argument, making it non-optional [1]. Conversely, if you want a parameter to be truly optional without a default value, you must use a nullable type [1][4]. Summary of rules for optional/named parameters: - To make a parameter optional with an implicit null default: Use a nullable type (e.g., {int? x}) [3][4]. - To provide a non-null default value: Assign a compile-time constant (e.g., {int x = 1}) [3][5]. - To make a named parameter mandatory: Use the 'required' keyword (e.g., {required int x}) [1][2]. - A required named parameter cannot have a default value [6]. Citations:
Fix the null-safe Dart signature for At Use Proposed nullable signature-NdkDataResponse<List<String>?> getPrivateUserRelays({Duration timeout});
+NdkDataResponse<List<String>?> getPrivateUserRelays({Duration? timeout});📝 Committable suggestion
Suggested change
🤖 Prompt for AI AgentsSource: MCP tools |
||||||
| ``` | ||||||
|
|
||||||
| An empty list and no list are different answers: | ||||||
|
|
||||||
| | emission | meaning | | ||||||
| | --- | --- | | ||||||
| | `([], relays)` | the event exists and holds no relay | | ||||||
| | `(null, relays)` | no kind 10013 event exists | | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use an unambiguous update date.
At Line 9,
05-08-2026can mean May 8, 2026 or August 5, 2026. Use ISO 8601 format, such as2026-05-08or2026-08-05, based on the intended date.🤖 Prompt for AI Agents