Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,9 @@ Two models: upstream's configuration file, and a small store for state that has

Within `mempool-config.json`:

**Enforced** — rewritten to a fixed value whenever the package writes the file: the whole `DATABASE` section bar the password (the bundled MariaDB on loopback), `MEMPOOL.BACKEND`, the two `MEMPOOL.POOLS_JSON` URLs (the bundled snapshot on loopback), `CORE_RPC.COOKIE_PATH`, LND's certificate and macaroon paths, Core Lightning's socket path, and the `SYSLOG`, `MAXMIND`, `REDIS`, `REPLICATION`, and `STRATUM` sections, all held off.
**Enforced** — rewritten to a fixed value whenever the package writes the file: the whole `DATABASE` section bar the password (the bundled MariaDB on loopback), the two `MEMPOOL.POOLS_JSON` URLs (the bundled snapshot on loopback), `CORE_RPC.COOKIE_PATH`, LND's certificate and macaroon paths, Core Lightning's socket path, and the `SYSLOG`, `MAXMIND`, `REDIS`, `REPLICATION`, and `STRATUM` sections, all held off.

**Derived** — written from live addresses by init on every start: `CORE_RPC.HOST`/`PORT`, `ELECTRUM.HOST`/`PORT`, `LND.REST_API_URL`, and `SOCKS5PROXY.HOST`/`PORT`/`ENABLED`. Each is a reactive read of the dependency's address over the LXC bridge, so init re-runs and the backend restarts precisely when an address changes — an install, an uninstall, a port change — and not on a routine dependency update. **An absent dependency resolves to nothing and its key is omitted entirely** rather than written as a placeholder that would fail to connect; the write heals when the dependency returns.
**Derived** — written from live addresses by init on every start: `CORE_RPC.HOST`/`PORT`, `ELECTRUM.HOST`/`PORT`, `LND.REST_API_URL`, and `SOCKS5PROXY.HOST`/`PORT`/`ENABLED`. Each is a reactive read of the dependency's address over the LXC bridge, so init re-runs and the backend restarts precisely when an address changes — an install, an uninstall, a port change — and not on a routine dependency update. **An absent dependency resolves to nothing and its key is omitted entirely** rather than written as a placeholder that would fail to connect; the write heals when the dependency returns. `MEMPOOL.BACKEND` is written alongside `ELECTRUM` — `electrum` when an indexer address resolved, `none` otherwise.

**Seeded once** — `DATABASE.PASSWORD`, generated at install for the bundled database.

Expand All @@ -106,12 +106,12 @@ Two defaults depart from upstream's:

Bitcoin is required; the rest are chosen, and each choice changes which dependency exists.

| Dependency | Kind | Required | Health checks |
| ------------------------ | --------- | ------------------------------------ | ----------------------------------- |
| `bitcoind` | `running` | Always | `bitcoind`, `sync-progress` |
| `fulcrum` _or_ `electrs` | `running` | Whichever the indexer action selects | The indexer's ready and sync checks |
| `lnd` _or_ `c-lightning` | `running` | Only with Lightning enabled | The node's ready and sync checks |
| `tor` | `running` | Only with the Tor proxy enabled | `tor` |
| Dependency | Kind | Required | Health checks |
| ------------------------ | --------- | -------------------------------------------------------- | ----------------------------------- |
| `bitcoind` | `running` | Always | `bitcoind`, `sync-progress` |
| `fulcrum` _or_ `electrs` | `running` | Whichever the indexer action selects; none with **None** | The indexer's ready and sync checks |
| `lnd` _or_ `c-lightning` | `running` | Only with Lightning enabled | The node's ready and sync checks |
| `tor` | `running` | Only with the Tor proxy enabled | `tor` |

**Bitcoin must be archival with transaction indexing.** Mempool raises a `critical` task against Bitcoin asking for `prune=0` and `txindex=true`, and that task re-raises whenever the settings stop matching — it is not a one-time prompt. Address lookups additionally need an Electrum indexer, which is what the indexer selection is for.

Expand Down Expand Up @@ -141,11 +141,11 @@ Five actions, all user-facing.

### Select Indexer

Chooses which Electrum server backs address lookups — Fulcrum or Electrs.
Chooses which Electrum server backs address lookups — Fulcrum, Electrs, or **None**. Nothing is preselected; the form cannot be submitted until you pick one.

- **What it changes:** `indexer` in `store.json`, and through it the package's optional dependency. The address itself is resolved by init on the next start.
- **Cost:** seconds, then a restart.
- **Repeat safety:** idempotent; switching indexers loses nothing, since neither stores Mempool's data.
- **Repeat safety:** idempotent; switching indexers loses nothing, since neither stores Mempool's data. **None** turns address lookups off and drops the dependency.

### Enable Lightning

Expand Down Expand Up @@ -187,14 +187,14 @@ Deletes the backend's on-disk mempool and RBF cache.

## Tasks

Two tasks, both raised at install and both `critical`.
Two tasks, both `critical` — the service will not start while either is unresolved.

| Task | On | Raised when | Cleared when |
| ------------------------ | ------------ | -------------------------------------------------------- | ------------------------ |
| Select Indexer | This package | At install | The action runs |
| Select Indexer | This package | No indexer choice has been made | The action runs |
| Bitcoin's Auto-Configure | `bitcoind` | Bitcoin's settings are not `prune=0` with `txindex=true` | Bitcoin's settings match |

Both are `critical` because Mempool is not functional without them: no indexer means no address lookups, and a pruned or unindexed node cannot answer the historical queries the explorer is built on.
Both are `critical` because Mempool cannot do what it is installed for until they are answered: address lookups need an indexer, and a pruned or unindexed node cannot answer the historical queries the explorer is built on. Choosing **None** answers the first deliberately — Mempool then runs without address lookups.

The Bitcoin task is the one to know about. It is registered with `once: false`, so it comes back if Bitcoin's configuration later stops matching — for example after restoring Bitcoin from a backup taken with pruning on.

Expand Down Expand Up @@ -229,7 +229,7 @@ Only configuration is backed up — `sdk.Backups.ofVolumes('config', 'startos')`

1. **The database is not backed up.** A restore replays settings, not history; any enabled indexing backfills again from scratch.
2. **Bitcoin must be archival with transaction indexing**, enforced by a re-raising `critical` task rather than merely recommended.
3. **An Electrum indexer is required for address lookups**, and which one is a choice with no default until the install task is run.
3. **An Electrum indexer is required for address lookups**, and which one is yours to choose — nothing is preselected, and **None** is a valid answer that leaves lookups off.
4. **Indexing is memory-hungry and slow**, and its progress is invisible at the default log level.
5. **Telemetry and the maxmind, syslog, redis, replication, and stratum integrations are held off.**
6. **Acceleration services are off by default**, and the frontend's accelerator upstream is redirected at the backend rather than `mempool.space`. The external data servers upstream ships go over clearnet unless the Tor proxy action is enabled.
Expand Down Expand Up @@ -287,7 +287,7 @@ actions:
- tor-proxy
- clear-backend-cache # only-stopped
tasks:
- { action: select-indexer, severity: critical }
- { action: select-indexer, severity: critical } # until a choice is made
- { action: autoconfig, severity: critical } # on bitcoind; re-raises
health_checks:
- mariadb # hidden
Expand Down
6 changes: 3 additions & 3 deletions instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
## What you get on StartOS

- A **Web UI** interface — the Mempool block explorer, visualizer, fee estimator, and REST/WebSocket API in one site.
- Address lookup powered by a separate StartOS Electrum indexer (Fulcrum or Electrs).
- Address lookup powered by a separate StartOS Electrum indexer (Fulcrum or Electrs), or left off.
- An optional **Lightning** explorer that pulls network data from a local LND or Core Lightning node.
- A bundled MariaDB sidecar; you do not configure a database.
- Everything needed for a first start bundled in — Mempool never has to reach the internet to come up, and does not contact GitHub at all.
Expand All @@ -19,7 +19,7 @@ Mempool needs Bitcoin, an Electrum-style indexer, and (optionally) a Lightning n

1. Install **Bitcoin** if you don't have it. Mempool posts a critical task on Bitcoin requiring `txindex` enabled and pruning disabled, with an autoconfig action attached — accept it. The task re-appears any time those conditions stop being met.
2. Install **Fulcrum** (recommended) or **Electrs**.
3. After installing Mempool, run the **Select Indexer** task that appears for Mempool and pick **Fulcrum** or **Electrs**.
3. After installing Mempool, run the **Select Indexer** task that appears for Mempool and pick **Fulcrum**, **Electrs**, or **None**. Nothing is picked for you, and Mempool will not start until you answer.
4. Optionally install **LND** or **Core Lightning**, then run **Enable Lightning** and pick the backend you want feeding the Lightning tab.
5. Start Mempool. It will wait until Bitcoin, the selected indexer, and (if enabled) the Lightning backend are healthy and synced.

Expand All @@ -31,7 +31,7 @@ Open the **Web UI** interface to reach Mempool. The home page shows the live mem

### Actions

- **Select Indexer** — switch the Electrum backend between Fulcrum and Electrs. Mempool's dependency set updates accordingly.
- **Select Indexer** — choose the Electrum backend: Fulcrum, Electrs, or None. Mempool's dependency set updates accordingly, and None leaves address lookups off.
- **Enable Lightning** — choose LND, Core Lightning, or none for the Lightning tab's data source. The selected node is mounted read-only. On a low-memory box the form carries a warning: the Lightning network sync is memory-hungry, and turning it on alongside Bitcoin and your indexer can tip such a box into out-of-memory crashes. You can still proceed.
- **Indexing and Performance** — tune backend behavior on a single form:
- **Performance Profile** — pick **Low-CPU** (default; polls bitcoind every 8s, projects 4 future blocks), **Balanced** (4s / 6 blocks), or **Responsive** (2s / 8 blocks; highest CPU). The Mempool backend rebuilds its block projection on every poll, so this is the main lever for CPU usage on low-power devices.
Expand Down
12 changes: 4 additions & 8 deletions startos/actions/selectIndexer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ const indexerInputSpec = InputSpec.of({
values: {
fulcrum: i18n('Fulcrum (recommended)'),
electrs: i18n('Electrs'),
none: i18n('None — address lookups disabled'),
},
default: 'fulcrum',
// The spec accepts null; only the SDK 2.0.9 builder signature does not.
default: null as any,
}),
})

Expand All @@ -34,13 +36,7 @@ export const selectIndexer = sdk.Action.withInput(
indexerInputSpec,

// optionally pre-fill the input form
async ({ effects }) => {
const indexer = await selectedIndexer(effects)

return {
indexer: indexer as any,
}
},
async ({ effects }) => ({ indexer: await selectedIndexer(effects) }),

// the execution function. Record the choice in StartOS state; init/watchHosts
// resolves the indexer's LXC-bridge address into ELECTRUM.HOST/PORT next start.
Expand Down
2 changes: 1 addition & 1 deletion startos/file-models/mempool-config.json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {

const mempoolSection = z.object({
// enforced
BACKEND: z.literal('electrum').catch('electrum'),
BACKEND: z.enum(['none', 'electrum']).catch('none'),
// configurable
OFFICIAL: z.boolean().catch(false),
NETWORK: z.enum(['mainnet', 'testnet']).catch('mainnet'),
Expand Down
3 changes: 2 additions & 1 deletion startos/file-models/store.json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ import { sdk } from '../sdk'
// init/watchTorProxy. Keeping intent here is what lets the proxy be switched off
// when tor is uninstalled without forgetting that the user asked for it.
const shape = z.object({
indexer: z.enum(['electrs', 'fulcrum']).optional().catch(undefined),
// Absent means the choice is unmade; 'none' is the user declining an indexer.
indexer: z.enum(['electrs', 'fulcrum', 'none']).optional().catch(undefined),
torProxy: z.boolean().catch(false),
})

Expand Down
1 change: 1 addition & 0 deletions startos/i18n/dictionaries/default.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ const dict = {
"Route Mempool's outbound internet requests through Tor. Useful where an ISP or national firewall blocks the endpoints Mempool reads from, and where the server's own name resolution is unreliable — a SOCKS proxy resolves hostnames at the proxy rather than locally.": 70,
'This requires the Tor service to be installed and running.': 71,
'This server could not resolve an external hostname. Mempool is otherwise unaffected — Bitcoin, the Electrum indexer, and the database are reached by address — but fiat exchange rates will be unavailable. Set explicit DNS servers under System > DNS on your server, and check any VPN or StartTunnel gateway you have configured: a gateway supplies its own resolver, which stops working whenever the tunnel does.': 72,
'None — address lookups disabled': 73,
} as const

export type I18nKey = keyof typeof dict
Expand Down
4 changes: 4 additions & 0 deletions startos/i18n/dictionaries/translations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ export default {
70: 'Enruta por Tor las solicitudes salientes de Mempool a internet. Util cuando un proveedor de internet o un cortafuegos nacional bloquea los puntos de acceso que Mempool consulta, y cuando la resolucion de nombres del propio servidor no es fiable — un proxy SOCKS resuelve los nombres en el proxy y no localmente.',
71: 'Esto requiere que el servicio Tor este instalado y en ejecucion.',
72: 'Este servidor no pudo resolver un nombre de host externo. Por lo demas Mempool no se ve afectado — Bitcoin, el indexador Electrum y la base de datos se alcanzan por direccion — pero los tipos de cambio fiat no estaran disponibles. Configure servidores DNS explicitos en Sistema > DNS de su servidor, y revise cualquier pasarela VPN o StartTunnel que tenga configurada: una pasarela aporta su propio resolutor, que deja de funcionar en cuanto el tunel deja de funcionar.',
73: 'Ninguno — busquedas de direcciones desactivadas',
},
de_DE: {
0: 'Mempool wird gestartet',
Expand Down Expand Up @@ -140,6 +141,7 @@ export default {
70: 'Leitet die ausgehenden Internetanfragen von Mempool ueber Tor. Nuetzlich, wenn ein Internetanbieter oder eine nationale Firewall die von Mempool abgefragten Endpunkte blockiert, und wenn die Namensaufloesung des Servers selbst unzuverlaessig ist — ein SOCKS-Proxy loest Hostnamen am Proxy auf statt lokal.',
71: 'Dies setzt voraus, dass der Tor-Dienst installiert ist und laeuft.',
72: 'Dieser Server konnte einen externen Hostnamen nicht aufloesen. Mempool ist davon ansonsten nicht betroffen — Bitcoin, der Electrum-Indexer und die Datenbank werden ueber die Adresse erreicht — aber Fiat-Wechselkurse sind nicht verfuegbar. Tragen Sie unter System > DNS auf Ihrem Server ausdrueckliche DNS-Server ein und pruefen Sie ein etwaiges VPN- oder StartTunnel-Gateway: ein Gateway liefert seinen eigenen Resolver, der ausfaellt, sobald der Tunnel ausfaellt.',
73: 'Keiner — Adressabfragen deaktiviert',
},
pl_PL: {
0: 'Uruchamianie Mempool',
Expand Down Expand Up @@ -210,6 +212,7 @@ export default {
70: 'Kieruje wychodzace zapytania internetowe Mempool przez Tora. Przydatne, gdy dostawca internetu lub krajowa zapora blokuje punkty koncowe, z ktorych Mempool korzysta, oraz gdy rozwiazywanie nazw na samym serwerze jest zawodne — proxy SOCKS rozwiazuje nazwy hostow po stronie proxy, a nie lokalnie.',
71: 'Wymaga to zainstalowanej i dzialajacej uslugi Tor.',
72: 'Ten serwer nie mogl rozwiazac zewnetrznej nazwy hosta. Poza tym Mempool dziala bez zmian — Bitcoin, indekser Electrum i baza danych sa osiagane po adresie — ale kursy walut fiat beda niedostepne. Ustaw jawne serwery DNS w System > DNS na swoim serwerze i sprawdz skonfigurowana brame VPN lub StartTunnel: brama dostarcza wlasny resolver, ktory przestaje dzialac, gdy tunel przestaje dzialac.',
73: 'Brak — wyszukiwanie adresow wylaczone',
},
fr_FR: {
0: 'Demarrage de Mempool',
Expand Down Expand Up @@ -280,5 +283,6 @@ export default {
70: "Achemine via Tor les requetes internet sortantes de Mempool. Utile lorsqu'un fournisseur d'acces ou un pare-feu national bloque les points d'acces que Mempool interroge, et lorsque la resolution de noms du serveur lui-meme n'est pas fiable — un proxy SOCKS resout les noms d'hotes au niveau du proxy et non localement.",
71: "Cela necessite que le service Tor soit installe et en cours d'execution.",
72: "Ce serveur n'a pas pu resoudre un nom d'hote externe. Mempool n'en est par ailleurs pas affecte — Bitcoin, l'indexeur Electrum et la base de donnees sont joints par adresse — mais les taux de change fiat seront indisponibles. Renseignez des serveurs DNS explicites dans Systeme > DNS sur votre serveur, et verifiez toute passerelle VPN ou StartTunnel configuree : une passerelle fournit son propre resolveur, qui cesse de fonctionner des que le tunnel cesse de fonctionner.",
73: "Aucun — recherches d'adresses desactivees",
},
} satisfies Record<string, LangDict>
12 changes: 6 additions & 6 deletions startos/init/taskSelectIndexer.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import { selectIndexer } from '../actions/selectIndexer'
import { i18n } from '../i18n'
import { sdk } from '../sdk'
import { selectedIndexer } from '../utils'

export const taskSelectIndexer = sdk.setupOnInit(async (effects, kind) => {
if (kind === 'install') {
await sdk.action.createOwnTask(effects, selectIndexer, 'critical', {
reason: i18n('Select which Electrum server to use for address lookups'),
})
}
export const taskSelectIndexer = sdk.setupOnInit(async (effects) => {
if (await selectedIndexer(effects)) return
await sdk.action.createOwnTask(effects, selectIndexer, 'critical', {
reason: i18n('Select which Electrum server to use for address lookups'),
})
})
10 changes: 8 additions & 2 deletions startos/init/watchHosts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ import {
* port-change (and on an indexer/backend selection change), never on a routine
* dependency update. An absent dependency resolves to `null` and is omitted from
* the config entirely (no fake placeholder address is written); the write heals
* automatically when the dependency returns.
* automatically when the dependency returns. `MEMPOOL.BACKEND` follows whether an
* Electrum address resolved, so the backend is never left pointed at upstream's
* own `ELECTRUM` defaults.
*/
export const watchHosts = sdk.setupOnInit(async (effects, _) => {
const indexer = await selectedIndexer(effects)
Expand All @@ -28,13 +30,17 @@ export const watchHosts = sdk.setupOnInit(async (effects, _) => {
// gated on selection. A `null` means the dependency is absent — omit its
// section rather than write an unreachable address.
const bitcoind = await bitcoindRpcBridge(effects)
const electrum = indexer ? await electrumBridge(effects, indexer) : null
const electrum =
indexer && indexer !== 'none'
? await electrumBridge(effects, indexer)
: null
const lndRest = lndEnabled ? await lndRestBridge(effects) : null

await configJson.merge(
effects,
{
...(bitcoind && { CORE_RPC: hostPort(bitcoind) }),
MEMPOOL: { BACKEND: electrum ? 'electrum' : 'none' },
...(electrum && {
ELECTRUM: { ...hostPort(electrum), TLS_ENABLED: false },
}),
Expand Down
Loading