handbook: narrow versioning design doc scope

handbook/engineering/design-documents/versioning.mdx

@@ -0,0 +1,335 @@
+---
+title: "Versioning"
+description: "How products and benefits get versions for organizations using declarative config."
+---
+
+<Info>
+
+**Created**: 2026-10-06
+
+**Last Updated**: 2026-10-20
+
+**Author**: Petru Rares Sincraian
+</Info>
+
+<Note>
+
+**Revision 2026-10-20.** Scope narrowed after review:
+
+- Versioning applies only to organizations using declarative config, and only to new organizations in v0.
+- Full freeze: no change reaches existing customers until they are explicitly moved.
+- Migrations (DSL, strategies, selectors) move to a separate design doc. v0 moves customers with a bulk subscription update and a new one-time order endpoint.
+- Meters are not versioned in v0.
+- Catalog membership is derived. `is_current` is not stored.
+- The draft lives in the API, config is its only writer, and the dashboard is read-only (review, publish, discard).
+- New sections: [References](#references) and [API surface](#api-surface).
+</Note>
+
+## Summary
+
+We want to let agents start making changes on the Polar side. Agents will write config files to create products and benefits. But we want agents to do that safely, for merchants and for customers.
+
+To achieve that, we propose to follow some design principles:
+
+1. Every change goes into a **draft catalog**. A member reviews it in the dashboard and publishes it, then it becomes the **live catalog**. Every new checkout goes through the live catalog.
+2. Every change in a product or benefit gets a **new row with a new ID**. Old rows don't change, and existing orders and subscriptions keep pointing to them.
+3. **Full freeze.** No change reaches existing customers until the merchant explicitly moves them to the new version.
+
+Versioning comes with declarative config (config as code). Organizations that use the dashboard only keep today's behavior. In v0, only new organizations can use declarative config.
+
+## Current state
+
+Today, changes behave differently depending on what changes:
+
+| Change | Effect on existing customers |
+|---|---|
+| Product price | Old price is archived. Existing subscriptions keep it. Merchants move them with the subscription update endpoint |
+| Product name, description, media | Changed in place, for everyone |
+| Product recurring interval | Locked, unless all prices are legacy |
+| Benefit properties (e.g. license key activation limit) | Changed in place. Existing grants are updated through `benefit.update` |
+| Meter credit units | Not applied to existing grants, but every subscription gets the new amount at its next renewal (the cycle reads the current benefit) |
+| Benefits added to or removed from a product | Granted or revoked for subscriptions, one-time orders, and seats |
+| Meter filter or aggregation | Locked once the meter has billed events. Other fields change in place |
+
+Prices are the only thing that is frozen per customer today. Everything else follows the product.
+
+## Goals
+
+- Provide a unified approach to updating billing entities for organizations using declarative config.
+- Make billing changes in a safe and predictable way: nothing changes for an existing customer without an explicit action.
+- Allow merchants to move existing customers to the latest version.
+
+## Out of scope
+
+- Customer notifications related to billing changes.
+- Migrations as declared config (selectors, strategies, scheduling, cancel). These get their own design doc, built on the move endpoints described here.
+- Meter versioning, and moving credit balances between meters.
+- Existing organizations opting in.
+- Editing catalog entities from the dashboard for organizations using declarative config.
+
+## Options
+
+| Option | We pick it? |
+|---|---|
+| **Catalogs**: immutable rows grouped by a Catalog entity | Yes |
+| **Stable ids with version tables**: the product id never changes and versions live in another table | No. Has better compatibility, but we need two models for every resource. We cover compatibility with `external_id` and publish-time re-pointing instead (see [References](#references)) |
+| **One catalog version per customer** | No. Purchases made at different times can't live side by side |
+| **New logical_id attribute** | No. `external_id` already plays this role |
+| **No versioning**: the merchant writes `pro_v2` for external id | No way |
+
+## Solution
+
+There are only 4 concepts that make all of this possible:
+
+1. **Catalog.** It contains the full list of products and benefits at a point in time. It has a per-organization incremental `version` and a state: `draft`, `live`, or `archived`. Every organization has at most one draft and one live catalog.
+2. **Copy-on-write** for products and benefits. Every change to a product or benefit creates a new row in the draft catalog. Once the catalog is published, those rows are locked.
+3. **`external_id` is the stable name.** It is required for every product, benefit, and meter. A row is unique by `(organization_id, external_id, version)`, where `version` is the catalog where the row was created.
+4. **Catalog membership is the source of truth.** `catalog_products` and `catalog_benefits` list the rows in each catalog. "Current" means "in the live catalog", and it is always derived from membership. We don't store an `is_current` flag. If performance requires one later, it is a cache enforced by a partial unique index on `(organization_id, external_id) WHERE is_current`.
+
+### What we store
+
+ACME sells a Pro product at $20 and a Starter at $10 in catalog 1. Then the merchant changes Pro to $25 and publishes catalog 2. Starter didn't change, so its row is reused in catalog 2.
+
+`catalogs`
+
+| id | version | status |
+|---|---|---|
+| `cat_1` | 1 | `archived` |
+| `cat_2` | 2 | `live` |
+
+`products`
+
+| id | external_id | version | price |
+|---|---|---|---|
+| `prod_A` | `pro` | 1 | $20 |
+| `prod_B` | `pro` | 2 | $25 |
+| `prod_C` | `starter` | 1 | $10 |
+
+`catalog_products`
+
+| catalog | external_id | product |
+|---|---|---|
+| 1 | `pro` | `prod_A` |
+| 1 | `starter` | `prod_C` |
+| 2 | `pro` | `prod_B` |
+| 2 | `starter` | `prod_C` |
+
+`subscriptions`
+
+| id | product_id | pays |
+|---|---|---|
+| `sub_1` (bought in catalog 1) | `prod_A` | $20, until moved |
+| `sub_2` (bought in catalog 2) | `prod_B` | $25 |
+
+Things to notice:
+
+- `prod_A` still exists and still costs $20. Reading `prod_A` always returns the $20 definition. An old id never redirects to the new one.
+- `prod_C` was created in catalog 1 and reused in catalog 2. It keeps `version = 1`, and it is in the live catalog.
+- A purchase reads its own row directly. We never search "which version does this customer have".
+
+### What creates a new row
+
+| Change at publication | Result |
+|---|---|
+| Any change to a product, including its name | New product row |
+| Any change to a benefit | New benefit row, plus new rows for the products granting it |
+| Benefits added to or removed from a product | New product row |
+| Same definitions applied again | No new catalog |
+
+A benefit change cascades to products because a product's benefit links are part of its definition. Grants are reconciled from `product.benefits` on subscription creation, reactivation, past-due recovery, product change, seat claims, and one-time order updates. If we changed `prod_A`'s links in place, existing customers would pick up the new benefit the next time one of those paths ran, and the freeze would break for some customers and not others.
+
+The cost is that editing a benefit shared by many products gives all of them new IDs, even though the merchant didn't touch them. The plan and the review screen show why each dependent product changed:
+
+```text
+~ pro       prod_A → prod_B · benefit discord-role changed
+```
+
+Meters are not versioned in v0. In organizations using declarative config, `plan` rejects changes to a meter's filter or aggregation once it has billed events, as today. To count usage differently, the merchant creates a new meter with a new `external_id`.
+
+### Main flows
+
+#### Making a change
+
+Config is the only way to change the catalog. The dashboard is read-only for catalog entities.
+
+```tsx
+import { defineConfig, usd } from "@polar-sh/polar"
+
+export default defineConfig({
+  meters: () => ({}),
+  products: ({ product, fixed }) => ({
+    pro: product("Pro").prices(fixed().amount(usd(25))).recurring("monthly"), // was usd(20)
+  }),
+})
+```
+
+```text
+~ catalog   1 → 2
+~ pro       prod_A → prod_B · price $20.00 → $25.00 · 168 subscriptions stay on prod_A
+```
+
+There are two ways to apply:
+
+- `apply` creates the draft and publishes it in one transaction.
+- `apply --draft` uploads the draft. A member reviews the diff and impact in the dashboard, then publishes or discards it. This is the flow for changes proposed by agents.
+
+Rules:
+
+- The draft lives in the API. Config is its only writer.
+- `apply --draft` replaces any existing draft. There are no conflicts, because nothing else edits the draft.
+- Draft rows cannot be sold, and are not visible through the regular API endpoints.
+- Every draft is complete and valid: it is the full next catalog, including dependent products.
+- Publishing is atomic.
+- There is no revert. To go back, the merchant applies the old definition, which publishes a new catalog.
+
+Editing catalog entities from the dashboard is future work. It would make the draft and the config file diverge, and solving that needs generating config back from server state into the merchant's own files.
+
+#### New sales
+
+Every new checkout uses the live catalog:
+
+```sql
+SELECT p.* FROM products p
+JOIN catalog_products cp ON cp.product_id = p.id
+JOIN catalogs c ON c.id = cp.catalog_id
+WHERE c.organization_id = :organization_id
+  AND c.status = 'live'
+  AND p.external_id = 'pro';
+```
+
+Checkout sessions created before a publish keep the row they were created with.
+
+#### Removing an entity
+
+Removing an entity means it is not in the next catalog. Existing purchases keep their rows, as with archiving today.
+
+```text
+Catalog 2: pro → prod_B, starter → prod_C
+Catalog 3:               starter → prod_C
+```
+
+#### Moving customers
+
+Publishing never moves existing customers. Moving them is a separate, explicit action, with two endpoints:
+
+- **Subscriptions.** A bulk subscription update: a loop over today's subscription update, which changes `product_id` to the target row with the requested proration behavior (`next_period`, or `immediate` with `prorate` or `invoice`).
+- **One-time orders.** A new endpoint that moves an order's benefits to a target product row. The order itself doesn't change: amounts, invoice, and product on the receipt stay as purchased.
+
+Both endpoints share how grants are moved. Today, a product change revokes benefits that are not on the new product and grants new ones. With versioning, every benefit edit is a new row, so that would revoke and reissue license keys, remove and re-add Discord roles and GitHub access, and debit then re-credit meter credits mid-cycle.
+
+Instead, grants are **re-pointed by benefit `external_id`**: if the old and new product both have a benefit with the same `external_id`, we keep the grant, point it to the new benefit row, and call the strategy's existing `update(grant, previous_properties)`. That is the path in-place benefit edits use today. Only benefits that were added or removed are granted or revoked.
+
+Example: the License key benefit allows 3 activations (`ben_1`). Catalog 2 allows 5 (`ben_2`):
+
+| | Before move | After move |
+|---|---|---|
+| Grant | `grant_1` → `ben_1` | `grant_1` → `ben_2` |
+| License key | `ABC-123`, 2 devices active | `ABC-123`, 2 devices active |
+| Activation limit | 3 | 5 |
+
+Meter credits are the exception, because they are consumed. Like today's subscription update, moving a subscription settles the usage on the old version, revokes the old credit grant, and grants the new credits prorated. Extra usage is billed at the new price.
+
+We considered blocking moves to a stricter benefit (e.g. 5 devices active, new limit of 3). We don't block in v0: today a merchant can already lower a limit in place for everyone, and a move is the same change made deliberately per customer.
+
+Declarative migrations (selectors, scheduling, cancel) are a separate design doc built on these two endpoints.
+
+#### What changes for merchants
+
+| | Today | With declarative config |
+|---|---|---|
+| Change a price | Old prices are archived. Existing subscriptions keep them | New product row in a new catalog. Existing subscriptions keep the old row |
+| Change a product name or description | Changes for all customers | New product row. Existing customers keep the old row |
+| Change a benefit | Changes for all customers | New benefit row and new rows for products granting it. Existing grants keep the old definition |
+| Change meter credit units | Applies to every subscription at its next renewal | Applies only to new purchases and moved customers |
+| Change a meter | Locked once billing starts | Same. Create a new meter instead |
+| Product id after a change | Same id | New id. `external_id` stays the same |
+| Edit from the dashboard | Live right away | Not possible. Review and publish drafts from config |
+| Past one-time buyers | Get benefit changes automatically | Get them only when moved |
+| Move existing customers | Subscription update, one by one | Bulk subscription update, and an endpoint for one-time orders |
+
+### References
+
+References made by merchants follow the live catalog. References made by purchases stay pinned to their row.
+
+| Reference | Follows |
+|---|---|
+| Subscriptions, orders, benefit grants, open checkout sessions | Pinned row |
+| Checkout links (`checkout_link_products`) | Live catalog. Re-pointed at publish |
+| Discounts (`discount_products`) | Live catalog. Re-pointed at publish |
+| Checkout creation through the API | Accepts product `external_id`, resolved to the live row |
+
+Re-pointing happens in the publish transaction: for every `external_id` that got a new row, we update `checkout_link_products.product_id` and `discount_products.product_id` from the old row to the new one. There is no lookup at read time. Pinning a checkout link to a specific version is not supported in v0.
+
+Discounts already applied to existing subscriptions or checkouts don't change.
+
+Merchant code that stores raw product IDs (hardcoded checkout products, entitlement checks such as `sub.product_id == PRO_ID`) goes stale after a publish. The docs tell merchants using declarative config to key on `external_id`.
+
+### API surface
+
+The API hides versions as much as possible:
+
+- List endpoints (`GET /products`, `GET /benefits`) return only the live catalog. There is no parameter to list other versions.
+- `GET /products/{id}` returns any published row, so a subscription's `product_id` can always be resolved.
+- Draft rows are only reachable through the catalog draft endpoints used by the review screen. They never show up in regular endpoints or in checkout.
+- Publishing fires `product.created` and `benefit.created` for each new row, and `product.updated` and `benefit.updated` for rows that leave the live catalog, as archiving does today. Consumers that key on `external_id` see a consistent picture.
+
+## Open questions
+
+1. **Meters and credits (M3).** How do we version a meter? The direction is independent balances per meter row, with the move endpoint carrying the remaining balance from the old meter to the new one. A balance shared across meter versions is ruled out: there is no clean answer to which version's filter consumes it.
+2. **Adoption.** How do existing organizations opt in? A `polar init` that assigns `external_id`s and generates a config where the first `plan` shows no changes is the likely direction.
+3. **Dashboard editing.** Can members edit drafts from the dashboard, and how do those edits get back into the config file?
+
+## Security considerations
+
+- **Data in and out:** config from merchants and agents, and plan reports with counts and amounts.
+- **Who can access it:** the existing read and write scopes. Publishing a draft and moving customers check permissions again. Moving subscriptions needs `subscriptions:write`. Moving one-time orders needs its own write permission.
+- **Data we store:** draft and published rows, and catalogs. No new personal data.
+- **Risks:** an agent moves many customers to higher prices. Mitigations: agents propose drafts that a member publishes, publishing never moves customers, and moves are explicit and idempotent.
+
+## Appendix: Milestones
+
+**POC goal.** Prove the core loop on one internal organization:
+
+1. Change Pro from $20 to $25 in config.
+2. `plan` shows `prod_A → prod_B`. `apply` publishes catalog 2.
+3. A new checkout sells `prod_B` at $25.
+4. The old subscription stays on `prod_A` at $20.
+5. A bulk subscription update moves it to `prod_B` at its next renewal.
+
+**POC scope (M1)**
+
+| In | Out (later milestones) |
+|---|---|
+| Products with fixed prices. Benefit links are copied as they are | Benefit edits and cascade, grant re-pointing, one-time moves (M2) |
+| `catalogs`, `catalog_products`, row `version`, derived live membership | Meters and credits (M3) |
+| Config `plan` / `apply`. Apply creates the draft and publishes it in one transaction | `apply --draft`, review, diff, publish, history (M4) |
+| Checkout and product lists read the live catalog | Checkout link and discount re-pointing, `external_id` on checkout, webhooks (M5) |
+| Bulk subscription update with `next_period` only | `immediate` moves (M5) |
+| Dashboard and API product edits are blocked for the flagged org | |
+
+**Milestones**
+
+| Milestone | Goal | Depends on | Done when |
+|---|---|---|---|
+| M0 Design Doc | Features and architecture agreed | — | Design doc reviewed |
+| M1 POC | The scope above | M0 | Demo passes |
+| M2 Benefits | Benefit versioning, cascade to products, grant re-pointing, one-time order moves | M1 | A license key and an OAuth benefit move without being reissued |
+| M3 Meters and credits | Resolve the open question, then build meter versioning and balance carry-over | M1 | Separate design decision |
+| M4 Dashboard review | `apply --draft`, read-only draft review, diff, publish, discard, history | M1 | An agent proposes a draft and a member publishes it |
+| M5 References and API | Publish-time re-pointing, `external_id` on checkout, webhooks, `immediate` moves | M1 | Checkout links and discounts sell the live row after a publish |
+| M6 Hardening | Analytics and the main flows end to end | M2, M4, M5 | Main flows work end to end |
+| M7 Release | Declarative config for new organizations, with docs | M6 | First external organization created with declarative config |
+
+## Appendix: How others do it
+
+| Provider | When the price changes | Existing customers | How to move them |
+|---|---|---|---|
+| Stripe | A new price is created, the old one is deactivated | Keep the old price | By hand, or with subscription schedules |
+| Orb | A new plan version is created | Stay on their version | Migrations, that can be scheduled and canceled |
+| Metronome | The rate card changes from a date | Stay on their rate card | By editing each contract |
+| Lago | The plan is edited in place | **Are repriced right away** | A new plan, and then a migration |
+| Chargebee | Only new subscriptions get the new price | Keep the old price | A bulk update tool |
+| Autumn | A new version is created, if the plan has customers | Stay on their version | Migrations from the CLI |
+| `void` branch (our prototype) | The whole config gets a new version | Stay on their version | Not possible |
+| **Polar today** | Old prices are archived, other fields change in place | Keep the old price, but get the other changes | Subscription update, one by one |
+| **This design** | A new catalog with new rows for changes | Keep their rows | Bulk subscription update and one-time order moves |