Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 37 additions & 14 deletions docs/conflicts.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,24 +33,47 @@ conflicts with the following logic:
Note that on a conflicting transaction, the delta-apply column will be
correctly calculated and applied.

To make a column a conflict-free delta-apply column, ensuring that the value
replicated is the delta of the committed changes (the old value plus or
minus any new value) to a given record, you need to apply the following
settings to the column: `log_old_value=true,
delta_apply_function=spock.delta_apply`. For example:
To make a column a conflict-free delta-apply column, ensuring that the
value replicated is the delta of the committed changes (the old value
plus or minus any new value) to a given record, attach a SECURITY LABEL
to the column via the `spock.delta_apply()` helper:

```sql
ALTER TABLE pgbench_accounts ALTER COLUMN abalance
SET (log_old_value=true, delta_apply_function=spock.delta_apply);
ALTER TABLE pgbench_branches ALTER COLUMN bbalance
SET (log_old_value=true, delta_apply_function=spock.delta_apply);
ALTER TABLE pgbench_tellers ALTER COLUMN tbalance
SET (log_old_value=true, delta_apply_function=spock.delta_apply);
SELECT spock.delta_apply('pgbench_accounts'::regclass, 'abalance');
SELECT spock.delta_apply('pgbench_branches'::regclass, 'bbalance');
SELECT spock.delta_apply('pgbench_tellers'::regclass, 'tbalance');
```

As a special safety-valve feature, if you ever need to re-set a
`log_old_value` column you can temporarily alter the column to
`log_old_value` is `false`.
To remove the marker, pass `to_drop => true`:

```sql
SELECT spock.delta_apply('pgbench_accounts'::regclass, 'abalance', to_drop => true);
```

Under the hood, `spock.delta_apply()` writes a row into `pg_seclabel`
with `provider = 'spock'` and `label = 'spock.delta_apply'`. The
binary-upgrade compatibility shim that translates legacy spock 5.x
reloptions during `pg_upgrade` writes the same canonical label, so
operators can audit the catalog uniformly:

```sql
SELECT * FROM pg_seclabel
WHERE provider = 'spock' AND label = 'spock.delta_apply';
Comment thread
danolivo marked this conversation as resolved.
```

### Upgrading from spock 5.x

Spock 5.x recorded the same intent as a pair of per-attribute reloptions
(`log_old_value=true, delta_apply_function=spock.delta_apply`). During
`pg_upgrade`, the binary-upgrade compatibility shim translates those
reloptions into the new `SECURITY LABEL` form automatically. Look for
`NOTICE: spock: rewrote ALTER TABLE … ALTER COLUMN … legacy options to
SECURITY LABEL` lines in `pg_upgrade.log` to audit the translation.
After the upgrade, `SELECT * FROM pg_seclabel WHERE provider = 'spock'
AND label = 'spock.delta_apply'` is the authoritative list of delta-apply
columns. See
[Binary-upgrade compatibility shims](internals-doc/binary-upgrade-compat-shim.md)
for the full design.

### Conflict Configuration Options

Expand Down
3 changes: 1 addition & 2 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,8 +165,7 @@ ALTER TABLE table_name ALTER COLUMN column_name SET NOT NULL;
Then configure the Delta-Apply column with the following command:

```sql
ALTER TABLE table_name ALTER COLUMN column_name
SET (log_old_value=true, delta_apply_function=spock.delta_apply);
SELECT spock.delta_apply('table_name'::regclass, 'column_name');
```

## Configuration Issues
Expand Down
3 changes: 3 additions & 0 deletions include/spock.h
Original file line number Diff line number Diff line change
Expand Up @@ -133,4 +133,7 @@ VALGRIND_PRINTF(const char *format,...)

extern void spock_init_failover_slot(void);

/* lives in src/spock_bucompat_5x.c -- used only under IsBinaryUpgrade */
extern void register_spock_compat_5x(void);

#endif /* SPOCK_H */
32 changes: 32 additions & 0 deletions patches/15/pg15-000-spock-patchset-version.diff
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Spock core-patchset: export patchset version via miscadmin.h and globals.c.

Adds SPOCK_CORE_PATCHSET_VERSION (compile-time constant) and
SpockCorePatchsetVersion (runtime global) to the standard places
PostgreSQL already uses for server-wide state. No new files.

--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -498,4 +498,11 @@
/* in executor/nodeHash.c */
extern size_t get_hash_memory_limit(void);

+/*
+ * Spock core-patchset identity. Bump the version when the patchset
+ * changes in a way visible to the extension binary.
+ */
+#define SPOCK_CORE_PATCHSET_VERSION 1
+extern PGDLLIMPORT int SpockCorePatchsetVersion;
+
#endif /* MISCADMIN_H */
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -114,6 +114,9 @@
bool IsBinaryUpgrade = false;
bool IsBackgroundWorker = false;

+/* Spock core-patchset identity. */
+int SpockCorePatchsetVersion = SPOCK_CORE_PATCHSET_VERSION;
+
bool ExitOnAnyError = false;

int DateStyle = USE_ISO_DATES;
32 changes: 32 additions & 0 deletions patches/16/pg16-000-spock-patchset-version.diff
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Spock core-patchset: export patchset version via miscadmin.h and globals.c.

Adds SPOCK_CORE_PATCHSET_VERSION (compile-time constant) and
SpockCorePatchsetVersion (runtime global) to the standard places
PostgreSQL already uses for server-wide state. No new files.

--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -510,4 +510,11 @@
/* in executor/nodeHash.c */
extern size_t get_hash_memory_limit(void);

+/*
+ * Spock core-patchset identity. Bump the version when the patchset
+ * changes in a way visible to the extension binary.
+ */
+#define SPOCK_CORE_PATCHSET_VERSION 1
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is smart idea in general. It does replace one patch with another though (but smaller). I am thinking one day we could have a spock-lite with reduced functionality that would work with plain Postgres. I will think about this some more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mason-sharp ,
It is trivial to guard our patches and the corresponding Spock machinery. That enables Spock to be used with upstream Postgres. But for production use, at least one patch (logical clock) is a requirement. So, for the safety and predictability, we should have a mechanism to detect patched postgres.

+extern PGDLLIMPORT int SpockCorePatchsetVersion;
+
#endif /* MISCADMIN_H */
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -114,6 +114,9 @@
bool IsBinaryUpgrade = false;
bool IsBackgroundWorker = false;

+/* Spock core-patchset identity. */
+int SpockCorePatchsetVersion = SPOCK_CORE_PATCHSET_VERSION;
+
bool ExitOnAnyError = false;

int DateStyle = USE_ISO_DATES;
32 changes: 32 additions & 0 deletions patches/17/pg17-000-spock-patchset-version.diff
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Spock core-patchset: export patchset version via miscadmin.h and globals.c.

Adds SPOCK_CORE_PATCHSET_VERSION (compile-time constant) and
SpockCorePatchsetVersion (runtime global) to the standard places
PostgreSQL already uses for server-wide state. No new files.

--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -525,4 +525,11 @@
/* in executor/nodeHash.c */
extern size_t get_hash_memory_limit(void);

+/*
+ * Spock core-patchset identity. Bump the version when the patchset
+ * changes in a way visible to the extension binary.
+ */
+#define SPOCK_CORE_PATCHSET_VERSION 1
+extern PGDLLIMPORT int SpockCorePatchsetVersion;
+
#endif /* MISCADMIN_H */
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -117,6 +117,9 @@
bool IsUnderPostmaster = false;
bool IsBinaryUpgrade = false;

+/* Spock core-patchset identity. */
+int SpockCorePatchsetVersion = SPOCK_CORE_PATCHSET_VERSION;
+
bool ExitOnAnyError = false;

int DateStyle = USE_ISO_DATES;
32 changes: 32 additions & 0 deletions patches/18/pg18-000-spock-patchset-version.diff
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Spock core-patchset: export patchset version via miscadmin.h and globals.c.

Adds SPOCK_CORE_PATCHSET_VERSION (compile-time constant) and
SpockCorePatchsetVersion (runtime global) to the standard places
PostgreSQL already uses for server-wide state. No new files.

--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -540,4 +540,11 @@
/* in executor/nodeHash.c */
extern size_t get_hash_memory_limit(void);

+/*
+ * Spock core-patchset identity. Bump the version when the patchset
+ * changes in a way visible to the extension binary.
+ */
+#define SPOCK_CORE_PATCHSET_VERSION 1
+extern PGDLLIMPORT int SpockCorePatchsetVersion;
+
#endif /* MISCADMIN_H */
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -120,6 +120,9 @@
bool IsUnderPostmaster = false;
bool IsBinaryUpgrade = false;

+/* Spock core-patchset identity. */
+int SpockCorePatchsetVersion = SPOCK_CORE_PATCHSET_VERSION;
+
bool ExitOnAnyError = false;

int DateStyle = USE_ISO_DATES;
43 changes: 40 additions & 3 deletions src/spock.c
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
#include "pgstat.h"

#include "spock_apply.h"

#if PG_VERSION_NUM >= 180000
#include "spock_conflict_stat.h"
#endif
Expand Down Expand Up @@ -979,7 +980,9 @@ spock_object_relabel(const ObjectAddress *object, const char *seclabel)

extoid = get_extension_oid(EXTENSION_NAME, true);
if (!OidIsValid(extoid))
elog(ERROR, "spock extension is not created yet");
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("spock extension is not created yet")));

/*
* Check: classId must be pg_class, objectId should an existing table and
Expand All @@ -1004,6 +1007,20 @@ _PG_init(void)
if (!process_shared_preload_libraries_in_progress)
elog(ERROR, "spock is not in shared_preload_libraries");

/*
* Runtime patchset check: if the server binary was built from a
* different patchset generation than this extension, refuse to
* start. An unpatched server never reaches here -- the dynamic
* linker fails on the missing SpockCorePatchsetVersion symbol.
*/
if (SpockCorePatchsetVersion != SPOCK_CORE_PATCHSET_VERSION)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("spock core patchset version mismatch: "
"server has v%d, extension expects v%d",
SpockCorePatchsetVersion,
SPOCK_CORE_PATCHSET_VERSION)));

DefineCustomEnumVariable("spock.conflict_resolution",
gettext_noop("Sets method used for conflict resolution for resolvable conflicts."),
NULL,
Expand Down Expand Up @@ -1293,6 +1310,23 @@ _PG_init(void)
NULL,
NULL);

/*
* Register the spock security label provider BEFORE the
* IsBinaryUpgrade early-return. The binary-upgrade compatibility
* shims synthesise SECURITY LABEL statements during pg_restore;
* those statements need the provider to be registered, otherwise
* ExecSecLabelStmt fails with "security label provider 'spock' is
* not loaded".
*/
register_label_provider(SPOCK_SECLABEL_PROVIDER, spock_object_relabel);

/*
* Install the 5.x -> 6.x binary-upgrade compatibility ProcessUtility
* hook. Self-gates on IsBinaryUpgrade -- nothing happens outside
* pg_upgrade.
*/
register_spock_compat_5x();

if (IsBinaryUpgrade)
return;

Expand Down Expand Up @@ -1326,8 +1360,11 @@ _PG_init(void)
prev_emit_log_hook = emit_log_hook;
emit_log_hook = log_message_filter;

/* Security label provider hook */
register_label_provider(SPOCK_SECLABEL_PROVIDER, spock_object_relabel);
/*
* Note: the security label provider is registered earlier in
* _PG_init, before the IsBinaryUpgrade early-return, so it is
* available during pg_upgrade for the binary-upgrade compat shims.
*/

#if PG_VERSION_NUM >= 180000
/* Spock replication conflict statistics */
Expand Down
2 changes: 0 additions & 2 deletions src/spock_apply_heap.c
Original file line number Diff line number Diff line change
Expand Up @@ -510,7 +510,6 @@ physatt_in_attmap(SpockRelation *rel, int attid)
return false;
}


static void
build_delta_tuple(SpockRelation *rel, SpockTupleData *oldtup,
SpockTupleData *newtup,
Expand Down Expand Up @@ -569,7 +568,6 @@ build_delta_tuple(SpockRelation *rel, SpockTupleData *oldtup,
}
}


/**
* This is called when there is a potential conflict that may be able to be resolved
* according to resolution rules
Expand Down
Loading
Loading