Skip to content

Generate a native ORCA plan for a replicated CTE in scalar subqueries#384

Open
Alena0704 wants to merge 1 commit into
OPENGPDB_STABLEfrom
bugfix-cross-slice-shared-scan-cte
Open

Generate a native ORCA plan for a replicated CTE in scalar subqueries#384
Alena0704 wants to merge 1 commit into
OPENGPDB_STABLEfrom
bugfix-cross-slice-shared-scan-cte

Conversation

@Alena0704

@Alena0704 Alena0704 commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

When a CTE over a DISTRIBUTED REPLICATED table is referenced from several
scalar subqueries, ORCA puts the SharedScan Producer and Consumer on
different slices. That cross-slice SharedScan used to hang.

Until now we just avoided the hang: FHasCrossSliceReplicatedCTEConsumer
detected this shape before DXL translation and fell back to the Postgres
planner. This change lets ORCA handle the scalar-subquery case natively,
so no fallback is needed: the replicated CTE is materialized once and
shared by all references inside ORCA's own plan.

The fix is in apply_shareinput_xslice (src/backend/cdb/cdbmutate.c). When
a cross-slice Consumer is found inside a SubPlan and the CTE source is a
replicated table, the Consumer gets its own local copy of the Producer's
subtree (Materialize + base Scan) with a fresh share_id, marked
SHARE_MATERIAL and placed in the Consumer's slice. The table is
replicated, so every segment already has the full data and the local copy
is equivalent -- this removes the cross-slice coordination. Other
Consumers of the same CTE in the same slice reuse this copy (tracked by
(orig_share_id, motId) -> new_share_id), so the CTE is materialized once
and read by all references. cleanup_orphaned_producers then drops the
original Producers that are no longer used. The reuse map and consumer
counts live in new ApplyShareInputContext fields in
src/include/nodes/relation.h.

The pre-DXL fallback check (CUtils::FHasCrossSliceReplicatedCTEConsumer)
was too broad -- it fired for every cross-slice replicated CTE Consumer.
Narrow it to the join case only: a CTE Consumer under a
duplicate-hazard / broadcast Motion (greengage 51fe92e), which still
can't be handled natively. The scalar-subquery case no longer matches
here and reaches the new materialization path instead.

@Alena0704

Copy link
Copy Markdown
Contributor Author

I decided to create about new pull request because we fixed that problem before with fall back walker and now I am developing approach making orca builds the query plan for cross slice scalar subqueries without breaking the semantics of checking cross slice pattern for other components with replicated tables (the regression test proved it).

FYI, the previous PR to fix this topic and discussion #370

@Alena0704 Alena0704 force-pushed the bugfix-cross-slice-shared-scan-cte branch from 82f0b49 to 26c2063 Compare June 1, 2026 04:34
@Alena0704 Alena0704 changed the title Fix cross-slice SharedScan hang for replicated CTE references Generate a native ORCA plan for a replicated CTE in scalar subqueries Jun 1, 2026
@Alena0704 Alena0704 changed the title Generate a native ORCA plan for a replicated CTE in scalar subqueries Generate a native orca plan for a replicated CTE in scalar subqueries Jun 1, 2026
@Alena0704 Alena0704 force-pushed the bugfix-cross-slice-shared-scan-cte branch from 26c2063 to 8eb4869 Compare June 1, 2026 04:36
@Alena0704 Alena0704 marked this pull request as draft June 1, 2026 04:36
@Alena0704 Alena0704 force-pushed the bugfix-cross-slice-shared-scan-cte branch 2 times, most recently from ac442fe to 92486db Compare June 3, 2026 05:32
When a CTE over a DISTRIBUTED REPLICATED table is referenced from several
scalar subqueries, ORCA puts the SharedScan Producer and Consumer on
different slices. That cross-slice SharedScan used to hang.

Until now we just avoided the hang: FHasCrossSliceReplicatedCTEConsumer
detected this shape before DXL translation and fell back to the Postgres
planner. This change lets ORCA handle the scalar-subquery case natively,
so no fallback is needed: the replicated CTE is materialized once and
shared by all references inside ORCA's own plan.

The fix is in apply_shareinput_xslice (src/backend/cdb/cdbmutate.c). When
a cross-slice Consumer is found inside a SubPlan and the CTE source is a
replicated table, the Consumer gets its own local copy of the Producer's
subtree (Materialize + base Scan) with a fresh share_id, marked
SHARE_MATERIAL and placed in the Consumer's slice. The table is
replicated, so every segment already has the full data and the local copy
is equivalent -- this removes the cross-slice coordination. Other
Consumers of the same CTE in the same slice reuse this copy (tracked by
(orig_share_id, motId) -> new_share_id), so the CTE is materialized once
and read by all references. cleanup_orphaned_producers then drops the
original Producers that are no longer used. The reuse map and consumer
counts live in new ApplyShareInputContext fields in
src/include/nodes/relation.h.

The pre-DXL fallback check (CUtils::FHasCrossSliceReplicatedCTEConsumer)
was too broad -- it fired for every cross-slice replicated CTE Consumer.
Narrow it to the join case only: a CTE Consumer under a
duplicate-hazard / broadcast Motion (greengage 51fe92e), which still
can't be handled natively. The scalar-subquery case no longer matches
here and reaches the new materialization path instead.
@Alena0704 Alena0704 changed the title Generate a native orca plan for a replicated CTE in scalar subqueries Generate a native ORCA plan for a replicated CTE in scalar subqueries Jun 3, 2026
@Alena0704 Alena0704 marked this pull request as ready for review June 3, 2026 08:30
@Alena0704 Alena0704 force-pushed the bugfix-cross-slice-shared-scan-cte branch from 92486db to 433247e Compare June 3, 2026 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant