erf.ibseb: distribute the ray-cast column map and the building labelling instead of replicating them

[asalmgren]

Follow-up to the surface energy balance on immersed-boundary building faces (#3960), recording work deliberately deferred from that PR so it is not lost when the PR merges. Sister issue to #4019.

Where it stands

IBFaceSet::build() needs a map of column tops for the ray cast and a connected-component labelling of the buildings. Both started as full-domain host arrays replicated on every rank; #3960 narrowed them to the bounding box of the built columns and then collapsed them to a single int array holding the index of the highest solid cell of each column, so the standing cost is now 4 bytes per built column per rank on the device plus one all-reduce, down from 24 bytes across three Real arrays plus a mask and two all-reduces.

What is left is the scaling, not the constant: for buildings spread across a city the bounding box is the whole domain, so every rank still holds a map of every column. At 4096 x 4096 that is ~67 MB per rank resident plus an all-reduce of the same width, multiplied by ranks per node.

The part that makes an exact fix possible

The ray walks are already bounded in height, and this is the fact worth not rediscovering:

• a rising ray stops once it passes the tallest column (ray_blocked, Source/ImmersedBoundarySEB/ERF_IBSEBSolar.H:42; ray_hit, :55);
• a descending ray stops at the ground (ray_hit, :50).

So a ray's horizontal reach never exceeds (z_max - z_ground) / tan(elevation). A halo sized from the shallowest elevation actually sampled is therefore exact, not an approximation — for shadows that elevation is the sun's, known every step; for view fractions it is set by view_n_el. Only the grazing tail, where the required halo would exceed the domain, needs a cap and a documented far-field rule, and at those elevations the direct beam is already negligible.

Concretely: put the column map on a k-collapsed MultiFab over grids[lev] with ceil((z_max - z_ground) / (dx * tan(elev_min))) ghost columns, FillBoundary it, and hand the ray cast a local Array4 instead of a replicated raw pointer.

The per-face lookups are not in the way: h_bid (the building id of the face's adjacent solid column) and h_hb (that column's top) both read a column within one cell of a face this rank owns, so a one-cell halo covers them.

The blocker

The building labelling is a global connected-components problem, currently a serial depth-first flood fill over the replicated mask. Distributing it means a local flood fill plus iterative boundary label exchange with union-find, converging in O(domain diameter / box width) rounds. Standard, but fiddly, and the reason this was kept out of #3960.

Two related items in the same family

1. Replicated BoxArray metadata. m_state_ba and m_xfer_ba are global BoxArrays built with AllGatherBoxes and held on every rank. Surface energy balance on immersed-boundary building faces (erf.ibseb), with immersed-boundary fixes for the forcing and the MRF/YSUNew schemes #3960 dropped the checkpoint block size from 8 to 4, which bought a 2x cell reduction but multiplied the box count by about 4 (8 boxes to 32 on the FaceStorage level). That is a good trade at these sizes, but at city scale it pushes the wrong way on replicated metadata; the block size should be revisited together with whatever distributes the column map.
2. Two full host passes over the blanking in build(). The bounding-box pass and the face-detection pass each walk every cell of every FAB with their own HostFab copy (Source/ImmersedBoundarySEB/ERF_IBFaceSet.cpp, the two MFIter loops near the top of build()). Pure initialisation cost and trivially fusable into one pass.

Verification to keep

Exec/CanonicalTests/SEB/Shortwave checks the shadow flag of every face against an independent ray cast in the checker (0/2616 mismatches, 738 and 644 shadowed at the two zenith angles), and Exec/CanonicalTests/SEB/Longwave checks the view fractions against an independent hemisphere sampling (0/2616 mismatches). Any change here should be held to those, on one and on several ranks.