DRA: Sharing Affinity for DRA Consumable Capacity

[ashvindeodhar]

Enhancement Description

KEP-5075 (Consumable Capacity) enables device sharing via allowMultipleAllocations and capacity counters. However, it assumes all claims are fungible—any claim can share the device with any other claim. Real-world sharing is often constrained. For example: "Share this NIC across 16 pods, but only if the pods request the same subnet." Today, there's no native way to express this constraint.
I'd like to propose adding a sharingAffinity field to the DRA device spec that allows drivers to declare which claim config parameters must match for sharing to be allowed.

The current workaround is painful where drivers must-

1. Publish devices with capacity=1 as "placeholders"
2. Wait for the first claim to determine the affinity value (e.g., subnet)
3. Dynamically update device attributes and expand capacity
4. Contract back to placeholder when all claims release.

This adds complexity to drivers, introduces race conditions, and requires the scheduler to re-evaluate devices after attribute changes.
Adding sharing affinity to the device spec would allow us to express the constraint in a native way.
I believe it would also improve the scheduling throughput since pods with matching affinity values can be scheduled immediately.

Other potential use cases:

• GPUs: Share across pods in same tenant/namespace
• FPGAs: Share only if pods use same bitstream
• RDMA: Share only for same partition key

/assign @ashvindeodhar
/cc @johnbelamaric @pohly @sunya-ch

• One-line enhancement description (can be used as a release note): Enable DRA devices to constrain sharing based on matching claim config parameters (sharing affinity)
• Kubernetes Enhancement Proposal: TBD
• Discussion Link:
• PRs by stage and milestone:
  • Alpha - v1.xx
    • KEP (k/enhancements) update PR(s):
    • Code (k/k) update PR(s):
    • Docs (k/website) update PR(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[ashvindeodhar]

/sig scheduling
/wg device-management

[ashvindeodhar]

Attaching a doc with more details on current gap, issue with workaround approach and how proposed changes resolve those. dra-sharing-affinity-proposal.md

[pohly]

There's one conceptual problem with this proposal:

        opaque:
          driver: networking.example.com
          parameters:
            subnet: "subnet-X"                     # Affinity value

These config parameters that the scheduler is meant to match on are opaque to it. It doesn't know the schema of the embedded JSON, how it's versioned and how to decode it.

We would have to add non-opaque configuration parameters (quite some heavy lifting!) or define a generic schema that DRA driver and scheduler then can decode. I'm leaning towards the later. The implementation of such a schema would be similar to https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/dynamic-resource-allocation/api/metadata.

[ashvindeodhar]

Thanks @pohly, this is very helpful! In the KEP (#5987), I identified this opaque config limitation and proposed a separate structured affinityValues field alongside the opaque config to give the scheduler a readable path. But your suggestion of a generic structured configuration schema is a cleaner approach — it eliminates the duplication entirely and removes the need for a built-in admission controller to map between opaque config and affinity values (SharingAffinityMapping), significantly simplifying that part of the design.

If I understand correctly, the idea would be to add a Structured option alongside Opaque in DeviceConfiguration:

   type DeviceConfiguration struct {
       Opaque     *OpaqueDeviceConfiguration
       Structured *StructuredDeviceConfiguration  // NEW
   }

   type StructuredDeviceConfiguration struct {
       Driver     string
       // Attributes uses the same types as ResourceSlice Devices
       Attributes map[QualifiedName]DeviceAttribute
}

This would give us a schema that both the scheduler and the driver can decode — using the same DeviceAttribute types already used in ResourceSlice. For parameters like subnet or RDMA PKey, the user specifies the value once in structured config, the scheduler reads it for affinity matching, and the driver reads it for device programming. No duplication needed.

Opaque config would remain available for complex driver-specific configuration that doesn't fit a flat key-value model, but for the common sharing affinity cases (subnet, PKey, bitstream ID), structured config alone would be sufficient.

Does this align with what you had in mind? Happy to revise the KEP in this direction.

One question: should StructuredDeviceConfiguration be proposed as part of this KEP (with sharing affinity as the motivating use case), or would you prefer it as a separate KEP that this one depends on?