Adding the dimension_slices query parameter and associated metadata

[JPBergsma]

This PR introduces the ability for a client to request a specific subset, or "slice", of items from list properties. This is primarily achieved through the addition of a new query parameter, dimension_slices, and associated metadata fields to describe slicing capabilities and reflect the requested slices in the response.
Note this new feature is distinct from the existing protocol for large property values (see #467 and follow-up PRs), as slicing is initiated by the client, not the server. This means that even when slicing is requested by the client, the server can still decide whether to return the whole data in line, or via the partial data protocol.

1. dimension_slices query parameter

• Purpose: Request only specific parts of list properties.
• Syntax: dimension_slices=dim_xxx[start:stop:step],...
  • The slice defaults are as follows: start=0, step=1, stop=null (meaning upto the last value in the list).
  • Note that, at variance with Python, stop is inclusive.

2. Metadata enhancements in the response

A new list_axes list of dictionaries describing each list dimension. Each dict must contain dimension_name (required).
Optional keys: length, sliceable (bool), available_slice (index range), requested_slice (returned in list_axes for each sliced property, reflecting the exact slice requested).

3. Shared dimensions

• A slice on a dimension name applies to all properties that use that same name.

Result: Clients can fetch only needed portions of list data, with servers clearly describing slicing capabilities and confirming applied slices in the response.

Note that it surpersedes the original proposal #452.

[rartino]

Thanks for writing this up!

Why did you move the whole section "Transmission of large property values"? It makes it tricky to see what has changed. Can you move it back?

Also, my interpretation of the workshop discussions was to make the property_ranges query parameter completely separate from, and orthogonal to, the mechanism for "Transmission of large property values" (this was a strong point of @sauliusg). Hence, it isn't clear to me why anything has to change in the "Transmission of large property values". Why can't we keep all documentation about this feature in the definition of the property_ranges query parameter?

[JPBergsma]

It seemed more logical to place the metadata section before the "Transmission of large property values" section, as the "Transmission of large property values" section depends on the metadata section but not the other way round. So it is more useful to read the metadata section before the "Transmission of large property values" section.

[JPBergsma]

You are right, I could still separate the property ranges query parameter and the extra metadata fields from the transmission of large data section, as each could still be used separate from the other. I'll try to do this next week.

[ml-evs]

Mostly formatting changes and suggesting rewordings -- I think I can go through and force most of them in if there are no objections

[rartino]

This PR appears a bit stalled and is important. @JPBergsma do you plan to continue working on this, or are you ok with me (or someone else) starting to merge changes into it?

[giovannipizzi]

I am very happy with this final version. If there are minor textual changes, feel free to apply and merge as if these are also approved by me!

I don't remember anything about it

@merkys asks me to dismiss this

[giovannipizzi]

Thanks, great work! Let's get it merged!