Skip to content

Commit 0a6617d

Browse files
pranz1996Pranjal Patel
andauthored
[Service Bus] Send server-timeout on management operations (#48563)
Management operations (peek, deferred receive, settlement over the management link, lock renewal, session state, session listing, schedule/cancel scheduled) now send `com.microsoft:server-timeout`: the caller's remaining time less a one second buffer, or 60 seconds when none was given, floored at zero and capped at the AMQP uint maximum. Previously no bound was sent, so a stalled service held the call until the AMQP link failed; it now raises a retryable `OperationTimeoutError`, surfacing after roughly four minutes at default retries. A settle over the management link can therefore raise `MessageLockLostError` where the call used to block. The data path is unchanged: `receive_messages`, receiver iteration and `send_messages` are unaffected, as is CBS token auth. Set in the base handler after the retry loop computes the remaining time, so it reflects the time left on the attempt. Both transports pass application properties through unchanged, so no transport change is needed. `REQUEST_RESPONSE_TIMEOUT` already held the correct key and was unused. Tests cover the arithmetic and its boundaries, that the property reaches the outgoing message, that the service's answer surfaces as a retryable `OperationTimeoutError`, and that the value carries the transport's own type. api.metadata.yml carries a gated parserVersion bump only; apiMdSha256 is unchanged, so no public API moved. Matches the .NET, Java and Go SDKs. Refs: AB#38822503 Co-authored-by: Pranjal Patel <pranjalpatel@microsoft.com>
1 parent f5f77ab commit 0a6617d

7 files changed

Lines changed: 300 additions & 3 deletions

File tree

‎sdk/servicebus/azure-servicebus/CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
### Other Changes
2525

2626
- When using the async `AmqpOverWebsocket` transport on Python 3.10 or later, `aiohttp>=3.14.0` is now recommended. Earlier `aiohttp` versions have a WebSocket heartbeat bug ([aio-libs/aiohttp#12030](https://github.com/aio-libs/aiohttp/pull/12030)) that can cause the connection to be dropped during long message processing, surfacing as a `SocketError` ("Cannot write to closing transport"). Python 3.9 users must upgrade Python to install an `aiohttp` release containing this fix. ([#44028](https://github.com/Azure/azure-sdk-for-python/issues/44028))
27+
- Management operations (peek, deferred receive, message settlement over the management link, lock renewal, session state, session listing, schedule/cancel) now send `com.microsoft:server-timeout`: the caller's remaining time less one second, or 60 seconds when none was given. Previously no bound was sent, so a stalled service held the call until the AMQP link failed; it now raises a retryable `OperationTimeoutError`, so a persistently stalled service surfaces after roughly four minutes at default retry settings. Matches the .NET, Java and Go SDKs.
2728

2829
## 7.14.3 (2025-11-11)
2930

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
apiMdSha256: 70b3fbd3239df511d17fde454a717f9ef0a11f125b096da26d9c815ab0120a17
2-
parserVersion: 0.3.30
2+
parserVersion: 0.3.31
33
pythonVersion: 3.14.0

‎sdk/servicebus/azure-servicebus/azure/servicebus/_base_handler.py‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,13 +25,19 @@
2525
OperationTimeoutError,
2626
SessionLockLostError,
2727
)
28-
from ._common.utils import create_properties, strip_protocol_from_uri, parse_sas_credential
28+
from ._common.utils import (
29+
create_properties,
30+
strip_protocol_from_uri,
31+
parse_sas_credential,
32+
get_server_timeout_ms,
33+
)
2934
from ._common.constants import (
3035
CONTAINER_PREFIX,
3136
MANAGEMENT_PATH_SUFFIX,
3237
TOKEN_TYPE_SASTOKEN,
3338
MGMT_REQUEST_OP_TYPE_ENTITY_MGMT,
3439
ASSOCIATEDLINKPROPERTYNAME,
40+
REQUEST_RESPONSE_TIMEOUT,
3541
)
3642

3743
if TYPE_CHECKING:
@@ -499,6 +505,10 @@ def _mgmt_request_response(
499505
except AttributeError:
500506
pass
501507

508+
application_properties[REQUEST_RESPONSE_TIMEOUT] = self._amqp_transport.AMQP_UINT_VALUE(
509+
get_server_timeout_ms(timeout)
510+
)
511+
502512
mgmt_msg = self._amqp_transport.create_mgmt_msg(
503513
message=message,
504514
application_properties=application_properties,

‎sdk/servicebus/azure-servicebus/azure/servicebus/_common/constants.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,14 @@
6161
PYAMQP_LIBRARY = "pyamqp"
6262
OPERATION_TIMEOUT = VENDOR + b":timeout"
6363

64+
# Bounds a management operation on the service side when the caller gave no timeout.
65+
DEFAULT_SERVER_TIMEOUT_MS = 60000
66+
# Subtracted from the caller's remaining time so the service answers before the client
67+
# gives up.
68+
SERVER_TIMEOUT_BUFFER_MS = 1000
69+
# The property is encoded as an AMQP uint, so cap at its maximum (about 49.7 days).
70+
MAX_SERVER_TIMEOUT_MS = 2**32 - 1
71+
6472
MANAGEMENT_PATH_SUFFIX = "/$management"
6573

6674
MGMT_RESPONSE_SESSION_STATE = b"session-state"

‎sdk/servicebus/azure-servicebus/azure/servicebus/_common/utils.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,9 @@
3737
DEAD_LETTER_QUEUE_SUFFIX,
3838
TRANSFER_DEAD_LETTER_QUEUE_SUFFIX,
3939
USER_AGENT_PREFIX,
40+
DEFAULT_SERVER_TIMEOUT_MS,
41+
SERVER_TIMEOUT_BUFFER_MS,
42+
MAX_SERVER_TIMEOUT_MS,
4043
)
4144
from ..amqp import AmqpAnnotatedMessage
4245

@@ -89,6 +92,23 @@ def utc_now():
8992
return datetime.datetime.now(timezone.utc)
9093

9194

95+
def get_server_timeout_ms(timeout: Optional[float]) -> int:
96+
"""Return the server-timeout for a management operation, in milliseconds.
97+
98+
This is a service-side bound, not a client-side one. It clamps at zero, since under
99+
a second of remaining time there is no room for the service to answer first, and at
100+
the AMQP uint maximum, since the value is encoded as one.
101+
102+
:param float or None timeout: The caller's remaining timeout in seconds, or None.
103+
:rtype: int
104+
:returns: The remaining time less the buffer, or the default if no timeout was given.
105+
"""
106+
if timeout is None:
107+
return DEFAULT_SERVER_TIMEOUT_MS
108+
capped = min(timeout, MAX_SERVER_TIMEOUT_MS / 1000)
109+
return max(int(capped * 1000) - SERVER_TIMEOUT_BUFFER_MS, 0)
110+
111+
92112
def build_uri(address, entity):
93113
parsed = urlparse(address)
94114
if parsed.path:

‎sdk/servicebus/azure-servicebus/azure/servicebus/aio/_base_handler_async.py‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,19 @@
1313
from ._transport._pyamqp_transport_async import PyamqpTransportAsync
1414
from .._base_handler import _generate_sas_token, BaseHandler as BaseHandlerSync, _get_backoff_time
1515
from .._common._configuration import Configuration
16-
from .._common.utils import create_properties, strip_protocol_from_uri, parse_sas_credential
16+
from .._common.utils import (
17+
create_properties,
18+
strip_protocol_from_uri,
19+
parse_sas_credential,
20+
get_server_timeout_ms,
21+
)
1722
from .._common.constants import (
1823
TOKEN_TYPE_SASTOKEN,
1924
MGMT_REQUEST_OP_TYPE_ENTITY_MGMT,
2025
ASSOCIATEDLINKPROPERTYNAME,
2126
CONTAINER_PREFIX,
2227
MANAGEMENT_PATH_SUFFIX,
28+
REQUEST_RESPONSE_TIMEOUT,
2329
)
2430
from ..exceptions import (
2531
ServiceBusConnectionError,
@@ -340,6 +346,10 @@ async def _mgmt_request_response(
340346
except AttributeError:
341347
pass
342348

349+
application_properties[REQUEST_RESPONSE_TIMEOUT] = self._amqp_transport.AMQP_UINT_VALUE(
350+
get_server_timeout_ms(timeout)
351+
)
352+
343353
mgmt_msg = self._amqp_transport.create_mgmt_msg( # type: ignore # TODO: fix mypy
344354
message=message,
345355
application_properties=application_properties,
Lines changed: 248 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
3+
"""Unit tests for the management server-timeout.
4+
5+
Management operations send the `com.microsoft:server-timeout` property, asking the service to
6+
bound the operation on its side. The value is the caller's remaining time less a one second
7+
buffer, so the service answers before the client gives up, or 60 seconds when the caller
8+
supplied no timeout.
9+
10+
Previously no bound was sent at all, so a stalled service could hold a management call until
11+
the AMQP link itself failed. Matches the .NET, Java and Go SDKs.
12+
"""
13+
14+
from unittest.mock import MagicMock
15+
16+
import struct
17+
18+
import pytest
19+
20+
from azure.servicebus._common import mgmt_handlers
21+
from azure.servicebus._common.constants import (
22+
ERROR_CODE_TIMEOUT,
23+
MAX_SERVER_TIMEOUT_MS,
24+
MGMT_RESPONSE_MESSAGE_ERROR_CONDITION,
25+
REQUEST_RESPONSE_TIMEOUT,
26+
)
27+
from azure.servicebus._common.utils import get_server_timeout_ms
28+
from azure.servicebus._transport._pyamqp_transport import PyamqpTransport
29+
from azure.servicebus.exceptions import OperationTimeoutError
30+
31+
32+
class TestServerTimeoutMillis:
33+
"""`get_server_timeout_ms` converts the caller's remaining time into the
34+
value advertised to the service."""
35+
36+
@pytest.mark.parametrize(
37+
"remaining_seconds,expected_ms",
38+
[
39+
(None, 60000), # no caller timeout: the default, not "no bound at all"
40+
(120, 119000),
41+
(60, 59000), # equals the default, but must still take the buffer path
42+
(10, 9000),
43+
(1.5, 500),
44+
(1, 0), # at the buffer, nothing left to give the service
45+
(0.5, 0),
46+
(-5, 0), # deadline already passed: clamped, never a negative uint
47+
],
48+
)
49+
def test_remaining_time_less_buffer(self, remaining_seconds, expected_ms):
50+
assert get_server_timeout_ms(remaining_seconds) == expected_ms
51+
52+
def test_wire_contract(self):
53+
# The key .NET and Java send, encoded as an unsigned int of milliseconds.
54+
assert REQUEST_RESPONSE_TIMEOUT == b"com.microsoft:server-timeout"
55+
encoded = PyamqpTransport.AMQP_UINT_VALUE(get_server_timeout_ms(None))
56+
assert encoded == {"TYPE": "UINT", "VALUE": 60000}
57+
58+
@pytest.mark.parametrize("remaining_seconds", [4294968, 5_000_000, 1e12, float("inf")])
59+
def test_capped_at_the_amqp_uint_maximum(self, remaining_seconds):
60+
# `timeout` is only bounded at zero, so a large value would overflow the uint encoder.
61+
result = get_server_timeout_ms(remaining_seconds)
62+
assert result <= MAX_SERVER_TIMEOUT_MS
63+
struct.pack(">I", result) # raises if it does not fit
64+
65+
def test_just_below_the_cap_is_not_clamped(self):
66+
assert get_server_timeout_ms(4294967) == 4294966000
67+
68+
69+
class TestManagementRequestSetsServerTimeout:
70+
"""The property must actually reach the outgoing management message, on both
71+
the associated-link and no-associated-link paths."""
72+
73+
def _make_handler(self):
74+
from azure.servicebus._base_handler import BaseHandler
75+
76+
captured = {}
77+
78+
def fake_create_mgmt_msg(message, application_properties, config, reply_to, **kwargs):
79+
captured.clear()
80+
captured.update(application_properties)
81+
return MagicMock()
82+
83+
handler = BaseHandler.__new__(BaseHandler)
84+
handler._amqp_transport = MagicMock()
85+
handler._amqp_transport.create_mgmt_msg = fake_create_mgmt_msg
86+
handler._amqp_transport.AMQP_UINT_VALUE = PyamqpTransport.AMQP_UINT_VALUE
87+
handler._amqp_transport.get_handler_link_name = lambda h: "link-1"
88+
handler._amqp_transport.mgmt_client_request = lambda *args, **kwargs: "response"
89+
handler._amqp_transport.TIMEOUT_ERROR = TimeoutError
90+
handler._open = lambda: None
91+
handler._handler = MagicMock()
92+
handler._config = MagicMock(encoding="UTF-8")
93+
handler._mgmt_target = "queue/$management"
94+
return handler, captured
95+
96+
def test_default_sent_when_caller_gave_no_timeout(self):
97+
# The gap this closes: previously no bound was sent at all.
98+
handler, captured = self._make_handler()
99+
handler._mgmt_request_response(b"op", {}, lambda *a: None, timeout=None)
100+
assert captured[REQUEST_RESPONSE_TIMEOUT] == {"TYPE": "UINT", "VALUE": 60000}
101+
102+
def test_remaining_time_less_buffer_sent(self):
103+
handler, captured = self._make_handler()
104+
handler._mgmt_request_response(b"op", {}, lambda *a: None, timeout=10)
105+
assert captured[REQUEST_RESPONSE_TIMEOUT] == {"TYPE": "UINT", "VALUE": 9000}
106+
107+
def test_clamped_below_buffer(self):
108+
handler, captured = self._make_handler()
109+
handler._mgmt_request_response(b"op", {}, lambda *a: None, timeout=0.4)
110+
assert captured[REQUEST_RESPONSE_TIMEOUT] == {"TYPE": "UINT", "VALUE": 0}
111+
112+
def test_associated_link_name_preserved(self):
113+
handler, captured = self._make_handler()
114+
handler._mgmt_request_response(b"op", {}, lambda *a: None, timeout=None)
115+
assert b"associated-link-name" in captured
116+
assert REQUEST_RESPONSE_TIMEOUT in captured
117+
118+
def test_sent_on_calls_without_an_associated_link(self):
119+
# list_sessions passes keep_alive_associated_link=False, starting from an empty map.
120+
handler, captured = self._make_handler()
121+
handler._mgmt_request_response(
122+
b"op",
123+
{},
124+
lambda *a: None,
125+
keep_alive_associated_link=False,
126+
timeout=None,
127+
)
128+
assert captured == {REQUEST_RESPONSE_TIMEOUT: {"TYPE": "UINT", "VALUE": 60000}}
129+
130+
131+
class TestServiceTimeoutResponse:
132+
"""The response half: the service's answer must surface as a retryable
133+
`OperationTimeoutError`, which rests on `com.microsoft:timeout` in `errorCondition`."""
134+
135+
@staticmethod
136+
def _response(condition):
137+
message = MagicMock()
138+
message.application_properties = {MGMT_RESPONSE_MESSAGE_ERROR_CONDITION: condition}
139+
return message
140+
141+
def test_timeout_condition_raises_retryable_operation_timeout_error(self):
142+
with pytest.raises(OperationTimeoutError) as exc_info:
143+
mgmt_handlers.default(408, self._response(ERROR_CODE_TIMEOUT), "The operation timed out.", PyamqpTransport)
144+
145+
assert exc_info.value._retryable is True
146+
147+
def test_success_returns_the_value_untouched(self):
148+
message = self._response(None)
149+
message.value = {"ok": True}
150+
assert mgmt_handlers.default(200, message, None, PyamqpTransport) == {"ok": True}
151+
152+
def test_uamqp_maps_the_same_condition(self):
153+
# Skip on uamqp itself: the transport module imports without it, but UamqpTransport is not defined.
154+
pytest.importorskip("uamqp", reason="uamqp not installed")
155+
from azure.servicebus._transport._uamqp_transport import UamqpTransport
156+
157+
with pytest.raises(OperationTimeoutError):
158+
mgmt_handlers.default(
159+
408,
160+
self._response(ERROR_CODE_TIMEOUT),
161+
"The operation timed out.",
162+
UamqpTransport,
163+
)
164+
165+
166+
class TestUamqpValueType:
167+
"""uamqp is the one place the value type changes: pyamqp uses a plain dict, uamqp an `AMQPuInt`."""
168+
169+
def test_transport_supplied_type_is_what_reaches_the_message(self):
170+
171+
from azure.servicebus._base_handler import BaseHandler
172+
173+
sentinel = object()
174+
captured = {}
175+
176+
def fake_create_mgmt_msg(message, application_properties, config, reply_to, **kwargs):
177+
captured.update(application_properties)
178+
return MagicMock()
179+
180+
handler = BaseHandler.__new__(BaseHandler)
181+
handler._amqp_transport = MagicMock()
182+
handler._amqp_transport.create_mgmt_msg = fake_create_mgmt_msg
183+
handler._amqp_transport.AMQP_UINT_VALUE = lambda ms: sentinel
184+
handler._amqp_transport.get_handler_link_name = lambda h: "link-1"
185+
handler._amqp_transport.mgmt_client_request = lambda *args, **kwargs: "response"
186+
handler._amqp_transport.TIMEOUT_ERROR = TimeoutError
187+
handler._open = lambda: None
188+
handler._handler = MagicMock()
189+
handler._config = MagicMock(encoding="UTF-8")
190+
handler._mgmt_target = "queue/$management"
191+
192+
handler._mgmt_request_response(b"op", {}, lambda *a: None, timeout=None)
193+
assert captured[REQUEST_RESPONSE_TIMEOUT] is sentinel
194+
195+
def test_real_uamqp_type_encodes_in_application_properties(self):
196+
uamqp = pytest.importorskip("uamqp", reason="uamqp not installed")
197+
from azure.servicebus._transport._uamqp_transport import UamqpTransport
198+
199+
value = UamqpTransport.AMQP_UINT_VALUE(get_server_timeout_ms(None))
200+
assert isinstance(value, uamqp.types.AMQPuInt)
201+
202+
message = UamqpTransport.create_mgmt_msg(
203+
message={"operation": "peek"},
204+
application_properties={REQUEST_RESPONSE_TIMEOUT: value},
205+
config=MagicMock(encoding="UTF-8"),
206+
reply_to="queue/$management",
207+
)
208+
209+
assert message.encode_message()
210+
211+
212+
class TestAsyncParity:
213+
"""The async management path is a separate implementation and can drift from the sync one."""
214+
215+
@pytest.mark.asyncio
216+
async def test_async_management_sets_server_timeout(self):
217+
from azure.servicebus.aio._base_handler_async import BaseHandler as AsyncBaseHandler
218+
219+
captured = {}
220+
221+
def fake_create_mgmt_msg(message, application_properties, config, reply_to, **kwargs):
222+
captured.clear()
223+
captured.update(application_properties)
224+
return MagicMock()
225+
226+
async def fake_request(*args, **kwargs):
227+
return "response"
228+
229+
async def fake_open():
230+
return None
231+
232+
handler = AsyncBaseHandler.__new__(AsyncBaseHandler)
233+
handler._amqp_transport = MagicMock()
234+
handler._amqp_transport.create_mgmt_msg = fake_create_mgmt_msg
235+
handler._amqp_transport.AMQP_UINT_VALUE = PyamqpTransport.AMQP_UINT_VALUE
236+
handler._amqp_transport.get_handler_link_name = lambda h: "link-1"
237+
handler._amqp_transport.mgmt_client_request_async = fake_request
238+
handler._amqp_transport.TIMEOUT_ERROR = TimeoutError
239+
handler._open = fake_open
240+
handler._handler = MagicMock()
241+
handler._config = MagicMock(encoding="UTF-8")
242+
handler._mgmt_target = "queue/$management"
243+
244+
await handler._mgmt_request_response(b"op", {}, lambda *a: None, timeout=None)
245+
assert captured[REQUEST_RESPONSE_TIMEOUT] == {"TYPE": "UINT", "VALUE": 60000}
246+
247+
await handler._mgmt_request_response(b"op", {}, lambda *a: None, timeout=10)
248+
assert captured[REQUEST_RESPONSE_TIMEOUT] == {"TYPE": "UINT", "VALUE": 9000}

0 commit comments

Comments
 (0)