Repository navigation
Commit e8d7a54
committed
runtime: wait the interval the service asked for, on every retry path
Microsoft's throttling guidance has one instruction before all the others:
If the response includes a Retry-After header, wait for the specified
interval before retrying. Don't retry the request immediately. Treat it as
the minimum time to wait.
The library honoured that in two of three places. `_send_with_retry` and
`_send_with_retry_async` (the parallel and async paths) return the parsed
`Retry-After` from their `on_failure` hook, so a throttled request there waits
what the service asked. `execute_query_retry` -- the sequential entry point, and
the one `docs/` and `examples/` recommend for a polite retry -- passed
`failure_callback=None` by default and therefore returned `None` from its hook,
which the retry loop reads as "no opinion" and replaces with the exponential
backoff. So `Retry-After: 120` was answered with `sleep(5)` and an immediate
re-send: the retry storm the header exists to prevent, emitted by the call the
documentation points you at.
Two changes, one rule each.
The default policy is now the server's. `runtime.retry.retry_after_policy(attempt,
ex)` is a two-argument adapter over the existing `retry_after_delay(ex)`, whose
signature did not match what the retry loops promise to call; the loops use it
when no `on_failure` was supplied, and `execute_query_retry` /
`execute_query_async_retry` fall back to it when `failure_callback is None`. A
callback you pass still wins -- the policy is a default, not a ceiling on what a
caller may decide.
`max_delay` now bounds whichever delay is chosen. It used to cap only
`backoff_delay`, so a caller who had already said "never wait longer than 30
seconds" was still handed an hour by a `Retry-After: 3600`. One helper,
`delay_before_attempt`, is the only place the choice is made, and it clamps the
server's value too: a caller with a ceiling keeps it, a caller without one waits
as long as the service asked. That is also the difference the new test pins --
`Retry-After: 600` with `max_delay=30` sleeps 30, and `Retry-After: 7` with no cap
sleeps 7.
Proved red first, as required: with the two library files stashed,
`test_retry_after_is_waited_by_default` and
`test_max_delay_bounds_the_wait_the_server_asks_for` fail with
`sleep(4.076850252856038)` where 30 was expected and `sleep(5)` where 7 was
expected; with the change they pass, and the 33 tests already in
`tests/unit/test_runtime_retry.py` -- including the one that pins "no
`Retry-After` means exponential backoff" -- are untouched.
Measured: `tests/unit` + `tests/tooling` 1012 passed, 5 subtests; offline suite
1015 passed / 983 skipped; pyright 0 errors, 0 warnings, 0 informations; ruff
clean; a live `ctx.web.get().execute_query_retry(max_retry=2)` against the team
site still answers the web title, so the untrottled path is unchanged.
Not changed, deliberately: the per-service request windows (Outlook's 10,000 per
10 minutes per mailbox, Teams' per-resource rates, Files' concurrent-write
limits) are documented as *thresholds*, not as signals a response carries. The
library reacts to the signals the service sends and does not simulate the token
bucket; mirroring those tables into code would give callers a number to believe
without giving the runtime anything to enforce. Where they do belong -- beside
the model, as `Limit(...)` rows -- the catalogs for Outlook/Excel/Reports/Teams
are missing and are a data change, not a code path.1 parent ae6f087 commit e8d7a54
4 files changed
Lines changed: 106 additions & 35 deletions
File tree
- office365/runtime
- tests/unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
378 | 378 | | |
379 | 379 | | |
380 | 380 | | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
381 | 398 | | |
382 | 399 | | |
383 | 400 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
170 | 170 | | |
171 | 171 | | |
172 | 172 | | |
173 | | - | |
174 | | - | |
175 | | - | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
179 | 179 | | |
180 | | - | |
181 | | - | |
| 180 | + | |
| 181 | + | |
182 | 182 | | |
183 | 183 | | |
184 | | - | |
| 184 | + | |
| 185 | + | |
185 | 186 | | |
186 | 187 | | |
187 | 188 | | |
188 | 189 | | |
189 | 190 | | |
190 | 191 | | |
191 | | - | |
| 192 | + | |
192 | 193 | | |
193 | 194 | | |
194 | 195 | | |
| |||
199 | 200 | | |
200 | 201 | | |
201 | 202 | | |
202 | | - | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
203 | 207 | | |
204 | 208 | | |
205 | 209 | | |
| |||
550 | 554 | | |
551 | 555 | | |
552 | 556 | | |
553 | | - | |
554 | | - | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
555 | 560 | | |
556 | 561 | | |
557 | 562 | | |
558 | 563 | | |
559 | | - | |
560 | | - | |
| 564 | + | |
| 565 | + | |
561 | 566 | | |
562 | 567 | | |
563 | | - | |
| 568 | + | |
| 569 | + | |
564 | 570 | | |
565 | 571 | | |
566 | 572 | | |
567 | 573 | | |
568 | 574 | | |
569 | | - | |
| 575 | + | |
570 | 576 | | |
571 | 577 | | |
572 | 578 | | |
| |||
577 | 583 | | |
578 | 584 | | |
579 | 585 | | |
580 | | - | |
| 586 | + | |
581 | 587 | | |
582 | 588 | | |
583 | 589 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
131 | 141 | | |
132 | 142 | | |
133 | 143 | | |
| |||
152 | 162 | | |
153 | 163 | | |
154 | 164 | | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
155 | 185 | | |
156 | 186 | | |
157 | 187 | | |
| |||
175 | 205 | | |
176 | 206 | | |
177 | 207 | | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
| 208 | + | |
| 209 | + | |
182 | 210 | | |
183 | 211 | | |
184 | 212 | | |
| |||
198 | 226 | | |
199 | 227 | | |
200 | 228 | | |
201 | | - | |
202 | | - | |
203 | | - | |
204 | | - | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
205 | 234 | | |
206 | 235 | | |
207 | 236 | | |
| |||
215 | 244 | | |
216 | 245 | | |
217 | 246 | | |
218 | | - | |
219 | | - | |
| 247 | + | |
| 248 | + | |
220 | 249 | | |
221 | 250 | | |
222 | 251 | | |
223 | | - | |
| 252 | + | |
| 253 | + | |
224 | 254 | | |
225 | 255 | | |
226 | 256 | | |
| |||
261 | 291 | | |
262 | 292 | | |
263 | 293 | | |
264 | | - | |
265 | | - | |
| 294 | + | |
| 295 | + | |
266 | 296 | | |
267 | 297 | | |
268 | 298 | | |
269 | 299 | | |
270 | 300 | | |
271 | 301 | | |
272 | | - | |
273 | | - | |
| 302 | + | |
| 303 | + | |
274 | 304 | | |
275 | 305 | | |
276 | 306 | | |
277 | | - | |
| 307 | + | |
| 308 | + | |
278 | 309 | | |
279 | 310 | | |
280 | 311 | | |
| |||
288 | 319 | | |
289 | 320 | | |
290 | 321 | | |
291 | | - | |
292 | | - | |
293 | | - | |
294 | | - | |
| 322 | + | |
| 323 | + | |
295 | 324 | | |
296 | 325 | | |
297 | 326 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
109 | 119 | | |
110 | 120 | | |
111 | 121 | | |
| |||
128 | 138 | | |
129 | 139 | | |
130 | 140 | | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
131 | 150 | | |
132 | 151 | | |
133 | 152 | | |
| |||
0 commit comments