[O365] - Prune stale listing URLs from cursor to recover from long outages

[ShourieG]

Type of change

• Bug

Proposed commit message

packages/o365: prune stale listing URLs from cursor to recover from long outages

A listing URL queued in cursor.todo_links before a prolonged outage
can age past the Management API's 7-day startTime window while parked
there, producing a permanent AF20055 retry loop: the listing-error
branch cannot evict it because its map-shaped error return is treated
by the CEL runtime as a freeze signal that discards cursor mutations.

Add a pre-flight filter that drops listing URLs whose embedded
startTime is older than now - maximum_age, tolerating both startTime
and starttime spellings (NextPageUri responses have been observed
with the lowercase form). Includes a system test that queues a
NextPageUri with a 2019 startTime against a 1h maximum_age and
asserts it is not followed.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Screenshots

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: f4f3c4c

Failed CI Steps

• Check integrations o365

History

cc @ShourieG