Fix/4222 sharepoint list item id collision - #4254
Open
falscherwiener1-svg wants to merge 4 commits into
Open
Conversation
Add per-item retry logic for transient HTTP status codes (429, 503) in the bulk ingestion sink. Previously, individual items that failed with transient errors inside a successful bulk response were sent directly to the ErrorMonitor without retry, which could trigger a premature TooManyErrors abort on a temporarily overloaded cluster. Changes: - Add TRANSIENT_STATUS_CODES constant (429, 503) - Add _determine_action() helper to extract the bulk action key from a response item - Add _extract_transient_failed_operations() to collect retryable items from a bulk response - Add _retry_transient_item_failures() with exponential backoff that retries only the failed items - Rewire _batch_bulk to call _retry_transient_item_failures before _process_bulk_response, so the ErrorMonitor only tracks failures after all retry attempts are exhausted - Remove the now-unused retryable import and the dead _bulk_api_call function that was never invoked Closes elastic#4002
…ns (elastic#4222) SharePoint list GUIDs are only unique within a site collection. When two site collections share a list GUID (e.g. via site templates or migration), list_item documents from different sites collide on the same _id and silently overwrite each other. Include site_id in the _id construction to guarantee global uniqueness.
Member
|
Please fill in the information in the PR body. Additionally, PR contains unrelated changes related to retrying bulk requests - they should not be here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes https://github.com/elastic/connectors-py/issues/###
Checklists
Pre-Review Checklist
config.yml.example)v7.13.2,v7.14.0,v8.0.0)Changes Requiring Extra Attention
Related Pull Requests
Release Note