fix(warehouse-sources): stop one unreadable HubSpot object failing the properties table - #95236
fix(warehouse-sources): stop one unreadable HubSpot object failing the properties table#95236posthog[bot] wants to merge 1 commit into
Conversation
…e properties table The properties lookup table fans out over every HubSpot object endpoint, and its guard skipped an object the portal cannot read by catching `requests.HTTPError`. Since 403s started being raised as `HubspotForbiddenError` (a `NonReportableError`, not an `HTTPError`), that guard no longer fires on 403 and a single unreadable object type fails the whole table. Catch `HubspotForbiddenError` too, which also covers the `HubspotMissingScopeError` subclass raised for scope-gated objects. The existing regression test injected `requests.HTTPError` directly, so it kept passing against an exception class the fetch path no longer raises. It now raises through `raise_for_hubspot_status`, so the skip is exercised against whatever the production path actually produces. Generated-By: PostHog Desktop Task-Id: 27f98539-efe2-40b1-a44a-780543bd4e58
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
🤖 CI report
|
|
Closing in favor of #95301, which fixes the same root cause. The two differ on one decision. This PR skips the 403 for every lookup table, #95301 also covers The root-cause analysis here was correct, and #95301 builds on it. |
1 participant
Problem
A HubSpot source's
propertiestable stops syncing entirely — statusFailed, zero rows, never completes — for any portal whose grant cannot read one of HubSpot's object types. Every other table on the same source syncs fine, and the customer has no way to opt out:get_properties_rowsfans out over every entry inHUBSPOT_ENDPOINTS, so the table ignores which tables were selected for sync.The connector already intends to tolerate this.
_iter_pageshas a_SKIPPABLE_STATUSES = (403, 404)guard whose comment says "Skipping keeps one unavailable object type from failing the table." The guard catchesrequests.HTTPError. Sinceraise_for_hubspot_statuswas introduced, a 403 is raised asHubspotForbiddenError, which subclassesNonReportableError(Exception)— notrequests.HTTPError. So the 403 half of that guard has been unreachable, and one unreadable object type takes the whole table down.The customer-facing error tells them to reconnect HubSpot, which does not help: the grant is fine for everything they selected, and reconnecting cannot make an object readable that their plan does not include.
Changes
_iter_pagesnow also skipsHubspotForbiddenError, so an object type the portal cannot read is logged and passed over instead of failing theproperties,pipelines,pipeline_stages, andownerstables.HubspotMissingScopeErrorsubclass too, so a scope-gated object such asleadsno longer fails thepropertiestable on connections that were never granted its optional scope.How did you test this code?
test_properties_skips_an_object_type_the_portal_cannot_readpreviously raisedrequests.HTTPErrordirectly from a patchedfetch_data, so it kept passing against an exception class the fetch path no longer raises — that mismatch is why this shipped. It now raises through the realraise_for_hubspot_status, so the skip is exercised against whatever the production path actually produces. Because it lets every non-dealspath 403, it coversHubspotMissingScopeError(vialeads) as well as the plain forbidden case.Verified the test fails on master and passes with the fix:
test_properties_skips_an_object_type_the_portal_cannot_readfails withHubspotForbiddenError: 403 Client Error: Forbidden for url: .../crm/properties/2026-03/contacts.ruff format --check,ruff check, andmypyare clean on the changed files. No manual end-to-end run against a real HubSpot portal was done.Automatic notifications
Docs update
None.
🤖 Agent context
Autonomy: Fully autonomous
Written by Claude Code during warehouse-sources ticket triage, following the repo's
triaging-warehouse-sync-ticketsskill and theticket-investigation-warehouse-sourcesskill from the PostHog skills store. The bug was found by readingmetadata.py,helpers.py, andscopes.py, then confirmed against production sync logs for an affected source.Two fixes were considered. Giving the unreadable object a
required_scopewas rejected:HubspotMissingScopeErrorsubclassesHubspotForbiddenError, so the properties job would still escape the guard and fail, just with different wording. Catching the exception type is what actually restores the intended skip, and it fixes the scope-gated case at the same time.No duplicate:
gh pr list --state open --searchacross "hubspot", "403 OR forbidden", and the module path found nothing touching this guard. The nearest neighbors — #91331 (HubSpot 401 classification,source.py) and #91423 (Decagon/Northpass 403 messaging) — share no root cause or files.Public artifact: the investigation drew on a customer support ticket and that customer's production sync logs. Nothing from either appears in this PR. The test fixtures are the ones already in the file, and the description describes the failure generically.