Repository navigation
Conversation
upload_file()'s `except BaseException` handler cleans up both the raw download and the converted parquet, but only parquet_file was pre-initialised. raw_path is first bound at the `_download_raw_with_retry` call, so any failure raised before that point made the handler itself raise `UnboundLocalError: cannot access local variable 'raw_path'`. That replaced the real exception with a misleading message, since `_reprocess` surfaces whatever the handler propagates — the operator saw "cannot access local variable 'raw_path'" instead of the actual cause. The temp file was also orphaned, as the cleanup aborted before running. Both names are now bound to None before the try block and the handler guards on raw_path, matching the existing parquet_file handling. Failures before the download now propagate their original type and message. Tests cover both paths: an error raised by _ensure_management_columns (before the download) propagates unchanged, and an error raised while hashing the download (after it) still removes the temp file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SyncEngine.upload_file()ends with anexcept BaseExceptionhandler that cleans up both the raw download and the converted parquet:parquet_file = Nonewas pre-initialised, butraw_pathwas not — its first binding is the_download_raw_with_retry()call partway into the block. Any failure raised before that point made the handler itself raiseUnboundLocalError: cannot access local variable 'raw_path'.Impact
The handler's own exception replaced the real one, because
_reprocesssurfaces whatever propagates. A failure inwriter._ensure_management_columns()(theALTER TABLE), inget_file_full(), or on a broken connection was reported to the operator as:which points at PySUS internals instead of the actual cause. The cleanup also aborted before running, so the temp file was orphaned on disk.
Fix
Both names are bound to
Nonebefore thetryblock, and the handler guards onraw_paththe same way it already did forparquet_file.Tests
Two tests added to
TestUploadFile:test_error_before_download_propagates_original— an error from_ensure_management_columnspropagates with its original type and messagetest_error_after_download_cleans_up_raw_file— an error while hashing the download still removes the temp fileI verified both fail with
UnboundLocalErroragainst the unpatched handler and pass after the fix.1735 passed, 6 skippedblackandisortclean