Skip to content

Commit 499ac31

Browse files
authored
Add some clarifications based on DPO feedback (#5012)
* Clarify the atomic publishing requirements * Close some race conditions * Describe conflict resolution and name reservation * Session status retention and other clarifications
1 parent 113a0a5 commit 499ac31

1 file changed

Lines changed: 134 additions & 4 deletions

File tree

‎peps/pep-0694.rst‎

Lines changed: 134 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -723,16 +723,79 @@ A publish attempt that fails *synchronously* (i.e. within the publish request it
723723
client as an :ref:`error response <session-errors>` and leaves the session in its current editable state; it
724724
does **not** move the session to ``error``.
725725

726+
.. _publishing-session-atomicity:
727+
728+
Atomic Publication and Conflicts
729+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
730+
731+
Publishing a session is atomic with respect to the release's filename namespace. Because published artifacts
732+
are immutable, the index **MUST** guarantee that it never publishes two files with the same name for the same
733+
release, even in the presence of concurrent uploads arriving through this API or the legacy API.
734+
735+
The point-in-time conflict check performed when a :ref:`file upload session is created <file-upload-session>`
736+
is best-effort: it reflects the published state at the moment of that request and does **not** guarantee the
737+
file will still be conflict-free at publish time, since the published state of the release can change while a
738+
session is open. For example, a file with the same name may be published through the legacy API, or through a
739+
subsequent session for the same, already-published ``name`` and ``version``. The authoritative conflict check
740+
is therefore performed atomically at publish time.
741+
742+
**Filename reservation.** When a client requests publication, the server **MUST** atomically reserve the
743+
filenames of all files in the session within the target release, and hold that reservation for the duration of
744+
the publish:
745+
746+
- While the reservation is held, any other attempt to upload a file with one of those filenames to the same
747+
release -- whether through this API or the legacy API -- **MUST** be rejected with a ``409 Conflict``,
748+
exactly as if the file were already published. The index **MUST** enforce this regardless of which upload
749+
path the conflicting request arrives through; this is a requirement on the index's shared published-filename
750+
namespace and does not otherwise modify the legacy API.
751+
752+
- The reservation governs conflict detection only; the reserved files remain :ref:`staged <staged-preview>`
753+
and **MUST NOT** become publicly visible until the publish succeeds.
754+
755+
- If the publish succeeds, the reservation converts to permanent publication. If it fails -- synchronously,
756+
leaving the session editable, or during deferred ``processing``, moving it to the ``error`` state -- the
757+
server **MUST** release the reservation, making those filenames available again.
758+
759+
A still-``open`` session does **not** reserve its filenames; the reservation is acquired only when publication
760+
is requested. For an immediate (``201 Created``) publish the reservation is held only for the duration of the
761+
single request and is effectively unobservable. The requirement is meaningful mainly for deferred (``202
762+
Accepted`` then ``processing``) publishes, where there is a real window between the index validating one staged
763+
artifact and committing the rest.
764+
765+
This closes that window at a deliberate, conservative cost: a concurrent upload **MAY** receive a ``409
766+
Conflict`` during a publish that ultimately fails, and then succeed on retry once the reservation is released.
767+
The index never publishes two files with the same name, at the cost of occasionally rejecting a concurrent
768+
upload that a later retry would allow.
769+
770+
**Reporting a publish-time conflict.** When the index detects a conflict while publishing -- whether a
771+
filename collision as above, or another precondition that held when the session was created but no longer
772+
holds, such as an exhausted quota or a revoked permission -- it reports the failure according to the
773+
:ref:`publishing session state machine <publishing-session-states>`:
774+
775+
- If the server is completing the publish synchronously, it **MUST** return an :ref:`error response
776+
<session-errors>` -- a ``409 Conflict`` for a filename collision -- identifying the conflicting file(s), and
777+
leave the session in its current editable state.
778+
779+
- If the server accepted the publish for deferred processing and detects the conflict during ``processing``,
780+
it **MUST** move the session to the ``error`` state and report the conflicting file(s) in the session's
781+
``notices`` (and, where the failure is attributable to a particular file, in that file's ``notices``).
782+
783+
In either case the client may delete or replace the offending file, or otherwise resolve the conflict, and
784+
publish again, or :ref:`cancel <publishing-session-cancellation>` the session.
785+
726786
.. _publishing-session-cancellation:
727787

728788
Publishing Session Cancellation
729789
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
730790

731791
To cancel a publishing session, a client issues a ``DELETE`` request to the ``session`` :ref:`link
732792
<publishing-session-links>` given in the :ref:`session creation response body <publishing-session-response>`.
733-
The server then marks the session as ``canceled``, and **SHOULD** purge any data that was uploaded as part of
734-
that session. Future attempts to access that session URL or any of the publishing session URLs **MUST**
735-
return a ``404 Not Found``.
793+
The server then marks the session as ``canceled`` and **SHOULD** purge any data that was uploaded as part of
794+
that session. Once that data is purged, the session's action and data-bearing URLs -- ``links.upload``,
795+
``links.publish``, ``links.extend``, ``links.stage``, and the individual file URLs -- **SHOULD** become
796+
unavailable and **MAY** return ``404 Not Found``. The session status URL (``links.session``) is instead
797+
retained as described in :ref:`publishing-session-retention`: it continues to report the ``canceled`` status
798+
for an index-specific period before it too **MAY** return ``404 Not Found``.
736799

737800
Cancellation is only permitted while the session is :ref:`open or in the error state
738801
<publishing-session-states>`. If the session is in the ``processing`` state (i.e. because a deferred
@@ -760,6 +823,31 @@ The server will respond to this ``GET`` request with the same :ref:`publishing s
760823
any changes to ``status``, ``expires-at``, or ``files`` reflected.
761824

762825

826+
.. _publishing-session-retention:
827+
828+
Session Status Retention
829+
~~~~~~~~~~~~~~~~~~~~~~~~~~
830+
831+
A session status URL is not guaranteed to remain valid indefinitely. Once a session reaches a terminal state
832+
-- ``published`` or ``canceled`` -- the server **SHOULD** continue to serve its :ref:`status URL
833+
<publishing-session-status>`, reporting the terminal ``status``, for an index-specific retention period, so
834+
that clients can reliably observe the final outcome of a session they did not watch to completion. After that
835+
retention period elapses, the server **MAY** return ``404 Not Found`` for the status URL. The length of the
836+
retention period is left to the index, but **SHOULD** be long enough to let a client that initiated or
837+
contributed to the session learn its outcome.
838+
839+
This retention applies only to the session *status*; the data-bearing and action URLs for a terminated session
840+
(for example ``links.stage`` and the individual file URLs) may become unavailable as soon as their underlying
841+
data is no longer needed, as described for :ref:`cancellation <publishing-session-cancellation>`.
842+
843+
Clients **MUST** be prepared for a session status URL to return ``404 Not Found`` once the session has
844+
terminated and its retention period has elapsed. A ``404`` on a previously valid session status URL is not in
845+
itself an error; clients **SHOULD** treat it as indicating that the session no longer exists. Note that once a
846+
terminated session has been purged, a request to :ref:`create a new session <publishing-session-multiple>` for
847+
the same ``name`` and ``version`` will succeed with a ``201 Created`` rather than returning the ``409
848+
Conflict`` that a still-live session would have produced.
849+
850+
763851
.. _publishing-session-extension:
764852

765853
Publishing Session Extension
@@ -884,7 +972,9 @@ A publishing session **MAY** be created for a ``name`` and ``version`` that has
884972
example to add wheels for additional platforms to an existing release. However, because published artifacts
885973
are immutable, if the ``filename`` in this request matches a file that has already been published for this
886974
release, the server **MUST** reject the request with a ``409 Conflict`` and **MUST NOT** overwrite the
887-
published file.
975+
published file. This check is best-effort and reflects the published state at the time of the request; the
976+
authoritative, atomic conflict check is performed at publish time, as described in
977+
:ref:`publishing-session-atomicity`.
888978

889979
If the server determines that upload should proceed, it will return a ``202 Accepted`` response, with the
890980
response body below. The :ref:`status <publishing-session-status>` of the publishing session will also
@@ -1160,6 +1250,15 @@ The client can query status of the file upload session by issuing a ``GET`` requ
11601250
<file-upload-session-response>`. The server responds to this request with the same payload as the file upload
11611251
session creation response, except with any changes ``status`` and ``expires-at`` reflected.
11621252

1253+
A file upload session has no existence independent of the publishing session it belongs to, and its status URL
1254+
is retained accordingly. While the parent publishing session is in a non-terminal state, the server
1255+
**SHOULD** keep each of its file upload session status URLs valid, reporting the file upload session's current
1256+
``status`` -- including a terminal ``canceled`` -- so that a client can observe the outcome of any file it
1257+
uploaded. Once the parent publishing session itself terminates, its file upload session URLs are retained no
1258+
longer than the parent's own :ref:`status URL <publishing-session-retention>` and **MAY** return ``404 Not
1259+
Found`` thereafter. As with cancellation, the data-bearing and mechanism portions of these URLs may become
1260+
unavailable as soon as the underlying file data is purged, independent of the status URL.
1261+
11631262
.. _file-upload-session-extension:
11641263

11651264
File Upload Session Extension
@@ -1642,6 +1741,16 @@ these constraints upfront is simpler when the project is known at session creati
16421741
project version. Allowing multiple projects would fundamentally change this model and
16431742
complicate the definition of what "publish" means for a session.
16441743

1744+
Even with the single-project restriction, this PEP still improves multi-project releases. A client releasing
1745+
several projects at once can fully :ref:`stage <staged-preview>` every project -- creating a publishing
1746+
session per project and uploading all of its artifacts -- before a final step runs through and
1747+
:ref:`publishes <publishing-session-completion>` each one. This gives the whole set a "stage everything, then
1748+
publish" workflow, even though each publish is still per-project and atomic on its own.
1749+
1750+
The single-project session also lays a foundation that a future "publish multiple projects" operation could
1751+
build on as a separate endpoint, coordinating the publication of several already-staged sessions, without
1752+
changing the per-session model defined here.
1753+
16451754

16461755
Why is the version required when creating a publishing session?
16471756
---------------------------------------------------------------
@@ -1706,6 +1815,27 @@ as experience is gained operating Upload 2.0.
17061815
Change History
17071816
==============
17081817

1818+
* TBD
1819+
1820+
* Add an **Atomic Publication and Conflicts** section. Specify that publication is atomic with respect to
1821+
the release's filename namespace: the server reserves the session's filenames for the duration of a
1822+
publish so that concurrent uploads (including through the legacy API) receive a ``409 Conflict``, and
1823+
releases the reservation if the publish fails. Clarify that the conflict check at file upload session
1824+
creation is best-effort and that the authoritative check happens atomically at publish time, reported
1825+
synchronously as a ``409 Conflict`` or, for a deferred publish, by moving the session to the ``error``
1826+
state with the reason in ``notices``.
1827+
* Add a **Session Status Retention** section. Specify that after a session reaches a terminal
1828+
(``published`` or ``canceled``) state, the server **SHOULD** keep serving its status URL reporting the
1829+
terminal status for an index-specific retention period, after which it **MAY** return ``404 Not Found``,
1830+
and that clients must be prepared for such a ``404``. Reconcile the cancellation rules accordingly: the
1831+
data-bearing and action URLs may become unavailable once purged, while the status URL is retained.
1832+
Tie file upload session status URL retention to the parent publishing session: while the parent is
1833+
non-terminal the server **SHOULD** keep its file upload session status URLs valid, and once the parent
1834+
terminates they are retained no longer than the parent's status URL.
1835+
* Expand the "Why is the project name required" FAQ to note that single-project sessions still improve
1836+
multi-project releases (all projects can be fully staged before a final step publishes each one) and lay a
1837+
foundation for a possible future "publish multiple projects" endpoint.
1838+
17091839
* `26-Jun-2026 <https://discuss.python.org/t/pep-694-pypi-upload-api-2-0-round-3/107923>`__
17101840

17111841
* Session actions now use dedicated endpoint links instead of an ``action`` key in request bodies.

0 commit comments

Comments
 (0)