Skip to content

feat: serverless worker#4445

Open
lennessyy wants to merge 29 commits intomainfrom
feat/serverless-worker-prerelease
Open

feat: serverless worker#4445
lennessyy wants to merge 29 commits intomainfrom
feat/serverless-worker-prerelease

Conversation

@lennessyy
Copy link
Copy Markdown
Contributor

@lennessyy lennessyy commented Apr 16, 2026

What does this PR do?

  • Adds the serverless worker feature

Notes to reviewers

┆Attachments: EDU-6216 feat: serverless worker

lennessyy and others added 3 commits April 16, 2026 16:06
* docs: Serverless Workers - Go SDK pages (2/4)

Add Go SDK Serverless Workers documentation including the lambdaworker
package guide for AWS Lambda, rewrite run-worker-process to focus on
long-lived Workers, and remove cloud-worker (content folded in).
Update sidebars, add redirect, and set broken links to warn for
cross-PR references.

Part of #4405.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: expand OTel section with context and links for serverless Go SDK page

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: link worker defaults to Go SDK reference, clarify ShutdownDeadlineBuffer is serverless-only

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address review feedback on Go SDK pages

- Replace ambiguous "Lambda deadline" with "configurable invocation
  deadline" on the AWS Lambda page (akhayam)
- Rewrite tautological "serverless compute" intro on the Go SDK
  serverless landing page (smuneebahmad)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add Serverless Workers production deployment guide (3/4)

Add deploy guide for serverless workers covering AWS Lambda deployment,
including the serverless-workers index and aws-lambda pages under
production-deployment/worker-deployments. Update sidebar navigation
and set onBrokenLinks/onBrokenAnchors to warn for cross-PR references.

Part of #4405.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address review feedback on Deploy guide

- Rename TLS env vars to TEMPORAL_TLS_CLIENT_CERT_PATH and
  TEMPORAL_TLS_CLIENT_KEY_PATH (smuneebahmad)
- Add HOME=/tmp env var to deploy command and env var table; needed
  for the SDK's config loader to resolve a user config directory in
  Lambda (smuneebahmad)
- Include TEMPORAL_API_KEY in deploy command so the auth path is
  complete (smuneebahmad)
- Remove --scaler-min-instances, --scaler-max-instances from
  create-version snippet (smuneebahmad)
- Remove --ignore-missing-task-queues from set-current-version
  snippet (smuneebahmad)
- Replace CloudFormation template stub with the real template from
  smuneebahmad, inline in a <details> block plus a downloadable file
  at /files/temporal-cloud-serverless-worker-role.yaml
- Update IAM parameter table to match real template params
  (AssumeRoleExternalId, LambdaFunctionARNs, RoleName)
- Add note flagging template is Cloud-scoped; self-hosted TBD
- Add sample aws cloudformation create-stack usage
- Use "invocation deadline" for consistency with Go SDK page

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: Serverless Workers - Evaluate pages (1/4)

Add the Evaluate section for Serverless Workers documentation:
- Serverless Workers overview page
- Interactive demo page with ServerlessWorkerDemo component
- Sidebar entry under Features
- Redirect from old demo URL
- Change onBrokenLinks/onBrokenAnchors to 'warn' for incremental PRs

Part 1 of 4, splitting PR #4405 into smaller PRs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address PM feedback on evaluate page

- Fix polling description: serverless workers still poll, the difference is lifecycle
- Tone down operational overhead claims: customers still deploy and configure
- Clarify long-running limitation applies to activities, not workflows

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address review feedback on Evaluate pages

- Reframe lifecycle description: Temporal invokes the Serverless Worker
  on demand (bchav, akhayam)
- Clarify operational overhead: offload invocation and scaling, but
  deployments remain the user's responsibility (bchav)
- Introduce "long-lived Workers" terminology and use consistently
- Sharpen Lambda execution limit wording and Cloud Run callout
  (akhayam, bchav)
- Remove Worker Versioning row from comparison table as too low-level
  (akhayam)
- Remove --scaler-min-instances, --scaler-max-instances, and
  --ignore-missing-task-queues from demo CLI snippets (smuneebahmad)
- Remove Min/Max Instances config fields from demo UI (smuneebahmad)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 16, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
temporal-documentation Ready Ready Preview, Comment Apr 29, 2026 2:29am

Request Review

@lennessyy lennessyy marked this pull request as ready for review April 17, 2026 01:54
@lennessyy lennessyy requested a review from a team as a code owner April 17, 2026 01:54
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 17, 2026

* docs: add Serverless Workers encyclopedia page and update related pages

Add the encyclopedia entry for Serverless Workers, update workers.mdx
and task-queues.mdx with serverless references, add the architecture
diagram, update sidebar, and add "Serverless Worker" to Vale terms.

Change onBrokenLinks/onBrokenAnchors to 'warn' to accommodate
cross-references to pages in other PRs in this series.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Remove differences section from encyclopedia page

The comparison table lives on the evaluate page now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix Worker invocation wording in encyclopedia

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address review feedback on encyclopedia page

- Use "traditional long-lived Worker" on first contrast, "long-lived
  Worker" thereafter (akhayam, smuneebahmad)
- Replace "triggers the compute environment" with "invokes the
  Serverless Worker on demand" (akhayam)
- Use "shuts down" instead of "exits" to match AWS Lambda runtime
  terminology (akhayam)
- Fix "serverless function" -> "Serverless Worker" for cross-provider
  accuracy
- Use "invocation deadline" for consistency with other pages
- Add Worker lifecycle section with new lifecycle diagram (addresses
  akhayam's suggestion to mirror AWS's Lambda lifecycle diagram)
- Explain Worker stop timeout and shutdown deadline buffer, including
  tuning guidance for long-running Activities and the consequences of
  raising one knob without the other

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: crop lifecycle diagram

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: add light/dark themed diagrams with figure captions

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…aram table

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…o Go SDK page

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ment

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lennessyy and others added 2 commits April 21, 2026 10:29
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: add Serverless Worker Autoscaling encyclopedia page

Add a dedicated page explaining how Temporal autoscales Serverless Workers
on AWS Lambda, covering scaling signals (backlog + sync match rate),
the push-based scaling flow, failure handling, and key constraints.

Also adds an Autoscaling section to the existing Serverless Workers
encyclopedia page linking to the new page, and a sidebar entry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* merge autoscaling info with existing docs

* copyedits

* copyedits

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Lenny Chen <lenny.chen@temporal.io>
…ation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lopedia

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add self-hosted setup page for serverless workers

Covers enabling the Worker Controller via dynamic config, configuring
AWS credentials for the Temporal server, and creating the Lambda
invocation role with a CloudFormation template adapted for self-hosted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add describe-stacks command to retrieve role ARN

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add WCI verification steps and ARN discovery tip

Add how to view WCI workflows using TemporalNamespaceDivision filter
to both the self-hosted setup page and the deploy guide. Add
aws sts get-caller-identity tip for finding the server's IAM ARN.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: remove WCI filter tip from deploy guide

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: mark IAM section as Cloud-only, link to self-hosted setup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add (Cloud only) to IAM heading

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: move Cloud-only callout to top of IAM section

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address PR review feedback on self-hosted setup page

- Remove first paragraph (Cloud comparison)
- Add brief overview of the three setup steps
- Use "Worker Controller Instance (WCI)" instead of "Worker Controller"
- Move dynamic config reference table out (belongs on dynamic config page)
- Replace verify section with Next steps linking to deploy guide

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* prettier

* docs: reference self-hosted setup page in deploy guide prerequisites

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Apply suggestions from code review

Co-authored-by: Stefan Richter <stefan@02strich.de>

* Apply suggestions from code review

Co-authored-by: Stefan Richter <stefan@02strich.de>

* copyedits

* copyedits

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Stefan Richter <stefan@02strich.de>
* feat: serverless worker - typescript

* docs: fix TS serverless worker review items

- Add HOME=/tmp to TS create-function env vars
- Fix makeOtelPlugins() to makeOtelPlugin() to match SDK export
- Add versioning behavior requirement note

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix ADOT layer instructions and add X-Ray permissions note

- Correct two-layer instruction to single ADOT Node.js layer (includes collector)
- Add AWS_LAMBDA_EXEC_WRAPPER env var requirement
- Add AWSXRayDaemonWriteAccess policy requirement with silent failure warning

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: revert to two-layer ADOT setup, keep permissions note

Revert to the original two-layer ADOT instruction from the SDK author.
The single-layer setup was not verified end-to-end. Keep the
AWSXRayDaemonWriteAccess and AWS_LAMBDA_EXEC_WRAPPER additions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add custom OTel collector config and setup instructions for TS

The default ADOT collector config does not route OTLP data to the traces
pipeline. Add the custom collector config YAML that wires OTLP to both
traces (X-Ray) and metrics (CloudWatch EMF). Document the required env
vars and IAM permissions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix OTel collector config to use logging exporter

The standalone ADOT collector layer (v0.40.0) does not support the
debug exporter. Replace with logging, which is the supported equivalent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update TS OTel to match sample (two layers, debug exporter)

Match the TS sample exactly:
- Two ADOT layers (JS layer + standalone collector)
- debug exporter (not logging)
- AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-instrument
- OPENTELEMETRY_COLLECTOR_CONFIG_URI (not _FILE)
- tracing-config Mode=Active
- IAM permissions for xray, cloudwatch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix AWS_LAMBDA_EXEC_WRAPPER to /opt/otel-handler for Node.js

The Node.js ADOT layer ships /opt/otel-handler, not /opt/otel-instrument
(which is the Python wrapper). Verified end-to-end: X-Ray traces confirmed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
lennessyy and others added 2 commits April 28, 2026 11:57
* feat: python serverless worker

* docs: add versioning behavior samples, packaging fixes, heading cleanup

- Add versioning behavior (PINNED) to Python and Go code samples
- Add VersioningBehavior import from temporalio.common for Python
- Fix Python packaging to zip deps first then add app files
- Change Go samples from AutoUpgrade to Pinned
- Add Python and TypeScript tabs to deploy guide
- Clean up deploy guide headings (remove articles)
- Add Serverless Worker link in deploy guide intro

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Revert "docs: add versioning behavior samples, packaging fixes, heading cleanup"

This reverts commit 6fe5cc5.

* docs: add Python versioning behavior samples and packaging fix

- Add versioning behavior (PINNED) example to Python SDK page and deploy guide
- Fix Python packaging: use --platform manylinux2014_x86_64 for Lambda
- Zip deps first, then add app files separately

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: restructure Python SDK page for readability

Highlight run_worker lines, explain configure callback right after
the code block, move versioning behavior below.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: explain deployment name and build ID with links to versioning

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Apply suggestions from code review

Co-authored-by: Lenny Chen <55669665+lennessyy@users.noreply.github.com>

* docs: add custom OTel collector config and setup instructions for Python

The default ADOT collector config does not route OTLP data to the traces
pipeline. Add the custom collector config YAML that wires OTLP to both
traces (X-Ray) and metrics (CloudWatch EMF). Document the required env
var and IAM permissions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix OTel collector config to use logging exporter

The standalone ADOT collector layer (v0.40.0) does not support the
debug exporter. Replace with logging, which is the supported equivalent.
Verified end-to-end: traces now appear in X-Ray.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update Python OTel to use language-specific ADOT layer

Use the ADOT Python layer (includes auto-instrumentation + collector)
instead of standalone collector layer. Matches the verified sample setup.
Use debug exporter (supported by the newer collector in the language
layer). Add AWS_LAMBDA_EXEC_WRAPPER and tracing-config Mode=Active.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: fix bad merge

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Member

@Sushisource Sushisource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just focused on the SDK / getting started instructions, but those parts are looking good. Thanks Lenny!

Comment on lines +78 to +91
| Setting | Lambda default |
|---|---|
| `MaxConcurrentActivityExecutionSize` | 2 |
| `MaxConcurrentWorkflowTaskExecutionSize` | 10 |
| `MaxConcurrentLocalActivityExecutionSize` | 2 |
| `MaxConcurrentNexusTaskExecutionSize` | 5 |
| `MaxConcurrentActivityTaskPollers` | 1 |
| `MaxConcurrentWorkflowTaskPollers` | 2 |
| `MaxConcurrentNexusTaskPollers` | 1 |
| `WorkerStopTimeout` | 5 seconds |
| `DisableEagerActivities` | Always true |
| Sticky cache size | 100 |
| `ShutdownDeadlineBuffer` | 7 seconds |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think maybe we can keep the paragraph above but omit the exact values just to avoid future drift, since I suspect we may end up changing them? I'm fine if you want to keep them too.

Applies to other langs too.

- For self-hosted deployments, complete the
[self-hosted setup](/production-deployment/worker-deployments/serverless-workers/self-hosted-setup) before following
this guide.
- Every Workflow must declare a [versioning behavior](/worker-versioning#versioning-behaviors).
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or the worker needs to set a default behavior

The underlying metrics and traces are the same ones the Python SDK emits in any environment.
For general observability concepts and the full list of available metrics, see [Observability - Python SDK](/develop/python/observability) and the [SDK metrics reference](/references/sdk-metrics).

from temporalio.common import WorkerDeploymentVersion
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is missing a ``` start fence

Install dependencies into a local directory for packaging. Use `--platform` to fetch Linux-compatible binaries for the Lambda runtime:

```bash
pip install --target ./package --platform manylinux2014_x86_64 --only-binary=:all: temporalio
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would probably be nice to show how to include the lambda-worker-otel optional dependency set if desired

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants