[Flex Consumption][Python] Worker shutdown during DrainMode after successful function execution is treated as an error

[moritzkoerber]

We have a Python Azure Function App running on Flex Consumption. A timer-triggered function runs once every minute and usually performs no work.

Occasionally, the function host enters DrainMode after a successful timer invocation/run. During the subsequent host shutdown, the Python worker exits with code 143 and this is reported as an Error-level WorkerProcessExitException, triggering an alert email.

Investigative information

• Timestamp: 2026-09-08T11:04:00.0019081Z
• Functions runtime: ~4
• Functions host version: 4.1053.200.26403
• Worker assembly: Microsoft.Azure.WebJobs.Script.Grpc, Version=4.1053.0.0
• Extension bundle configuration: [4.*, 5.0.0)
• Invocation Id: 06dbbe9e-d06d-4cff-a09a-5556d1e8fefe
• Region: West Europe
• Hosting plan: Azure Functions Flex Consumption
• OS: Linux
• Language: Python 3.12 (v2 programming model)
• Always-ready instances: not configured
• maximum scale out 1

We can provide the function app name and function name privately if necessary.

python exited with code 143 (0x8F)
Language Worker Process exited. Pid=64.

Error output:

Microsoft.Azure.WebJobs.Script.Workers.WorkerProcessExitException: at Microsoft.Azure.WebJobs.Script.Workers.WorkerProcess.ThrowIfExitError
(Microsoft.Azure.WebJobs.Script.Grpc, Version=4.1053.0.0,
 Culture=neutral, PublicKeyToken=null:
 /_/src/WebJobs.Script.Grpc/ProcessManagement/WorkerProcess.cs:276) at Microsoft.Azure.WebJobs.Script.Workers.WorkerProcess.OnProcessExited
(Microsoft.Azure.WebJobs.Script.Grpc, Version=4.1053.0.0,
 Culture=neutral, PublicKeyToken=null:
 /_/src/WebJobs.Script.Grpc/ProcessManagement/WorkerProcess.cs:231)

The successful invocation and the later worker shutdown are separate events. In the example below, the invocation completed successfully, and the host entered DrainMode approximately 40 seconds later:

timestamp	message	severityLevel	itemType	customDimension
2026-09-08T11:04:00.0019081Z	Executing 'Functions.myfunc' (Reason='Timer fired at 2026-09-08T11:04:00.0015964+00:00', Id=06dbbe9e-d06d-4cff-a09a-5556d1e8fefe)	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Function.myfunc","prop__invocationId":"06dbbe9e-d06d-4cff-a09a-5556d1e8fefe","InvocationId":"06dbbe9e-d06d-4cff-a09a-5556d1e8fefe","prop__functionName":"Functions.myfunc","EventId":"1","EventName":"FunctionStarted","prop__reason":"Timer fired at 2026-09-08T11:04:00.0015964+00:00"}
2026-09-08T11:04:00.0020912Z	Trigger Details: ScheduleStatus: {"Last":"2026-09-08T11:03:00.0002346+00:00","Next":"2026-09-08T11:04:00+00:00","LastUpdated":"2026-09-08T11:03:00.0002346+00:00"}	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Function.myfunc","InvocationId":"06dbbe9e-d06d-4cff-a09a-5556d1e8fefe","prop__ScheduleStatus":"{"Last":"2026-09-08T11:03:00.0002346+00:00","Next":"2026-09-08T11:04:00+00:00","LastUpdated":"2026-09-08T11:03:00.0002346+00:00"}"}
2026-09-08T11:04:00.4233598Z	Executed 'Functions.myfunc' (Succeeded, Id=06dbbe9e-d06d-4cff-a09a-5556d1e8fefe, Duration=421ms)	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Function.myfunc","prop__invocationId":"06dbbe9e-d06d-4cff-a09a-5556d1e8fefe","InvocationId":"06dbbe9e-d06d-4cff-a09a-5556d1e8fefe","prop__functionName":"Functions.myfunc","EventId":"2","EventName":"FunctionCompleted","prop__status":"Succeeded","prop__executionDuration":"421"}
2026-09-08T11:04:40.0390727Z	DrainMode mode enabled	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Microsoft.Azure.WebJobs.Host.DrainModeManager"}
2026-09-08T11:04:40.0392468Z	Calling StopAsync on the registered listeners	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Microsoft.Azure.WebJobs.Host.DrainModeManager"}
2026-09-08T11:04:40.0398882Z	Stopping the listener 'Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener' for function 'myfunc'	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Host.Startup","prop__functionName":"myfunc","prop__listenerName":"Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener"}
2026-09-08T11:04:40.0642991Z	Stopped the listener 'Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener' for function 'myfunc'	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Host.Startup","prop__functionName":"myfunc","prop__listenerName":"Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener"}
2026-09-08T11:04:40.066106Z	Call to StopAsync complete, registered listeners are now stopped	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Microsoft.Azure.WebJobs.Host.DrainModeManager"}
2026-09-08T11:04:40.2945768Z		3	exception	{"ProcessId":"44","HostInstanceId":"af740deb-...","prop__{OriginalFormat}":"Language Worker Process exited. Pid=64.","FormattedMessage":"Language Worker Process exited. Pid=64.","LogLevel":"Error","Category":"Worker.rpcWorkerProcess.python.b2ab90d5-dda2-4b9e-afe9-3b5da860a678"}
2026-09-08T11:04:40.2946516Z	Language Worker Process exited. Pid=64.	3	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Error","Category":"Worker.rpcWorkerProcess.python.b2ab90d5-dda2-4b9e-afe9-3b5da860a678"}
2026-09-08T11:04:40.295268Z	Stopping JobHost	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Microsoft.Azure.WebJobs.Hosting.JobHostService"}
2026-09-08T11:04:40.2954379Z	Job host stopped	1	trace	{"ProcessId":"44","HostInstanceId":"af740deb-...","LogLevel":"Information","Category":"Host.Startup"}
2026-09-08T11:04:52.6341332Z	Initializing Warmup Extension.	1	trace	{"ProcessId":"45","HostInstanceId":"848fc292-ce4e-4c38-b6e6-759b37308148","LogLevel":"Information","Category":"Host.Triggers.Warmup"}

Timer trigger:

@myfunc_bp.timer_trigger(
    arg_name="timer",
    schedule="0 * * * * *",
    run_on_startup=False,
    use_monitor=True,
)

Known workarounds

None known. Limiting maximum app scale-out to one instance did not prevent the issue.

Expected behavior

The worker shutdown should not be reported as a function error or trigger a failure alert when the function invocation itself completed successfully.

[adrieellucas]

We are experiencing exactly the same issue described here.

In our case, the Function App is running Node.js rather than Python. The WorkerProcessExitException with exit code 143 and the DrainMode sequence are the same as described in this issue.

Additionally, starting yesterday, we also began seeing:

Microsoft.Azure.WebJobs.Host.FunctionTimeoutAbortException

We have seen this same FunctionTimeoutAbortException behavior in Functions Consumption scenarios as well. In our case, it appears to occur mostly when the function is triggered by Service Bus, at least in the majority of the cases we have observed.

[choogeveen-integro]

We see what looks like the same scale-in/drain behavior, but since a host update with a more serious consequence: invocations that start right around the moment the instance is drained are aborted after 1–3 seconds with FunctionTimeoutAbortException: Worker channel is shutting down. Aborting function. This matches what adrieellucas describes above (Service Bus triggers in particular).

Environment

• Hosting plan: Flex Consumption, West Europe, Linux
• Language: Python 3.14 (v1 programming model), instance memory 2048 MB
• Always-ready: configured only for the HTTP group; the affected functions run on on-demand instances
• Host version: 4.1054.250.26428 (rolled out to our app on 2026-09-17 ~00:00Z; before that 4.1053.200.26403)

What changed with the host version

Host version	Period (our app)	python exited with code 143 (distinct instances)	Worker channel is shutting down. Aborting function.
4.1052.300.26370	2026-08-19 → 2026-08-27	49	0
4.1053.200.26403	2026-08-26 → 2026-09-17	992	0
4.1054.250.26428	2026-09-17 → now	0	3 invocations (2 incidents)

On 4.1053 we had exactly the symptom from this issue: exit code 143 on every scale-in, logged as an error, but no invocation was ever interrupted. Since 4.1054 the 143 errors are gone, and instead we now get aborted invocations. We had not seen this exception once in the 90 days before.

Incident 1 — 2026-09-21 (Service Bus queue trigger)

Time (UTC)	Event
09:33:29.906	Previous invocation on this instance: Executed ... (Succeeded, Duration=7466ms)
09:34:07.439	Executing 'Functions.<servicebus_function>', DeliveryCount: 1 (≈38 s after the previous invocation finished)
09:34:10.888	FunctionTimeoutAbortException: Worker channel is shutting down. Aborting function. → Executed ... (Failed, Duration=3443ms)
09:34:10.907	Message processing error (Action=ProcessMessageCallback, ...); last log line ever written by this instance
09:34:41.644	Message redelivered on another instance, DeliveryCount: 2 → Succeeded

The function was healthy and mid-way through its work: no Python exception, and functionTimeout is nowhere near 3 s. The ≈40 s idle gap before the instance goes away matches the DrainMode timing in the original report.

Invocation Id: 103921fa-cebf-41d3-860b-8efc44ecbfaa

Incident 2 — 2026-09-18 08:30 UTC (two instances within one second)

• Timer trigger fired at 08:30:00.003 and was aborted after 1655 ms with the same exception (Invocation Id 8b2b6a98-decc-47d3-be6d-515e66c42a9d).
• Service Bus queue trigger started at 08:30:01.409 and was aborted after 1383 ms (Invocation Id 29a45a33-b4dd-42ad-bae5-d6c6c742eed3). The message was not abandoned, so it was only redelivered after the 5-minute lock expired.
• Both instances had been up for ~35 minutes and never logged again afterwards.

Why this matters
The docs state that function executions get a grace period of up to 60 minutes during scale-in on Flex Consumption. Here, invocations are killed 1–3 seconds after they start. It looks like a race: the instance is selected for scale-in while idle, but a listener still accepts a new message or timer tick before (or while) the listeners are stopped, and the worker channel is then shut down without waiting for that invocation.

This can lead to partially executed work followed by a retry, which is considerably worse than the false-positive error alert this issue was originally about.

Expected behavior

• Once an instance is selected for drain/scale-in, listeners should stop accepting new work before anything is torn down; and
• an invocation that is already running should be allowed to complete (within the documented grace period) before the worker channel is shut down.

adrieellucas, which host version were you on when the FunctionTimeoutAbortException started on 2026-09-15? That would help confirm whether this correlates with the 4.1054 rollout.

Happy to share the function app name and full traces privately with the team.