Suppress additional polling logs - #11947
Rohit Ranjan (RohitRanjanMS) wants to merge 1 commit into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: fc802fbd-2254-4fb7-81a2-58c1da953c68
There was a problem hiding this comment.
Pull request overview
This PR extends the existing log-category suppression filter to reduce high-volume polling noise by dropping Debug/Trace logs for additional well-known listener categories, while preserving Information+ events. It builds on the earlier telemetry noise suppression work from #11603.
Changes:
- Added four additional polling-related logging categories to the suppressed-category set (Debug/Trace only).
- Expanded unit test coverage to assert suppression behavior (Debug/Trace suppressed; Information+ allowed) for the newly added categories.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/WebJobs.Script/Extensions/ScriptLoggingBuilderExtensions.cs |
Adds new high-volume polling categories to the suppression allowlist (Info+) / denylist (Debug/Trace). |
test/WebJobs.Script.Tests/Extensions/ScriptLoggingBuilderExtensionsTests.cs |
Adds unit test cases validating the new categories are suppressed at Debug/Trace and allowed at Information+. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| "Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener", | ||
| "Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener", | ||
| "Microsoft.Azure.WebJobs.EventHubs.EventHubProducerClientImpl", | ||
| "Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener", |
There was a problem hiding this comment.
Did you double-check what we'll lose from this? Is it just the polling logs? We should still have the option to do a { Category, EventName } pair for these if there's something we want to be more targeted with.
There was a problem hiding this comment.
Yeah, I think the category/eventname approach would be safer. All of the listener start/stop lifecycle logs appear to be logged with this category, e.g. https://github.com/Azure/azure-sdk-for-net/blob/ca61f202024f99d33f4693fe3176d54092f41205/sdk/storage/Microsoft.Azure.WebJobs.Extensions.Storage.Common/src/Shared/Queues/QueueListener.cs#L173. We don't want to lose those.
| "Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener", | ||
| "Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener", | ||
| "Microsoft.Azure.WebJobs.EventHubs.EventHubProducerClientImpl", | ||
| "Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener", |
There was a problem hiding this comment.
Ah subtle point here - this source you're adding Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener is for older versions of the extension - the current QueueListener in newer storage extension is in namespace Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener. The code you're modifying here already filters Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener - has that been released yet? We don't want to be filtering all these QueueListener debug logs out
| "Host.Triggers.Kafka", | ||
| "Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessor", | ||
| "Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener", | ||
| "Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener", |
There was a problem hiding this comment.
The logger for this listener is here https://github.com/Azure/azure-sdk-for-net/blob/ca61f202024f99d33f4693fe3176d54092f41205/sdk/storage/Microsoft.Azure.WebJobs.Extensions.Storage.Blobs/src/Listeners/BlobListener.cs#L35 and appears only to be used to log listener start/stop lifecycle events. Those are important and also aren't high volume, so why do we want to suppress them?
Surveying prod logs, I see the following event names being logged using this source: "BlobAlreadyProcessed", "PollBlobContainer", "BlobDoesNotMatchPattern", "BlobMessageEnqueued", "ContainerDoesNotExist", "InitializedScanInfo", "ScanBlobLogs", "FunctionNotFound", "BlobHasNoETag"
Some of those are clearly noisy, but some are useful for debugging and infrequent. I worry that simply throwing all of these away is going to hamper our ability to investigate issues.
| // suppressed to reduce noise in the FunctionsLogs table. | ||
| private static readonly HashSet<string> _suppressedCategories = new(StringComparer.Ordinal) | ||
| { | ||
| "Host.Triggers.Kafka", |
There was a problem hiding this comment.
Tsuyoshi Ushio (@TsuyoshiUshio) to review what types of logs we emit using this category and whether anything important would be lost
| private static readonly HashSet<string> _suppressedCategories = new(StringComparer.Ordinal) | ||
| { | ||
| "Host.Triggers.Kafka", | ||
| "Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessor", |
There was a problem hiding this comment.
I notice that for EventHub, you're not suppressing Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener. If we're inconsistent across extensions wrt. suppression of their listener start/stop lifecycle events as we will be with these changes, I wonder if that will cause confusion - we'll no longer know which extensions we can count on these logs for in investigations.
Issue describing the changes in this PR
Follow-up to #11603.
Suppresses Debug and Trace logs from these additional high-volume polling categories while preserving Information and higher:
Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessorMicrosoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListenerHost.Triggers.KafkaMicrosoft.Azure.WebJobs.Host.Queues.Listeners.QueueListenerPull request checklist
IMPORTANT: Currently, changes must be backported to the
in-procbranch to be included in Core Tools and non-Flex deployments.in-procbranch is not required