Skip to content

Suppress additional polling logs #11947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Rohit Ranjan (RohitRanjanMS) wants to merge 1 commit into
base: dev
Choose a base branch
from
+20 −0
Open

Suppress additional polling logs #11947

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/WebJobs.Script/Extensions/ScriptLoggingBuilderExtensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,12 @@ public static class ScriptLoggingBuilderExtensions
// suppressed to reduce noise in the FunctionsLogs table.
private static readonly HashSet<string> _suppressedCategories = new(StringComparer.Ordinal)
{
"Host.Triggers.Kafka",

@mathewc Mathew Charles (mathewc) Sep 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tsuyoshi Ushio (@TsuyoshiUshio) to review what types of logs we emit using this category and whether anything important would be lost

"Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessor",

@mathewc Mathew Charles (mathewc) Sep 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice that for EventHub, you're not suppressing Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener. If we're inconsistent across extensions wrt. suppression of their listener start/stop lifecycle events as we will be with these changes, I wonder if that will cause confusion - we'll no longer know which extensions we can count on these logs for in investigations.

"Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener",
"Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener",

@mathewc Mathew Charles (mathewc) Sep 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger for this listener is here https://github.com/Azure/azure-sdk-for-net/blob/ca61f202024f99d33f4693fe3176d54092f41205/sdk/storage/Microsoft.Azure.WebJobs.Extensions.Storage.Blobs/src/Listeners/BlobListener.cs#L35 and appears only to be used to log listener start/stop lifecycle events. Those are important and also aren't high volume, so why do we want to suppress them?

Surveying prod logs, I see the following event names being logged using this source: "BlobAlreadyProcessed", "PollBlobContainer", "BlobDoesNotMatchPattern", "BlobMessageEnqueued", "ContainerDoesNotExist", "InitializedScanInfo", "ScanBlobLogs", "FunctionNotFound", "BlobHasNoETag"

Some of those are clearly noisy, but some are useful for debugging and infrequent. I worry that simply throwing all of these away is going to hamper our ability to investigate issues.

"Microsoft.Azure.WebJobs.EventHubs.EventHubProducerClientImpl",
"Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener",

@brettsam Brett Samblanet (brettsam) Aug 27, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you double-check what we'll lose from this? Is it just the polling logs? We should still have the option to do a { Category, EventName } pair for these if there's something we want to be more targeted with.

@mathewc Mathew Charles (mathewc) Sep 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think the category/eventname approach would be safer. All of the listener start/stop lifecycle logs appear to be logged with this category, e.g. https://github.com/Azure/azure-sdk-for-net/blob/ca61f202024f99d33f4693fe3176d54092f41205/sdk/storage/Microsoft.Azure.WebJobs.Extensions.Storage.Common/src/Shared/Queues/QueueListener.cs#L173. We don't want to lose those.

@mathewc Mathew Charles (mathewc) Sep 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah subtle point here - this source you're adding Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener is for older versions of the extension - the current QueueListener in newer storage extension is in namespace Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener. The code you're modifying here already filters Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener - has that been released yet? We don't want to be filtering all these QueueListener debug logs out

"Host.Executor"
};

Expand Down
16 changes: 16 additions & 0 deletions test/WebJobs.Script.Tests/Extensions/ScriptLoggingBuilderExtensionsTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,14 @@ public void Filter_RespectsMinLevelAndCategory(string category, LogLevel actualL
[InlineData("Microsoft.Azure.WebJobs.EventHubs.EventHubProducerClientImpl", LogLevel.Trace)]
[InlineData("Host.Executor", LogLevel.Debug)]
[InlineData("Host.Executor", LogLevel.Trace)]
[InlineData("Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessor", LogLevel.Debug)]
[InlineData("Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessor", LogLevel.Trace)]
[InlineData("Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener", LogLevel.Debug)]
[InlineData("Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener", LogLevel.Trace)]
[InlineData("Host.Triggers.Kafka", LogLevel.Debug)]
[InlineData("Host.Triggers.Kafka", LogLevel.Trace)]
[InlineData("Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener", LogLevel.Debug)]
[InlineData("Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener", LogLevel.Trace)]
public void Filter_SuppressedCategory_DebugAndTrace_ReturnsFalse(string category, LogLevel level)
{
Assert.False(ScriptLoggingBuilderExtensions.Filter(category, level, LogLevel.Trace));
Expand All @@ -74,6 +82,14 @@ public void Filter_SuppressedCategory_DebugAndTrace_ReturnsFalse(string category
[InlineData("Microsoft.Azure.WebJobs.EventHubs.EventHubProducerClientImpl", LogLevel.Error)]
[InlineData("Host.Executor", LogLevel.Information)]
[InlineData("Host.Executor", LogLevel.Critical)]
[InlineData("Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessor", LogLevel.Information)]
[InlineData("Microsoft.Azure.WebJobs.EventHubs.Listeners.EventHubListener.PartitionProcessor", LogLevel.Warning)]
[InlineData("Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener", LogLevel.Information)]
[InlineData("Microsoft.Azure.WebJobs.Extensions.Storage.Blobs.Listeners.BlobListener", LogLevel.Error)]
[InlineData("Host.Triggers.Kafka", LogLevel.Information)]
[InlineData("Host.Triggers.Kafka", LogLevel.Critical)]
[InlineData("Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener", LogLevel.Information)]
[InlineData("Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener", LogLevel.Warning)]
public void Filter_SuppressedCategory_InformationAndAbove_ReturnsTrue(string category, LogLevel level)
{
Assert.True(ScriptLoggingBuilderExtensions.Filter(category, level, LogLevel.Trace));
Expand Down
Loading