Skip to content

test: add benchmark coverage for ExecutorEventHandler performance#6344

Open
sachin9058 wants to merge 3 commits intomindersec:mainfrom
sachin9058:test/executor-benchmark
Open

test: add benchmark coverage for ExecutorEventHandler performance#6344
sachin9058 wants to merge 3 commits intomindersec:mainfrom
sachin9058:test/executor-benchmark

Conversation

@sachin9058
Copy link
Copy Markdown
Contributor

@sachin9058 sachin9058 commented Apr 11, 2026

Summary

This PR adds benchmark coverage for ExecutorEventHandler.HandleEntityEvent to evaluate its performance characteristics.

Currently, there is no benchmark for this critical execution path, making it difficult to understand throughput and memory allocation behavior or detect performance regressions over time. This change introduces both sequential and parallel benchmarks to provide a baseline for future performance analysis.

The benchmarks use a minimal stub executor to isolate handler logic and avoid external dependencies.

No additional dependencies are required for this change.

Fixes #6343

Testing

  • Ran benchmarks locally using:
    go test ./internal/engine -bench=.

Benchmark results:

BenchmarkHandleEntityEvent-12 1803787 635.4 ns/op 840 B/op 14 allocs/op
BenchmarkHandleEntityEventParallel-12 2365395 499.8 ns/op 838 B/op 14 allocs/op

  • Verified both sequential and parallel benchmarks execute successfully
  • Confirmed allocation reporting (allocs/op, B/op)
  • Ensured lint checks pass (make lint-fix)

The parallel benchmark shows improved throughput compared to sequential execution, indicating the handler scales well under concurrent workloads.

No functional changes were introduced, as this PR only adds benchmark tests.
No functional changes were introduced, as this PR only adds benchmark tests.

@sachin9058 sachin9058 requested a review from a team as a code owner April 11, 2026 13:23
@coveralls
Copy link
Copy Markdown

coveralls commented Apr 11, 2026

Coverage Status

coverage: 59.539%. remained the same — sachin9058:test/executor-benchmark into mindersec:main

@evankanderson
Copy link
Copy Markdown
Member

I'm not sure this is especially useful -- the problem with EvalEntityEvent is the user-defined work from policy evaluation, remediation, etc which is triggered by the entity event, not the message handling itself.

@sachin9058
Copy link
Copy Markdown
Contributor Author

@evankanderson Thanks, that makes sense.

You're right that the main cost likely comes from the evaluation and remediation logic triggered by the event, rather than the message handling itself.

I added this as a baseline for the handler, but I agree it would be more useful to benchmark the full execution path, including policy evaluation and remediation.

I can follow up with a more representative benchmark that captures the end-to-end flow if that would be helpful.

@sachin9058 sachin9058 force-pushed the test/executor-benchmark branch from 0ce49fa to 8391f35 Compare April 13, 2026 09:01
@sachin9058
Copy link
Copy Markdown
Contributor Author

@evankanderson

I’ve updated the benchmark to include a lightweight simulation of the execution path instead of a no-op executor. It now includes some structured processing and branching to better approximate evaluation work, while still keeping it deterministic and isolated from external dependencies.

I’ve also kept the original baseline benchmarks for comparison.

Happy to adjust further if you’d prefer a different approach or a deeper integration with the evaluation pipeline.

@evankanderson
Copy link
Copy Markdown
Member

I think this is fine, but the real cost of these executions is user-defined based on the number of policies and the ingestion of git and/or rest resources, including data sources.

We do set a timeout for the user executions, but I believe it may be close to 5 minutes.

@sachin9058 sachin9058 closed this Apr 14, 2026
@sachin9058 sachin9058 reopened this Apr 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add benchmark coverage for ExecutorEventHandler to evaluate performance characteristics

3 participants