⚡️ Add MLIR mapping performance benchmark

[burgholzer]

🤖 AI text below 🤖

Summary

• add deterministic frontier-heavy and routing-heavy MLIR mapping workloads
• add a paired A/B runner with randomized within-pair order, raw samples, robust summaries, and bootstrap confidence intervals
• record source-worktree state, executable SHA-256 digests, and compiler/CMake metadata
• document the timing boundary and the separate correctness and route-quality validation requirements

Motivation

Follow-up to #1930. The local harness used during that review was useful for separating genuine mapping-pass improvements from behavior changes. Publishing it provides a shared starting point for iterating on representative workloads, measurement methodology, and performance-regression evaluation.

The benchmark is opt-in through BUILD_MQT_CORE_BENCHMARKS and does not change production behavior.

Validation

• configured and built mqt-core-mlir-mapping-eval in release mode with MLIR 22.1.3
• exercised both scenarios through the paired runner
• passed all applicable repository hooks for the changed files, including CMake formatting, clang-format, Ruff, ty, Markdown, policy, and lockfile checks
• independently reviewed the exact branch head

Draft iteration points

• workload representativeness and additional mapping/routing scenarios
• route-quality metrics alongside timing
• cross-platform build and metadata handling
• suitable CI integration, if any, as a separate decision

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[burgholzer]

@MatthiasReumann FYI: this is what GPT-5.6-sol came up with when asked for a benchmarking suite for the mapping pass.
I haven't looked too deeply myself, but I would be interested in how that compares to what you have set up locally and whether this might be something to further extend.

[MatthiasReumann]

Thanks @burgholzer! Looks really promising 👀

I've left two comments - nothing dramatic - with some additional ideas.

[MatthiasReumann]

Instead of wrapping pm.run we could also use MLIR's pass instrumentation. In my local benchmarks I implemented it as follows.

struct PassExecutionStats : public mlir::PassInstrumentation {
  PassExecutionStats(Statistics &stats) : stats_(&stats) {}

  void runBeforePass([[maybe_unused]] mlir::Pass *pass,
                     mlir::Operation *op) override {
    nswapsPrev_ = countSwaps(op);
    t0_ = std::chrono::steady_clock::now();
  }

  void runAfterPass([[maybe_unused]] mlir::Pass *pass,
                    mlir::Operation *op) override {
    const auto t1 = std::chrono::steady_clock::now();
    const auto duration =
        std::chrono::duration_cast<std::chrono::milliseconds>(t1 - t0_).count();
    const auto nswapsAfter = countSwaps(op);

    stats_->emplace("time", duration);
    stats_->emplace("nswaps", nswapsAfter - nswapsPrev_);
  }

private:
  std::size_t countSwaps(mlir::Operation *op) {
    std::size_t cnt{0};
    op->walk([&cnt](mlir::qco::SWAPOp) { ++cnt; });
    return cnt;
  }

  Statistics *stats_;
  std::chrono::steady_clock::time_point t0_;
  std::size_t nswapsPrev_;
};

int main() {
  ...
  pm.addInstrumentation(
          std::make_unique<support::PassExecutionStats>(stats));
}

Not sure if this brings any real benefit, but I think mlir-timing is also implemented this way.

[MatthiasReumann]

I really like this idea! Eventually it would be really awesome if we could simply use mqt-bench in Python to generate QASM circuits and run the mapping pass one those. Maybe this is even possible today - Codex could probably implement this with relative ease!

[MatthiasReumann]

If this was possible, we could also think about using this setup as a test-framework instead of the current unit-tests.